Preheat oven to 350°F (175°C). Grease two 9-inch round cake pans and line bottoms with parchment paper.
Sift together the cake flour, baking powder, and fine sea salt in a large bowl. Whisk to combine.
Beat softened butter until smooth, then gradually add sugar and beat on medium-high for 4-5 minutes until pale and fluffy.
Add eggs one at a time, mixing well after each. Add vanilla and almond extracts and scrape down the bowl.
Alternate adding dry ingredients and milk, mixing on low speed just until combined. Do not overmix.
Gently fold in the rainbow jimmies with a rubber spatula using 8-10 slow strokes.
Divide batter evenly between prepared pans. Bake for 28-33 minutes until a toothpick comes out clean.
Cool in pans for 10 minutes, then invert onto a wire rack. Cool completely before frosting.
Beat softened butter for 3-4 minutes until pale and fluffy. Gradually add powdered sugar, then vanilla, salt, and heavy cream. Beat until smooth.
Place one cake layer on a plate. Spread with frosting, top with second layer. Apply a thin crumb coat and chill for 20-30 minutes.
Apply final layer of frosting and smooth sides and top. Decorate with additional sprinkles. Serve immediately or refrigerate.