Preheat your oven to 350°F (175°C) and position the rack in the center. Grease a 9x5-inch loaf pan and line it with parchment paper, leaving overhang on the long sides for easy removal.
In a medium bowl, combine 1/3 cup flour, 1/3 cup sugar, 1/2 teaspoon cinnamon, and 1/4 teaspoon nutmeg. Cut in 1/4 cup cold cubed butter with a pastry cutter until the mixture resembles coarse, damp sand. Refrigerate while you make the batter.
In a medium bowl, whisk together 1 3/4 cups flour, baking powder, baking soda, 3/4 teaspoon cinnamon, and salt until evenly combined. Set aside.
In a large mixing bowl, beat 1/2 cup room-temperature butter and 1 cup sugar with an electric hand mixer on medium speed for 2-3 minutes until light and fluffy. Scrape down the bowl as needed.
With the mixer on low, add the egg and mix until just combined, about 20-30 seconds. Add vanilla and sour cream, mixing until the mixture is smooth.
Pour the dry ingredients into the wet ingredients and gently fold with a rubber spatula until no dry flour streaks remain. Do not overmix. Gently fold in the chopped peaches with 4-5 careful folds until evenly distributed.
Scrape the batter into the prepared loaf pan and smooth the top with a spatula. Remove the chilled crumble topping from the refrigerator and scatter it evenly over the batter, gently pressing it down so it adheres.
Bake on the center rack for 60-70 minutes, until a wooden skewer inserted into the center comes out clean or with just a few moist crumbs. Tent with foil after 45 minutes if the topping is browning too quickly.
Let the bread cool in the pan for 15 minutes, then use the parchment paper overhangs to lift the loaf out and transfer to a wire cooling rack. Cool for an additional 30-45 minutes before slicing with a serrated knife.