Preheat oven to 350°F (175°C). Grease a 9x5-inch loaf pan and line with parchment paper, leaving overhang on the long sides to use as a sling.
If using fresh peaches, blanch, peel, and dice into 1/2-inch chunks. Thaw and thoroughly pat dry frozen peaches, or drain and dry canned peaches. Set on paper towels to draw out excess moisture.
In a large bowl, mash the bananas with a fork or potato masher until mostly smooth with a few small lumps.
Add cooled melted butter to the bananas and stir well. Mix in eggs one at a time, then add brown sugar, milk, and vanilla until the wet mixture is smooth and cohesive.
In a medium bowl, whisk together flour, baking soda, baking powder, salt, cinnamon, and nutmeg until fully combined and lump-free.
Add dry ingredients to wet mixture in two additions, folding gently with a silicone spatula just until no dry streaks remain. Do not overmix.
Gently fold in the prepared peach chunks and walnuts (if using) with just a few strokes to distribute evenly.
Pour batter into the prepared loaf pan and spread evenly. Optional: arrange peach slices on top and sprinkle with brown sugar and cinnamon.
Bake for 55-65 minutes, or until a toothpick inserted in the center comes out clean or with a few moist crumbs. Tent with foil if top browns too quickly. Cool in pan for 15 minutes, then use parchment to lift out and cool completely on a wire rack before slicing.