Preheat oven to 350°F (175°C). Grease a 9×5-inch loaf pan with butter or cooking spray, and optionally line with parchment paper leaving an overhang on the sides.
Peel bananas and mash in a large bowl with a fork or potato masher until mostly smooth with a few small lumps, yielding about 1½ cups of mashed banana.
Add brown sugar, oil, yogurt (or buttermilk), and vanilla extract to the mashed bananas; whisk until well combined, smooth, and creamy.
In a separate bowl, whisk together flour, baking soda, baking powder, salt, cinnamon, and nutmeg for about 30 seconds until evenly combined with no lumps.
Pour the dry ingredients into the wet and gently fold with a rubber spatula just until no dry streaks remain — do not overmix; the batter should be thick and slightly lumpy.
If using, fold in nuts and/or chocolate chips with a few gentle strokes, reserving a small handful to sprinkle on top.
Pour batter into the prepared loaf pan, smooth the top, sprinkle reserved mix-ins if using, and tap the pan gently on the counter 2–3 times to release air bubbles.
Bake for 50–60 minutes until a toothpick inserted in the center comes out clean or with a few moist crumbs; tent loosely with foil if the top browns too quickly.
Cool in the pan on a rack for 10–15 minutes, then remove and cool completely (1–2 hours) before slicing for clean cuts and the best texture.