Preheat the oven to 375°F (190°C). Grease a 2-quart or 9x13-inch baking dish with softened butter or cooking spray.
Cut the bread into 1-inch cubes. If using fresh bread, dry it in a 300°F oven for 10 minutes. Place cubes in the dish and drizzle with melted butter, tossing gently to coat.
Peel and slice bananas into 1/4-inch rounds. Scatter evenly over the buttered bread cubes, tucking some between the pieces.
In a medium bowl, whisk the eggs, then add milk, granulated sugar, brown sugar, vanilla, cinnamon, nutmeg, and salt. Whisk until smooth and fully combined.
Pour the custard evenly over the bread and bananas. Gently press the bread down with a spatula to submerge. Let stand at room temperature for at least 15 minutes.
Bake uncovered on the center rack for 40 to 45 minutes until golden brown and the center is set. A thermometer should read 160°F (71°C) in the center. Tent with foil if browning too quickly.
Remove and let rest for 10 minutes while making the sauce. In a small saucepan over medium heat, combine brown sugar, butter, and cream. Simmer gently for 2-3 minutes until thickened, then stir in vanilla and a pinch of salt.
Scoop the warm pudding into bowls and drizzle generously with the warm brown sugar vanilla sauce. Add optional toppings like vanilla ice cream, whipped cream, or toasted nuts.