If using fresh peaches, blanch them in boiling water for 30-60 seconds, then transfer to ice water. Peel, pit, and slice into 1/2-inch thick slices. Toss with sugar, lemon juice, and cinnamon. Refrigerate for at least 30 minutes. If using canned peaches, drain well and pat dry.
Cut the pound cake into 1-inch cubes. If using fresh soft cake, let it sit uncovered for 20-30 minutes to firm up slightly.
In a large bowl, whisk the instant pudding mix with cold whole milk for about 2 minutes until smooth and beginning to thicken. In a separate bowl, whip 1 cup of cold heavy cream to soft peaks, then gently fold it into the pudding mixture.
In a chilled bowl, beat 2 cups of cold heavy cream with powdered sugar and vanilla extract until stiff peaks form. Refrigerate until ready to use.
In the trifle bowl, arrange half of the pound cake cubes in an even layer. Drizzle with half of the soaking liquid and let it soak in for 1-2 minutes.
Spread half of the vanilla custard over the cake layer. Top with half of the macerated peaches and their syrup, pressing a few slices against the glass sides for visual effect.
Repeat the layers with the remaining pound cake, soaking liquid, custard, and peaches. Press a few peach slices against the glass for presentation.
Top with the whipped cream, spreading into an even, slightly mounded layer. Garnish with reserved peach slices, toasted almonds, and mint sprigs if desired. Cover and refrigerate for a minimum of 4 hours, preferably overnight.