Preheat the oven to 350 degrees F. Spray a 9 x 13 pan with nonstick spray.
PREPARE THE CAKE:
In a large bowl, use an electric mixer to combine the sugars with the butter, beating until fluffy. Add the eggs and vanilla and beat until well-combined. In a separate bowl, whisk together the flour, baking soda, baking powder, cinnamon, salt and nutmeg. Add the dry ingredients and buttermilk alternately to the wet mixture, making sure the batter is mixed well.
Scrape the batter into the prepared pan. Place peach wedges decoratively on top of the batter. Bake 30 to 35 minutes (depending on size of the pan) or until a wooden pick inserted near the center comes out clean. Let cool.
PREPARE THE GLAZE:
In a medium bowl, use an electric mixer to combine the cream cheese with the powdered sugar. Mix until smooth. Add the vanilla and 1 tablespoon of milk. You want a drizzling consistency, so add a little more milk if you don't have that yet. Drizzle the glaze over the cooled cake. Keep covered at room temperature, or cover and refrigerate.
Notes
*To make your own buttermilk (aka sour milk), combine 2/3 cup milk with 1/2 teaspoon vinegar. Let stand for 5 minutes before adding it to the recipe.
*No need to peel the peaches unless you really want to. The recipe is perfectly delicious when the peel is left on.