Preheat oven to 350 degrees F. Line a 9x13 pan with parchment paper and spray with nonstick cooking spray. Set aside.
Mix cinnamon, ginger, cloves, and nutmeg together in a large bowl. Remove 1/2 teaspoon for the frosting (“Reserved Spices”). To the large bowl, mix in flour, baking powder, baking soda and salt. Set aside.
With a stand or hand mixer, beat butter and sugar over medium high speed for a full 4 minutes until very light and fluffy, scraping down sides as needed. With mixer running on medium, add vanilla followed by eggs, then egg yolk, ONE at a time, beating just until the yellow disappears after each egg.
Gradually add flour mixture to butter/sugar mixture in thirds, alternately with buttermilk in between each third, beating until just combined after each addition. Stir in apples and walnuts just until combined. Spoon batter into prepared pan.
Bake at 350 degrees F until a wood pick inserted near the center of cake comes out clean, approximately 45-50 minutes. **NOTE: While the cake is baking is the perfect time to make your Caramel Sauce!**
Let cake cool in pan on wire rack for 5 minutes then poke holes about an inch apart with the back of a straw all over the cake. Pour 1 1/2 cups caramel over cake and spread into holes with a spatula. Cover and refrigerate overnight (see notes about frosting head).
Spiced Frosting: Using a stand or hand mixer, beat the cream cheese, butter, sour cream, vanilla, salt and reserved 1/2 teaspoon spices at medium high speed until well combined. Add confectioners’ sugar 1/2 cup at a time then beat until very fluffy, about 2 minutes. Add more sugar if needed to reach desired consistency. Evenly spread frosting over cake and sprinkle with toffee bits.
If serving Caramel Apple Poke Cake immediately after frosting, then let cake sit at room temperature BEFORE frosting to come to room temperature. Alternatively, you can frost the cake and return to refrigerator then bring to room temperature before serving.