Preheat oven to 325°F (170°C). Spray a 9-inch springform pan or 3-inch tall-sided round cake pan with baking spray with flour. Line bottom of pan with parchment paper.
In the bowl of a stand mixer fitted with the paddle attachment, beat butter and sugar at medium speed until fluffy, 3 to 4 minutes, stopping to scrape sides of bowl. Add oil, vanilla, salt, and almond extract, beating just until combined. Add eggs and egg yolks, one at a time, beating just until combined after each addition.
In a medium bowl, whisk together flour and baking powder. With mixer on low speed, gradually add flour mixture to butter mixture alternately with buttermilk, beginning and ending with flour mixture, beating just until combined after each addition.
In another medium bowl, beat cream with a mixer at high speed until soft peaks form, about 3 minutes. Fold whipped cream into batter. Pour batter into prepared pan.
Bake until light golden brown and a wooden pick inserted in center comes out clean, about 40 to 45 minutes. Let cool in pan for 10 minutes. Remove from pan, and let cool completely on a wire rack.
Using a serrated knife, cut cooled cake in half horizontally. Reserve ¼ Amaretto Pastry Cream; spread remaining Amaretto Pastry Cream onto first cake layer. Top with remaining cake layer. Spread remaining Amaretto Pastry Cream on sides of cake; press almonds onto sides of cake. Pour Chocolate Ganache over top of cake. Refrigerate until ganache is set, about 15 minutes. Serve at room temperature. Cover and refrigerate for up to 5 days.
Recipe by Bake from Scratch at https://staging.bakefromscratch.com/amaretto-boston-cream-pie/