Preheat oven to 350°F (180°C). Line a 13x9-inch baking pan with parchment paper, letting excess extend over sides of pan.

In a large bowl, stir together granulated sugar and melted butter until combined. Add eggs, vanilla paste, and orange zest, stirring until well combined.

In a medium bowl, whisk together flour, baking powder, salt, and baking soda. Gradually add flour mixture to sugar mixture alternately with buttermilk, beginning and ending with flour mixture, stirring just until combined after each addition. Pour batter into prepared pan, smoothing top with a small offset spatula.

Bake until cake is puffed but top is still wet and pale, about 18 minutes. Gently press plums into top of cake, and sprinkle with sliced almonds. Bake until edges are golden brown and a wooden pick inserted in center comes out clean, 15 to 20 minutes more. Garnish warm cake with turbinado sugar, if desired. Let cool in pan for 15 minutes. Using excess parchment as handles, remove from pan, and let cool completely on a wire rack.