Preheat oven to 325°F (170°C).

In the bowl of a stand mixer fitted with the paddle attachment, beat butter and sugar at medium speed until fluffy, 3 to 4 minutes, stopping to scrape sides of bowl. Add eggs, one at a time, beating well after each addition.

In a medium bowl, whisk together flour, graham cracker crumbs, baking powder, and salt. In a small bowl, whisk together milk and vanilla. With mixer on low speed, gradually add flour mixture to butter mixture alternately with milk mixture, beginning and ending with flour mixture, beating just until combined after each addition.

Spray a Nordic Ware 6-Cup Charlotte Cake Pan with baking spray with flour. Using a pastry brush, spread any excess spray in pan, blotting brush with a paper towel as needed. Spread batter into prepared pan. Tap pan on a kitchen towel-lined counter several times to settle batter and release any air bubbles.

Bake until golden and a wooden pick inserted in center comes out clean, 35 to 40 minutes. Let cool in pan for 10 minutes. Invert cake onto a wire rack, and let cool completely.

Transfer cake to a serving plate. Fill with Key Lime Custard, spreading with an offset spatula.

Place Swiss Meringue in a large pastry bag fitted with a ½-inch open star piping tip (Ateco #825). Pipe meringue as desired. Using a handheld kitchen torch, carefully brown meringue as desired. Serve immediately.