Preheat oven to 375° F. Line 3 baking sheets with parchment paper.

For cookies: In a large bowl, beat butter and sugar with a mixer at medium speed until fluffy, 3 to 4 minutes, stopping to scrape sides of bowl. Add eggs, one at a time, beating well after each addition. In a medium bowl, whisk together flour, baking powder, and salt. In a small bowl, combine milk and extracts. Gradually add flour mixture to butter mixture alternately with milk mixture, beginning and ending with flour mixture, beating just until combined after each addition.

Place dough in a piping bag fitted with a Wilton 2A round tip. Using even pressure, pipe 2½-inch rounds 1½ inches apart on prepared pans.

Bake until edges are lightly golden, 10 to 12 minutes. Let cool completely on wire racks.

For vanilla glaze: Place wire racks on top of rimmed baking sheets. Transfer cookies to racks. In a medium bowl, whisk together confectioners’ sugar, milk, and vanilla. Spoon glaze over cookies, and lightly tap racks to remove any excess glaze.

For chocolate glaze: In a medium bowl, whisk together confectioners’ sugar, cocoa, and milk. Place chocolate glaze in a piping bag fitted with a Wilton 2A round tip. Pipe chocolate glaze over half of each cookie, and lightly tap racks to remove any excess glaze. Let set for 5 minutes.