Preheat the oven to 375ºF. Line 2 baking sheets with parchment paper.

Cut each of the marshmallows in half horizontally.

In a bowl, whisk together the flour, cocoa powder, baking soda, and salt. Set aside.

In the bowl of a stand mixer, or a large bowl if making by hand, beat the butter and sugar together until it is light and fluffy, at least 2 minutes. Add in the egg, milk, and vanilla extract.

Gradually add in the flour mixture and beat until it is completely combined.

Us a 1 3/4-inch cookie scoop to make balls of dough and place them on the prepared baking sheets, at least 2 inches apart.

Bake, one sheet at a time, rotating halfway through the cook time, for about 8-10 minutes, or until the cookies are set. Remove from the oven and immediately place a marshmallow half on top of the cookie. Return to the oven for 2 more minutes. Remove from the oven and allow the cookies to cool completely.

To make the frosting, melt the butter in a medium saucepan. Once melted, remove from the heat and whisk in the cocoa powder, followed by the powdered sugar, milk, and vanilla. Whisk until smooth.

Spread about 1 tablespoon of the frosting over the top of each cookie, covering the marshmallow.

Allow the cookies to set up completely before serving.