Whisk the egg yolks in a medium-size saucepan.
Add sugar and milk to the yolks.
Place your saucepan over medium to low heat.
Stir frequently with a rubber spatula. To test if the filling is done, there is a little trick. Take your finger and run it on the spatula, and the mixture should not run into the line.
Take off the stove and let the filling cool. It should settle into two layers.
Transfer the filling to a mixing bowl, and while it is cooling, mix the two layers. Then add in small chunks of the room temperature butter.
The filling should thicken like buttercream. Keep mixing for 8-10 minutes. As the butter cools, it thickens.
Place the first layer of the cake on a plate and frost the top of the cake.
Add another layer on top of the first cake and top with frosting.
Repeat if you are making a three-layer cake.
Frost the sides of the mini-cake. If you want a smooth frosting, make sure to place the cake on a revolving cake stand and frost with a metal spatula.