Simmer the whole potatoes, with the skin on, in salt water over medium heat until they are tender all the way through when pierced by a knife, about an hour. Do not allow the water to reach a violent boil, as this will cause the skin to break apart and the potatoes will absorb too much water.
In a small saucepan, heat the olive oil and minced garlic over medium-low heat and cook for 10-12 minutes. You want the garlic to slowly cook through without browning.
Meanwhile, when the potatoes are almost done, mix the milk and cream together and heat until hot, but not bubbling.
Drain the water from the potatoes, and add 1/4 of the warm milk and cream mixture. Use a potato masher to begin the mashing.
Add half the butter and continue mashing. Slowly pour in half of the remaining milk and cream, then the cooked garlic and olive oil, and the rest of the butter. The remaining milk, cream, and butter can be added for flavor and consistency, as you wish.
Season the potatoes to taste and serve warm.
Note: If you are not serving the potatoes immediately, you will want to be sure to save some of the warm milk and cream mixture, and butter, to add right before serving.