Coffee, vanilla, and cinnamon are all obvious choices that can spice up any dessert you'd like, but Bon Appétit says that its favorite way to spice up an olive oil cake is by adding sugar infused with rosemary. The fresh, piney notes of rosemary make for a great complement to the sweet richness of an olive oil cake.