Of course, you should ensure it still works before incorporating it into your batter, and you can test your baking powder's efficiency by stirring it into hot water. It is still effective and ready to use if bubbles or fizz appear, but if not, then you should pick up a new box.