Preheat oven to 325 degrees F. Coat a 13 X 9-inch casserole dish with cooking spray and set aside.
Cut baguette into 1/2-inch cubes and spread along the bottom of the dish.
In a large bowl, combine eggs, buttermilk, maple syrup, vanilla extract, cinnamon, and nutmeg. Mix well and pour over bread.
Cover with plastic wrap and press down lightly, so the bread absorbs the egg mixture. Refrigerate at least 2 hours (or overnight) before baking.
Bake for 45 minutes, or until eggs are set.
When removed from oven, sprinkle with brown sugar and let stand for 10-15 minutes before serving