Swordfish, like tuna and marlin, are very meaty fish that don’t benefit from being overcooked. The topping works well, with the citrus flavors cutting through the richness of the fish.
Combine the extra-virgin olive oil, lemon zest and juice, orange zest and juice, chile, scallions, garlic, capers, olives, mint, and chives in a bowl and leave the flavors to develop for 20 minutes. Season with salt and pepper, to taste.
Meanwhile, bring a pot of water to boil over high heat. Cook the new potatoes for 12-15 minutes until nearly tender, then add the broccoli and cook for another 4 minutes. Carefully drain and season with salt and pepper, to taste. Keep warm.
Heat a griddle pan or large frying pan over high heat until almost smoking. Season the swordfish with salt and pepper, to taste, and spray a fine mist of olive oil over the surface of the fish. Cook for 1 minute on each side.
Turn off the heat and allow the steaks to sit for 3 minutes in the residual heat. Serve with the new potatoes and broccoli, with the citrus salsa scattered over the fish.