As far as I'm concerned, the criteria for a good potato salad is for the potatoes to have textural character, for it to be tangy with plenty of mayo, and for there to be added elements that bring other textures and flavors, particularly heat. Oh, never peel them. There's absolutely nothing wrong with leaving the skin on.
This potato salad takes a cue from one Pies 'n' Thighs used to make back in '06 when its first incarnation was practically under the Williamsburg Bridge (left). Back then, the little hole-in-the-wall made a potato salad that was well-vinegared and included a much-underused ingredient, small-chopped pickles. I love the new place, but they don't seem to ever make their potato salad with pickles anymore.
It's a shame. Chopped pickles add crunch, tang, and vinegar. This spicy purple potato salad goes one better by adding color, and increasing the flavor and textural components, using several types of onion and, for zip, jalapeño and chile pepper. Don't worry, the heat mostly disappears into the mayo... kind of.
Put some water on. When it's boiling add salt. Add potatoes and boil just until you can pierce with ease, then drain and let cool. Do NOT overcook! Run them under cold water if you're worried they're still cooking.
Toss the shallot and red onion into a large bowl. Flatten 3 celery stalks with the back of your knife. Julienne them, then mince. Slice the last stalk crosswise for thin half-moons and toss in. Add 2 minced jalapeños and chile pepper, cornichons, and ½ container Kewpie mayonnaise.
Cut potatoes to about the same size. Toss in the bowl with other ingredients and adjust salt, then add vinegar and pepper.