How to Judge a BBQ Contest
Today on The Daily Meal
Walk the grounds of a barbecue competition like the Kingsford Invitational Barbecue Competition and you will repeatedly hear certain phrases about who made the best barbecue. From judges to spectators, snippets of conversations include the same questions: Did it blend "heat and sweet"? Did it make you want to smile? Was there a "command of heat"? Was there a perfect rib/bone connection? Was it sweet, spicy, and sexy?
The Daily Meal, fresh off its Ultimate BBQ Road Trip, headed to the Kingsford Invitation Barbecue Competition, which pits the best-of-the-best in the barbecue world, chosen from eight top-tier regional contests like Memphis in May and the Jack Daniel’s World Championships, against each other, to see how the pros judge barbecue contests. Their insight may surprise casual observers.
The first thing to realize about judging a barbecue contest is that pit creations and rules do not necessarily correlate to the barbecue found in a local restaurant; such concentrated flavors are too labor-intensive for the normal palate and too expensive to produce daily.
"Judging barbecue in a competition setting is a complete 180 degrees from the typical backyard experience," admits Scott Jones, a certified Kansas City Barbeque Society (KCBS) judge. "It’s a pretty intense proposition, especially when trying to stay objective. I suspect if judges are honest about the process, personal preferences always creep in."
As a result, most competitors are there to impress the judges with "orgasmic," in-the-moment creations, according to Brad Orrison, judge at the 2012 Kingsford Invitational Barbecue Competition and owner of The Shed BBQ & Blues Joint in Ocean Springs, Miss.
It’s about making the best on competition day, and how the judges’ palates taste it that day.
"I want to blow you, as a judge, away with one bite," said Tommy Houston, owner of Checkered Pig BBQ in Martinsville, Va., and one of the eight competitors at Kingsford Invitational Barbecue Competition.
Barbecue Rating Systems: A Primer
There are many types of rating systems, however, which vary from region to region, and some are pretty complex. In a typical Kansas City Barbeque Society competition, for example, there are three categories: appearance, taste, and tenderness.
Sounds simple, right?
But read further, because the rating method on the Kansas City Barbeque Society’s website looks like a complicated algorithm: "The scoring system is from nine to two… nine excellent, eight very good, seven above average, six average, five below average, four poor, three bad, and two inedible. The weighted factors for the point system are: appearance -0.5714, taste -2.2858, and tenderness -1.1428."
Confusing? When most people enjoy something, they say things like "Man, that tasted good," or "Wow, that was awesome." Not, "Tastes like 2.2858."
In Memphis, there is a different system called Memphis Barbecue Network, which focuses only on pork categories: whole hog, pork shoulder, and pork rib.
"In on-site judging, the teams are evaluated on their personal and site appearance, their presentation to the judges, the appearance of the entry, the tenderness and texture of their entry, and, finally, the flavor of their entry. In blind judging, only the appearance, tenderness, and flavor of each entry are evaluated. Each judge also assigns a score for Overall Impression to each entry, which takes into account the judge’s entire experience with that entry," according to the Memphis Barbecue Network’s judging guidelines.
Less perplexing, to be sure, but still a bit vague.
"I think that it is totally subjective to what the judge thinks is good barbecue, not what the competitor thinks. Too many times competitors think that they have the best stuff only to find out that what they made wasn’t good," said Richard Wachtel of the website Grilling with Rich.
"An interesting point to think about is that you can go to a barbecue competition in, let’s say, Virginia, win the competition and then put together the same exact flavor profiles and do the same exact thing in, let’s say North Carolina, which is only a couple of hours away, and come in dead last. Totally subjective," said Wachtel.
At the Kingsford Invitational, a new rating system was invented specifically for this contest, in which contestants competed for $50,000. The group of seven judges, six main and one tiebreaker, participated in double blind-box judging and scoring on a 14-point Hedonic Scale, co-created by one of the judges, Meathead Goldwyn, and based on a system Goldwyn had originally brought to the beverage world for taste comparisons.
"We needed a scale that works better and codifies repeatability while also bringing a simplicity to the process," said Goldwyn. "One number is all you need."
How does Kingsford judging differ fundamentally from other contests?
First, it allows judges to compare samples with one another. Second, the 14-point Hedonic Scale, with scores from 0 to 13 (from 12-13 for "like extremely" to 0-1 for "dislike extremely"), was developed by Goldwyn with Dr. Harry Lawless of Cornell University. In the Kingsford Invitational Scoring System, these particular ratings are "simple and easy. There is no arithmetic… There are no substitutions. Scales of 9 or 10 are easily ignored by judges who may choose to apply their own 10-point 'Miss America' scale."
"Is it an objective or subjective process? How does one define measurability when rating different types of barbecue from different regions with different styles? There is so much fragmentation across regions with variable sanctioning bodies," said Kingsford judge Tuffy Stone of Q Barbecue in Midlothian, Va., of the considerations made during judging.
At some point hairs will have to be split.
"Smoke is like salt and pepper to me," Stone observes. "I need nuance, moisture. One lost art is cooking the skin of a chicken. You have to cook chicken to render it delicate so that the skin is easy to bite through. This means seasoning chicken in just the right way. Remember: judges get one bite, typically, so a lot of flavor has to be in that bite."
The Kingsford Invitational is unique in that it aims to take the champions from all regions and rate them with attempted parity, despite disparate scoring systems.
Is it fair? "If we win, it’s fair," said a chuckling member of the Motley Que Crew, part of the chosen eight at Kingsford.
An Insider’s Look at Barbecue Judging at the Kingsford Invitational
There are four main categories at Kingsford: chicken, ribs, pork shoulder/pork butt/whole hog, and brisket. For the judging, each dish is placed with no garnish in a simple to-go box.
"It’s meat in a box," said judge Brad Orrison, "but it’s artwork, glazed, sticky, orange barbecue meat from heaven!"
Judges cleaned their palates with saltine crackers and grapes between 50 to 60 overall bites.
Harry Soo, one of the more charismatic and animated of the Kingsford judges, wildly gesticulated about his love of barbecue. His website, SlapYoDaddyBBQ.com, gives some indication of Soo's humor, but he is an engineer by vocation and an award-winning BBQ Grand Champion by avocation. "We are looking for creativity outside the norm," he noted about the judging process. "Bright flavors, not dull ones."
When addressing the appearance of a dish, for example, Soo explained, "You eat with your eyes, but the brain knows not what the tongue wants, so judging can be tricky." He also had advice for the competitors, indicating that they would win or lose in the last 15 minutes of the contest so he cautioned that they need to get a good night’s sleep before the big day and pace themselves for those last critical 15 minutes.
"We sleep like homeless people for these contests," said Soo, with a beaming smile, "and we spend our life savings just to come here. Barbecue is therapy for us." For that reason he believes that pitmasters need to adapt to what judges want across all contests, given that flavor profiles are coming from the region, the individual, the county, and the state.
Participants also need to deliver in all conditions, whether it be rain, snow, or wind. Furthermore, they must have persistence and consistency yet variability from contest to contest. A lot of barbecue nowadays also makes the mistake of saturating the meat with sauce, which should be avoided.
"The flavor profile has changed and so has the competition. My dad, for example, couldn’t win today," said Kingsford judge Amy Mills, known as The Heiress, who grew up with her famous restaurateur father, Mike Mills of 17th Street Barbecue in Murphysboro, Ill., with whom she co-wrote Peace, Love and Barbecue.
What are Mills’ main criteria for judging?
"It looked good, tasted good, and you wanted to eat more." She also said that there should be just a kiss of smoke in the dish because after a certain point, meat doesn’t absorb smoke anymore.
The emotion of the contestants cannot be forgotten either. Closer to turn-in time, words became limited. Facial expressions changed.
"Don’t try to engage them in conversation. There’s a lot of heart and money and pulling of trailers to get here," said Stone. "It’s taken seriously."
Crowning a Winner
This year’s winner of the Kingsford Invitational Barbecue Competition was Yazoo’s Delta Q led by Melissa Cookston, who not only won the $5,000 One Bite Challenge on the first day of competition but also won three of the four Kingsford categories (except for chicken), making them the landslide Grand Champion with a prize of $50,000 in the competition’s inaugural year.
"The Hedonic scoring system adapted by Meathead and Dr. Harry Lawless is based upon reputable science and experience and was ideal for this small contest," said veteran judge and barbecue author Ardie Davis, of the book America’s Best Ribs. "I liked the one-number score plus the opportunity to discuss each entry after we finished scoring. The judges gave serious attention to the job at hand, and in my view we rendered a fair and impartial judgment."
Clearly, bringing subjectivity like taste and appearance¬¬ to a contest means that judging leans closer to an art than a science. Nevertheless, these sharp Kingsford Invitational judges are probably the closest to "taste scientists" around with their historical palates, BBQ champions themselves at the top of their game judging their peers, also at their apices. As evidenced by the new scoring system, the judging of barbecue is constantly adapting as the sport grows.
It seems like this is one situation where it’s maybe good that the smoke doesn’t clear.
Mark Damon Puckett has written for Saveur and Greenwich Magazine. He is the author of The Reclusives, YOU with The Ill-usives, and The Killer Detective Novelist (October 2012), all available on amazon.com and bn.com. Please visit him at www.markdamonpuckett.com.
Be a Part of the Conversation
Join the Daily Meal's Community and Share your Thoughts