BeerBorg

I'd like some feedback on how we perceive our numeric competition results. Here's what I'm after, and why:

I won third place in my category with a score of 32.

In a later competition, I entered a different beer in the same category. I scored 36.5. I did not get a podium spot.

I am entering a beer in two competitions this fall. It's a recipe I entered in a different competition a couple years ago, but the bottle was a gusher, which was a disappointment and, as one judge put it, they weren't able to fairly judge the beer I wanted them to judge. Even so, it scored a 33.

But these three incidents cause me to reflect. I'm naturally thrilled and proud that I got a ribbon. At the same time, that beer was adjudicated as "low very good." Same with the gusher. Then there's the "high very good" that didn't get a ribbon. So, philosophically, where would you stand with these results?

For me, it's a matter of knowing that every competition is different, with different judges, so some subjectivity is inevitable. That being said, every attempt at fairness and consistency in judging to style is made. Ultimately, I think the higher the score, the finer the points of contention, as it should be. And this is where the comments become all the more important. A poorly-made beer (infections, bad process, barely to style, e.g.) is easy to judge when it comes to flaws, but a well-made beer require more finesse when it comes to evaluating. Carbonation level, small levels of bittering hops, a bit too dark, etc., can drop a good beer by half a point or so. At the same time, by mentioning it in the comments, a brewer can get inside the head of the judge better. One of the comments on the 36.5 (from the judge who scored it a 37) was "Maybe reduce the amount of littering hops and this would be a great Dark American Lager."

I'd be interested in your experiences, and how you balance the numbers with the comments.

Beer judging is a lot like marching band judging. You can't compare contest to contest. Once a judge, or in this case, judges, set a number, every beer is then judged in relation to that one. The second beer is either better than the first, or worse. Then, determined how much better or worse. Judges must always manage the numbers, but sometimes they get away from you.

In other words, on the 32 point beer, the judge may have set a lower number as the average threshold. Then, yours was deemed better than average, but not by much. This gives him lots of room for something to blow him away, which maybe didn't happen. I'd guess your beer was early in the tasting, when the number was being set. Then, as the others couldn't pass it in quality, the numbers are set low.

This works the other way too. Set your early number high and scores become inflated. They are more relative than standard. It'd be helpful to see all the scores in a category, but alas, they are not usually given.


Sent from my iPad using Tapatalk

Congrats on the ribbon Dave. I havent entered any comps this year and even last year was slow but yrs previous I have entered quite a few. I have seen the same thing happen in many competitions. I think my first ribbon was a 2nd place for a Blonde Ale and scored a 34, maybe it was a 36 which sounds about right to me. Having a higher scoring beer not place can be confusing but it just shows just how tight the scoring can be between the spots. A perfect score being 50, you do not see a lot of beers in the 40's so that 1st, 2nd, 3rd placement is being fought out in the mid to high 30's....usually unless you are talking the best of the best or NHC or something.

Sometimes how you present the beer entry to the judges makes all the differsnce too, eppecially with specialty categories. I had a "Coffee Chocolate Porter" in one competition and it did not place but scored a 38. All 3 judges wrote comments of nice beer, but flavor is more Cocoa than chocolate and coffee in tbe background. I took that same beer and entered it as a "Porter with Cocoa and Coffe" and it scored a 43 and my highest score yet. It was a different comp and 2 weeks later, but I believe how it was presented made a the difference. One thing to consider is when they group entries together. You might have your American Dark Lager going against an awesome German Bock or Baltic Porter that just nails the style. Its not fair to compare the 2 or 3 in the same category, but many times thats how it goes and otherwise your beer may have placed.

I look over the scoresheets and take them for what they are...then I take the comments with a grain of salt. Most judges are good, have good intentions and make great comments and suggestions. Others...well they try or they dont know what the hell they are doing or talking about.

Mr.BandGuy, actually, my award-winning 32 Dark Lager was the first tasted in a field of 13, so what you say makes sense. And the 36.5 Dark Lager was first in a field of 6.

MadBrewer, your point about the various eligible beers in a category is well-taken. I got third place for my Dark American Lager. First place was a Munich Dunkel and second place was a Schwarzbier. And I can live with those lagers being a little more difficult to get right than something as nebulous as an American Amber.

Obviously, the numbers will change from comp to comp and from judge to judge. I'm entering my South Ferry Steam Beer in two comps this fall, and I'll be interested in the results. And I also know I'll be going up against American Amber Ales and American Brown Ales. Maybe I'll get lucky, and there will only be 3 entries in that category!

Thanks for you responses.

FedoraDave wrote:I'd like some feedback on how we perceive our numeric competition results. Here's what I'm after, and why:

I won third place in my category with a score of 32.

In a later competition, I entered a different beer in the same category. I scored 36.5. I did not get a podium spot.

I think you're way over thininking this. Congratulations as both scores are darn good and pretty close and to each other. The way I see it with your score of 32 two beers scored higher and with your 36.5 at least 3 beers scored higher. Take the judges scores and be proud of it, take their notes and expand on it. Like you said "every attempt at fairness and consistency in judging to style is made. Ultimately, I think the higher the score, the finer the points of contention, as it should be." Adjust the recipe or not, brew and enter again.

Well, I'm actually not wondering why my 32 got a ribbon and my 36.7 didn't. Different comps, different judges. It just goes to show ya.

I guess what I didn't completely make clear is wanting to know what would make you happy as far as a numeric score. Being in the 30s means a beer is "very good." But is that good enough? I haven't tweaked any of my recipes, no matter what they scored, because I was always pleased with the results. This also goes for a pilsner that I thought was superb, but only scored in the high 20s. I understand that pilsners are more challenging and touchy, and any small flaw can be magnified. Still, I'm not changing the recipe at all.

To me, the comments have more meaning than the numbers.

Now I'm confused. You're first post seemed to be all about the numbers, but you're last post says it's more about the comments.

Well, some brewers pay more attention to the numbers and some pay more attention to the comments and how that can lead them to improve the recipe. I guess it goes with one of the phrases you like to quote, which is, brewers choice.

To me, numbers are numbers. It's the content of the comments that matters. But even then, you have to learn to read what they are really saying. Because the number may guide them in their comments, or vice versa. For example, If they just drank a Pilsner with perfect aroma made of the right amount of spicy hops combined with the right amount of Pilsner malt, they will write a high number there and say why. The next beer might have less of the Pils malt profile so they write a number lower than what they did for the previous beer, and their comments will follow. But that doesn't mean that the aroma in the second beer is bad, it's just not as good as the first one judged.

Everyone likes to get a ribbon, but it doesn't solidify the fact that it's a great beer. If all the beers entered in that category in that particular competition are subpar, it just means that yours is better than the others, but it's still subpar. The guy that only cares about the ribbon thinks he brewed a great beer becasue he took second place, but he really didn't brew a great beer. But the guy that pays more attention to the comments will realize that the beer needs a lot of work even though there is a ribbon to show for it.

Now I bet you're confused right???


Sent from my iPhone using Tapatalk

Well, I'm always interested and intrigued by both the numbers and the comments. They both serve their individual purposes. To me, the number is a point on a scale that places the evaluation in a range somewhere. The finer points within that range are sort of minor, since a "very good" beer can easily score a 32 in one competition and in another competition, that same beer, from the same batch, can score a 37. It's still "very good," but the five point difference only means the scale has some slide to it.

The comments, on the other hand, serve a more personal purpose. One judge scores the beer a 32 and says, "A little too malty for this style; maybe add more bittering hops or mash at a higher temperature." Another judge scores the beer a 36 and says, "Good malt/hops balance, but detected some plastic notes in the finish. Maybe too high a fermentation temperature." These are more specific and even more personal, and just as valid.

For the purposes of this thread, however, I'm more interested in having four separate judges in two separate competitions judge a beer that didn't score too badly before, but was tainted by the gushing. It affected pretty much every category, yet it scored a 33. Assuming I don't have another gusher, and given a little extra care I took in my process, I'm interested in what my score will be. So I thought I'd just solicit some opinions about how people approach the numbers, or perhaps might approach the numbers in this particular case.

In these competitions there is so much subjective evaluation happening, most competitions require that there be at least one ranked BJCP judge at each table and usually 3 per table, categories that may not have enough entries get lumped in with another category so there is some more of that subjectivity in play, in the local comps around here the judges need to be within a couple points of each other if there is any big disparity, before moving to the next beer they need to discuss their scores and reasoning till it can be brought into a closer point range. That said I have had a beer that scored 42.5 and did not medal, a friend had been one of the judges ( I didn't know till got the scoresheet) said it was in the final 4 before they named the top 3 for American Ales, can't imagine how good the 3 ahead of me were.

Dawg LB Steve wrote:In these competitions there is so much subjective evaluation happening, most competitions require that there be at least one ranked BJCP judge at each table and usually 3 per table, categories that may not have enough entries get lumped in with another category so there is some more of that subjectivity in play, in the local comps around here the judges need to be within a couple points of each other if there is any big disparity, before moving to the next beer they need to discuss their scores and reasoning till it can be brought into a closer point range. That said I have had a beer that scored 42.5 and did not medal, a friend had been one of the judges ( I didn't know till got the scoresheet) said it was in the final 4 before they named the top 3 for American Ales, can't imagine how good the 3 ahead of me were.

I would have to agree about the subjective nature of any competition, but especially one related to sensory perception on the part of the judges. The latest competition I entered had 3 judges sampling 16 beers from 4 different categories; IPA, Stout, Brown Ale and a Holdiay Ale. My entry was a Brown Ale which was bumped from 1st place by a Brown Ale the judges felt were more to style. Our take away from the experience was never have a competition where there is more than 1 style of beer. Not exactly a one for one match with most competitions, but if like me after a few IPAs most people's taste buds have pretty well been fried for the day.

So....I've never entered my beer in a competition (and may never), BUT I do judge Taekwondo competitions and MrBandGuy's comments ring true for me.

There are similarities here:
- We judge groups of martial artists in groups ranging from 2-6, all performing the same form (other martial arts call them different things). Each form is typically 1-2 minutes long.
- We have 3 judges giving scores for each competitor who performs by themselves, so one after the other
- For each group, each judge must remain consistent although they may vary by group by group depending on skill set
- It CAN be somewhat subjective, although it shouldn't be

I don't know how beer scoring is done (i.e. starting at 50 and getting marked down for each flaw), but we start at 10.0 for TKD and do mark down for each error. We try to avoid ties, so as MrBandGuy said, the first performer (beer) typically sets the bar and everybody either scores above/below that. If there is a tie, it comes down (for me anyway) to some intangibles like how their belt is tied, how clean and presentable their uniform looks or how they bow (to show respect).

Sometimes that first athlete is really good, but if you score them a 10.0 you leave no room for anybody that follows to be better so you have to leave room.

So yeah, from the outside looking in, I'd say those comments sum up the way I *think* it works. Again, I'm the first to admit that I've not entered competitions so this is me speculating.

It makes sense, though. I know Dawg has entered many competitions, and he had a brown ale score disappointingly in one, and that same batch medaled in another. So yes, some of it is subjective. It also has to start somewhere, as Kealia pointed out. For those who haven't been in competitions, there are five categories, Aroma (12), Appearance (3), Flavor (20), Mouthfeel (5), and Overall Impression (10). So they don't start at 50 and work down. One judge may give a beer 8/12 for aroma, and the other may give it 7/12, etc. through each category. Most of my scoresheets indicate very close scores in most categories, and only 0.5 or 1 point aggregate total between the two (Judge A giving me a total of 35 and Judge B giving a 33, so my final would be 34, e.g.). All of these numbers are, as pointed out, comparative to what went on before, and interestingly, both my American Dark Lagers were the first sampled in their flight, so I was the bellwether against which all the other entries were scaled in their scoring.

Even so, and maybe especially since they were the first ones tasted, both beers fell within the range of "Very Good" examples of the style, one in the lower end, and one nearly breaking through to "Excellent," which pleases me no little and more than somewhat, especially since that placement is in a wider numeric range, and, hence, less subjective. After all, if a beer is really a stinker, for whatever reason, they're not going to be able to score it high in any category, lowering the overall score.

This discussion has been very helpful to me in getting a good mindset and better understanding of interpreting the numbers. Combined with the comments (when they're legible), the judge's sheets are helpful and very encouraging. The thing to always keep in mind is that the judges want all the contestants to do well, and are aiming their comments in that direction. Just to have one judge write "Nice job!" at the end of his comments still gives me a great feeling years later.

Havent done a comp in a long time, but I always considered my numbers to me +/- 5 pts. Everything is relative. If the previous beer sucked, yours will be really good... etc. A 5K can be a daunting race sometimes, but after I ran a 1/2 marathon a 5K seems like a warmup.

If you've entered the same recipe several times, maybe just take out your high and low scores. That seems fair to me. You middle numbers are probably a more accurate representation.

Feedback is where it's at... and even there I'd probably throw out the high and low comments.