`Judging EVOO Competitions Takes More than Just Tasting

Fairs, Competitions

Judging EVOO Competitions Takes More than Just Tasting

Aug. 24, 2010
Richard Gawel

Recent News

By Dr. Richard Gawel

A cou­ple of weeks ago I con­ducted a work­shop for the judges from the Royal Perth Extra Vir­gin Olive Oil Show. It was the 4th such work­shop con­ducted annu­ally at the request of the West­ern Aus­tralian Olive Coun­cil. The judg­ing work­shop pro­gram evolved out of the belief that whilst show judges are expe­ri­enced EVOO tasters, the act of show judg­ing requires a vari­ety of skills to bring it all together.

  • Sen­sory — of course.
  • Lin­guis­tic- for writ­ing use­ful com­ments for exhibitors, and
  • Inter­per­sonal – as judges work in small teams of 3 or 4.

Every year I con­cen­trate on build­ing a par­tic­u­lar skill required by show judges. Themes have included – mea­sur­ing judge con­sis­tency, writ­ing effec­tive com­ments for the ben­e­fit of exhibitors, and under­stand­ing and deal­ing with diver­sity of EVOO styles and char­ac­ters.

Judge Con­sis­tency

One of the most impor­tant require­ments of a good show judge is the abil­ity to score con­sis­tently. Agree­ing with your­self is a pretty good qual­ity to have. One would hope that a judge that gives an oil a gold medal score when tasted on one occa­sion would give it another gold medal score if they were asked to score it on another occa­sion. While it would be nice to think that this could be done by all show judges all the time, I have to say that believ­ing so is a lit­tle naïve. A cou­ple of years ago I mea­sured the con­sis­tency of over 500 wine show judges (and poten­tial wine show judges) and found that a sur­pris­ing num­ber found scor­ing con­sis­tently was a tricky busi­ness (Gawel and God­den, 2008). In my opin­ion, EVOO judg­ing is more dif­fi­cult than wine judg­ing (I’ve done quite a bit of both) as the dif­fer­ence between an aver­age qual­ity and a high qual­ity EVOO is much smaller than between the aver­age and high qual­ity wine. So it’s no sur­prise that con­sis­tency scor­ing EVOO’s for qual­ity isn’t as easy as many peo­ple might think.


So with the aim of mea­sur­ing the con­sis­tency of the olive oil judges I pre­sented them with 12 EVOO’s which they scored for qual­ity. Later on that day, they were pre­sented with 15 oils, 12 of which were the same as those scored in the morn­ing – but in a dif­fer­ent order of course. The judges were unaware that oils were being pre­sented for a sec­ond time as I led them to believe that we were con­duct­ing a dif­fer­ent exer­cise (I can be a com­plete bas­tard!). I then cor­re­lated the scores given on the first occa­sion with those given on the sec­ond to the same oils. The results of the exer­cise allowed the judges to bench­mark them­selves against their peers.

Appre­ci­at­ing Style Dif­fer­ences

Every year I spend a lot of time at these work­shops pre­sent­ing diverse sets of EVOO’s – from dif­fer­ent vari­eties and blends – early and later har­vest – from dif­fer­ent regions – mild through to very robust . All this is done to impress upon judges that it is bad form to penal­ize an oil because it either has or doesn’t have, a par­tic­u­larly flavour char­ac­ter­is­tic. The world of EVOO is won­der­fully diverse – dif­fer­ent fla­vor pro­files and dif­fer­ent lev­els of bit­ter­ness and pun­gency are nec­es­sary. Why? – because despite what some pro­duc­ers might tell you, no sin­gle style of EVOO can hope to be per­fectly suited for use in every sin­gle culi­nary dish. While some styles might have a lim­ited appli­ca­tion com­pared with other styles, they all have their place out there’ in one way or other. But there are high and low qual­ity exam­ples of each style, and rec­og­niz­ing this is the point of the judg­ing exer­cise.

So here were the results of the 2009 Perth Show which took place two weeks after the work­shop. I’ve given the aver­age score for oils within polyphe­nols ranges. The higher the polyphe­nols level, the more bit­ter and/or pep­pery the oil (gen­er­ally speak­ing), so the polyphe­nol level is a pretty good indi­ca­tor of style. I’ve also turned it around and given the aver­age polyphe­nols level for each medal ranges, best of class and best of show.

Polyphe­nol RangeAver­age Score
<20013.7 Bronze
200 – 29913.8 Bronze
300 – 39913.4 Bronze
>40013.6 Bronze
MedalAver­age Polyphe­nols(mg/kg)
No Medal350
Best of Class*410
Best of Show310

*aver­age of the high­est gold medal scor­ing oil in each class

Lastly, if you’re like me and like see­ing things in pic­tures, here is the plot of awarded score against polyphe­nol con­tent.

The cor­re­la­tion between polyphe­nols level and score was a mea­ger 0.005 which was as close to the mag­i­cal ran­dom, no rela­tion­ship what­so­ever’ value of 0 as you would hope to get. In short on all the mea­sures, the amount of bitterness/pepperyness had no bear­ing what­so­ever on the scores awarded to the oils. This is the way it should be. Good oils should be awarded based on qual­ity not style. Fluke per­haps? – well I just spent some time on the flight back from the 2010 Royal Perth Show and cal­cu­lated that the cor­re­la­tion between score and polyphe­nols level again showed that there was no rela­tion­ship between score and polyphe­nols level (aka style) – the cor­re­la­tion was 0.001

Writ­ing Descrip­tions

Unlike EVOO shows else­where, the major­ity of Aus­tralian shows pro­vide com­men­tary on each sub­mit­ted oil via the pub­lished show results book­let. The com­ments are writ­ten by the head judges of each panel, and are based on their opin­ion and those of the other two judges. How­ever, the com­ments can get a bit jar­gonny at times, and while they clearly mean some­thing to the judges who wrote them, they may not mean much at all to the exhibitor. On the other side of the coin, the com­ments can be so generic that they mean lit­tle at all.

So in one work­shop I pre­sented a series of com­ments writ­ten by judges in another show (The Aus­tralian National), and simul­ta­ne­ously pre­sented the oils to which they referred. The judges attend­ing the work­shop were asked to match the com­ments to the oils after tast­ing them. We then dis­cussed at length exactly what aspects of the com­ment made it easy to iden­tify the oil, and what aspects of the com­ment was con­fus­ing or resulted in an incor­rect match of descrip­tion to oil. The logic here was, well if you can iden­tify the oil from its descrip­tion then the descrip­tion must be infor­ma­tive. After pick­ing apart the value of the pro­vided descrip­tions, the judges rewrote the com­ments in a way which they felt made for the briefest yet most infor­ma­tive com­ment. While I’d admit that the art of writ­ing brief but use­ful com­men­tary on oils is a work in progress, it was a good start.


Gawel, R. and God­den, P.W. (2008) Eval­u­a­tion of the con­sis­tency of wine qual­ity assess­ments from expert wine tasters. Aus­tralian Jour­nal of Grape and Wine Research, 14, 1 – 9.


Richard Gawel’s blog is Slick Extra Vir­gin. Repro­duced with per­mis­sion.

Related News