Feature spotlight: Why Evalato’s score normalization matters 

Evalato’s unique score normalization feature ensures unparalleled judging objectivity by means of a mathematical formula. Sound too complicated? If math was not your favorite subject at school, this article is for you. If formulas are your cup of tea, however, our score normalization help doc has you covered. 

Why you need score normalization

In a perfect world, your awards judges will always be absolutely 100% objective. In ours, that is much less likely. Note that this is not a question of intention, since your judges are probably planning to be as objective as can be, it is a question of their background, life getting in the way, and other assorted factors all blending in to result in unintentional bias. Imagine this: 

• Judge 1 steps on a Lego on their way to the computer on evaluation day only to find out that Netflix canceled their favorite show after season 1 (which ended on a cliffhanger);
• Judge 2 received the marriage proposal of their dreams the night before and views the world through rose-colored glasses; 
• Judge 3 is a renowned expert in their field with years of experience spotting the tiniest of mistakes from a mile away, who’s also a stickler for the Oxford comma;
• Judge 4 is a promising industry newcomer who is so, so happy to be part of your jury they want to spread the joy, and who believes that punctuation is not that big of a deal in entries for The Midwest’s Best Gardening Equipment Awards. 

While the above is an obvious exaggeration, it shows that bias is possible — and not for a specific entry (a different issue altogether, usually the result of a conflict of interest) but for all the entries a specific judge evaluates.  

Now, if all judges evaluate all entries, then it balances out because all entries will get the exact same treatment, which takes care of the bias by default. But if you have Judge 1 and Judge 3 evaluate one group of entries, and Judge 2 and Judge 4 evaluate another, then the first group is likely to end up with lower grades overall, the quality of the entries notwithstanding. 

To sum up, if your judges evaluate subsets of entries, then their more positive or negative view — aka unintentional bias — gives some entries an unfair advantage or disadvantage, skewing the results. This is exactly where score normalization comes in. 

How score normalization works (no math, as promised) 

Score normalization is a setting in the Evalato awards judging software that, as the name suggests, normalizes or standardizes the scores given by all judges, removing any unintentional bias. More specifically: 

• The calculation is done entirely by Evalato when the setting is turned on; 
• The setting doesn’t change or determine the winner; 
• It levels the playing field, regardless of whether the entries ended up in a group evaluated by Judge 1 and Judge 3, rather than Judge 2 and Judge 4 from the example above. 

The result is a fairer competition, where every group, subgroup, or category of entries gets the same treatment and the same bias, or rather, lack thereof.  

Objectivity for the win

Score normalization is one of Evalato’s evaluation features that ensure objectivity, along with:

• The option to add an NDA or T&C that your judges need to agree to before evaluation, meaning, you can easily insulate your awards program from conflict of interest;
• Abstention if a judge has any concerns about evaluating an entry, which is yet another way to sidestep any possible conflict of interest; 
• Judge feedback — you can enable this option which will not only make your awards a learning experience, adding to their value but will also give every judge the option to justify a low (or a high) grade, removing any doubt about bias toward a specific entry. 

At the end of the day 

In a nutshell, score normalization represents one of the core principles of our awards management software — to allow customers to pick the worthiest winners. If you want to see how it works in practice, you can head over to Evalato to test it for free. Or if you want to quiz our team about it, you can book a demo.