← Scoring Guidelines (Deutsch) Executive Committee schaut sich die IYPT-Stätten an →

grading

As promised last time, let’s think about the grading itself a bit. Back in 2010, without the help of our new Scoring Guidelines , the distribution of grades looked like this:

The diagram, like this description taken from Ilya’s archive, shows the distribution parameters (population mean and standard deviation) for grades of individual jurors at the 23rd IYPT (2010.) The grades are averaged across all stages, performances, and PFs. The data clearly reveals that the individual grading standards vary among jurors, and result in a marked spectrum of average grades in the range from ca. 6 to ca. 8. The relative standard deviations, not shown here, vary in the range from ca. 0.1 to ca. 0.3. The marks above 8—9 and below 5—6 are statistically rare events.

One of the SG’s goals is to make it harder for jurors to just give a 7 all the time. Besides being a nice prime number, 7 also is a mark weak teams are happy about and strong teams aren’t too sad about (well, as long as it’s not the finals). Clearly this is not what we want for a tournament. Ideally the overall mean grade in a tournament is somewhere close to 5.5 with all the grades used – e.g. with a reasonable standard deviation. With jurors only giving marks from 6 to 8, we effectively narrow the results so much, that eventually randomness and rounding effects become much to influential. The SG certainly help to avoid this, with results from our national YPT in Austria showing that the standard deviation has improved a lot in the past two years and most of the jurors giving grades as low as 1 and as high as 10 over the course of the tournament.

update: a few more words from Ilya :

[01:14:06] Martchenko, Ilya: http://iypt.de/?p=1222
[01:14:59] Martchenko, Ilya: if you want to be completely open, you can also write 
that it's *very* important to have all jurors staying centered around one, uniform grade
[01:15:06] Martchenko, Ilya: i.e. 5
[01:16:11] Martchenko, Ilya: if it's not happening, we may easily have (and always had) the following situation:
[01:16:33] Timotheus Hell: c'mon, i posted this like 5min ago. How can you be that fast?!  
[01:17:02] Martchenko, Ilya: in the fight room A by a pure coincidence we have those jurors 
who put slighly higher grades (right leg of the spectrum) than in the fight room B
[01:17:17] Martchenko, Ilya: ** c'mon, it's by job to monitor mass media  

This entry was posted in blog. Bookmark the permalink.
Posted on July 9., 2012    0:06 by Timotheus Hell
← Scoring Guidelines (Deutsch) Executive Committee schaut sich die IYPT-Stätten an →

One Response to grading

1. Felix says:

   July 11., 2012 at 22:15

   Nice Post

   Reply