Closing the Loupe : Part 1 – boring statistics


With the recent International Loupe Award judging coming to a close, Marianne and I are glad to announce that between us, we scored:

1 Gold award

7 Silver awards

4 Bronze awards.

This is a result we are very happy with and gives us a base to work from for the 2013 awards! While we don’t enter many competitions, we do believe in progressing our photography in every step of creating an image. Perhaps striving for better competition results might provide us with the catalyst to consider different capture and post process techniques we have yet to perfect. We never want to stagnate and be pleased with our current status!

The competition judging has been marred by claims of massive score variation and heated discussion regarding the validity of the individual judges for the landscape category in particular. I can make no comment about the judges other than to say that they are all experienced and very well credentialled photographers.  Some of their CVs do not contain landscape photography and this was the cause for consternation.

For those who were unaware of these awards, the judging system this year was designed to reveal a layer of transparency behind the scoring. Each of the judges were not allowed to mark entrants from their own geographical area, each of the judges gave their scores entirely independent of each other, and each of the judges were allowed to give feedback if they felt it was warranted. There was no panel to discuss scenarios of wildly varying scores. The scores were averaged to give a final score.  I believe that there was quite a large variation of scores for individual images experienced by many photographers including ourselves. While most people believe that it was the lower score which was the deviation from the others, I was surprised to find that many of our images had an outlying high score!

Just for fun I ran our scores through some basic statistics (I’d like to emphasise the word ‘fun’ in that the intention was nothing more than an intellectual exercise)

I chose to highlight scores with a variation (according to standard deviation) of >5.5 as the category separation between bronze/silver/gold/platinum was 5-6.

In those images with a high variation: I highlighted in red , images whose lower score was disproportionately low. I highlighted in green, images whose upper score was disproportionately high.

Rough statistics for our images

Here are some of my conclusions:

1. Marianne’s scores were on average 3 better than Dylan’s

2. Marianne’s scores would have caused less controversy with a jury style judging panel with a difference of scores 10.5  vs 12.4

3. Four of our scores were ‘dragged down’ from a lower scoring outlier

4. Three of our scores were ‘pulled up’ from a higher scoring outlier

5. One image had scores varying widely equally from the average score

6. Tighter selection of Marianne’s images won out over Dylan’s “spray and pray” submission approach!

So, was the judging really as unfair as many of us perceived? After standing back and looking at this objectively , I think I made an emotional mountain out of a molehill .  Interested to hear the thoughts of those who entered !

Stay tuned for the next post : The actual images and limited feedback regarding the feedback !

-D

Advertisements

Posted on November 30, 2012, in Photography, Random Musings and tagged , , , , . Bookmark the permalink. 8 Comments.

  1. Too much time on your hands, like it though….well done Marianne!

  2. Very interesting analysis – thanks for sharing… I entered three and received three bronzes… judge 3 didn’t like me – for one image he/she was 10 points lower than the next judge, while on the second he/she was 20 points lower! Other two judges were within two or three points of each other across all three images.

    Re the “feedback” – one image had none (that’s fine), another had a really useful suggestion on it, and the third just said “great image, wrong forum” – even though it was a landscape in the landscape category. Not really sure how to work with that comment, but anyway…

    I’m certainly glad I entered, and like you, I feel it does give me a base to work from next time, along with the one very helpful comment that will really help me improve that specific image.

    • On their facebook site, the ILA now attempt to explain that ‘great photo wrong forum’ line. Strange but it apparently was an instruction that the judges could use it for certain images with its implied meaning. There was only written feedback for 1 of the images out of 12, and then a few with crop suggestions. I will put these up in the next post

  3. I really think that we cant get too “navel gazing” about judging. The fact is that we are all different with different biases and different expectations. Judges are just people like us. I submit to my local club competition monthly and experience varying judges with varying outcomes. I have been disappointed to receive 3rd place (or no place) for some entries only to have them awarded (club) Image of the Year. Even if they were judged by a computerised judging app we would probably still disagree with the outcomes 🙂
    Maybe next year they need to do the same as Gymnastics competition and discard the highest and lowest score. That would require more judges though and cut down the profit for the event 🙂

    Love ya work

    • Yep , agree with you there – there really isn’t a basis to argue subjective opinion. I think the problem is,many have then turned on the judges themselves (and their qualifications) which has set off a wildfire ! (one which I am gently walking away from)

  4. I’d like to see 5 judges scoring each image to potentially make this outlying score less signifucant

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: