Most rating systems for events rely on a simple matrix, giving you a series of statements which you rate according to how much you agree with them on a 1-5 scale. Despite it being a common methodology it does throw some interesting conundrums;
- which way should the scale go? is 1 best or is 5 best? I always thought five was best until I got to the Netherlands and found that my performance is graded 1-5 with 1 being “you’re exceptional, and you’re in the wrong job” and 5 being “you’re a loser and you’re in the wrong job”
- when you work with a multinational audience cultural differences come into play. The Dutch are more likely to use the whole scale, and North Americas tend to only grade across the top three scores
- it doesn’t help people like me who always want to score things at 0.5 of something, so 3.5 or 4.5 but never a round number
TED does something that gets past these issues, and still returns useful assessment data. Their system is very simple and very appealing – especially for those of us whose brains are wired for words rather than numbers.
They give me selection of words and let me choose three, or the same word three times if I want.
The data collected still tells them what people thought of the video, you can label it as persuasive, funny, obnoxious or just OK. By giving a selection rather than open fields it shepherds the viewers towards useful responses.
The way the data is presented it also very cool, and very simple. A tag cloud lets me see that most people rated this talk as fascinating. Beautiful, informative and inspiring were also common ratings. Just a few people found it confusing or unconvincing.
We’ve just added the “amazon star” rating to some of our content, it was a big deal for us. But this is a whole other level, and it’s giving me more ideas.
Postscript: Sadly this rating system has been abandoned by TED