How useful is wisdom of the crowd?

I checked out the entries on Wikipedia and found there’s a book named The Wisdom of Crowds by James Surowiecki and another entry on just wisdom of the crowd (note the “s”). The idea is that groups of individuals are able to provide better solutions, answers or opinions than a single individual.

People raising hands
[image by Michael Jay]

Now Surowiecki stated 4 criteria needed for it to work:

  • Diversity of opinion
  • Independence
  • Decentralisation
  • Aggregation

The first two points are somewhat countered by herd behaviour, where people are (easily) influenced by other people. Based on my observations of people around me, and my personal experience, I found that it’s challenging to come up with your own original idea, thought or opinion. Then it’s also challenging to defend that originality against questioning by other people, or self doubt.

The other two points seem related. The more you decentralise, the more you need to aggregate better. It’s like specialists and generalists (using the terms loosely here). The more you specialise in something, the more work you need to tap on other fields of knowledge.

Aannyway, I was thinking about independent thought, the phrase I heard that’s “wisdom of the crowd” and crowdsourcing (though after reading its meaning, it’s not what I wanted). I was thinking if you’re working on something really hard, would the crowd give you better answers? Or would you (or an individual knowledgeable in that field) be a better choice?

Previously, I talked about using heuristics to match image textures. I was using a few tests to determine if a given texture image matched any of the textures in a sample set. This is like using wisdom of the crowd. No one individual test could match with a reasonably high enough accuracy rate. Let’s compare my matching algorithm with the criteria above.

I’ve got a few different tests, so “diversity of opinion” is covered. I’ve got “independence” because each test doesn’t rely on another. I’ve got “decentralisation” because each test is an “expert” on the particular test algorithm used. And I’ve got “aggregation” when I used some kind of average of the tests’ results.

What happens if the problem is so hard, so unheard of, that you don’t know whose opinion to seek?

Diversity dilutes the final (if any) solution if unrelated opinions are brought on. And for an uncommon problem, independent thinkers might get swayed by the first seemingly correct answer (thus following the herd), simply because the problem is so hard. And there’s not much expert knowledge if the problem is so rare (there goes “decentralisation”).

And whose answer are you going to trust, or how are you going to consolidate answers (aggregation)? When answers are uncertain, bias comes into play. A person’s reputation, your feelings towards a person, the way a person presented his solution.

Remember how I used weights on the results of my texture matching algorithm? Not all opinions are created equal…

Of course, all this could be another barber’s paradox, because there’s no such problem! *smile*

Ok, the whole point of this article is I want you to increase your reliance on your opinions, intuition, thoughts and feelings. I’m sure you have a healthy dose of humility and self-awareness such that when you’re truly stumped, you will find someone knowledgeable and ask. It’s when you form a habit of asking others before coming up with your own conclusions first that I’m worried about.