Wisdom of the Crowd for lesson plan

[srfriggen]

Hello,

I'm in an aspiring high school math teacher who is in his first math-teaching course, and we have split into groups that must teach the class for 2 weeks a different subject. My group is "data analysis and probability." I was trying to come up with a fun lesson for the class and thought using the Wisdom of the Crowd technique may be cool. Something as simple as guessing the number of jelly beans in a jar. Of course I know the sample size would have to be more than 20 people, but that's not the issue; we can always make it a project and put the jar outside the classroom and collect data from passers by.

My question really is about how exactly does this technique relate to data analysis and probability. I mean, it's pretty clear how collecting data and analyzing it works in this experiment (learning how to average results), but what exactly does this experiment tell us about the nature of probability? Would need to build a lesson based around the core concept.

From the research I've done on this it seems there are some conflicting views on what exactly is going on.

Any advice on how to incorporate this into a fun learning experience would be greatly appreciated.

 

[Stephen Tashi]

I'm in an aspiring high school math teacher who is in his first math-teaching course, and we have split into groups that must teach the class for 2 weeks a different subject. My group is "data analysis and probability." I was trying to come up with a fun lesson for the class

Is your course about teaching or about curriculum development - or both? Coming up with a lesson is curriculum development.

Of course I know the sample size would have to be more than 20 people

Why? Is this a rule of thumb known to practitioners of "Wisdom Of The Crowd"? In a statistical analysis, you can't put requirements on sample size until you define your goal mathematically.

My question really is about how exactly does this technique relate to data analysis and probability.

Trying to relate a real life situation to probability is a subjective process, so there no unique answer to that question. In general, the bare facts (data) , do no give enough information to supply the "givens" for a mathematical analysis.

If you merely want to display the data, you could teach the conventions of histograms, how to to compute a sample mean, etc. That subject is "descriptive statistics". This doesn't teach anything about probability.

If you want to use mathematics to analyze a real life problem from data, you must make assumptions about how the data is generated. (That's a good lesson in itself.) In applying probability theory to analyze data, the assumptions specify a probability model, or family of models, for the process that generates the data.

Two major divisions of statistical analysis are 1) Estimation 2) Hypothesis Testing.

In Estimation, you assume a probability model for how the data is generated that is a function of some unknown parameters. (For example, you might assume the data consists of independent random samples from the same lognormal distribution with unknown mean and variance.) You use the data to estimate the unknown parameters.

In Hypothesis Testing, you assume a probability model for the data, compute the probability of some aspect of the data based on that assumption and "accept" or "reject" the probability model based on how probable the model makes that aspect.

To me, the fundamental question to ask about the Wisdom Of The Crowd method is how well it does compared to other methods. I don't think that question can be investigated in a convincing manner by a single application of the method. You'd have to have many jars of jelly beans and apply the wisdom of the crowd to each of them.

A jelly bean counting example could be used to teach estimation. Fill a jar with jelly beans many times and (objectively) count the numbers of beans on each fill. Assume a probability distribution with unknown parameters for how many beans it takes to fill a jar. Estimate the parameters from the data.

 

[FactChecker]

My two cents:
You could compare the mean, median, and modes of the set of guesses with individual guesses. If you did enough experiments, you could compare the variation of the group mean, median, modes with the variation of individual guesses. You could try different experiments where the crowd is or is not allowed to discuss before coming to a group guess. Is that combined guess better than the median of individual guesses? I'm not sure if the statistics involved is very deep, but it would still be interesting.

 

[srfriggen]

Thank you both for your responses. I appreciate all of the input, and will share this with my group. right now we are in the infant stages of planning this. in fact, we haven't even met yet with our professor, I just saw a bbc special with marcus du sautoy and he did the jelly bean example and i was really curious. (btw the class is both teaching and curriculum development).

one question still remains for me though. how does the wisdom of the crowd work? what is the theory behind it? if you polled a group of people individually to guess how many jelly beans are in a jar, the mean converges to the exact number as you increase sample size. what is going on there? does anyone know why that works? i would like to have an explanation if i was going to use that method alone or compare it with other methods.

 

[FactChecker]

if you polled a group of people individually to guess how many jelly beans are in a jar, the mean converges to the exact number as you increase sample size. what is going on there? does anyone know why that works? i would like to have an explanation if i was going to use that method alone or compare it with other methods.

The Central Limit Theorem states that the average of a sample approaches the mean of a distribution as long as the original distribution has a well defined mean and variance. I don't know the proof. We have to assume that the mean of their guesses will be the same as the correct answer. There are management theories on how to organize groups for the best results (such as, don't let leaders dominate)

 

[Stephen Tashi]

We have to assume that the mean of their guesses will be the same as the correct answer.

That's the important point. Understanding a mathematical model for sociological phenomena like "Wisdom of the Crowd" requires understanding probability theory as a prerequisite. It isn't a good illustration of probability theory per se. Probability theory doesn't say that the average of observed estimates is "correct". it only says that the average of observed estimates is probably close to the population average of the estimates. An estimator of something can be a "biased" estimator, which roughly means that its population average may be "incorrect".

 

[noowutah]

Jaynes gives a good example for this. If a billion Chinese people estimate the height of the Chinese emperor and their estimates are unbiased, then you will get an awfully good approximation for the height of the Chinese emperor. Unfortunately, most people will think that the emperor is taller than he really is, and despite the large sample size the sample average won't do much for you. The crowd isn't always wise. Francis Galton noticed that people are pretty unbiased when it comes to estimating the weight of a bull at a fair. James Surowiecki has written a nice book about wise crowds.

 

[FactChecker]

I do not believe that there is a mathematical proof that the "Wisdom of the Crowd" leads to the correct answer. In fact, there are many counterexamples. What the "Wisdom of the Crowd" can claim is that a well run group process can get better results than an individual (even expert) person. Management books are full of techniques and processes that try to arrive at more accurate group decisions.