- #1

jobyts

- 218

- 58

You should upgrade or use an alternative browser.

- Thread starter jobyts
- Start date

- #1

jobyts

- 218

- 58

- #2

Vanadium 50

Staff Emeritus

Science Advisor

Education Advisor

- 29,533

- 14,962

why do they say

Who is they?

- #3

- 18,135

- 10,965

Because "they" refers in this case to people who don't know what they are talking about.

- #4

jobyts

- 218

- 58

Pretty much all US media says that. As an example,

http://edition.cnn.com/2016/10/26/politics/cnn-poll-of-polls-october/

EDIT: Not sure if it is all media or just cnn.

http://edition.cnn.com/2016/10/26/politics/cnn-poll-of-polls-october/

The CNN Poll of Polls, which does not have a margin of error, includes the USA Today/Suffolk poll; the ABC News Poll conducted October 21-24; the CNN/ORC poll conducted October 20-23; the Quinnipiac University poll conducted October 17-18; and the Fox News poll conducted October 22-25.

EDIT: Not sure if it is all media or just cnn.

Last edited:

- #5

- 18,135

- 10,965

That doesn't make it right.Pretty much all US media says that.

- #6

Student100

Education Advisor

Gold Member

- 1,652

- 417

Pretty much all US media says that. As an example,

http://edition.cnn.com/2016/10/26/politics/cnn-poll-of-polls-october/

EDIT: Not sure if it is all media or just cnn.

I interpret that as "due to constraints error can't be estimated" not "we're perfect."

Last edited:

- #7

jobyts

- 218

- 58

That doesn't make it right.

Care to elaborate a bit more? Are you implying the margin of error should get multiplied and cnn is incorrectly and ignorantly stating that there is no margin of error.

The 3 outcome I can think of are

1. margin of error gets multiplied on each inclusion of individual polls.

2. margin of error does not change if you consider the sum of all the groups just as one larger group.

3. margin of error reduces with multiple groups, since the question bias is reduced, the dates of each individual poll is different (so a specific date sampling error is reduced), sample space is larger etc.

Statistical evidence would be the key to know how well they engineered their sampling criteria.

- #8

Vanadium 50

Staff Emeritus

Science Advisor

Education Advisor

- 29,533

- 14,962

- #9

- 18,135

- 10,965

- #10

jim mcnamara

Mentor

- 4,688

- 3,630

However @phinds hidden assumption is correct - science reporting in the general media stinks, IMO. People come away with weird ideas. As if this were a revelation...

- #11

Vanadium 50

Staff Emeritus

Science Advisor

Education Advisor

- 29,533

- 14,962

I interpreted the OP's original statement "why do they say poll of polls has no margin of error" as meaning there was no error.

So did I, which is why I wanted him to tell us where he got this from.

- #12

Borek

Mentor

- 29,167

- 3,838

You cannot lump data from many disparate methodologies into one amorphous blob and make sense of it.

I wonder if that's true. Chebyshev's inequality gives some upper limit to the variance (with very weak assumptions). Perhaps some analogous model could be used for a case where you mix data from different sources.

Could be the upper limit would be so huge it is not worth calculating though.

- #13

- 22,178

- 3,316

How does Chebyshev help at all??I wonder if that's true. Chebyshev's inequality gives some upper limit to the variance (with very weak assumptions). Perhaps some analogous model could be used for a case where you mix data from different sources.

Could be the upper limit would be so huge it is not worth calculating though.

- #14

Bystander

Science Advisor

Homework Helper

Gold Member

- 5,418

- 1,503

"Poll of polls?" Isn't that on a par with combining the forecast tracks for hurricanes?

- #15

russ_watters

Mentor

- 22,050

- 9,148

Yes, and I would expect the result to have a narrower error margin, even if it is tough to quantify."Poll of polls?" Isn't that on a par with combining the forecast tracks for hurricanes?

- #16

Borek

Mentor

- 29,167

- 3,838

How does Chebyshev help at all??

I am not saying it helps directly. I just wonder if there is no similar inequality that would say something about the upper limit of errors for the poll of polls.

- #17

collinsmark

Homework Helper

Gold Member

- 3,236

- 2,114

It is possible to combine the results of individual polls to obtain a meta-poll. It's also possible to calculate a margin of error for that meta-poll, and that margin of error will have a expected, general trend to be [itex] \frac{1}{\sqrt{N}} [/itex] of that of an individual poll, where [itex] N [/itex] is the number of individual polls (this assumes that the individual polls are polling the same thing [i.e., apples-to-apples] and that their individual margins of error are comparable).

As a simplified example, suppose we have [itex] N [/itex] polls for comparison. For the sake of simplicity, suppose all polls have identical margins of error, which is proportional to the the poll's standard deviation which I'll call [itex] \sigma [/itex]. (This is essentially saying that each poll is an equally valid predictor, even though each poll might give a unique prediction.)

We'll treat each poll as a random variable [itex] x_n [/itex] with a mean [itex] \mu_n [/itex] and a standard deviation [itex] \sigma_n [/itex], where [itex] \sigma_n = \sigma [/itex]: the same value for all polls.

Summing the results of all the polls into the random variable [itex] y [/itex],

[itex] y = x_1 + x_2 + x_3 + \dots + x_{N-1} + x_N [/itex]

gives the mean,

[itex] \mu_y = \mu_1 + \mu_2 + \mu_3 + \dots + \mu_{N-1} + \mu_N [/itex]

and variance

[itex] \sigma^2_y = \sigma^2_1 + \sigma^2_2 + \sigma^2_3 + \dots + \sigma^2_{N-1} + \sigma^2_N [/itex]

but since the individual variances are all the same in this simple example, we can say,

[itex] \sigma^2_y = N \sigma^2. [/itex]

and standard deviation

[itex] \sigma_y = \sqrt{N} \sigma. [/itex]

But in the end, we're not really interested in the sum but rather the average. So we scale the mean of the sum and the standard deviation of the sum by [itex] N [/itex].

[itex] \mu_{ave} = \frac{\mu_y}{N} [/itex]

[itex] \sigma_{ave} = \frac{\sigma_{ave}}{N} = \frac{\sqrt{N} \sigma}{N} = \frac{\sigma}{\sqrt{N}} [/itex]

And that last one is the kicker. It shows that when you combine multiple random variables, the*average* tends to reduce the "noise" by an amount [itex] \frac{1}{\sqrt{N}} [/itex].

----

The simple example above has a glaring limitation that it assumes that all the individual polls are created equal. In the real world that is not the case. Still, statisticians have mathematical tools to weigh the individual polls before combining, but that is getting out of the scope of this thread.

My point was just to say that it is possible to combine the results of polls into a meta-poll and still obtain statically significant results. The claim in the original post that the meta-poll has "no margin of error" is not true.

(This of course assumes that the individual polls are comparing apples-to-apples. It doesn't make any sense to combine a poll for the 2016, US presidential election with another poll regarding favorite ice-cream flavors, for example.)

As a simplified example, suppose we have [itex] N [/itex] polls for comparison. For the sake of simplicity, suppose all polls have identical margins of error, which is proportional to the the poll's standard deviation which I'll call [itex] \sigma [/itex]. (This is essentially saying that each poll is an equally valid predictor, even though each poll might give a unique prediction.)

We'll treat each poll as a random variable [itex] x_n [/itex] with a mean [itex] \mu_n [/itex] and a standard deviation [itex] \sigma_n [/itex], where [itex] \sigma_n = \sigma [/itex]: the same value for all polls.

Summing the results of all the polls into the random variable [itex] y [/itex],

[itex] y = x_1 + x_2 + x_3 + \dots + x_{N-1} + x_N [/itex]

gives the mean,

[itex] \mu_y = \mu_1 + \mu_2 + \mu_3 + \dots + \mu_{N-1} + \mu_N [/itex]

and variance

[itex] \sigma^2_y = \sigma^2_1 + \sigma^2_2 + \sigma^2_3 + \dots + \sigma^2_{N-1} + \sigma^2_N [/itex]

but since the individual variances are all the same in this simple example, we can say,

[itex] \sigma^2_y = N \sigma^2. [/itex]

and standard deviation

[itex] \sigma_y = \sqrt{N} \sigma. [/itex]

But in the end, we're not really interested in the sum but rather the average. So we scale the mean of the sum and the standard deviation of the sum by [itex] N [/itex].

[itex] \mu_{ave} = \frac{\mu_y}{N} [/itex]

[itex] \sigma_{ave} = \frac{\sigma_{ave}}{N} = \frac{\sqrt{N} \sigma}{N} = \frac{\sigma}{\sqrt{N}} [/itex]

And that last one is the kicker. It shows that when you combine multiple random variables, the

----

The simple example above has a glaring limitation that it assumes that all the individual polls are created equal. In the real world that is not the case. Still, statisticians have mathematical tools to weigh the individual polls before combining, but that is getting out of the scope of this thread.

My point was just to say that it is possible to combine the results of polls into a meta-poll and still obtain statically significant results. The claim in the original post that the meta-poll has "no margin of error" is not true.

(This of course assumes that the individual polls are comparing apples-to-apples. It doesn't make any sense to combine a poll for the 2016, US presidential election with another poll regarding favorite ice-cream flavors, for example.)

Last edited:

- #18

mheslep

Gold Member

- 360

- 728

IIRC, only in the case that the polls are i) statistically independent, ii) the errors Gaussian. A big IF.It is possible to combine the results of individual polls to obtain a meta-poll. It's also possible to calculate a margin of error for that meta-poll, and that margin of error will have a expected, general trend to be 1√N1N \frac{1}{\sqrt{N}} of that of an individual poll,

- #19

Student100

Education Advisor

Gold Member

- 1,652

- 417

- #20

Vanadium 50

Staff Emeritus

Science Advisor

Education Advisor

- 29,533

- 14,962

- #21

collinsmark

Homework Helper

Gold Member

- 3,236

- 2,114

The errors doIIRC, only in the case that the polls are i) statistically independent, ii) the errors Gaussian. A big IF.

Of course we're assuming a certain level of statistical independence in the individual polls -- such as we can assume that a given individual poll is not going to base its results on that of another individual poll. Each individual poll is conducted individually. That's not an unrealistic assumption.

- #22

collinsmark

Homework Helper

Gold Member

- 3,236

- 2,114

Is it not? Assuming that we're careful on how percentage and percentage points are used here, I think your method of combining is correct.

What I mean by that is assuming that the "40" and "60" are in units of percent, and the margin of error is +/- 1 percentage point then I think the method of combination is okay. Otherwise just get rid of the percent sign. Combine 40 units +/- 1 unit with 60 units +/- 1 unit to make 50 units +/- 0.7 units.

If something seems wrong here it is because you know right away that one or both of the original data points (original polls, if you will) represents a statistical fluke. Assuming the methodology that went into determining the margin of error to be +/- 1 unit is correct, it is very, very improbable that the initial data points will be different by a whopping 20 units. It's not necessarily impossible, but very improbable.

Initial data points like that would make me question the validity of the original 40 unit and 60 unit values, and the claim that their margin of error is +/- 1 unit. It's a hint that there might be something wrong there. Is it possible that these values are actually correct and are merely the result of random chance? Yes, it is possible, yet very improbable.

Share:

- Replies
- 9

- Views
- 441

- Replies
- 15

- Views
- 812

- Replies
- 18

- Views
- 1K

- Replies
- 102

- Views
- 6K

- Replies
- 7

- Views
- 545

- Replies
- 5

- Views
- 985

- Replies
- 9

- Views
- 930

- Replies
- 14

- Views
- 1K

- Last Post

- Replies
- 17

- Views
- 531

- Replies
- 12

- Views
- 1K