Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Poll of Polls

  1. Oct 29, 2016 #1
    If each individual poll has a margin of error, when you combine all the polls, why do they say poll of polls has no margin of error? Shouldn't the margin of error get multiplied when you combine many polls?
     
  2. jcsd
  3. Oct 29, 2016 #2

    Vanadium 50

    User Avatar
    Staff Emeritus
    Science Advisor
    Education Advisor

    Who is they?
     
  4. Oct 29, 2016 #3

    phinds

    User Avatar
    Gold Member
    2016 Award

    Because "they" refers in this case to people who don't know what they are talking about.
     
  5. Oct 29, 2016 #4
    Pretty much all US media says that. As an example,
    http://edition.cnn.com/2016/10/26/politics/cnn-poll-of-polls-october/

    EDIT: Not sure if it is all media or just cnn.
     
    Last edited: Oct 29, 2016
  6. Oct 29, 2016 #5

    phinds

    User Avatar
    Gold Member
    2016 Award

    That doesn't make it right.
     
  7. Oct 30, 2016 #6

    Student100

    User Avatar
    Education Advisor
    Gold Member

    Last edited: Oct 30, 2016
  8. Oct 30, 2016 #7
    Care to elaborate a bit more? Are you implying the margin of error should get multiplied and cnn is incorrectly and ignorantly stating that there is no margin of error.

    The 3 outcome I can think of are
    1. margin of error gets multiplied on each inclusion of individual polls.
    2. margin of error does not change if you consider the sum of all the groups just as one larger group.
    3. margin of error reduces with multiple groups, since the question bias is reduced, the dates of each individual poll is different (so a specific date sampling error is reduced), sample space is larger etc.

    Statistical evidence would be the key to know how well they engineered their sampling criteria.
     
  9. Oct 30, 2016 #8

    Vanadium 50

    User Avatar
    Staff Emeritus
    Science Advisor
    Education Advisor

    I agree with Student100: what they seem to be saying is that they are not estimating the margin of error, not that they are estimating it and that it is zero.
     
  10. Oct 30, 2016 #9

    phinds

    User Avatar
    Gold Member
    2016 Award

    I interpreted the OP's original statement "why do they say poll of polls has no margin of error" as meaning there was no error. If they're just saying they can't figure out what it is, that's a different story.
     
  11. Oct 30, 2016 #10

    jim mcnamara

    User Avatar

    Staff: Mentor

    In meta-analysis papers in Biology you will often see something to the effect 'no margin of error determined'. Since a poll of polls (seems a bit illogical to me) is really analyzing multiple resultsets of polling data sets that seems a fair statement to me. You cannot lump data from many disparate methodologies into one amorphous blob and make sense of it.

    However @phinds hidden assumption is correct - science reporting in the general media stinks, IMO. People come away with weird ideas. As if this were a revelation....
     
  12. Oct 30, 2016 #11

    Vanadium 50

    User Avatar
    Staff Emeritus
    Science Advisor
    Education Advisor

    So did I, which is why I wanted him to tell us where he got this from.
     
  13. Oct 31, 2016 #12

    Borek

    User Avatar

    Staff: Mentor

    I wonder if that's true. Chebyshev's inequality gives some upper limit to the variance (with very weak assumptions). Perhaps some analogous model could be used for a case where you mix data from different sources.

    Could be the upper limit would be so huge it is not worth calculating though.
     
  14. Oct 31, 2016 #13

    micromass

    User Avatar
    Staff Emeritus
    Science Advisor
    Education Advisor
    2016 Award

    How does Chebyshev help at all??
     
  15. Oct 31, 2016 #14

    Bystander

    User Avatar
    Science Advisor
    Homework Helper
    Gold Member
    2016 Award

    "Poll of polls?" Isn't that on a par with combining the forecast tracks for hurricanes?
     
  16. Oct 31, 2016 #15

    russ_watters

    User Avatar

    Staff: Mentor

    Yes, and I would expect the result to have a narrower error margin, even if it is tough to quantify.
     
  17. Oct 31, 2016 #16

    Borek

    User Avatar

    Staff: Mentor

    I am not saying it helps directly. I just wonder if there is no similar inequality that would say something about the upper limit of errors for the poll of polls.
     
  18. Oct 31, 2016 #17

    collinsmark

    User Avatar
    Homework Helper
    Gold Member

    It is possible to combine the results of individual polls to obtain a meta-poll. It's also possible to calculate a margin of error for that meta-poll, and that margin of error will have a expected, general trend to be [itex] \frac{1}{\sqrt{N}} [/itex] of that of an individual poll, where [itex] N [/itex] is the number of individual polls (this assumes that the individual polls are polling the same thing [i.e., apples-to-apples] and that their individual margins of error are comparable).

    As a simplified example, suppose we have [itex] N [/itex] polls for comparison. For the sake of simplicity, suppose all polls have identical margins of error, which is proportional to the the poll's standard deviation which I'll call [itex] \sigma [/itex]. (This is essentially saying that each poll is an equally valid predictor, even though each poll might give a unique prediction.)

    We'll treat each poll as a random variable [itex] x_n [/itex] with a mean [itex] \mu_n [/itex] and a standard deviation [itex] \sigma_n [/itex], where [itex] \sigma_n = \sigma [/itex]: the same value for all polls.

    Summing the results of all the polls into the random variable [itex] y [/itex],

    [itex] y = x_1 + x_2 + x_3 + \dots + x_{N-1} + x_N [/itex]

    gives the mean,

    [itex] \mu_y = \mu_1 + \mu_2 + \mu_3 + \dots + \mu_{N-1} + \mu_N [/itex]

    and variance

    [itex] \sigma^2_y = \sigma^2_1 + \sigma^2_2 + \sigma^2_3 + \dots + \sigma^2_{N-1} + \sigma^2_N [/itex]

    but since the individual variances are all the same in this simple example, we can say,

    [itex] \sigma^2_y = N \sigma^2. [/itex]

    and standard deviation

    [itex] \sigma_y = \sqrt{N} \sigma. [/itex]

    But in the end, we're not really interested in the sum but rather the average. So we scale the mean of the sum and the standard deviation of the sum by [itex] N [/itex].

    [itex] \mu_{ave} = \frac{\mu_y}{N} [/itex]

    [itex] \sigma_{ave} = \frac{\sigma_{ave}}{N} = \frac{\sqrt{N} \sigma}{N} = \frac{\sigma}{\sqrt{N}} [/itex]

    And that last one is the kicker. It shows that when you combine multiple random variables, the average tends to reduce the "noise" by an amount [itex] \frac{1}{\sqrt{N}} [/itex].

    ----

    The simple example above has a glaring limitation that it assumes that all the individual polls are created equal. In the real world that is not the case. Still, statisticians have mathematical tools to weigh the individual polls before combining, but that is getting out of the scope of this thread.

    My point was just to say that it is possible to combine the results of polls into a meta-poll and still obtain statically significant results. The claim in the original post that the meta-poll has "no margin of error" is not true.

    (This of course assumes that the individual polls are comparing apples-to-apples. It doesn't make any sense to combine a poll for the 2016, US presidential election with another poll regarding favorite ice-cream flavors, for example.)
     
    Last edited: Oct 31, 2016
  19. Oct 31, 2016 #18

    mheslep

    User Avatar
    Gold Member

    IIRC, only in the case that the polls are i) statistically independent, ii) the errors Gaussian. A big IF.
     
  20. Oct 31, 2016 #19

    Student100

    User Avatar
    Education Advisor
    Gold Member

    Seems like the easiest way is to take the raw data from each sample, apply your own methodology, and recalculate error. Your sample size is larger than any one poll, but still realitivity small compared to the entire voting population. You could then look at one polls methodology, apply it to the sample, and your MOE should then be improved compared to that poll.
     
  21. Oct 31, 2016 #20

    Vanadium 50

    User Avatar
    Staff Emeritus
    Science Advisor
    Education Advisor

    Also, what do you do if you get one poll that measures 40 +/- 1% and the other that measures 60 +/- 1%? I don't think 50 +/- 0.7% is the right answer.
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Have something to add?
Draft saved Draft deleted



Similar Discussions: Poll of Polls
  1. If there is a poll (Replies: 5)

  2. This is a poll (Replies: 10)

  3. A poll (Replies: 3)

  4. Scientific Polling (Replies: 3)

  5. The happiness poll: (Replies: 27)

Loading...