Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

A Rough Guide to Spotting Bad Science

  1. Jan 5, 2015 #1
    Spotting-Bad-Science-v2-vps80w-724x1024.png
     

    Attached Files:

  2. jcsd
  3. Jan 5, 2015 #2

    Bystander

    User Avatar
    Science Advisor
    Homework Helper
    Gold Member
    2016 Award

    Looks like something that'd be worth pinning here and there around the forum, barring copyright problems, of course.
     
  4. Jan 6, 2015 #3
    I agree, I was thinking the same thing.
     
  5. Jan 6, 2015 #4

    Garth

    User Avatar
    Science Advisor
    Gold Member

    Now let's see....

    The Announcement of the BICEP2 B Mode polarisation signal in the CMB. First Direct Evidence of Cosmic Inflation

    1. Sensationalised Headlines.

    The announcement went global and was headline news here on the BBC Inflation: A compact guide to big science and British newspapers.

    2. Misinterpreted Results Constraint on the primordial gravitational waves from the joint analysis of BICEP2 and Planck HFI 353 GHz dust polarization data
    5. Speculative Language Worse than using the speculative 'may', could', and 'might' is to use definite language when you are wrong, or perhaps, not yet right. From the CfA announcement:
    We now know there was no evidence (yet discovered), no confirmation of gravitational waves from inflation, and therefore no confirmation (yet) of the deep connection between QM and GR,

    6&7. Sample Size Too Small & Unrepresentative Samples BICEP2 looked at a relatively small area of sky, PLANCK looked at the whole sky and found dust, if you look at the the crude map of the PLANCK findings from the CNES site you will see the boxed area and that there was a quieter (dark blue=less dust signal) part of the sky closer to the South Celestial Pole. (Might BICEP2/3 repeat the experiment looking at this part of the sky?)

    12. Journals and Citations That's the point - there weren't any! The Havard-Smithsonian Center for Astrophysics went public before they had published in a peer reviewed reputable journal.

    My point? It is not only crackpots who are guilty of 'Bad Science'.

    Garth
     
    Last edited: Jan 6, 2015
  6. Jan 6, 2015 #5

    Doug Huffman

    User Avatar
    Gold Member

    I would add ad-hockery.
     
  7. Jan 6, 2015 #6
    Part of that guide could be applied to politicians
     
  8. Jan 6, 2015 #7

    Garth

    User Avatar
    Science Advisor
    Gold Member

    O come on, there would be nobody left.....

    Garth
     
  9. Jan 6, 2015 #8

    OmCheeto

    User Avatar
    Gold Member
    2016 Award

    Perhaps we should ( @RonL ) add a poster of how to ( @RonL ) spot good science. hmmmm?

    Scientific.method.jpg
     
  10. Jan 6, 2015 #9

    RonL

    User Avatar
    Gold Member

    I'm brushing up on how to spot bad science, watching "Warehouse 13" for the third time around on Netflix, just can't figure out why Joanne Kelly makes me feel like Kaw-Liga ?:oldconfused::oldlove:
     
  11. Jan 6, 2015 #10

    OmCheeto

    User Avatar
    Gold Member
    2016 Award

    That's somewhat humorous, as I've been brushing up on my science by watching the 1st season of the Twilight Zone. (1959 edition. good stuff! episode 7 sparked an idea on how to get to Mars and back.)

    I do not know what "Warehouse 13" is, who Joanne Kelly is, nor what a, um, "Kaw-Liga" is supposed to be.

    Anyways..... This was an experiment in how the new forum software works. Did you get 1 or 2 alerts?
     
  12. Jan 6, 2015 #11

    RonL

    User Avatar
    Gold Member

    I got the two alerts, they took me to my profile, does that mean they were successful ?
    Warehouse 13 is a fun spoofy Si-Fi TV series, Kaw-Liga was a wooden indian, Joanne Kelly is just HOT :oldlove:.
     
    Last edited: Jan 6, 2015
  13. Jan 6, 2015 #12

    dlgoff

    User Avatar
    Science Advisor
    Gold Member

    How about in a Signature?
     
  14. Jan 7, 2015 #13
    You wandered into that thread last June and you still remember it? That in itself puts you into a category of intellect that leaves me thinking "How the H-ll does he do that." :D
     
  15. Jan 7, 2015 #14

    OmCheeto

    User Avatar
    Gold Member
    2016 Award

    It's a long running joke between Drakkith and myself. I'm pretty sure it started before that post.

    I liken it to jumping into the deep end of the pool, and realizing that I'm just going to have everyone eventually laugh at me, trying to get out, not knowing how to swim, so I ask the nearest person; "Please get me out of here".
    Which he did that day.

    @Drakkith , do you remember the first time? I surely don't.
     
  16. Jan 7, 2015 #15

    RonL

    User Avatar
    Gold Member

    I'm so slow, :( I think I just figured it out....... If I do @Drakkith , and @OmCheeto , you both get an alert to look at this post, no matter what forum it might be in ? right :):rolleyes:

    ps. If I make some comment and have the word here, how do I make that a link, that clicking on it opens some intended place or document ? :confused:
     
    Last edited: Jan 7, 2015
  17. Jan 7, 2015 #16

    OmCheeto

    User Avatar
    Gold Member
    2016 Award

    Bad.
    No. That means there's a bug in the software. This is why I used to write all of my own.
    I googled those terms this morning. My responses: If you say so, yes, and yes. :)

    That is correct.
     
  18. Jan 7, 2015 #17

    RonL

    User Avatar
    Gold Member

    Just witnessed an example of how fast our mods are at clearing out bad science, :nb) @Doug Huffman, I didn't have time to like your reply:eek::D
    Something about a fusion engine ?
     
  19. Jan 7, 2015 #18

    WWGD

    User Avatar
    Science Advisor
    Gold Member

    How about having, e.g., Basketball players vouching for tooth paste. Why would I prefer brand X just because Shak
    is sponsoring it --tho he does have shiny teeth.

    My prof. used to refer to non-credible results using small sample sizes as "the law of small numbers".

    Maybe you can summarize good research when you see the researcher making an honest effort to falsify , i.e., find as many flaws as possible in his/her own research results. Reminds me of Metallica (decrying the opposite) : "fighting the truth, winning is all " (Eye of the Beholder)

    EDIT : I mean the researcher, before espousing a theory, should try to disprove it in every possible way, find and address any possible flaw, and after all this fails, the theory stands. I think this is the method some followed in the middle ages; when a theory was presented, all the arguments addressing why the theory could be wrong would be presented and countered.
     
    Last edited: Jan 7, 2015
  20. Jan 7, 2015 #19

    mfb

    User Avatar
    2016 Award

    Staff: Mentor

    To be fair, this is common practice if the field is dominated by large collaborations. They have their internal review processes and get close to 100% approval rate from journals anyway.
    Conference talks or similar presentations are the usual way to make important results public, everything else appears on arXiv long before a journal publishes it.


    I disagree with point 5 in the first post. Good scientists won't use "we are sure that Y" unless they are really, really certain (an extremely rare case). They will prefer "X is evidence for Y", "X suggests Y", "X could mean Y" or "we observed X" (and leave the conclusions out).
     
  21. Jan 7, 2015 #20

    Drakkith

    User Avatar

    Staff: Mentor

    I think it involved a bottle of tequila, three waitresses from a nearby diner, a lopsided pool table, and a hot tub. And I'm pretty sure I remember lawn gnomes coming into the picture at some point in the night... didn't you leave the left half of your pants on one?
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook




Similar Discussions: A Rough Guide to Spotting Bad Science
  1. Bad Science on TV (Replies: 36)

  2. Rough Science (Replies: 3)

  3. Bad Science in Movies (Replies: 2)

  4. Bad science reporting (Replies: 8)

Loading...