Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Criticising the scientific method

  1. Oct 12, 2008 #1
    Criticising the scientific method !!

    Hi all

    Quoting a part of prof. Michael Shermer's book: Why people believe weird things?, about some problems of scientific thinking, made me a bit confused.

    "Let us suppose that an ichthyologist is exploring the life of the ocean. He casts a net into the water and brings up a fishy assortment. Surveying his catch, he proceeds in the usual manner of a scientist to systematise what it reveals. He arrives at two generalisations: No sea-creature is less than two inches long. (2) All sea-creatures have gills. These are both true of his catch, and he assumes tentatively that they will remain true however often he repeats it.

    In applying this analogy, the catch stands for the body of knowledge which constitutes physical science, and the net for the sensory and intellectual equipment which we use in obtaining it. The casting of the net corresponds to observation; for knowledge which has not been or could not be obtained by observation is not admitted into physical science.

    An onlooker may object that the first generalisation is wrong. "There are plenty of sea-creatures under two inches long, only your net is not adapted to catch them." The icthyologist dismisses this objection contemptuously. "Anything uncatchable by my net is ipso facto outside the scope of icthyological knowledge. In short, "what my net can't catch isn't fish." Or — to translate the analogy — "If you are not simply guessing, you are claiming a knowledge of the physical universe discovered in some other way than by the methods of physical science, and admittedly unverifiable by such methods. You are a metaphysician. Bah!
    "

    My question is:
    Is this really explaining the scientific method ?!! If yes, so how we're relying on such method ?!!!!!!!!!!
    What is the appropriate way to criticize these claims and ensure that the scientific method is the best knowledge method that does not have such holes ?!!


    Thank you !


    Hadeka
     
    Last edited: Oct 12, 2008
  2. jcsd
  3. Oct 12, 2008 #2

    D H

    User Avatar
    Staff Emeritus
    Science Advisor

    Re: Criticising the scientific method !!

    The post you quoted (I don't deign to lend it support by quoting it yet again) employs two logical fallacies, minimum. He first set up a parody of the scientific method in his ichthyologist example, and then proceeded to argue against this parody. He then extended this false description of the scientific method to the physical sciences. This is a red herring argument.

    Scientists know that their measuring devices and the measurements from them are incomplete and somewhat erroneous. That does not mean the scientific method itself is flawed. It merely means scientists should strive to improve their techniques (which they do, by means of research into new and improved measuring devices), double check their results (which they do, by means of peer review and independent replication of experiments), and be careful of overgeneralization (which they try to do).

    The quoted argument does have one thing right: Science treats explanations which cannot be possibly observed as inadmissible to the body of science. Science is predicated on the existence of reality. For example, an argument that we are all in a computer simulation is untestable and unscientific.
     
  4. Oct 12, 2008 #3

    Bill_B

    User Avatar
    Gold Member

    Re: Criticising the scientific method !!

    It might be an oversimplification, but I think it's generally accurate.

    Of course it sounds silly to believe that there are no sea-creatures smaller than 2 inches long. But from the perspective of the analogy, it's a reasonable conclusion to draw.

    The key is this statement -
    By definition, the body of knowledge is EVERYTHING we know about a topic. If everything we know about sea-life is in the net in front of us (meaning we've never seen a tadpole or a minnow in pond, etc.), then we don't know very much. And if all of the fish we see are longer than 2 inches, then as far as we know (from observation) that's true for all sea-life, until we observe otherwise. It's important to note that the 2inch fish assumption is tentative -

    which means it could change at a later time. The Scientific Method is an iterative process. (Like washing your hair - lather, rinse, repeat)

    After thinking about it a little bit, we can postulate that there might be smaller fish that fall through the holes in the net. We then create a new test (build a tighter net), and gather more data using the new test (catch more fish with the new net). If we catch something shorter than 2 inches, then we've just disproved (or modified) our original hypothesis. Rinse, repeat. Welcome to the scientific method.

    The problem is when things are outside of what is currently testable. If a person says to us "I KNOW that there are smaller fish" without any evidence, and with no way to test and observe (maybe we don't have the technology to build a better net), then it can't be added to the body of knowledge. After all, with no observable evidence, what proof does he have? How can he know? Maybe he just made it up. We can't apply the scientific method to something that isn't testable.

    Once something becomes testable (maybe we invent a new technology, or maybe someone simply thinks up a new test that works), then the scientific method can be applied, and the data could become part of the body of knowledge.

    Honestly, I'm not sure what claims you want to criticize or what holes you see. The analogy that Shermer used could have been a little clearer, but I think it works. And if done correctly, the scientific method works, albeit slowly (lots of iterations).
     
  5. Oct 12, 2008 #4

    D H

    User Avatar
    Staff Emeritus
    Science Advisor

    Re: Criticising the scientific method !!

    The first is correct. The latter is not. Because the author could not argue with the scientific method as it is, he (or she) intentionally oversimplified and overgeneralized to the extent as to make a parody - a straw man - of the scientific method. While this overly simplified and overly generalized parody is subject to attack, what he is attacking is not the scientific method. He is attacking his own parody of the method. In short, the author constructed a straw man argument.
     
  6. Oct 12, 2008 #5
    Re: Criticising the scientific method !!

    By my understanding of the scientific method that story does not depict it at all. The "scientist" in that story looks at a bunch of information and makes some guesses about the implications of patterns he sees. That might qualify as scholarship but not as science.

    A scientific approach would begin with the hypotheses about the size and gills of sea-dwelling creatures. Then there would be design and execution of experiments to rigorously test those hypotheses through direct, counterfactual, and other methods. The experiments would need to be thoroughly documented so that they could be accurately reproduced by other scientists. The initial findings and experimental method would be published and reviewed by other scientists.

    If the method and hypotheses were deemed sound by the scientific community, at that point the hypothesis wouldn't be regarded as fact but simply some interesting initial findings. In subsequent years other scientists would reproduce the experiments, perhaps with some adjustments to the experimental method. If the subsequent experiments produced the same results there would be more and more confidence in the hypotheses.

    But the hypotheses would never reach the status of "truth" like you might have in religion or philosophy. You never really have more than "confirmed findings" in science: an idea, the results of earlier tests of that idea, and instructions on how to reproduce those tests to examine the subject yourself.

    (So I think it's pretty obvious that the two hypotheses from the story - that no sea-dwelling creature is less than two inches long and that all sea-dwelling creatures have gills - would never successfully pass through the rigor of the scientific method as I've described it. (Unless, as Bill_B suggests, the net is really only a metaphor for everything that's directly perceivable. But that would be dumb, as if Shermer were trying to explain science with the language of dream interpretation or something.))
     
    Last edited: Oct 12, 2008
  7. Oct 12, 2008 #6

    Bill_B

    User Avatar
    Gold Member

    Re: Criticising the scientific method !!

    I think we disagree on what the argument is about. I don't believe Shermer is trying to argue against the scientific method (or any strawman facsimile thereof) at all. The way I read it, he is talking about why scientists don't automatically believe any willy-nilly idea offered to them, but instead require testable evidence. I think my wife has a copy of the book - when she gets home I'll see if she knows where it is. Perhaps there's more context that would be helpful.


    I also think that the reason the analogy exists is to attempt to make the explanation more accessible to an average audience (to which his books are usually targeted). I'll concede that the analogy may not be the best possible one, but I think you're reading too much into it (or perhaps taking it too literally).
     
  8. Oct 12, 2008 #7

    Bill_B

    User Avatar
    Gold Member

    Re: Criticising the scientific method !!

    I agree that it doesn't qualify as the whole of science, but I think it is very much a part of the scientific method. You have to be able to analyze the data and draw conclusions, as a basis for your next hypothesis. If your hypothesis is disproved, you drew an incorrect (or an incomplete) conclusion.

    How do you begin with a hypothesis? Do you think up totally a new hypothesis without basing it on some preexisting body of knowledge? Don't you usually start with a question about a peculiarity or pattern in existing data?

    Except as noted, I don't disagree with your description of the scientific method - it's a more realistic picture of the way science works in the real world. However, I don't think Shermer was trying to build an exhaustive model of the scientific method, but a layman's model that could be used to explain why scientists often dismiss non-testable phenomena as hokum.

    I agree that if taken literally, it wouldn't stand up to rigor. But it's an analogy, it's not supposed to be taken literally.

    And yes, the way I read it, the net is a metaphor for our observational tools and processes - microscopes, accelerometers, our eyes, etc., while everything in the net (the catch) is a metaphor for everything we know about science.

    As far as I can tell, all Shermer is trying to say is that because of the scientific method, if something can't be tested, we can't say it is "known" (in the usual scientific meaning of the term).

    And reading it again, I think he may also be trying to point out the problems that may arise if a scientist entirely dismisses a hypothesis that cannot be tested at the present time (but might be in the future). Unfortunately, I'd need more context to be sure. I'll see if my wife can find her copy of the book. Or perhaps the original poster can post more snippets.
     
  9. Oct 12, 2008 #8
    Re: Criticising the scientific method !!

    Well ... It is written here:
    http://www.skywise711.com/Skeptic/WPBWT/index.html
    Twenty-five Fallacies That Lead Us to Believe Weird Things.


    It's number three: Equipment Constructs Results, which is under the section of: Problems of scientific thinking.


    Thanks to all of you !

    Looking for more participation to shed more light on the topic..

    Many thanks



    Hadeka.
     
  10. Oct 12, 2008 #9
    Re: Criticising the scientific method !!

    @Bill_B: Yeah, I think all of what you're saying is true there. Analysis is of course part of the scientific method - but it's pretty much part of every epistemological approach. In the excerpt above there's no distinction made between observation and analysis and it appears to me that science is presented as no different from deep thought. Within the excerpt it doesn't mention testing hypotheses or conclusions at all.

    ...okay, I've just noticed that I was probably confused by an unmatched quotation mark in that last paragraph. I hadn't realized that the scientist was the one calling the onlooker a metaphysician; initially I thought it was the other way around. So what you're saying makes more sense.

    Still though, I think the answer to hadeka's question "Is this really explaining the scientific method ?!!" is that no, that excerpt is not explaining the scientific method. It's more explaining what the epistemological domain of science is (with a confusing and inexact analogy, in my opinion).
     
  11. Oct 12, 2008 #10
    Re: Criticising the scientific method !!

    In reading the original source it appears that Bill_B's inference of the purpose of the story is dead-on: the title of that section is "Equipment Constructs Results" and the story is preceded with the assertion that scientific understanding of the nature of intelligence has been screwed up by the inexact and limited nature of IQ tests.

    So hadeka, I would maintain that the excerpt is not trying to explain the scientific method. It's making a different point entirely, that the conclusions of a scientific investigation can be flawed by insufficient experimental design that does not account for the limitations of the experimental equipment. (Which is basically what Bill_B said, I think.)
     
  12. Oct 12, 2008 #11
    Re: Criticising the scientific method !!

    Shermer Fallacy #1: Imply that there's a singular scientific method.

    There's no such thing. Lots of scientific discoveries did not go through the prototypical scientific method.
     
  13. Oct 12, 2008 #12

    Bill_B

    User Avatar
    Gold Member

    Re: Criticising the scientific method !!

    I don't think that's what I said, but I wish it was :biggrin: since I think you hit the nail on the head
     
  14. Oct 14, 2008 #13

    CRGreathouse

    User Avatar
    Science Advisor
    Homework Helper

    Re: Criticising the scientific method !!

    If the existence of sea creatures below 2" is supposed to represent things beyond the researcher's equipment, but within the realm of science, then the experiment is simply inadequate to determine the existence of small sea creatures and the claim 'such knowledge is metaphysical' is wrong. If it's supposed to represent things literally beyond the ability of science, then the claim 'no creatures under 2" exist' is not falsifiable and cannot be claimed under the scientific method.

    Either way the analogy fails as a criticism of the scientific method. If it's a criticism of researchers drawing untoward conclusions, though, it could work.
     
  15. Oct 14, 2008 #14
    Re: Criticising the scientific method !!

    I agree with this ... We cannot prove that there are no sea creatures smaller than 2 inches except by using experiments that prove that such creatures does not exist [ A net that can catch sea creatures that are less than 2 inches long will either prove or disprove that such creatures exist ]

    I mean, if i want to prove that there are no sea creatures less than 2 inches long, i should use a net that can catch such creatures to either prove or disprove their existence.

    Thank you :D
     
    Last edited: Oct 14, 2008
  16. Oct 14, 2008 #15

    Ivan Seeking

    User Avatar
    Staff Emeritus
    Science Advisor
    Gold Member

    Re: Criticising the scientific method !!

    You cannot prove the general case that something does not exist.
     
  17. Oct 14, 2008 #16

    HallsofIvy

    User Avatar
    Staff Emeritus
    Science Advisor

    Re: Criticising the scientific method !!

    Yes, but the crucial part of the analogy is when someone objects to the Ichthyologist that there are sea creatures under 2 inches long and he asserts that any thing he cannot catch in his net is not a sea creature. That is a Parody. As D H said, any good scientist- or anyone using the scientific method- would respond "That may well be true but it hasn't been proved. Do you have any suggestions for improving the net?"

    Which, in the analogy it wasn't. That was where Schermer "cheated". He said, at first, that the concusions were "tentative" but later had the Ichtyologist say, '"Anything uncatchable by my net is ipso facto outside the scope of icthyological knowledge. In short, "what my net can't catch isn't fish."'



     
  18. Oct 14, 2008 #17

    arildno

    User Avatar
    Science Advisor
    Homework Helper
    Gold Member
    Dearly Missed

    Re: Criticising the scientific method !!

    I would say, though, that NO ONE who did not have other means of obtaining evidence than the ichtyologist in question are to be given more authority than that ichtyologist.
    For example, a blind desert dweller has even less intelligent to say about life in the ocean than the ichtyologist..
     
  19. Oct 14, 2008 #18
    Re: Criticising the scientific method !!

    That's not scientific either, though - that's giving the ichthyologist a priori authority simply because he's the one who obtained the evidence. Him collecting the evidence doesn't make his conclusions / guesses about the nature of sea life any more authoritative.
     
  20. Oct 20, 2008 #19
    Re: Criticising the scientific method !!

    Wow, this is getting a little too philosophical for me. If you want a physical explanation for why scientific predictions are often flawed, it is called entropy. If you don't have a complete set of data points, but only say, a and b, you can't make that good of a prediction about what will happen "between" a and b, because the statistics of the system are unknown. At least, that's the way I look at it.
     
  21. Oct 20, 2008 #20
    Re: Criticising the scientific method !!

    What are you talking about?
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Have something to add?



Similar Discussions: Criticising the scientific method
  1. The Scientific Method (Replies: 55)

  2. Scientific Method? (Replies: 17)

  3. Scientific Method? (Replies: 24)

  4. The Scientific Method (Replies: 19)

Loading...