Psych and Social sciences are the harder science

[Pythagorean]

Psych and Social sciences are the "harder" science

Fanelli D (2010) “Positive” Results Increase Down the Hierarchy of the Sciences. PLoS ONE 5(4): e10068. doi:10.1371/journal.pone.0010068


http://www.plosone.org/article/info:doi/10.1371/journal.pone.0010068

 

[Pyrrhus]

Fanelli D (2010) “Positive” Results Increase Down the Hierarchy of the Sciences. PLoS ONE 5(4): e10068. doi:10.1371/journal.pone.0010068


http://www.plosone.org/article/info:doi/10.1371/journal.pone.0010068

I have been discussing this repeatedly, and extensively every few months on this forum, and elsewhere. Someone wrote it in a paper, and published it. Now, I can send them this paper, haha.

 

[Pythagorean]

Do you any anecdotes to offer? I feel like I've had some pretty robust physics professors, but I've read very little of their published work and I'm probably not capable of an objective critique. I have been considering a psychology PhD for sometime (I did my undergrad in physics; finishing an MS in computational neuroscience currently).

 

[D H]

Six comments.

1. Someone will inevitably post the XKCD comic on this topic. I'll leave that task to someone else.

2. This proves the adage "Politics ain't rocket science. It's much harder."

3. On a more serious note, there are a lot of different ways to interpret this paper. One is that the soft sciences are indeed soft. They are much more subject to confirmation biases than are the hard sciences.

4. "This study analysed 2434 papers published in all disciplines and that declared to have tested a hypothesis." They did this by searching papers (titles only? titles+abstracts? full text?) for the phrase "test* the hypothes*". I don't recall seeing that phrase in the articles I read. I just looked at several 'test of physics' type papers as a test of the hypothesis "physicists don't use the phrase 'test* the hyothes*'". That phrase is just not there for the most part.

5. Some disciplines use the phraseology tested for in this paper, others don't. Some may use it preferentially to report positive results, others tend to use it more when reporting negative results. There is no accounting for this potential bias in the paper.

6. Data. Where's the data? They analyzed 2434 papers. Which ones, and how did they classify them?

 

[Pyrrhus]

Six comments.

1. Someone will inevitably post the XKCD comic on this topic. I'll leave that task to someone else.

2. This proves the adage "Politics ain't rocket science. It's much harder."

3. On a more serious note, there are a lot of different ways to interpret this paper. One is that the soft sciences are indeed soft. They are much more subject to confirmation biases than are the hard sciences.

4. "This study analysed 2434 papers published in all disciplines and that declared to have tested a hypothesis." They did this by searching papers (titles only? titles+abstracts? full text?) for the phrase "test* the hypothes*". I don't recall seeing that phrase in the articles I read. I just looked at several 'test of physics' type papers as a test of the hypothesis "physicists don't use the phrase 'test* the hyothes*'". That phrase is just not there for the most part.

5. Some disciplines use the phraseology tested for in this paper, others don't. Some may use it preferentially to report positive results, others tend to use it more when reporting negative results. There is no accounting for this potential bias in the paper.

6. Data. Where's the data? They analyzed 2434 papers. Which ones, and how did they classify them?

Good comments.

I have done meta-analyses in the past. In fact, I published one too. The main problem is selection bias. The data is selected by the researcher, and this is very crucial. The researcher must be careful to have searched the literature in depth, and even if done so quite exhaustively, there are also "grey publications" that are likely to be ignored. However, I suspect that even if bias is present is likely too not throw the results to far off.

 

[Pyrrhus]

Do you any anecdotes to offer? I feel like I've had some pretty robust physics professors, but I've read very little of their published work and I'm probably not capable of an objective critique. I have been considering a psychology PhD for sometime (I did my undergrad in physics; finishing an MS in computational neuroscience currently).

Yes... Search Mathematical Models in Economics in this forum. Most posters seem to think that Math only "works" in Physics. A very ridiculous claim. I don't know about PhD in Psychology. The only area of psychology that I read periodically is psychometrics. I may read a bit about perception.

 

[Pythagorean]

I heard a Nature podcast from an economist criticizing the way economists use mathematical models (compared to physicists). The nature of the criticism was basically that economists don't have much for controlled experiment, so elegance becomes a determining factor of what kind of math survives, which isn't scientifically sound.

I am not familiar with modern economic theory, so I have no opinion, just regurgitating the podcast hearsay.

 

[Pyrrhus]

I heard a Nature podcast from an economist criticizing the way economists use mathematical models (compared to physicists). The nature of the criticism was basically that economists don't have much for controlled experiment, so elegance becomes a determining factor of what kind of math survives, which isn't scientifically sound.

I am not familiar with modern economic theory, so I have no opinion, just regurgitating the podcast hearsay.

Noted, and I disagree. Not willing to fight this mathematic modeling argument anymore. Also, economists do have controlled experiments for some cases, but for most we are left with estimating models by controlling for as much as we can...

 

[M65]

The issue of controlled experimentation can be dealt with in two major ways I can think of.

One is agent-based modeling. Here you can create and recreate an environment as many times as you like, with controls. Here's a great example of a very successful agent-based sim (sorry can't post links yet):

www (dot) u (dot) arizona (dot) edu/~mlittler/artanasazi.htm

The other, more obvious one, is to do experiments with real people à psychology.

More generally speaking, this artificial distinction between "hard" and "soft" sciences on grounds of whether experiments can be done, complexity, etc. is nonsense. "Real" physical systems can easily become as complex as more well-known social complex systems. (See e.g. the three-body problem.)

To paraphrase Duke Ellington: there are two kinds of science: good science and bad science. As the late Hans Eysenck pointed out, there are concepts in psychology, like reinforcement learning, that are on far sounder empirical footing than some of the more speculative areas in modern physics.

So trying to rank sciences in terms of "hardness" seems pretty silly.

ps first post here; hello

 

[FredoX]

Do you any anecdotes to offer? I feel like I've had some pretty robust physics professors, but I've read very little of their published work and I'm probably not capable of an objective critique. I have been considering a psychology PhD for sometime (I did my undergrad in physics; finishing an MS in computational neuroscience currently).

Yes please!
We need more physicists and mathematicians in psychological science.
But be prepared to have papers rejected because you used "vague terms like stability, self-organization and attractor landscape" (yes this is an actual quote from a reviewer on a paper I co-authored). I have seen papers by colleagues rejected in the Journal of Mathematical Psychology because "the mathematics you use is too difficult for our readers" when basic differential calculus was used. Yes you read correctly. That journal actually has mathematics in its title.

I believe psychology as a science is in serious trouble and I will provide some arguments below. This serious trouble is slowly being revealed to the general public and it is probably best captured by a satirical piece published in the high impact journal: perspectives on psychological science by Arina Bones (http://pps.sagepub.com/content/7/3/307.full.pdf, a poster version is available at her website: http://www.projectimplicit.net/arina/B2012.pdf )

It discusses the paper that started this thread and links it to some recent painful events for psychological science being the publication of the paper by social psychologist Daryl Bem in which he claims to have found evidence for the existence of Psi (http://dbem.ws/FeelingFuture.pdf ) and the largest scientific fraud committed by Dutch social psychologist and media darling Diederik Stapel.

My concerns:
- If you are a physicisist, read the paper by Daryll Bem, published in one of the highest impact journals of psychology and try stay sane while wondering how such a paper could ever be published in the name of science. Bem claims to have found evidence that his students can feel the occurence of future events above chance, especially men detecting erotic events. He claims this ability may have evolved because there is an evolutionary advantage for males to know where future possibilities for reproduction may occur. (no I am not making this up).
The editors claim the paper has been rigurously reviewed and it has been accepted because there seemed nothing wrong with the methods used. The fact that these results and their interpretation would turn almost all scientific disciplines from physics to evolutionary theory upside down (tear down that LHC!) is apparently not important. The paper should have been rejected on theoretical grounds. This concern may be summarised as: there is NO theory evaluation in psychology, all so called theories are actually theories of construction (that save the phenomena) they are not theories of principles (in the sense that Einstein proposed).

- if psychological science is indeed is the harder science, then a first year of psychology at graduate level should at least include a course in the major achievements of physics so we could learn from their efforts to understand the behaviour of non-living matter. Instead, most schools of psychology do not venture beyond teaching probability theory and inferential statistics dating back to the 1900s. The study of behaviour is the study of change, why not at least start to teach the mathematics of change? Summarising: many subdisciplines of psychological science are completely detached from contemporary developments in philosophy, mathematics, physics and biology... And this cannot be maintained for much longer. Engineers and computer scientist will take over most of the study of human behaviour.

- psychological science as a discipline is not capable to initiate a Solvay-like conference in which a consensus is achieved on a formalism describing what it actually is the discipline is studying. What is cognition, what isn't? What are the relevant levels of a system one should study? What are the interesting phenomena one should study? Currently (and unknown to many psychologist) humans are studied as ergodic systems. Certainly a questionable premise. It may be hard to believe for a physicist, but in psychological science I can claim to study mental representations of letters hypothesised to be involved in reading, without having to specify what these representations constitute of, how they relate to neural activity, or how they may be described or modeled mathematically. Interpreting results as if they existed is sufficient. Summarising: without a consensus formalism like the quantum formalism about the system psychology is studying, the interesting levels in the system that should be studied psychological science will not advance to produce theories of principles like the physical sciences have.

So is it a harder science in terms of object of study? Yes it probably is, but few working in the field realize what that actually entails: studying and learning from other disciplines of science. At least embrace mathematics as a tool to formalise theories and evaluate their predictions and empirical accuracy.

Any tips on how to achieve this are very welcome :)

 

[chill_factor]

I read outside interpretations of Daryll Bem's work and it seems he is making the case for predictive "modeling" and extrapolation by the brain subconsciously, not that people can actually see the future.

If that's the case, would his argument be more secure?

 

[Tzikin]

My concerns:
- If you are a physicisist, read the paper by Daryll Bem, published in one of the highest impact journals of psychology and try stay sane while wondering how such a paper could ever be published in the name of science. Bem claims to have found evidence that his students can feel the occurence of future events above chance, especially men detecting erotic events. He claims this ability may have evolved because there is an evolutionary advantage for males to know where future possibilities for reproduction may occur. (no I am not making this up).
The editors claim the paper has been rigurously reviewed and it has been accepted because there seemed nothing wrong with the methods used. The fact that these results and their interpretation would turn almost all scientific disciplines from physics to evolutionary theory upside down (tear down that LHC!) is apparently not important. The paper should have been rejected on theoretical grounds. This concern may be summarised as: there is NO theory evaluation in psychology, all so called theories are actually theories of construction (that save the phenomena) they are not theories of principles (in the sense that Einstein proposed).

The highest quality clinical evidence of Evidence Based Medicine and Psychology has rejected the traditional model of academic science and returned to the fundamental principles under which science is supposed to be practised. It is prioritising the evidence over the theory and really does not care about such matters as who is upset by the evidence.

The academic scientific model has been criticised as extremely open to subjective abuse for the medical profession to maintain a monopoly in health for the sake of power and profit. In evidence based medicine, nobody really understands how clinical psychology, physiotherapy or acupuncture works scientifically but the evidence says that they do. The reasoning behind preferring exercise to bed rest or avoiding therapist dependency is backed by hard evidence but is more common sense than scientific. And the studies based on this type of 'theory' are saving lives.

Is there any scientific explanation for the placebo effect?

So the very best in clinical medicine and psychology is harder. It is always the result of multidisciplinary teamwork and subject to multidisciplinary review because getting it right is a matter of life and death. Objectivity excludes group interests and how people feel about their life's work collapsing under the weight of the evidence. Objectivity should apply equal scepticism to old and accepted theories as old.

These are all fundamental failings of the fundamental scientific process and the ethical logic under which it is supposed to be practised. Modern clinical medicine and psychology is doing something about it.

However, you have chosen an example that goes somewhat beyond the evidence in its speculative argument and that would probably not have been published in clinical psychology. More rational explanations are non-verbal communication or simple determination and desire being involved in a self-fulfilling prophecy.

This is actually the worst of two worlds and a left over of the traditional academic scientific system of practice and is slowly being erased in psychology. It is a culture in which those who collect the data suddenly have the divine right of the person with funding to interpret it at a subjective whim. It is extremely common in academic science too. One ivory tower can contradict another's hard evidence as a result of peer rather than multidisciplinary review.

An example is that the idea of infinite missing links is not consistent with atomistic Mendelean genetics where a single gene can holistically change a creature. Geneticists are now backing Lamarck over Darwin but the evolutionists are entrenched. And many geneticists are in denial of the existence of epigenetics when their belief that DNA is the be all and end all of inheritance is threatened. I know. I have tried talking to them about it.

Just look at science's historical reputation in the resistance to changing direction in the face of the evidence. Clinical evidence is being followed in weeks regardless of the tradition in cultures where Evidence Based Medicine is practised culturally and in the best of psychology too. It is also these areas that are learning to deal with subjective evidence which requires an extra level of rigour if the grain of truth is to be found within the colouration of reporting. It deals with the vastly more complex situations of real life and interactive systems that the academic scientist has the luxury of ignoring.

 

[dmacfour]

I'd like to see Psychology become a harder science. I'm studying for a masters in Human Factors Psychology (engineering psychology), and I feel very much in the dark. One of my professors did an undergrad in Mechanical Engineering before he did a PhD in psych, and the concepts he teaches are extremely valuable to psychology, but frequently go over my head. He always complains about psych undergrads needing to at least take calculus. The lack of any significant math requirements for my undergrad are really starting to hamper my education.

I'd like to see the BS in psychology require much more math and bio than it did, and be geared toward people wanting to go on to grad school. At the very least, it'd reduce the saturation of psych degrees in the market, and make psych look more respectable.

 

[Digitalism]

I think the real question is how do we bring the rigorous methodology of the hard sciences to the soft sciences? I think the best way is to combine big data with sensors to get reams of objective data on biological, cognitive, mechanical and affective states and correlate them to environmental variables

 

[Pythagorean]

The highest quality clinical evidence of Evidence Based Medicine and Psychology has rejected the traditional model of academic science and returned to the fundamental principles under which science is supposed to be practised. It is prioritising the evidence over the theory and really does not care about such matters as who is upset by the evidence.

The academic scientific model has been criticised as extremely open to subjective abuse for the medical profession to maintain a monopoly in health for the sake of power and profit. In evidence based medicine, nobody really understands how clinical psychology, physiotherapy or acupuncture works scientifically but the evidence says that they do. The reasoning behind preferring exercise to bed rest or avoiding therapist dependency is backed by hard evidence but is more common sense than scientific. And the studies based on this type of 'theory' are saving lives.

Is there any scientific explanation for the placebo effect?

So the very best in clinical medicine and psychology is harder. It is always the result of multidisciplinary teamwork and subject to multidisciplinary review because getting it right is a matter of life and death. Objectivity excludes group interests and how people feel about their life's work collapsing under the weight of the evidence. Objectivity should apply equal scepticism to old and accepted theories as old.

These are all fundamental failings of the fundamental scientific process and the ethical logic under which it is supposed to be practised. Modern clinical medicine and psychology is doing something about it.

However, you have chosen an example that goes somewhat beyond the evidence in its speculative argument and that would probably not have been published in clinical psychology. More rational explanations are non-verbal communication or simple determination and desire being involved in a self-fulfilling prophecy.

This is actually the worst of two worlds and a left over of the traditional academic scientific system of practice and is slowly being erased in psychology. It is a culture in which those who collect the data suddenly have the divine right of the person with funding to interpret it at a subjective whim. It is extremely common in academic science too. One ivory tower can contradict another's hard evidence as a result of peer rather than multidisciplinary review.

An example is that the idea of infinite missing links is not consistent with atomistic Mendelean genetics where a single gene can holistically change a creature. Geneticists are now backing Lamarck over Darwin but the evolutionists are entrenched. And many geneticists are in denial of the existence of epigenetics when their belief that DNA is the be all and end all of inheritance is threatened. I know. I have tried talking to them about it.

Just look at science's historical reputation in the resistance to changing direction in the face of the evidence. Clinical evidence is being followed in weeks regardless of the tradition in cultures where Evidence Based Medicine is practised culturally and in the best of psychology too. It is also these areas that are learning to deal with subjective evidence which requires an extra level of rigour if the grain of truth is to be found within the colouration of reporting. It deals with the vastly more complex situations of real life and interactive systems that the academic scientist has the luxury of ignoring.

That's an interesting perspective; I hadn't thought about the weight on evidence over theory in the "soft" sciences and how it contributed to perception of hardness/softness. Of course, that doesn't stop people (especially in theoretical neurosciences) from speculating about mechanisms.

Though as far as this paper is concerned, DH's point #4 pretty much disqualifies the paper as being rigorous. I still haven't read this paper, I was just amused by the title and thought I'd share.

There's a paper that touches on a plausible "neuroscience of the placebo effect" with respect to pain. Just a grazing, really, but:
http://www.jneurosci.org/content/26/2/559.short

There was a lot of talk about "top down" regulation of pain in my undergraduate neuroscience courses years ago, so it's not terribly surprising. But the placebo effect is inevitably an umbrella term for many different kinds of (apparently) top-down regulation. You'd expect different signalling pathways to be involved in placebo effects that don't involve pain.

 

[jduster]

Psychology and social sciences still aren't hard sciences due to the former's enigmatic nature and the latter's subjectivity.

You can use as many objective means you want to test things, but if they are unobjectifiable, it doesn't matter.

 

[CubicFlunky77]

I guess I'm looking at this trivially, and no offense to anyone, but the fact that there is even a paper that seeks to compare the degree of relevance between the social and physical sciences suggests that these 'social scientists' (implying the authors of this paper, not all the people in the social sciences) have plenty of time on their hands.

This paper digs itself into a rather humorous little rut:

Although math provides the basis for the physical sciences, it is not one in and of itself. Interestingly enough, math is entirely omitted from the study even though all physical/biological phenomena would be impossible to assess without it. Thus, physical/biological phenomena would only exist philosophically. Units of measurement and empirical exactitude cannot be explained philosophically (from a social science perspective, which digresses from the pure mathematic basis of philosophy upheld by Euclid/Aristotle/Plato etc...) and thus the very tools that the authors used in this paper to support their hypothesis (statistical/empirical techniques) would not exist.

I was wondering how social scientists might respond to my argument. (Do not get me wrong, I do respect people in the social sciences, but I believe divorcing social science issues from physical science and math topics can be detrimental)

 

[lpetrich]

Here is a possible issue: treatment of negative results.

In the harder sciences, such results may be easier to spin as a sort of positive result. Like an upper limit. For instance, Particle Data Group reports on numerous upper limits on various searched-for processes. That may not be very feasible elsewhere.