Placebo Effects: Psychology’s Fundamental Flaw? Why active controls are not enough
Simon Oxenham covers the best and the worst from the world of psychology and neuroscience. Formerly writing with the pseudonym "Neurobonkers", Simon has a history of debunking dodgy scientific research and tearing apart questionable science journalism in an irreverent style. Simon has written and blogged for publishers including: The Psychologist, Nature, Scientific American and The Guardian. His work has been praised in the New York Times and The Guardian and described in Pearson's Textbook of Psychology as "excoriating reviews of bad science/studies”.
Follow Simon on Twitter
Like Simon on Facebook
Follow Simon on Google+
Subscribe via Email
Subscribe via RSS
Contact Simon directly by Email
A new paper published in Perspectives in Psychological Science (open access) suggests there is “a fundamental design flaw that potentially undermines any causal inference” in much psychology research. The paper is very accessible and well worth a read. The flowchart below describes the hypothesis:
Way back in the 1950’s a phenomenon called the Hawthorne effect (AKA the observer effect) was observed, an experiment to discover whether workers in the Hawthorne Works plant were more productive under increased lighting, found that productivity increased whenever the workers were being observed, regardless of the variable in question. The effect is a common cause of the placebo effect, which in most aspects of medicine is effectively handled by the gold standard of double blind randomised controlled trials (RCT’s). Unfortunately this is not always possible in psychology, not least because it is often impossible to blind an individual to the intervention, let alone the experimenter.
The problem is currently sometimes handled in psychology by using an active control, in which the control group are led to believe they are being experimented on too. The researchers in the new paper complain that this is not good enough, because all too often a difference remains in the participant’s expectations - an expectation effect. I’ve described studies where active controls might not be well chosen before, they are everywhere, in such cases expectation effects may be the cause of the results found, which would completely alter the conclusion of a study.
To demonstrate the severity of the problem, the researchers address video-gaming and brain-training research, not because the field is an example of poor research but because it is a rare example of psychology research that actually uses active placebos. The original research in question demonstrates a benefit in performance on vision and attention tasks following playing an action video game (Unreal Tournament) rather than Tetris or The Sims. The researchers in the new study asked people (admittedly only using Amazon’s Mechanical Turk) what improvements they would expect to see, following playing Unreal Tournament or following playing Tetris or The Sims. The result was that people expected the very same benefits as were found in the original research. More visual processing benefits were expected by participants from playing Unreal Tournament than from playing Tetris or The Sims, furthermore, more such benefits were expected by the participants from Tetris than The Sims whilst participants thought more benefits would be achieved in story-telling from The Sims and participants thought more benefits to visual rotation would be obtained from Tetris than from Unreal Tournament.
Could it be that in reality there is no real effect that can be generalised outside of the laboratory? Could it be that while they were being observed the gamers in the original experiment unconsciously let their behaviour be moulded by their own prior expectations?
If this is the case, then this is only the tip of the iceberg. Vast amounts of psychology experiments do not use active controls, let alone active controls that sufficiently avoid expectation effects. This will be no easy problem to fix, the researchers have some suggestions, many of which will likely be seen as impractical by researchers, but some of which may be taken up – the paper itself is valuable reading if you are a researcher yourself. The problem requires a major sea change in the way things are done, even with the best efforts of researchers, the problem will never go away entirely. You’d therefore be well advised to always keep the possibility of expectation effects in the back of your mind when you are reading a piece of psychology research. The question you want to ask yourself is this - these results might look very impressive on paper, but what are they compared to and will they still apply to me in my living room rather than to someone who knows they are being experimented on, and might think they know why, in a far away laboratory under the watchful gaze of a scientist in a metaphorical white coat?
The paper has also been well discussed in a blog post by Daniel Simons one of the paper’s authors, as well as a post by Ed Yong, a post by Micah Allen and an article in the Washington Post by Daniel Willingham. On the project’s own Open Science Framework page you can even do the experiment as if you were a participant yourself!
Boot W.R., Simons D.J., Stothart C. & Stutts C. (2013). The Pervasive Problem With Placebos in Psychology: Why Active Control Groups Are Not Sufficient to Rule Out Placebo Effects, Perspectives on Psychological Science, 8 (4) 445-454. DOI: 10.1177/1745691613491271 (PDF)
Image Credit: Shutterstock/Lawrence Wee
Both schizophrenics and people with a common personality type share similar brain patterns.
- A new study shows that people with a common personality type share brain activity with patients diagnosed with schizophrenia.
- The study gives insight into how the brain activity associated with mental illnesses relates to brain activity in healthy individuals.
- This finding not only improves our understanding of how the brain works but may one day be applied to treatments.
It's a development that could one day lead to much better treatments for osteoporosis, joint damage, and bone fractures.
- Scientists have isolated skeletal stem cells in adult and fetal bones for the first time.
- These cells could one day help treat damaged bone and cartilage.
- The team was able to grow skeletal stem cells from cells found within liposuctioned fat.
Gut bacteria play an important role in how you feel and think and how well your body fights off disease. New research shows that exercise can give your gut bacteria a boost.
- Two studies from the University of Illinois show that gut bacteria can be changed by exercise alone.
- Our understanding of how gut bacteria impacts our overall health is an emerging field, and this research sheds light on the many different ways exercise affects your body.
- Exercising to improve your gut bacteria will prevent diseases and encourage brain health.
SMARTER FASTER trademarks owned by The Big Think, Inc. All rights reserved.