What is Big Think?  

We are Big Idea Hunters…

We live in a time of information abundance, which far too many of us see as information overload. With the sum total of human knowledge, past and present, at our fingertips, we’re faced with a crisis of attention: which ideas should we engage with, and why? Big Think is an evolving roadmap to the best thinking on the planet — the ideas that can help you think flexibly and act decisively in a multivariate world.

A word about Big Ideas and Themes — The architecture of Big Think

Big ideas are lenses for envisioning the future. Every article and video on bigthink.com and on our learning platforms is based on an emerging “big idea” that is significant, widely relevant, and actionable. We’re sifting the noise for the questions and insights that have the power to change all of our lives, for decades to come. For example, reverse-engineering is a big idea in that the concept is increasingly useful across multiple disciplines, from education to nanotechnology.

Themes are the seven broad umbrellas under which we organize the hundreds of big ideas that populate Big Think. They include New World Order, Earth and Beyond, 21st Century Living, Going Mental, Extreme Biology, Power and Influence, and Inventing the Future.

Big Think Features:

12,000+ Expert Videos

1

Browse videos featuring experts across a wide range of disciplines, from personal health to business leadership to neuroscience.

Watch videos

World Renowned Bloggers

2

Big Think’s contributors offer expert analysis of the big ideas behind the news.

Go to blogs

Big Think Edge

3

Big Think’s Edge learning platform for career mentorship and professional development provides engaging and actionable courses delivered by the people who are shaping our future.

Find out more
Close

Placebo Effects: Psychology’s Fundamental Flaw? Why active controls are not enough

July 18, 2013, 10:27 AM
Lightbulb-smashed-640

A new paper published in Perspectives in Psychological Science (open access) suggests there is “a fundamental design flaw that potentially undermines any causal inference” in much psychology research. The paper is very accessible and well worth a read. The flowchart below describes the hypothesis:

Way back in the 1950’s a phenomenon called the Hawthorne effect (AKA the observer effect) was observed, an experiment to discover whether workers in the Hawthorne Works plant were more productive under increased lighting, found that productivity increased whenever the workers were being observed, regardless of the variable in question. The effect is a common cause of the placebo effect, which in most aspects of medicine is effectively handled by the gold standard of double blind randomised controlled trials (RCT’s). Unfortunately this is not always possible in psychology, not least because it is often impossible to blind an individual to the intervention, let alone the experimenter.

The problem is currently sometimes handled in psychology by using an active control, in which the control group are led to believe they are being experimented on too. The researchers in the new paper complain that this is not good enough, because all too often a difference remains in the participant’s expectations - an expectation effect. I’ve described studies where active controls might not be well chosen before, they are everywhere, in such cases expectation effects may be the cause of the results found, which would completely alter the conclusion of a study.

To demonstrate the severity of the problem, the researchers address video-gaming and brain-training research, not because the field is an example of poor research but because it is a rare example of psychology research that actually uses active placebos. The original research in question demonstrates a benefit in performance on vision and attention tasks following playing an action video game (Unreal Tournament) rather than Tetris or The Sims. The researchers in the new study asked people (admittedly only using Amazon’s Mechanical Turk) what improvements they would expect to see, following playing Unreal Tournament or following playing Tetris or The Sims. The result was that people expected the very same benefits as were found in the original research. More visual processing benefits were expected by participants from playing Unreal Tournament than from playing Tetris or The Sims, furthermore, more such benefits were expected by the participants from Tetris than The Sims whilst participants thought more benefits would be achieved in story-telling from The Sims and participants thought more benefits to visual rotation would be obtained from Tetris than from Unreal Tournament.  

Could it be that in reality there is no real effect that can be generalised outside of the laboratory? Could it be that while they were being observed the gamers in the original experiment unconsciously let their behaviour be moulded by their own prior expectations?

If this is the case, then this is only the tip of the iceberg. Vast amounts of psychology experiments do not use active controls, let alone active controls that sufficiently avoid expectation effects. This will be no easy problem to fix, the researchers have some suggestions, many of which will likely be seen as impractical by researchers, but some of which may be taken up – the paper itself is valuable reading if you are a researcher yourself. The problem requires a major sea change in the way things are done, even with the best efforts of researchers, the problem will never go away entirely. You’d therefore be well advised to always keep the possibility of expectation effects in the back of your mind when you are reading a piece of psychology research. The question you want to ask yourself is this - these results might look very impressive on paper, but what are they compared to and will they still apply to me in my living room rather than to someone who knows they are being experimented on, and might think they know why, in a far away laboratory under the watchful gaze of a scientist in a metaphorical white coat?

 

The paper has also been well discussed in a blog post by Daniel Simons one of the paper’s authors, as well as a post by Ed Yong, a post by Micah Allen and an article in the Washington Post by Daniel Willingham. On the project’s own Open Science Framework page you can even do the experiment as if you were a participant yourself!

 

Reference:

Boot W.R., Simons D.J., Stothart C. & Stutts C. (2013). The Pervasive Problem With Placebos in Psychology: Why Active Control Groups Are Not Sufficient to Rule Out Placebo Effects, Perspectives on Psychological Science, 8 (4) 445-454. DOI:  (PDF)

To keep up to date with this blog you can follow Neurobonkers on TwitterFacebookRSS or join the mailing list. 

Image Credit: Shutterstock/Lawrence Wee

 

 

 

Placebo Effects: Psychology...

Newsletter: Share: