Placebo Effects: Psychology’s Fundamental Flaw? Why active controls are not enough

A new paper published in Perspectives in Psychological Science (open access) suggests there is “a fundamental design flaw that potentially undermines any causal inference” in much psychology research. The paper is very accessible and well worth a read. The flowchart below describes the hypothesis:


Way back in the 1950’s a phenomenon called the Hawthorne effect (AKA the observer effect) was observed, an experiment to discover whether workers in the Hawthorne Works plant were more productive under increased lighting, found that productivity increased whenever the workers were being observed, regardless of the variable in question. The effect is a common cause of the placebo effect, which in most aspects of medicine is effectively handled by the gold standard of double blind randomised controlled trials (RCT’s). Unfortunately this is not always possible in psychology, not least because it is often impossible to blind an individual to the intervention, let alone the experimenter.

The problem is currently sometimes handled in psychology by using an active control, in which the control group are led to believe they are being experimented on too. The researchers in the new paper complain that this is not good enough, because all too often a difference remains in the participant’s expectations - an expectation effect. I’ve described studies where active controls might not be well chosen before, they are everywhere, in such cases expectation effects may be the cause of the results found, which would completely alter the conclusion of a study.

To demonstrate the severity of the problem, the researchers address video-gaming and brain-training research, not because the field is an example of poor research but because it is a rare example of psychology research that actually uses active placebos. The original research in question demonstrates a benefit in performance on vision and attention tasks following playing an action video game (Unreal Tournament) rather than Tetris or The Sims. The researchers in the new study asked people (admittedly only using Amazon’s Mechanical Turk) what improvements they would expect to see, following playing Unreal Tournament or following playing Tetris or The Sims. The result was that people expected the very same benefits as were found in the original research. More visual processing benefits were expected by participants from playing Unreal Tournament than from playing Tetris or The Sims, furthermore, more such benefits were expected by the participants from Tetris than The Sims whilst participants thought more benefits would be achieved in story-telling from The Sims and participants thought more benefits to visual rotation would be obtained from Tetris than from Unreal Tournament.  

Could it be that in reality there is no real effect that can be generalised outside of the laboratory? Could it be that while they were being observed the gamers in the original experiment unconsciously let their behaviour be moulded by their own prior expectations?

If this is the case, then this is only the tip of the iceberg. Vast amounts of psychology experiments do not use active controls, let alone active controls that sufficiently avoid expectation effects. This will be no easy problem to fix, the researchers have some suggestions, many of which will likely be seen as impractical by researchers, but some of which may be taken up – the paper itself is valuable reading if you are a researcher yourself. The problem requires a major sea change in the way things are done, even with the best efforts of researchers, the problem will never go away entirely. You’d therefore be well advised to always keep the possibility of expectation effects in the back of your mind when you are reading a piece of psychology research. The question you want to ask yourself is this - these results might look very impressive on paper, but what are they compared to and will they still apply to me in my living room rather than to someone who knows they are being experimented on, and might think they know why, in a far away laboratory under the watchful gaze of a scientist in a metaphorical white coat?

The paper has also been well discussed in a blog post by Daniel Simons one of the paper’s authors, as well as a post by Ed Yong, a post by Micah Allen and an article in the Washington Post by Daniel Willingham. On the project’s own Open Science Framework page you can even do the experiment as if you were a participant yourself!

Reference:

Boot W.R., Simons D.J., Stothart C. & Stutts C. (2013). The Pervasive Problem With Placebos in Psychology: Why Active Control Groups Are Not Sufficient to Rule Out Placebo Effects, Perspectives on Psychological Science, 8 (4) 445-454. DOI:  (PDF)

To keep up to date with this blog you can follow Neurobonkers on TwitterFacebookRSS or join the mailing list. 

Image Credit: Shutterstock/Lawrence Wee

​There are two kinds of failure – but only one is honorable

Malcolm Gladwell teaches "Get over yourself and get to work" for Big Think Edge.

Big Think Edge
  • Learn to recognize failure and know the big difference between panicking and choking.
  • At Big Think Edge, Malcolm Gladwell teaches how to check your inner critic and get clear on what failure is.
  • Subscribe to Big Think Edge before we launch on March 30 to get 20% off monthly and annual memberships.
Keep reading Show less

Why are so many objects in space shaped like discs?

It's one of the most consistent patterns in the unviverse. What causes it?

Videos
  • Spinning discs are everywhere – just look at our solar system, the rings of Saturn, and all the spiral galaxies in the universe.
  • Spinning discs are the result of two things: The force of gravity and a phenomenon in physics called the conservation of angular momentum.
  • Gravity brings matter together; the closer the matter gets, the more it accelerates – much like an ice skater who spins faster and faster the closer their arms get to their body. Then, this spinning cloud collapses due to up and down and diagonal collisions that cancel each other out until the only motion they have in common is the spin – and voila: A flat disc.

Freud is renowned, but his ideas are ill-substantiated

The Oedipal complex, repressed memories, penis envy? Sigmund Freud's ideas are far-reaching, but few have withstood the onslaught of empirical evidence.

Mind & Brain
  • Sigmund Freud stands alongside Charles Darwin and Albert Einstein as one of history's best-known scientists.
  • Despite his claim of creating a new science, Freud's psychoanalysis is unfalsifiable and based on scant empirical evidence.
  • Studies continue to show that Freud's ideas are unfounded, and Freud has come under scrutiny for fabricating his most famous case studies.

Few thinkers are as celebrated as Sigmund Freud, a figure as well-known as Charles Darwin and Albert Einstein. Neurologist and the founder of psychoanalysis, Freud's ideas didn't simply shift the paradigms in academia and psychotherapy. They indelibly disseminated into our cultural consciousness. Ideas like transference, repression, the unconscious iceberg, and the superego are ubiquitous in today's popular discourse.

Despite this renown, Freud's ideas have proven to be ill-substantiated. Worse, it is now believed that Freud himself may have fabricated many of his results, opportunistically disregarding evidence with the conscious aim of promoting preferred beliefs.

"[Freud] really didn't test his ideas," Harold Takooshian, professor of psychology at Fordham University, told ATI. "He was just very persuasive. He said things no one said before, and said them in such a way that people actually moved from their homes to Vienna and study with him."

Unlike Darwin and Einstein, Freud's brand of psychology presents the impression of a scientific endeavor but ultimately lack two of vital scientific components: falsification and empirical evidence.

Psychoanalysis

Freud's therapeutic approach may be unfounded, but at least it was more humane than other therapies of the day. In 1903, this patient is being treated in "auto-conduction cage" as a part of his electrotherapy. (Photo: Wikimedia Commons)

The discipline of psychotherapy is arguably Freud's greatest contribution to psychology. In the post-World War II era, psychoanalysis spread through Western academia, influencing not only psychotherapy but even fields such as literary criticism in profound ways.

The aim of psychoanalysis is to treat mental disorders housed in the patient's psyche. Proponents believe that such conflicts arise between conscious thoughts and unconscious drives and manifest as dreams, blunders, anxiety, depression, or neurosis. To help, therapists attempt to unearth unconscious desires that have been blocked by the mind's defense mechanisms. By raising repressed emotions and memories to the conscious fore, the therapist can liberate and help the patient heal.

That's the idea at least, but the psychoanalytic technique stands on shaky empirical ground. Data leans heavily on a therapist's arbitrary interpretations, offering no safe guards against presuppositions and implicit biases. And the free association method offers not buttress to the idea of unconscious motivation.

Don't get us wrong. Patients have improved and even claimed to be cured thanks to psychoanalytic therapy. However, the lack of methodological rigor means the division between effective treatment and placebo effect is ill-defined.

Repressed memories

Sigmund Freud, circa 1921. (Photo: Wikimedia Commons)

Nor has Freud's concept of repressed memories held up. Many papers and articles have been written to dispel the confusion surrounding repressed (aka dissociated) memories. Their arguments center on two facts of the mind neurologists have become better acquainted with since Freud's day.

First, our memories are malleable, not perfect recordings of events stored on a biological hard drive. People forget things. Childhood memories fade or are revised to suit a preferred narrative. We recall blurry gists rather than clean, sharp images. Physical changes to the brain can result in loss of memory. These realities of our mental slipperiness can easily be misinterpreted under Freud's model as repression of trauma.

Second, people who face trauma and abuse often remember it. The release of stress hormones imprints the experience, strengthening neural connections and rendering it difficult to forget. It's one of the reasons victims continue to suffer long after. As the American Psychological Association points out, there is "little or no empirical support" for dissociated memory theory, and potential occurrences are a rarity, not the norm.

More worryingly, there is evidence that people are vulnerable to constructing false memories (aka pseudomemories). A 1996 study found it could use suggestion to make one-fifth of participants believe in a fictitious childhood memory in which they were lost in a mall. And a 2007 study found that a therapy-based recollection of childhood abuse "was less likely to be corroborated by other evidence than when the memories came without help."

This has led many to wonder if the expectations of psychoanalytic therapy may inadvertently become a self-fulfilling prophecy with some patients.

"The use of various dubious techniques by therapists and counselors aimed at recovering allegedly repressed memories of [trauma] can often produce detailed and horrific false memories," writes Chris French, a professor of psychology at Goldsmiths, University of London. "In fact, there is a consensus among scientists studying memory that traumatic events are more likely to be remembered than forgotten, often leading to post-traumatic stress disorder."

The Oedipal complex

The Blind Oedipus Commending His Children to the Gods by Benigne Gagneraux. (Photo: Wikimedia Commons)

During the phallic stage, children develop fierce erotic feelings for their opposite-sex parent. This desire, in turn, leads them to hate their same-sex parent. Boys wish to replace their father and possess their mother; girls become jealous of their mothers and desire their fathers. Since they can do neither, they repress those feelings for fear of reprisal. If unresolved, the complex can result in neurosis later in life.

That's the Oedipal complex in a nutshell. You'd think such a counterintuitive theory would require strong evidence to back it up, but that isn't the case.

Studies claiming to prove the Oedipal complex look to positive sexual imprinting — that is, the phenomenon in which people choose partners with physical characteristics matching their same-sex parent. For example, a man's wife and mother have the same eye color, or woman's husband and father sport a similar nose.

But such studies don't often show strong correlation. One study reporting "a correction of 92.8 percent between the relative jaw width of a man's mother and that of [his] mates" had to be retracted for factual errors and incorrect analysis. Studies showing causation seem absent from the literature, and as we'll see, the veracity of Freud's own case studies supporting the complex is openly questioned today.

Better supported, yet still hypothetical, is the Westermarck effect. Also called reverse sexual imprinting, the effect predicts that people develop a sexual aversion to those they grow up in close proximity with, as a mean to avoid inbreeding. The effect isn't just shown in parents and siblings; even step-siblings will grow sexual averse to each other if they grow up from early childhood.

An analysis published in Behavioral Ecology and Sociobiology evaluated the literature on human mate choice. The analysis found little evidence for positive imprinting, citing study design flaws and an unwillingness of researchers to seek alternative explanations. In contrast, it found better support for negative sexual imprinting, though it did note the need for further research.

The Freudian slip

Mark notices Deborah enter the office whistling an upbeat tune. He turns to his coworker to say, "Deborah's pretty cheery this morning," but accidentally blunders, "Deborah's pretty cherry this morning." Simple slip up? Not according to Freud, who would label this a parapraxis. Today, it's colloquially known as a "Freudian slip."

"Almost invariably I discover a disturbing influence from something outside of the intended speech," Freud wrote in The Psychopathology of Everyday Life. "The disturbing element is a single unconscious thought, which comes to light through the special blunder."

In the Freudian view, Mark's mistaken word choice resulted from his unconscious desire for Deborah, as evident by the sexually-charged meanings of the word "cherry." But Rob Hartsuiker, a psycholinguist from Ghent University, says that such inferences miss the mark by ignoring how our brains process language.

According to Hartsuiker, our brains organize words by similarity and meaning. First, we must select the word in that network and then process the word's sounds. In this interplay, all sorts of conditions can prevent us from grasping the proper phonemes: inattention, sleepiness, recent activation, and even age. In a study co-authored by Hartsuiker, brain scans showed our minds can recognize and correct for taboo utterances internally.

"This is very typical, and it's also something Freud rather ignored," Hartsuiker told BBC. He added that evidence for true Freudian slips is scant.

Freud's case studies

Sergej Pankejeff, known as the "Wolf Man" in Freud's case study, claimed that Freud's analysis of his condition was "propaganda."

It's worth noting that there is much debate as to the extent that Freud falsified his own case studies. One famous example is the case of the "Wolf Man," real name Sergej Pankejeff. During their sessions, Pankejeff told Freud about a dream in which he was lying in bed and saw white wolves through an open window. Freud interpreted the dream as the manifestation of a repressed trauma. Specifically, he claimed that Pankejeff must have witnessed his parents in coitus.

For Freud this was case closed. He claimed Pankejeff successfully cured and his case as evidence for psychoanalysis's merit. Pankejeff disagreed. He found Freud's interpretation implausible and said that Freud's handling of his story was "propaganda." He remained in therapy on and off for over 60 years.

Many of Freud's other case studies, such "Dora" and "the Rat Man" cases, have come under similar scrutiny.

Sigmund Freud and his legacy

Freud's ideas may not live up to scientific inquiry, but their long shelf-life in film, literature, and criticism has created some fun readings of popular stories. Sometimes a face is just a face, but that face is a murderous phallic symbol. (Photo: Flickr)

Of course, there are many ideas we've left out. Homosexuality originating from arrested sexual development in anal phase? No way. Freudian psychosexual development theory? Unfalsifiable. Women's penis envy? Unfounded and insulting. Men's castration anxiety? Not in the way Freud meant it.

If Freud's legacy is so ill-informed, so unfounded, how did he and his cigars cast such a long shadow over the 20th century? Because there was nothing better to offer at the time.

When Freud came onto the scene, neurology was engaged in a giddy free-for-all. As New Yorker writer Louis Menand points out, the era's treatments included hypnosis, cocaine, hydrotherapy, female castration, and institutionalization. By contemporary standards, it was a horror show (as evident by these "treatments" featuring so prominently in our horror movies).

Psychoanalysis offered a comparably clement and humane alternative. "Freud's theories were like a flashlight in a candle factory," anthropologist Tanya Luhrmann told Menand.

But Freud and his advocates triumph his techniques as a science, and this is wrong. The empirical evidence for his ideas is limited and arbitrary, and his conclusions are unfalsifiable. The theory that explains every possible outcome explains none of them.

With that said, one might consider Freud's ideas to be a proto-science. As astrology heralded astronomy, and alchemy preceded chemistry, so to did Freud's psychoanalysis popularize psychology, paving the way for its more rapid development as a scientific discipline. But like astrology and alchemy, we should recognize Freud's ideas as the historic artifacts they are.

Photo by Alina Grubnyak on Unsplash
Mind & Brain

Do human beings have a magnetic sense? Biologists know other animals do. They think it helps creatures including bees, turtles and birds navigate through the world.

Keep reading Show less