from the world's big
The evidence for evidence-based therapy is not as clear as we thought
Scientists often find that they cannot replicate prior findings.
Over the past decade, many scholars have questioned the credibility of research across a variety of scientific fields.
Some of these concerns arise from cases of outright fraud or other misconduct. More troubling are difficulties in replicating previous research findings. Replication is cast as a cornerstone of science: we can trust the results originating in one lab only if other labs can follow similar procedures and get similar results. But in many areas of research – including psychology – scientists have found that too often they cannot replicate prior findings.
As psychologists specialising in clinical work (Alexander Williams) and methodology (John Sakaluk), we wondered what these concerns mean for psychotherapy. Over the past 50 years, therapy researchers have increasingly embraced the evidence-based practice movement. Just as medicines are pitted against placebos in research studies, psychologists have used randomised clinical trials to test whether certain therapies (eg, 'exposure therapy', or systematically confronting what one fears) benefit people with certain mental-health conditions (eg, a phobia of spiders). The treatment-for-diagnosis combinations that have amassed evidence from these trials are known as empirically supported treatments (ESTs).
We wondered, though: is the credibility of the evidence for ESTs as strong as that designation suggests? Or does the evidence-base for ESTs suffer from the same problems as published research in other areas of science? This is what we (with our coauthors, the US psychologists Robyn Kilshaw and Kathleen T Rhyner) explored in our study published recently in the Journal of Abnormal Psychology.
The Society of Clinical Psychology – or Division 12 of the American Psychological Association – has done the arduous work since the 1990s of establishing a list of more than 70 ESTs. They have continued to update the ESTs listed, and the evidence cited for them, to the present day. We conducted a 'meta-scientific review' of these ESTs. Across a variety of statistical metrics, we assessed the credibility of the evidence cited by the Society for every EST on their list. We examined measures related to statistical power, which indicates plausibility of the reported data given the sample sizes of the experiments. We computed Bayesian indices of evidence that shows how probable the results were, assuming the therapies actually helped those receiving them. We even looked at rates of misreported statistics – if a study reports, say, '2 + 2 = 5', we know that there must be a problem with at least some of the numbers. All told, we analysed more than 450 research articles. What we found is a study in contrasts.
Around 20 per cent of ESTs performed well across a majority of our metrics (eg, problem-solving therapy for depression, interpersonal psychotherapy for bulimia nervosa, the aforementioned exposure therapy for specific phobias). This means not only that the therapies have been subjected to clinical trials, but that the evidence produced from these clinical trials seems credible and supports the claim that the EST will help people. We also found a 'murky middle': 30 per cent of ESTs had mixed results across metrics, performing neither consistently well nor poorly (eg, cognitive therapy for depression, interpersonal psychotherapy for binge-eating disorder).
That leaves 50 per cent of ESTs with subpar outcomes across most of our metrics (eg, eye-movement desensitisation and reprocessing for PTSD, interpersonal psychotherapy for depression). In other words, although these ESTs seemed to work based on the claims of the clinical trials cited by the Society of Clinical Psychology, we found the evidence from these trials lacked statistical credibility. For these ESTs, the relevant research results are sufficiently ambiguous that we cannot be sure that they really do work better than other forms of therapy.
There is a large, dense body of literature showing that psychotherapy usually helps those who seek it out. Our results don't challenge that conclusion. What does it mean, though, if the evidence behind the therapies thought to be best supported by research is not as strong as one would hope?
One conclusion we draw is that we might be in need of what we're calling 'psychological reversal'. The term, a version of what the US medical scholars Vinay Prasad and Adam Cifu called medical reversal, argues for desisting from the use of psychological practices if they are found to be ineffective, inadvertently harmful or more expensive to employ than equally effective alternatives. If some ESTs lack credible evidence that they are superior to simpler, less costly and time-consuming forms of therapy, shifting resources towards the latter group of treatments will benefit therapy clients and all those bearing the costs of mental-health care.
The other conclusion is a lesson in humility for those who provide therapy (one of the authors of this article among them). For close to a century, psychologists have debated the 'dodo bird hypothesis'. Deriving its name from the proclamation of the Dodo Bird in Alice in Wonderland ('Everybody has won and all must have prizes!'), the dodo bird hypothesis suggests that different forms of psychotherapy perform equally well, and that this is because of the common factors of all therapies (eg, they all provide clients with a rationale for the therapy). The existence of ESTs seems to refute the hypothesis, demonstrating that some therapies do work better than others for certain mental-health conditions. We put forward a different possibility: the 'do not know' bird hypothesis. Given the problems with credibility we found across many clinical trials, we contend that we currently do not know in many cases if some therapies perform better than others. Of course, this also means we do not know if the majority of therapies are equally effective, and, if such equality exists, we do not know if it owes to common factors. When it comes to comparing psychotherapies, therapists could do worse than to channel every philosophy undergrad: when someone purports one therapy works better than another, wonder aloud: 'How do we know?'
Psychotherapy could be on the verge of a renaissance. Research on mental-illness treatment can benefit greatly from the lessons psychology has learned about credibility. For example, investigators can ensure that their studies have sufficient power; that is, enough participants in a clinical trial to reliably detect if a psychotherapy works. They can also practise open science by making their datasets publicly available so that other researchers can verify that a trial's statistics are reported accurately; and/or preregister their therapy trials, specifying in advance their methods and hypotheses, which makes the research process transparent and helps prevent the burying of negative findings.
Ethical therapists can continue to engage in practice that is evidence-based, not eminence-based, rooting their therapies in scientific evidence rather than their own conjecture or that of senior colleagues. They can also continue the routine outcome measurement many already employ: solicit therapy clients' feedback early and often, be open to surprise about what's working and what's not, and adjust accordingly. Clients can ask their therapists upfront if they will offer the opportunity for such mutual assessment of their progress.
Therapy helps the vast majority of those who receive it. Happily – if the discipline embraces reform in research, and cultivates a humble, flexible approach to therapy – it could help even more.
This article was originally published at Aeon and has been republished under Creative Commons. Read the original article.
- Psychotherapy is not harmless: on the side effects of CBT - Big Think ›
- Freud is renowned, but his ideas are ill-substantiated - Big Think ›
Join The Daily Show comedian Jordan Klepper and elite improviser Bob Kulhan live at 1 pm ET on Tuesday, July 14!
The team caught a glimpse of a process that takes 18,000,000,000,000,000,000,000 years.
- In Italy, a team of scientists is using a highly sophisticated detector to hunt for dark matter.
- The team observed an ultra-rare particle interaction that reveals the half-life of a xenon-124 atom to be 18 sextillion years.
- The half-life of a process is how long it takes for half of the radioactive nuclei present in a sample to decay.
Gender and sexual minority populations are experiencing rising anxiety and depression rates during the pandemic.
- Anxiety and depression rates are spiking in the LGBTQ+ community, and especially in individuals who hadn't struggled with those issues in the past.
- Overall, depression increased by an average PHQ-9 score of 1.21 and anxiety increased by an average GAD-7 score of 3.11.
- The researchers recommended that health care providers check in with LGBTQ+ patients about stress and screen for mood and anxiety disorders—even among those with no prior history of anxiety or depression.
Study findings<p>For the study, <a href="https://link.springer.com/article/10.1007/s11606-020-05970-4" target="_blank">published in the Journal of General Internal Medicine</a><em>, </em>Flentje and her team evaluated survey responses from nearly 2,300 individuals who identified as being in the lesbian, gay, bisexual, transgender, and queer (LGBTQ+) community. Most of the participants were white, while nearly 19 percent identified as a racial or ethnic minority. Multiple genders were represented with cisgender women (27.2 percent) and men (24.6 percent) making up a majority of the participants. Sixty-three percent had been assigned female at birth. For the most part, participants identified their sexual orientations as queer (40.3 percent), gay (36.5 percent), and bisexual (30.3 percent).</p><p>The JGIM study participants were recruited from the 18,000-participant <a href="https://pridestudy.org/" target="_blank">PRIDE Study</a> (Population Research in Identity and Disparities for Equality), which is the first large-scale, long-term national study focusing on American adults who identify as LGBTQ+. It conducts annual questionnaires to understand factors related to health and disease in this population. </p><p>Participants filled out an annual questionnaire (starting in June 2019) and a COVID-19 impact survey this past spring. Flentje noted that on an individual level, some people may not have experienced a big change in anxiety or depression levels, but for others there was. Overall, depression increased by a <a href="https://patient.info/doctor/patient-health-questionnaire-phq-9" target="_blank">PHQ-9 score</a> of 1.21, putting it at 8.31 on average. Anxiety went up by a <a href="https://www.mdcalc.com/gad-7-general-anxiety-disorder-7" target="_blank">GAD-7</a> score of 3.11 to an average of 8.89. Interestingly, the average PHQ-9 scores for those who screened positive for depression at the first 2019 survey decreased by 1.08. Those who screened negative for depression saw their PHQ-9 scores increase by 2.17 on average. As for anxiety, researchers detected no GAD-7 change among the study participants who screened positive for anxiety in the first survey, but did see an overall increase of 3.93 among those who had initially been evaluated as negative for the disorder. </p>
Risks among gender and sexual minorities<span style="display:block;position:relative;padding-top:56.25%;" class="rm-shortcode" data-rm-shortcode-id="fc3fd1ae68b77bbbf58a6995638d6d65"><iframe type="lazy-iframe" data-runner-src="https://www.youtube.com/embed/EnUqDjCqg0A?rel=0" width="100%" height="auto" frameborder="0" scrolling="no" style="position:absolute;top:0;left:0;width:100%;height:100%;"></iframe></span><p>The LGBTQ+ community is a vulnerable population to mental health concerns because of their fear of stigmatization and previous discriminatory experiences.</p> <p>Previous research by the Human Rights Campaign has found "that LGBTQ Americans are more likely than the <a href="https://medicalxpress.com/tags/general+population/" target="_blank">general population</a> to live in poverty and lack access to adequate medical care, paid <a href="https://medicalxpress.com/tags/medical+leave/" target="_blank">medical leave</a>, and basic necessities during the pandemic," said researcher Tari Hanneman, director of the health and aging program at the campaign.</p> <p>"Therefore, it is not surprising to see this increase in anxiety and depression among this population," Hanneman said in the release. "This study highlights the need for <a href="https://medicalxpress.com/tags/health+care+professionals/" target="_blank">health care professionals</a> to support, affirm and provide <a href="https://medicalxpress.com/tags/critical+care/" target="_blank">critical care</a> for the LGBTQ community to manage and maintain their mental health, as well as their physical health, during this pandemic."</p>
What should health care providers do?<p>The authors of the study recommend that health care providers check in with LGBTQ+ patients about stress and screen for mood and anxiety disorders in members of that community—even among those with no prior history of anxiety or depression.</p><p>As cases of COVID-19 continue to mount, the sustained social distancing, potential isolation, economic precariousness, and personal illness, grief, and loss are bound to have increased and varied impacts on mental health. Effective treatments may include individual therapy and medications as well as more large-scale coronavirus support programs like peer-led groups and mindfulness practices. </p><p>"It will be important to find out what happens over time and to identify who is most at risk, so we can be sure to roll out public health interventions to support the mental health of our communities in the best and most effective ways," said Flentje.</p>
What we know about black holes is both fascinating and scary.
- When it comes to black holes, science simultaneously knows so much and so little, which is why they are so fascinating. Focusing on what we do know, this group of astronomers, educators, and physicists share some of the most incredible facts about the powerful and mysterious objects.
- A black hole is so massive that light (and anything else it swallows) can't escape, says Bill Nye. You can't see a black hole, theoretical physicists Michio Kaku and Christophe Galfard explain, because it is too dark. What you can see, however, is the distortion of light around it caused by its extreme gravity.
- Explaining one unsettling concept from astrophysics called spaghettification, astronomer Michelle Thaller says that "If you got close to a black hole there would be tides over your body that small that would rip you apart into basically a strand of spaghetti that would fall down the black hole."