Has Psychotherapy Research “Progressed”?
The goal of this section is to reflect on the notion of “progress” in psychotherapy research. We begin with a quote:
“Research in psychotherapy over the past year has been concerned primarily with providing answers to practical problems such as the prediction of outcome, the selection of patients for a particular kind of therapy, and the comparative evaluation of different psychotherapeutic techniques.”
I conjecture that most experts would agree with this characterization of the field as it stands today: the above sentence could be included in any recently published synthesis of psychotherapy research, where it would hardly cause much controversy. It just happens that these words were written seventy years ago, in a critique provided by Worchel (1955). It is one of the first reviews in clinical and health psychology indexed in APA PsycInfo®.
The quotation above suggests, albeit anecdotally, that the main problems in psychotherapy research seem to remain unresolved, and that this has been the case for many decades. This is a troubling observation. Progress is often seen as a hallmark of any “mature” science: as empirical evidence accumulates, we gain a deeper understanding of a subject matter, which in turn provides the basis for further inquiries. It is questionable if psychotherapy research passes this test.
We have already discussed evidence that many, if not most patients do not respond sufficiently well to psychological treatment (see Figure 1). Meta-analytic studies also show that treatment effects have not appreciably increased over the last 50 years (Cuijpers, Harrer, Miguel, et al., 2023; Johnsen & Friborg, 2015; Ljótsson et al., 2017); nor has our understanding of why they work, and for whom (Cuijpers, Reijnders, et al., 2019). If the goal of psychotherapy research is to discover increasingly efficacious treatments for mental disorders, and reasons why they work, progress is difficult to discern.
Dissatisfaction with the state of psychological science is not new. In his landmark 1978 paper, Paul E. Meehl likened advances in clinical psychology and related disciplines to fashion trends: periods of enthusiasm are soon followed by “disillusionment as the negative data come in, a growing bafflement about inconsistent and unreplicable empirical results” after which “people just sort of lose interest […] and pursue other endeavors” (Meehl, 1978). This non-cumulative character of “soft” psychology (i.e., clinical, counseling, social) remains a commonly diagnosed problem (Eronen & Bringmann, 2021; Lilienfeld, 2010; Proulx & Morey, 2021; Zagaria et al., 2020).
In psychotherapy research, “innovative” treatments and techniques are proposed with clockwork regularity, each with their underlying theory and rationale; but these approaches seldom prove to be superior to existing formats, after which research interest cools down considerably.
Meehl’s diagnosis continues to exert a strong influence on the field, even more so against the backdrop of a widespread “replication crisis” starting from 2010 onwards (Maiers, 2022; Maxwell et al., 2015; Oberauer & Lewandowsky, 2019; Simmons et al., 2011; Tackett et al., 2019). Frequently named remedies for this issue equally draw on Meehl: abandoning null hypothesis significance testing in favor of “severe” tests (Mayo, 2021; Claesen et al., 2022); and the use of “rich mathematics” (Meehl, 1978, p. 825) to establish better theories of mental disorders and their treatment (Robinaugh et al., 2021).
A common theme appears to be, like in previous decades, that psychology lacks “depth” (in terms of theory) and “precision” (in terms of predicting behaviors, symptoms, or therapy outcomes). The last years have thus seen a long list of articles calling for more and better theory-building (Eronen & Bringmann, 2021; Muthukrishna & Henrich, 2019; Oberauer & Lewandowsky, 2019; Berkman & Wilson, 2021). These pleas are often connected to hopes that computational advances will allow to produce “formal” theories of mental illness and their treatment, providing greater explicative and predictive power (Borsboom et al., 2021; Haslbeck et al., 2022).
As of now, the real-world impact of such endeavors remains open. Borsboom and colleagues (2021) concede that computational models are much akin to structural equation modeling (SEM) approaches, which have been in use for decades. Some philosophers also argue that such models, even if they purport to be mathematically “precise”, remain reductionist by principle (Cartwright, 1999, chap. 5).
Instead, multiple theories, methods and models may need to be combined flexibly to understand context-dependent entities like mental disorders (“nomological pluralism”; Cartwright, 1994). Maatman (2021) argues that formal modelling may even worsen psychology’s theory crisis, since it decontextualizes models from the “fuzzy” and verbally mediated world from which they originate. At a later stage, we will revisit this context-sensitivity as a defining feature of mental disorders.
Discussions like the one above focus on the development of “theory” as a sign of scientific progress. Whether this is the only way to measure advances in “applied” sciences is debatable, and it could make us overlook aspects in which psychotherapy research has demonstrably progressed.
Firstly, most treatments are now subjected to much more rigorous quantitative investigation than ever before. RCTs, despite their avoidable and unavoidable limitations (Harrer, Cuijpers, et al., 2023), are the main reason we know about the limited effects of psychological treatment in the first place, and have helped to weed out treatments that are ineffective or even dangerous (Lilienfeld, 2007). These trials have fed into meta-analytic research, which has gained tremendously in importance as the basis for treatment guidelines worldwide.
“Working factors” and other theoretical underpinnings of psychotherapies may remain largely undetermined, but the general efficacy of various treatments is now firmly established; this evidence has undeniably led to increases in the number of patients routinely receiving psychological treatment.
In the last chapter, we also discussed that the role of technology in psychological treatments remains underemphasized. Since the late 1990s, for example, psychotherapy researchers have been leading in the development and evaluation of Internet-based and digital interventions. It is difficult to see these formats as mere “fashion trends” in Meehl’s sense, and their use in routine settings is likely to increase in the future.
Overall, it would be short-sighted to view psychotherapy research as a mere succession of fads; even though its successes remain incremental and incomplete.
It is clear that questions surrounding the progress of psychotherapy research will be very difficult to resolve. Such debates also offer frustratingly little to scientists and practitioners on the ground. We may consider a third option: that our idea of “progress” itself is misguided; at least when we adopt it naively from the “hard” sciences (viz., physics, chemistry) into our discipline.
Following 20th century philosophy, this would lead to a “quietist” view on the theoretical development of mental health research (McDowell, 2009; Rorty, 2007; Wright, 1994, p. 202). To elaborate on this point, it may be helpful to consider what philosophers of science trained in this tradition have to say about mental disorders, and their treatment.
In “Mad Travelers”, Hacking (1998) provides an account of what he terms transient mental illness, “an illness that appears at a time, in a place, and later fades away” (p.1). His analysis focuses on a strange epidemic of psychogenic fugues reported at the turn of 19th century. These fugues occurred in specific people (i.e., skilled laborers or merchantmen, but not in farmers or “country folk”) across certain regions (France, Germany and Italy; but not in Great Britain or the United States), and would involve patients travelling hundreds of kilometers in a trance state, often by train, without any recollection of doing so.
Hacking uses these “mad travelers” merely as an illustration of the transient, contextual character of mental disorders: earlier examples may be found in “acedia”, a type of spiritual depression that regularly befell monks (in contrast to the layman’s “melancholia”; Daly, 2007); or in reports of psychogenic “dancing plagues” that traveled along established trading routes (Waller, 2008, 2009).
In his analysis, Hacking asserts that mental illness may be best understood using the concept of “ecological niches”. These niches arise from the fact that mental disorders are categorized, understood and treated based on their symptoms or “signs”, which are themselves a complex product of biological, psychological, sociocultural elements that interact over time (“symptoms aplenty, yes, but different congeries of symptoms in different decades […], with no determinate medical entity from which they emanate”, Hacking, 1998, p. 9).
This is not a socially constructivist stance: Hacking’s niches are “not just social, not just medical, not just coming from the patient, not just from the doctors, but from the concatenation of an extraordinarily large number of diverse type of elements which for a moment provide a stable home for certain types of manifestation of illness.” (ibid, p. 13).
Hacking sees this as a defining feature of mental disorders, from which the perceived “ail-ments” mental health research emanate. Drawing on Wittgenstein’s “Philosophical Investiga-tions” (1953, §371; cf. Wakefield, 2014), he writes (p. 10):
“Wittgenstein said that in psychology there are experimental methods and conceptual confusion. We have more than that for the mental illnesses. We have the clinical methods of medicine, psychiatry, psychology; we have the innumerable variants of and deviations from psychoanalysis; have systems of self-help, group help, and counselors including priests and gurus; we have the statistical methods of epidemiology and population genetics; we have the experimental methods of biochemistry, neurol-ogy, pathology, and molecular biology; we have the theoretical modeling of cognitive science; and we have conceptual confusion.”
He shares his skepticism that scientific progress, as classically conceived, is attainable in these disciplines, or even desirable:
“Perhaps all our problems will be erased when we have enough objective scientific knowledge. I have another view. We do have a limitless reservoir of ignorance, but we also have conceptual con-fusions that new knowledge seldom helps relieve. There are a number of reasons for this, but I am especially impressed by the way that scientific knowledge about ourselves – the mere belief system – changes how we think of ourselves, the possibilities that are open to us, the kinds of people we take ourselves and our fellows to be. Knowledge interacts with us and with a larger body of practice and ordinary life. This generates socially permissible combinations of symptoms and disease entities.”
Hacking’s thoughts share a family resemblance, as he mentions, with Kuhn’s idea of research paradigms (Kindi, 2012). Existing taxonomies of mental disorders provide, for some time, a stable conceptual framework to accommodate (biological, behavioral, mental) manifestations of “disease”. These paradigms are productive, in the sense of suggesting direct pathways for treatment, once patients are correctly “sorted”.
This idea reappears in older classification systems as it does in newer nosological projects, such as the Hierarchical Taxonomy of Psychopathology (HiTOP; Kotov et al., 2017). A historic example may be French psychiatrists' efforts to pigeon-hole 19th century fugeurs as “epileptic” or “hysteric”, following the taxonomy of their time, in order to find a treatment. Jean-Martin Charcot (1825-1893) writes on his patient Mén: “if the man’s flights are equivalent to epileptic seizures, then I can treat him” (Hacking, 1998, p. 36; emphasis mine).
Similar to Kuhn’s scientific revolutions, taxonomies have to be broken up if some phenomena (clinical, social, or scientific) cannot be fully integrated anymore. This leads to new disorder groups, treatment approaches, and research paradigms. Such changes, in turn, influence what is regarded as “real” about mental health problems: a dysfunction in brain chemistry, for example (Jefferson, 2022; Moncrieff et al., 2023); or automatized thoughts (Beck, 2008); or “psychological inflexibility” (Levin et al., 2014), disturbances in the “Bayesian brain” (Feldmann et al., 2023), or “critical slowing down” of a symptom network (Van De Leemput et al., 2014).
These paradigms shape what disease markers “appear” in a population and how treatments, psychological or otherwise, are motivated. They are contingent, superseding older taxonomies as time progresses, as well as coexisting with others (Hacking, 1998, p. 38).
What do we take from this analysis of the field, as provided by an eminent contemporary philosopher of science? We need not follow Hacking’s view in its totality. I believe it does show, however, our underappreciation of the enormous context-sensitivity displayed by mental disorders, and their treatments. Many paradigms acknowledge this, although mostly as a “nuisance parameter” that their more precise (formal, mathematical) methods will “explain away”.
The idea I want to follow in this thesis is that, in mental health research, context sensitivity is central to the matter: it is an irreducible characteristic of the field, deserving study in and of itself.
The next chapter pursues the notion that, in everyday research, this context sensitivity reappears as heterogeneity. I will briefly review heterogeneity as a pervasive feature in the presentation and course of mental disorders, and in how patients respond to psychological treatment. I will also attempt to provide a more formal and statistical definition of heterogeneity.
This includes the concept of exchangeability, a foundational idea in statistics that also bears relevance to meta-analysis. Lastly, I will examine arguments that heterogeneity, if taken seriously, has catastrophic consequences for quantitative social science, including psychotherapy research.