1 Introduction
Declare the past, diagnose the present, foretell the future.
– Hippocrates of Kos
La probabilità non esiste se non per me in funzione del grado d’ignoranza in cui momentaneamente mi trovo.
– de Finetti, Probabilismo
1.1 Mental Disorders: Prevalence & Unmet Need for Treatment
Mental disorders are one of the leading causes of disability worldwide, affecting 10–19% of all individuals in any given year (Kessler et al., 2009). Overall, one in three people are estimated to develop a mental disorder once in their life (Steel et al., 2014; Chesney et al., 2014; Vigo et al., 2016; Kessler et al., 2009). Suffering from mental illness is associated with numerous negative outcomes for the individual and society, including labor market marginalization (Niederkrotenthaler et al., 2014, 2016), worse educational attainment (Eisenberg et al., 2009; Hysenbegasi et al., 2005; Kessler et al., 1995), limited role functioning (Fergusson & Woodward, 2002), negative somatic health outcomes (Hare et al., 2014; Wang et al., 2020), as well as early mortality (Cuijpers & Smit, 2002) and suicide (Nock et al., 2009). As a consequence, mental illness causes an enormous financial burden, with global costs adding up to 4.7 trillion dollars per year (Arias et al., 2022). It is estimated that by 2030, depression alone will become the leading cause of disease burden worldwide (WHO, 2011).
There is a range of evidence-based treatments for common mental disorders, including both psychological treatments (e.g., cognitive-behavioral therapies; Carpenter et al., 2018; Cuijpers et al., 2023; Karyotaki et al., 2021; Mendes et al., 2008; Olatunji et al., 2013; Papola et al., 2023) and pharmacotherapy (Cipriani et al., 2018; Leucht et al., 2013). For depressive disorders, many treatment guidelines recommend combination therapy as a first-line treatment, whereby evidence-based psychological treatment is provided alongside pharmacotherapy (e.g., Bundesärztekammer et al., 2022). In the long run, psychological treatment may be more effective than antidepressants when the two are compared directly (Furukawa, Shinohara, et al., 2021), and newer guidelines reflect this finding (Brohan et al., 2024). For anxiety disorders, psychological treatments (including exposure-based approaches) are often recommended as first-line treatment (Bandelow et al., 2022; Katzman et al., 2014). For psychotic disorders, pharmacological treatment typically takes precedence, with some psychosocial interventions (e.g., psychoeducation, cognitive-behavioral therapy for psychosis) recommended as part of a holistic intervention concept (Keepers et al., 2020; Hasan et al., 2020).
Despite the availability of effective treatments, the population-level impact of current mental health care is still suboptimal. Evidence-based treatments are established in many health care systems, and the number of patients who receive them has increased in recent decades (De Graaf et al., 2012; Filatova et al., 2019; Jorm et al., 2017; Kessler et al., 2005; Kowitz et al., 2014; Steffen et al., 2020; Walters et al., 2012). Nevertheless, there are no signs that the prevalence of mental disorders has decreased, not even in countries with well-developed mental health infrastructures (Baxter et al., 2014; Bretschneider et al., 2018; Ferrari et al., 2013; GBD 2019 Mental Disorders Collaborators, 2022; Richter et al., 2019; Steel et al., 2014). Different reasons have been named for this “treatment-prevalence paradox” (Ormel et al., 2019, 2022). Some interpreters have noted that mental illness is becoming less stigmatized, increasing the willingness to report mental health complaints among patients, and improving clinicians’ sensitivity in routine care (Busfield, 2012; Baxter et al., 2014). More critically, others argue that the expansion of “therapy culture” (Baxter et al., 2014) and mental health awareness may lead some individuals to overinterpret milder forms of distress as signs of mental illness, which ultimately amplifies their own symptomatology. The “prevalence inflation hypothesis” conjectures that destigmatization efforts could have a paradoxical effect on the reported prevalence of mental health problems; but empirical confirmation of this theory is still lacking (Foulkes & Andrews, 2023).
It is also important to note that, despite gradual improvements, the treatment gap among individuals who suffer from mental disorders is still enormous: even in high-income countries, less than half are currently receiving minimally adequate care (Alonso et al., 2018; Evans-Lacko et al., 2018; Thornicroft et al., 2017); and treatment rates are even lower in low and middle-income countries (Ndetei et al., 2023). Infrastructural barriers are often named as a reason for this unmet need for treatment. In many low and middle-income countries, for example, there is less than one mental health professional per 100’000 individuals (Rathod et al., 2017). However, this cannot account for the low uptake rates observed even when mental health interventions are freely available to entire populations, as is the case in high-income countries (Cuijpers et al., 2010). Attitudinal barriers are therefore increasingly studied as an important contributor to the global treatment gap, including patients’ preference to self-manage their own symptoms (Andrade et al., 2014; Ebert et al., 2019).
Another problem is that even the best-studied treatments only show modest benefits (Cuijpers, Miguel, Ciharova, et al., 2023; Kamp et al., 2024; Khan et al., 2021). It has been estimated that, even under 100% coverage with evidence-based treatment, only 28% of the disease burden of mental illness could actually be avoided (Andrews et al., 2004). Over the last two decades, the prevention of mental disorders has therefore become a relevant research field, especially for depressive disorders (Cuijpers et al., 2012). Meta-analytic evidence shows that psychological interventions can reduce the incidence of major depression within a clearly circumscribed time frame, but their long-term benefits are less certain (Cuijpers et al., 2021; Rigabert et al., 2020).
Thus, the current state of global mental health care gives a mixed picture. The relevance of expanding evidence-based mental health care on a global scale is now almost universally recognized, and clinical research is proliferating at unprecedented speeds (Chen et al., 2023; Liu et al., 2024; López-Muñoz et al., 2003; Wang et al., 2024; Zale et al., 2021); on the other hand, there seems to be no discernable impact of such efforts on the proportion of people living with a mental disorder, the global disease burden, or the effectiveness of treatments received in routine care (Ormel et al., 2020; Ormel & Emmelkamp, 2023).
A recent meta-analysis on the response rates of psychological treatment across eight mental disorders illustrates this issue: for each indication, pooled response rates among the treated were well below 50%; yet, for most disorders, patients are still at least twice as likely to respond under treatment than control (Cuijpers, Miguel, Ciharova, Harrer, et al., 2023; see Figure 1.1). This transports a nuanced message: on the one hand, treatments do not lead to meaningful change among the majority of patients. On the other hand, receiving a psychological intervention still strongly increases a patients’ chances of improving, indicating that their widespread adoption is all but imperative.
Questions regarding the benefits of psychological treatment are not a new phenomenon. Debates concerning if, how, when and why psychotherapies are effective accompany the field since its inception. The next chapter provides a brief history of psychological treatment research; it traces back main developments within the last 50 years, and discusses unresolved controversies. This historical review builds the basis, in the following chapter, to assess the extent to which psychotherapy research has “progressed” over the last decades.
Note. Forests plots are based on data reported in Cuijpers, Miguel, Ciharova, Harrer, Basic, …, Karyotaki (2024). Absolute and relative out-comes of psychotherapies for eight mental disorders: a systematic review and meta-analysis. World Psychiatry, 23(2), 267-275.
1.2 A Short History of Psychotherapy Research
Today, many people may find the relevance of psychological treatment in mental health care to be all but self-explanatory. Psychological therapies are now delivered all around the world, and recent WHO guidelines strongly recommend their implementation for six mental disorder groups, especially in low- and middle-income countries (LMICs; WHO, 2023). In many Western countries, psychological therapies are an established part of routine care for many decades. This “success story” hides the fact that psychotherapy, as conceived today, is a fairly new invention. Its empirical investigation is even younger, and did not emerge before the middle of the 20th century.
In his seminal work on the history of psychiatry, Ellenberger (1994, chap. 2) traces back the origins of psychological treatment to 18th century “animal magnetism”. This movement was associated with figures such as Franz Mesmer (1734-1815) and the Marquis de Puységur (1751-1825). Deeply non-scientific by modern standards, animal magnetism focused on the role of a “magnetizer” or “mesmerizer”, who would use eye contact, hand gestures, or magnets to reestablish the proper flow of “magnetic life fluids” (Lebensmagnetismus) to resolve patients’ psychological distress (Lopez, 1993; Schott, 2005). In some individuals, this treatment would cause a state of altered consciousness or trance. Leaving behind Mesmer’s magnetic theory, it was increasingly recognized that this trance could be induced by suggestion alone, and could be harnessed for therapeutic intervention (Hull, 1929; Perry, 1978). This led to the development of 19th century Hypnotism, which conceptualized trance as a type of artificially induced somnambulism (Ellenberger, 1994, p. 71, 90, 101).
The developments starting with Mesmer have been described as a “psychological turn” (Crabtree, 2019). Mesmerism and Hypnotism contained ideas that remain influential in psychodynamic theory today: first, the powerful and suggestive relationship (“rapport”) between the “therapist” (Mesmerizer, Hypnotizer) and their patient (Ellenberger, 1994, p. 76); as well as dipsychism, the notion that the human mind contains a second, subconscious layer malleable by intervention (Abramowitz & Torem, 2018, Ellenberger, 1994, p. 145). It is this intellectual context that Pierre Janet (1859-1947), Josef Breuer (1842-1925), Sigmund Freud (1856-1939) and others built and expanded on, ultimately leading to the development of psychoanalysis, and of psychodynamic psychotherapy. With many extensions and revisions in the 20th century, this type of treatment remains relevant in patient care until today.
Another strand of psychotherapy research can be traced back to the origins of behavioral psychology. Ivan Pavlov’s (1849-1936) groundbreaking research on classical conditioning (Windholz, 1997), or Watson’s “Little Albert” study (see Fridlund et al., 2012; Harris, 1979, for a critical summary) described methods to induce “experimental neurosis” without resorting to any mentalistic theory. Wampold and Imel (2015, p. 20) see a first attempt to leverage conditioning for therapeutic means in the works of Joseph Wolpe (1915-1997). Like other influential figures in behavioral therapy, Wolpe was a psychiatrist who become dissatisfied with psychoanalysis, and turned to learning theory instead. In “Psychotherapy by Reciprocal Inhibition” (1968), Wolpe described his method of systematic desensitization, in which patients are subjected to fear-provoking stimuli, which are then conditioned by progressive relaxation. In behavioral therapies, such exposure techniques have remained a quintessential tool in the treatment of anxiety disorders (Craske et al., 2014). Behavioral therapies were further enriched by the “cognitive turn” in the mid-20th century. Aaron Beck (1921–2021) and Albert Ellis (1913–2007) highlighted the importance of challenging automatic thoughts and irrational beliefs in psychotherapy, particularly for treating depression (Dowd, 2004; Rosner, 2014). In recent decades, cognitive-behavioral therapies were further enhanced by “third-wave” approaches, which putting greater emphasis on cognitive processes instead of their particular content, and including mindfulness and acceptance-based techniques (Hayes & Hofmann, 2021).
There are also other traditions that influenced modern psychotherapy, even though their contribution may be less commonly known. In the United States, for example, “talk therapies” with a spiritual background were immensely popular at the turn of the 19th century (Wampold & Imel, 2015, p. 17). Introduced in 1906, the Emmanuel movement in Boston was a first attempt to combine medical practice with individual and group therapies aimed at improving patients’ mental health, including those with less than severe mental disorders (e.g., depression, anxiety, or alcohol dependence; Caplan, 1998). There is also a rich history of humanistic treatments, emphasizing the unique experience and self-determination of each patient, as well as the importance of universal positive regard. This includes logotherapy (Viktor Frankl), Carl Rogers’ person-centered therapy, or Gestalt therapy. Some techniques of these schools continue to be practiced today, even if under the umbrella of eclectic and modern cognitive-behavioral treatments (Fernández-Álvarez & Fernández-Álvarez, 2019).
The history of psychotherapy is often presented in the way I did in the preceding sections; i.e., as a line of competing theories and resulting techniques that a trained therapist should or should not implement in a specific, face-to-face setting. But there is also a “second” history, arguably less commonly told, revolving around psychotherapy’s connection with technology. Starting from the 1940s, for example, Rogers’ group already used audiotapes to generate and test hypotheses, and to train new counsellors (Rogers, 1942). The value of administering psychotherapy via telephone has already been underlined in the 1960s (Rosenblum, 1969), and there is now robust evidence supporting its efficacy (Castro et al., 2020). There is a long, albeit underrepresented tradition in which technological advances were used to provide psychotherapeutic contents outside the ”classic” face-to-face setting, including bibliotherapy and other forms of self-guided treatment (Cuijpers, 1997; Jack & Ronan, 2008).
Lewinsohn’s “Coping with Depression” course (Lewinsohn & Clarke, 1984), for instance, has been publicly disseminated through technological means early on, including via personal computers, CD-ROMs, or television (Cuijpers et al., 2009). The efficacy of this approach in treating and preventing depression has been demonstrated in many studies (ibid.). These technological advances have culminated in the proliferation of Internet-based interventions, arguably one of the more consequential developments in psychotherapy research within the last two decades. The effects of Internet and smartphone-based interventions has since been demonstrated in hundreds of clinical trials (Moshe et al., 2021), and digital therapies are now a part of routine care in many countries. In the future, recent advances in cloud computing and sensor technology may improve the scope and flexibility of these interventions even further. I will return to this topic later (see “Methodological Innovations” section).
In the beginning of the 20th century, the dominant format to examine psychotherapy effects was the case study (Strupp & Howard, 1992). Such examinations had a particular importance among psychoanalytic researchers, and include famous examples such as Breuer’s “Anna O.” (Reeves, 1982) or Freud’s “Little Hans” (Wolpe & Rachman, 1960). Clinical case descriptions continue to be an important method of inquiry in this field, but their merits are not uncontested, even among psychodynamic researchers (Fonagy, 2013). In modern psychotherapy research, case reports are recognized as providing weak evidence at best (Guyatt et al., 1995). Historically, however, case reports can be viewed as attempts to align psychotherapy with the standards and practices of medicine (Wampold & Imel, 2015, p. 23). This tendency is also recognizable in early psychoanalytic theory, which provided a description of mental illness consistent with the medical model of disease: patients’ symptoms are explicable by a common underlying cause (i.e., repressed trauma), which is targeted by specific therapeutic actions (e.g., free association; ibid., p. 18). Alignment with medicine thus secured an “ecological niche” for psychological therapy, turning it into a scientific, “civilized”, and generally trustworthy endeavor.
Psychotherapy research’s complex relationship with medicine is also reflected by its methodology. In the middle of the 20th century, researchers were increasingly involved in finding more objective ways to examine the effects of psychological treatment. A major development in this respect was the introduction of matched placebo controls, as suggested by Rosenthal & Frank (1956). The idea of comparing therapies to a “therapeutically inert” condition was adopted from medicine: implicitly, it conceptualizes psychotherapy as a medical treatment with a specific agent, the effect of which must be disentangled from unspecific or contextual factors. “Psychological” and pill placebos are common comparators in psychotherapy studies until today, but their heuristic value remains contested (Kirsch et al., 2016).
Another major innovation was the adoption of randomized controlled trials (RCTs) in the field. This design had been introduced into medicine in the 1940s, based on a landmark double-blind study on streptomycin in pulmonary tuberculosis conceived by Austin Bradford Hill (1897-1991) (Crofton, 2006), who himself drew on previous works by R. A. Fisher (Armitage, 2003). In his “Design of Experiments” (1935), Fisher had laid the foundations for modern experimental design, and explicated the statistical tools to draw inferences from them (Lehman, 2011). In the second half of the 20th century, RCTs also became popular in psychotherapy research (Basterfield & Lilienfeld, 2020).
RCTs are now the dominant design in psychotherapy research. The concept of “evidence-based medicine” (EBM) has contributed strongly to this development from the 1990s onwards, emphasizing the role of randomized trials to establish “evidence-supported treatments” (ESTs) (Chambless & Hollon, 1998). Hundreds of RCTs have shown the efficacy of psychotherapies for various mental disorders since that time. This evidence was crucial in dispelling beliefs that psychological treatments were largely ineffective (Cuijpers, Karyotaki, et al., 2019), and in securing their place in various treatment guidelines. Cognitive-behavioral treatments were among the first to be subjected to rigorous quantitative examination; for disorders like depression, their evidence-base now surpasses most other formats by orders of magnitude (Cuijpers, Miguel, Harrer, et al., 2023). Some see the development of ESTs as a more recent attempt to legitimize “psychological” therapies by treating them as a medical treatment, and by applying equivalent methods to study their effects (Wampold & Imel, 2015, p. 27). This innovation cycle reverted with the introduction of meta-analyses in medicine. We will return to this point later (see Article 1).
Randomized controlled evidence has been crucial to show that psychological treatment can be effective. However, other longstanding debates remain unsettled, the most notorious of which surrounds the “Dodo-Bird” verdict (“all have won, and all must have prizes”). Initially coined by Rosenzweig (1936) almost 90 years ago, this dictum refers to the fact that all bona fide therapies result in equivalent benefits, regardless of their therapeutic rationale. This observation has often (but not always) been corroborated by modern effectiveness research, spawning recurrent discussions among scholars. The Dodo-Bird retains its relevance due to the historic emphasis on specific, theory-driven “techniques” in most psychotherapeutic schools. Equivalent therapy outcomes could mean that therapeutic techniques do not target the underlying mechanism purported by their inventors: the developed rationale is in fact a fiction, and effects across all therapies emerge due to purely contextual factors (e.g., positive regard, expectations, remoralization). Following the medical model, these factors would be regarded as placebo effects.
While rejecting the term “placebo”, some theorists conceptualize psychotherapy along these lines. An early example is Frank & Frank’s (1993) “Persuasion & Healing”, originally published in 1961. This work develops the argument that psychotherapies work by providing patients with a coherent explanation for their symptoms (which need not be true), based on which salubrious activities are motivated. More recently, Wampold (2001) drew on this tradition, developing a “contextual model” of psychotherapy. This theory rejects “medical model”-type explanations of psychotherapy effects, emphasizing the social context in which psychotherapy occurs. All effective therapies are thought to operate through a common set of pathways: relationship and social belonging; creation of treatment expectations; and goal-oriented actions (Wampold & Imel, 2015, p. 53). This model is intuitively appealing, but cannot accommodate all empirical findings (Cuijpers, Reijnders, et al., 2019). More recently, component network meta-analyses have unearthed several specific treatment components that may drive treatment effects, e.g. for affective disorders (Furukawa, Suganuma, et al., 2021; Furukawa et al., 2024; Miklowitz et al., 2021; Pompoli et al., 2018). Lastly, Wampold’s and others rejection of “the medical model” may also build up a straw man. In modern psychiatry, the biopsychosocial model of mental illness is almost universally accepted, as is the need for multimodal treatments. The latter are based on the understanding that there are multiple ways to conceptualize mental health problems (Huda, 2022).
Cuijpers, Reijnders & Huibers (2019) argue that current evidence is simply insufficient to conclude that psychotherapies work through specific techniques, common factors, or a combination of both. Cuijpers (2023) also argues that the “Dodo-Bird” debate itself may be misguided: given the dramatic treatment gap for mental disorders worldwide, it may be more relevant to determine what is “at least” needed to make psychological treatment effective, to ensure that interventions can be more efficiently disseminated.
This concludes our brief historical review. Tracing back the history of the field, we see that applications of “psychological” treatment were already firmly established by the turn of the 20th century, with precursors that reach back even further. What has changed, at times drastically, is how these cures where motivated, how individuals explained that their treatments work, and the methods to verify their effects. Many of these redefinitions were shaped by broader scientific and societal trends at the time, and by developments in other disciplines (e.g., neurology, physiology, cognitive science). A guiding theme is psychotherapy’s connection with medicine, which continues to have a profound impact on the research methodology practiced in the field. Across these twists and turns, it remains open if psychotherapy research has indeed “progressed” as a science. This question will be examined in the following chapter.
1.3 Has Psychotherapy Research “Progressed”?
The goal of this section is to reflect on the notion of “progress” in psychotherapy research. We begin with a quote:
“Research in psychotherapy over the past year has been concerned primarily with providing answers to practical problems such as the prediction of outcome, the selection of patients for a particular kind of therapy, and the comparative evaluation of different psychotherapeutic techniques.”
I conjecture that most experts would agree with this characterization of the field as it stands to today: the above sentence could be included in any recently published synthesis of psychotherapy research, where it would hardly cause much controversy. It just happens that these words were written seventy years ago, in a critique provided by Worchel (1955). It is one of the first reviews in clinical and health psychology indexed in APA PsycInfo®.
The quotation above suggests, albeit anecdotally, that the main problems in psychotherapy research seem to remain unresolved, and that this has been the case for many decades. This is a troubling observation. Progress is often seen as a hallmark of any “mature” science: as empirical evidence accumulates, we gain a deeper understanding of a subject matter, which in turn provides the basis for further inquiries. It is questionable if psychotherapy research passes this test. We have already discussed evidence that many, if not most patients do not respond sufficiently well to psychological treatment (see Figure 1.1). Meta-analytic studies also show that treatment effects have not appreciably increased over the last 50 years (Cuijpers, Harrer, Miguel, et al., 2023; Johnsen & Friborg, 2015; Ljótsson et al., 2017); nor has our understanding of why they work, and for whom (Cuijpers, Reijnders, et al., 2019). If the goal of psychotherapy research is to discover increasingly efficacious treatments for mental disorders, and reasons why they work, progress is difficult to discern.
Dissatisfaction with the state of psychological science is not new. In his landmark 1978 paper, Paul E. Meehl likened advances in clinical psychology and related disciplines to fashion trends: periods of enthusiasm are soon followed by “disillusionment as the negative data come in, a growing bafflement about inconsistent and unreplicable empirical results” after which “people just sort of lose interest […] and pursue other endeavors” (Meehl, 1978). This non-cumulative character of “soft” psychology remains a commonly diagnosed problem (Eronen & Bringmann, 2021; Lilienfeld, 2010; Proulx & Morey, 2021; Zagaria et al., 2020). In psychotherapy research, “innovative” treatments and techniques are proposed with clockwork regularity, each with their underlying theory and rationale; but these approaches seldom prove to be superior to existing formats, after which research interest cools down considerably.
Meehl’s diagnosis continues to exert a strong influence on the field, even more so against the backdrop of a widespread “replication crisis” starting from 2010 onwards (Maiers, 2022; Maxwell et al., 2015; Oberauer & Lewandowsky, 2019; Simmons et al., 2011; Tackett et al., 2019). Frequently named remedies for this issue equally draw on Meehl: abandoning null hypothesis significance testing in favor of “severe” tests (Mayo, 2021; Claesen et al., 2022); and the use of “rich mathematics” (Meehl, 1978, p. 825) to establish better theories of mental disorders and their treatment (Robinaugh et al., 2021). A common theme appears to be, like in previous decades, that psychology lacks “depth” (theory) and “precision” (prediction). The last years have thus seen a long list of articles calling for more and better theory-building (Eronen & Bringmann, 2021; Muthukrishna & Henrich, 2019; Oberauer & Lewandowsky, 2019; Berkman & Wilson, 2021). These pleas are often connected to hopes that computational advances will produce formal theories of mental illness and treatment (Borsboom et al., 2021; Haslbeck et al., 2022).
As of now, the real-world impact of such endeavors remains open. Borsboom and colleagues (2021) concede that computational models are much akin to structural equation modeling (SEM) approaches. Some philosophers also argue that such models, even if mathematically “precise”, remain reductionist (Cartwright, 1999, chap. 5). Instead, multiple theories and models may need to be combined flexibly to understand context-dependent entities like mental disorders (“nomological pluralism”; Cartwright, 1994). Maatman (2021) argues that formal modelling may even worsen psychology’s theory crisis, since it decontextualizes models from the “fuzzy” and verbally mediated world from which they originate.
Discussions like the one above focus on the development of “theory” as a sign of scientific progress. If this is the only way to measure advances in “applied” sciences is debatable, and it could make us overlook aspects in which psychotherapy research has demonstrably progressed. Firstly, most treatments are now subjected to much more rigorous quantitative investigation than ever before. RCTs, despite their limitations (Harrer, Cuijpers, et al., 2023), are the main reason we know about the limited effects of psychological treatment, and have helped to weed out treatments that are ineffective or even dangerous (Lilienfeld, 2007).
These trials have fed into meta-analytic research, which has gained tremendously in importance as the basis for treatment guidelines worldwide. “Working factors” and other theoretical underpinnings of psychotherapies may remain largely undetermined, but the general efficacy of various treatments is now firmly established; this evidence has undeniably led to increases in the number of patients routinely receiving psychological treatment. Since the late 1990s, psychotherapy researchers have been leading in the development and evaluation of Internet-based and digital interventions. It is difficult to see these formats as mere “fashion trends” in Meehl’s sense, and their use in routine settings is likely to increase in the future.
It is clear that questions surrounding the progress of psychotherapy research will be very difficult to resolve. Such debates also offer frustratingly little to scientists and practitioners on the ground. We may consider a third option: that our idea of “progress” itself is misguided; at least when we adopt it naively from the “hard” sciences (physics, chemistry). Following 20th century philosophy, this would lead to a “quietist” view on the theoretical development of mental health research (McDowell, 2009; Rorty, 2007; Wright, 1994, p. 202).
In “Mad Travelers”, Hacking (1998) provides an account of transient mental illness, “an illness that appears at a time, in a place, and later fades away” (p.1). His analysis focuses on a strange epidemic of psychogenic fugues at the turn of 19th century. These fugues occurred in specific people (skilled laborers or merchantmen, but not farmers or “country folk”) across certain regions (France, Germany, Italy; but not Great Britain or the United States), and involved patients travelling hundreds of kilometers in a trance state, often by train, without recollection. Hacking uses these “mad travelers” as an illustration of the contextual character of mental disorders: earlier examples include “acedia” (spiritual depression among monks), or psychogenic “dancing plagues” along trading routes (Waller, 2008, 2009).
Note. Dr. Philippe Tissié hypnotizes Albert Dadas, a famous Bordeaux „fugeur” who would walk up to 70 kilometers per day in a state of complete forgetfulness (1891; British Library). Center: “Accidia” (acedia) in Hieronymus Bosch’s “The Seven Deadly Sins and the Four Last Things” (c. 1500; Wikimedia). Right: Engraving of three individuals afflicted by the “dancing plague”, by Hendrik Hondius, after Brueghel the Elder (1564, Wikimedia). Dancing manias are reported all throughout the Middle Ages. They would involve large groups of people dancing in agony for days or weeks, some dying from exhaustion. Waller (2009) argues that these plagues were “psychogenic” in nature, and cannot be explained as manifestations of, e.g., ergotism (“Saint Anthony’s fire”).
In his analysis, Hacking asserts that mental illness may be best understood using the concept of “ecological niches”. These niches arise from the fact that mental disorders are categorized, understood and treated based on their symptoms or “signs”, which are themselves a complex product of biological, psychological, sociocultural elements that interact over time (“symptoms aplenty, yes, but different congeries of symptoms in different decades […], with no determinate medical entity from which they emanate”, Hacking, 1998, p. 9). This is not a socially constructivist stance: Hacking’s niches are “not just social, not just medical, not just coming from the patient, not just from the doctors, but from the concatenation of an extraordinarily large number of diverse type of elements which for a moment provide a stable home for certain types of manifestation of illness.” (ibid, p. 13).
Hacking sees this as a defining feature of mental disorders, from which the perceived “ailments” mental health research emanate. Drawing on Wittgenstein’s “Philosophical Investigations” (1953, §371) (cf. Wakefield, 2014), he writes (p. 10):
“Wittgenstein said that in psychology there are experimental methods and conceptual confusion. We have more than that for the mental illnesses. We have the clinical methods of medicine, psychiatry, psychology; we have the innumerable variants of and deviations from psychoanalysis; have systems of self-help, group help, and counselors including priests and gurus; we have the statistical methods of epidemiology and population genetics; we have the experimental methods of biochemistry, neurol-ogy, pathology, and molecular biology; we have the theoretical modeling of cognitive science; and we have conceptual confusion.”
He shares his skepticism that scientific progress, as classically conceived, is attainable in these disciplines, or even desirable:
“Perhaps all our problems will be erased when we have enough objective scientific knowledge. I have another view. We do have a limitless reservoir of ignorance, but we also have conceptual con-fusions that new knowledge seldom helps relieve. There are a number of reasons for this, but I am especially impressed by the way that scientific knowledge about ourselves – the mere belief system – changes how we think of ourselves, the possibilities that are open to us, the kinds of people we take ourselves and our fellows to be. Knowledge interacts with us and with a larger body of practice and ordinary life. This generates socially permissible combinations of symptoms and disease entities.”
Hacking’s thoughts share a family resemblance, as he mentions, with Kuhn’s idea of research paradigms (Kindi, 2012). Existing taxonomies of mental disorders provide, for some time, a stable conceptual framework to accommodate (biological, behavioral, mental) manifestations of “disease”. These paradigms are productive, in the sense of suggesting direct pathways for treatment, once patients are correctly “sorted”. This idea reappears in older classification systems as it does in newer nosological projects, such as the Hierarchical Taxonomy of Psychopathology (HiTOP; Kotov et al., 2017). A historic example may be French psychiatrists’ efforts to pigeon-hole 19th century fugeurs as “epileptic” or “hysteric”, following the taxonomy of their time, in order to find a treatment. Jean-Martin Charcot (1825-1893) writes on his patient Mén: “if the man’s flights are equivalent to epileptic seizures, then I can treat him” (Hacking, 1998, p. 36; emphasis mine).
Similar to Kuhn’s scientific revolutions, taxonomies have to be broken up if some phenomena (clinical, social, or scientific) cannot be fully integrated anymore. This leads to new disorder groups, treatment approaches, and research paradigms. Such changes, in turn, influence what is regarded as “real” about mental health problems: a dysfunction in brain chemistry (Jefferson, 2022; Moncrieff et al., 2023); or automatized thoughts (Beck, 2008); or “psychological inflexibility” (Levin et al., 2014), disturbances in the “Bayesian brain” (Feldmann et al., 2023), or “critical slowing down” of a symptom network (Van De Leemput et al., 2014). These paradigms shape what disease markers “appear” in a population and how treatments, psychological or otherwise, are motivated. They are contingent, superseding older taxonomies as time progresses, as well as coexisting with others (Hacking, 1998, p. 38).
What do we take from this analysis of the field, as provided by an eminent contemporary philosopher of science? We need not follow Hacking’s view in its totality. I believe it does show, however, our underappreciation of the enormous context-sensitivity displayed by mental disorders, and their treatments. Many paradigms acknowledge this, although mostly as a “nuisance parameter” that their more precise (formal, mathematical) methods will “explain away”. The idea I want to follow in this thesis is that, in mental health research, context sensitivity is central to the matter: it is an irreducible characteristic of the field, deserving study in and of itself.
The next chapter pursues the notion that, in everyday research, this context sensitivity reappears as heterogeneity. I will briefly review heterogeneity as a pervasive feature in the presentation and course of mental disorders, and in how patients respond to psychological treatment. I will also attempt to provide a more formal and statistical definition of heterogeneity. This includes the concept of exchangeability, a foundational idea in statistics that also bears relevance to meta-analysis. Lastly, I will examine arguments that heterogeneity, if taken seriously, has catastrophic consequences for quantitative social science, including psychotherapy research.
1.4 Heterogeneity, Exchangeability & Meta-analysis
1.4.1 Heterogeneity of Mental Disorders
Today, most psychiatric disorders are viewed as latent constructs identified by a collection of “characteristic” features, better known as symptoms. Commonly used diagnostic systems, including the Diagnostic and Statistical Manual of Mental Disorders (DSM-5), define mental disorders as “polythetic” constructs, i.e., as checklists of necessary and sufficient symptoms that, once fulfilled, result in a specific diagnosis (Krueger & Bezdjian, 2009; Regier, 2007). A characteristic feature for most disorder types is that some, but not all listed symptoms need to be fulfilled to make a diagnosis. For major depressive disorder, for example, the DSM-5 lists nine diagnostic criteria, only five of which need to obtain for a diagnosis to be made.
Many authors have pointed to the limitations of this classification system, which has been called a “Chinese menu”-type approach to diagnosis (Regier, 2007; Whooley, 2010). A major problem of current taxonomies is their failure to capture the, at times enormous, heterogeneity of symptom profiles within postulated disease entities. For depression, for example, there are more than 1,000 unique symptom combinations leading to a diagnosis of major depression (Fried & Nesse, 2015); for PTSD, this number is 636,120 (Galatzer-Levy & Bryant, 2013). For more than half of all mental disorders, current classification systems can lead to the paradoxical situation in which patients can receive the same diagnosis without sharing a single symptom (Olbert et al., 2014). A further problem of the polythetic approach is that it provides all but fuzzy boundaries between diagnoses due to symptom overlap, inflating their apparent similarity, and leading to many patients fulfilling the criteria for multiple disorders (Forbes et al., 2023).
Against this backdrop, it is not too surprising that clinical heterogeneity has been found to be a defining feature of many mental disorders, ranging across patients, populations, and age groups (Fried & Nesse, 2015; Goldberg, 2011; Nandi et al., 2009; Steinhausen, 2009).
Following Nunes et al. (2020b), this clinical heterogeneity can be viewed as a type of multimodality: existing disorder classifications are, in fact, sets comprised of many distinct clusters that appear with relative abundance, but cannot be differentiated a priori in a robust or reliable way. In part, this ignorance may be a logical consequence of the polythetic approach itself, which maintains to be “etiologically agnostic” for most disorders; i.e., not based on a single well-defined pathogenic pathway, which is often unclear (Castiglioni & Laudisa, 2015; McLennan & Braunberger, 2018). This relates to the often-stated equifinality of mental illness, whereby a wide range of developmental pathways can lead to one and the same disease phenotype (Cicchetti & Rogosch, 1996); as well as multifinality, which describes the reverse case. Kendler (2021) brings the problem to the point:
“We assume that constructs, such as schizophrenia or alcohol use disorder, exist but we can only ob-serve the signs, symptoms, and course of illness that we postulate result from these disorders. De-spite years of research, we cannot explain or directly observe the pathophysiologies of major mental health disorders that we could use to define essential features. […] [A] major criticism of our current nosologic efforts has been the limited progress made in moving from descriptive to etiologically based diagnoses.”
Kendler also alludes to issues we discussed above: in lack of a clearly identifiable causal pathway to define what is “real” about a mental disorder, and left with all but a descriptive analysis of “congeries of symptoms” (Hacking), existing taxonomies have to be seen as no more than preliminary. He states that:
“While reality is an unforgiving criterion, empirical adequacy is more flexible, can be progressively im-proved upon, and is subject to varying constructions. […] Indeed, Emil Kraepelin, was tentative in his diagnostic conclusions, experimenting with various nosologic categories over his career, and repeatedly revising earlier formulations. Furthermore, near the end of his life, he proposed several quite differ-ent approaches to conceptualizing psychiatric illness, the most famous being that of an organ register. […]
Anyone familiar with the history of psychiatry can name many diagnoses, popular at earlier times (eg, monomania, masturbatory insanity, hysteria), that have since been rejected. Even ardent DSM advocates are unlikely to claim that all current diagnoses are the final word.”
1.4.2 Heterogeneity of Treatment Effects