Implementation factors and mechanisms of impact in the context of IAPT Norway: A process evaluation

(1)

Implementation factors and mechanisms of impact in the context of IAPT Norway:

A process evaluation

Linn Vathne Lervik

Department of Health Promotion, Division of Physical and Mental Health,

Norwegian Institute of Public Health

Thesis submitted for the degree of Ph.D., Department of Psychology,

Faculty of Social Sciences, University of Oslo

2021

(2)

© Linn Vathne Lervik, 2021

Series of dissertations submitted to the Faculty of Social Sciences, University of Oslo No. 874

ISSN 1564-3991

reproduced or transmitted, in any form or by any means, without permission.

Cover: Hanne Baadsgaard Utigard.

Print production: Reprosentralen, University of Oslo.

(3)

1

Acknowledgements

The present thesis forms part of the larger “Randomized controlled trial of Prompt Mental Health Care (PMHC)” research project, which was initiated by the Norwegian Institute of Public Health and funded by the Norwegian Research Council’s High-Quality and Reliable Diagnostics, Treatment and Rehabilitation (BEHANDLING) research program.

This thesis has come to life through the contributions of many people to whom I am truly grateful. I wish to thank some of them here. First, I would like to express my gratitude to all the clients and team members who participated in the randomized controlled trial of the PMHC services in Sandnes and Kristiansand. This research could not have been done without your contributions and

commitment. Thank you for sharing your experiences with me in the hope of benefiting others.

I would particularly like to thank my main supervisor, Professor Robert Smith. Thank you for believing in me and for helping me pursue my academic dream by sharing your time, knowledge, and wisdom, both in general and in relation to the world of statistics. You are a truly dedicated researcher and a great inspiration to me. Having you by my side throughout my studies has been indispensable.

I also wish to thank my co-supervisor, Dr. Marit Knapstad. Your academic insights and social support have been immensely valuable to me. You are a true role model, both as a researcher and as a person. We have known each other for decades now, for which I am truly grateful, and I hope we will continue to be in each other’s lives in the future. Further, I want to express my gratitude to my other co-supervisor, Professor Asle Hoffart from the University of Oslo and Modum Bad Psychiatric Center.

Thank you for sharing your expertise as a researcher and a clinician. In time, I hope to pursue a similar path.

This Ph.D. journey has allowed me the opportunity to participate in a variety of scientific environments. I want to thank all my colleagues at the Norwegian Institute of Public Health, especially the Department of Health Promotion and the Department of Child Health and

Development, for sharing their work and knowledge with me. I also wish to thank the University of Oslo for providing me with valuable courses and seminars that have helped to develop and prepare me as a scholar. Moreover, special thanks are due to all the fellow Ph.D. students that I have become acquainted with along the way.

During the final year of my Ph.D. studies, the COVID-19 pandemic struck. This led to some extra challenges, and my home office became the site of my everyday life. The support I received from family, friends, and good neighbors was all the more important during this period. To my friend Anne, thanks for all the telephone calls; I cherish your friendship, and your faith in me means a lot.

(6)

4 To my parents, Anita and Gunnar, I am forever grateful for your eternal support and unconditional love. You have both stood beside me all my life and laid the foundation for who I am today. I hope I can do the same for my children.

To my parents-in-law, Randi and Inge, thanks for taking such good care of my family while I have been busy with my research as well as of all of us when we are together. It means a lot to me. To my children, Jenny, Embla and Frikk, thanks for helping me to be mindful and reminding me of the need to balance work and everyday life. I am so grateful to have you all in my life.

Last but certainly not least, dear Jørn, my rock in life. Thank you so much for your support, patience, and faith in me and my work. Thank you for making me laugh and reminding me of what is important in life. I am forever grateful, and I could not have completed this thesis without you. I look forward to the rest of our life together.

Oslo, May 2021 Linn Vathne Lervik

(7)

5

Abbreviations

APA = American Psychiatric Association AB = Avoidance behavior

AIC = Akaike information criterion BIC = Bayesian information criterion CBT = Cognitive behavioral therapy CFA = Confirmatory factor analysis CFI = Comparative fit index CLPM = Cross-Lagged Panel Model COG = Cognitions

CTACS = Cognitive Therapy Adherence and Competence Scale CTS = Cognitive Therapy Scale

ECV = Explained common variance EFA = Exploratory factor analysis ES = Effect size

GAD-7 = Generalized Anxiety Disorder 7-Item Scale GP = General practitioner

ICC = Intraclass correlation

IAPT = Improving Access to Psychological Therapies IPS = Individual placement and support

IQR = Interquartile range

LCM-SR = Latent Curve Model with Structured Residuals MAR = Missing at random

MNAR = Missing not at random NDH = Norwegian Directory of Health NHS = National Health Service

PHQ-9 = Patient Health Questionnaire-9 PMHC = Prompt Mental Health Care QoL = Quality of life

RCT = Randomized controlled trial SD = Standard deviation

RI-CLPM = Random Intercept Cross-Lagged Panel Model RMSEA = Root mean square error of approximation SAD = Social anxiety disorder

(8)

6 SPCQ = Social Probability and Cost Questionnaire

SPIN = Social Phobia Inventory

SRMR = Standardized root mean square residual TAU = Treatment as usual

WHO = World health organization

(9)

7

Summary

The English Improving Access to Psychological Therapies (IAPT) program and its Norwegian successor, the Prompt Mental Health Care (PMHC) program, are fairly new primary-care services designed to target the treatment gap in relation to anxiety and depression. In delivering the two programs, interdisciplinary teams offer a low-threshold, stepped-care approach consisting of both low-intensity (guided self-help and group-based psychoeducation) and high-intensity (face-to-face, individual) treatment approaches, with cognitive behavioral therapy (CBT) being the main treatment approach provided. Although a number of evaluations of the IAPT and PMHC programs have been conducted and shown positive results, most such studies have focused on treatment outcomes, while far fewer have considered aspects related to process evaluation. The aim of the present thesis is to generate new knowledge and insights concerning the implementation factors and mechanisms of impact associated with the PMHC services. This overarching aim is concretized and addressed in three articles. In the remainder of this summary, a short overview of each paper is provided, and some overall conclusions are drawn.

The research discussed in Paper 1 addressed several aspects of the United Kingdom’s Medical Research Council (MRC) guidance on process evaluation. More specifically, it considered the fidelity of the recommended practices, the content of the PMHC services, and the participants’ responses to and interactions with those services. Both qualitative and quantitative data were collected to

facilitate the investigation, and the main analyses were descriptive and thematic in nature. The main findings indicated adherence to several key aspects, namely reaching the target group, being low threshold, having relatively short waiting times (the median waiting time between initial contact and the start of treatment was 27 days; the interquartile range [IQR] was 18–39 days), having no waiting list, and frequently allowing for self-referral (33.3%). However, some aspects were implemented less well, namely the low provision of guided self-help (received by only 1.0% of clients), the limited focus on work participation (from low to some degree of focus in 70.8% of sick-listed clients), little

collaboration with other services (no collaboration in the case of 85.3% of clients), and challenges regarding future service development. These quantitative results harmonized with the results of the therapist interviews, and the latter provided more in-depth insights into the facilitators and barriers of implementation. The results also revealed a high degree of treatment satisfaction across the client groups and the severity levels at baseline (mean = 3.93; standard deviation [SD] = 0.71; range = 1–5).

The research considered in Paper 2 explored aspects related to treatment fidelity, as operationalized as CBT competence/adherence and alliance, as well as their roles as intervention mechanisms with regard to client outcomes from the PMHC services. More specifically, the psychometric properties of

(10)

8 the Cognitive Therapy Adherence and Competence Scale (CTACS) were used to rate the PMHC treatment sessions, while the scale’s associations with client outcomes were investigated by means of multilevel analyses. The psychometric problems of the CTACS were identified in terms of the interrater reliability and divergent and structural validity. The findings indicated that the mean level of CBT competence was slightly lower than the predefined level of sufficiency (3.0). In general terms, no evidence of relationships between either CBT competence or alliance and client outcomes was found.

Finally, the research discussed in Paper 3 explored the specific client mechanisms within the PMHC services. By focusing on one of the main problems identified in the PMHC sessions, namely social anxiety disorder (SAD), the temporal associations between social-anxiety-related avoidance behavior (AB) and social anxiety cognitions (COG), as well as their impacts on the general symptoms of anxiety, were investigated by means of structural equation modeling. Indications were found that AB

predicted COG, but not vice versa. Furthermore, unlike COG, AB was found to predict clients’ general level of anxiety. The effects remained significant even after controlling for depression.

In sum, a comprehensive process evaluation of the implementation aspects and mechanisms of impact of the PMHC services is provided in the present thesis. The findings align with those of previous evaluations, indicating that the PMHC services manage to adhere to the key aspects of the program (i.e., reaching the target group, being low threshold and short term, having relatively short waiting times). Across the groups, the clients reported being content to very content with the PMHC services. However, the need for improvements with regard to some crucial aspects was highlighted (i.e., increase the provision of guided self-help, enhance collaboration with other services, focus on work during treatment, and increase CBT competence). In terms of the mechanisms of impact within the PMHC services, none of the investigated intervention mechanisms, that is, CBT competence and alliance, were found to be significant contributors to the client outcomes. Notably, several

psychometric problems with the CTACS were identified, which may have contributed to the lack of impact on the part of those mechanisms and which could limit the utility of the scale. Indications of significant client mechanisms within the PMHC services were found. When clients with symptoms related to SAD reported changes in their AB, it predicted changes in their COG on the next occasion.

Targeting the AB of primary-care clients who report symptoms of SAD may, therefore, be vital to ensuring the optimal effect of relatively short-term CBT among this group. Thus, insights regarding adherence to the PMHC guidelines, the implementation challenges, and the facilitation factors are also provided in this thesis, together with information on methodological aspects to consider when investigating relevant mechanisms.

(11)

9 Still, there exists a need for further research on several aspects of the PMHC services before it will be possible to confidently determine that the program serves its intended functions. Among other issues, it is important to consider potential organizational and treatment changes and effects, in addition to developing a better understanding of the mechanisms of impact, for example, the possible adverse effects for clients. On a broader level, the degree to which the PMHC services are both cost-efficient and capable of contributing to closing the treatment gap concerning depression and anxiety among the Norwegian population remains unclear and so warrants further investigation.

It is hoped that this thesis will inspire and prove useful for policymakers, Norwegian municipalities, team members, and other countries seeking to implement the IAPT/PMHC approaches.

(12)

10

Norsk sammendrag [Summary in Norwegian]

Det engelske programmet Improving Access to Psychological Therapies (IAPT) og den norske etterfølgeren Rask Psykisk Helsehjelp (engelsk oversettelse: Prompt Mental Health Care (PMHC)) er relativt nye primærhelsetjenester som ble startet for å adressere behandlingsgapet for angst og depresjon i befolkningen. Tverrfaglige team tilbyr en lavterskel, trinnvis tilnærming bestående av lavintensiv (veiledet selvhjelp og gruppebasert psykoedukasjon) og høyintensiv (individuell ansikt-til- ansikt) behandling. Metoden som benyttes er hovedsakelig basert på Kognitiv adferdsterapi (CBT).

Selv om flere evalueringer av både IAPT og PMHC har blitt gjennomført og vist positive resultater, har de fleste studier satt søkelys på utfall, og i mindre grad aspekter relevant for prosessevaluering.

Målet med denne avhandlingen er å gi ny kunnskap og innsikt om implementeringsfaktorer og virkningsmekanismer for PMHC tjenestene. Dette overordnede målet konkretiseres og belyses i tre artikler. I det følgende presenteres en kort oppsummering av hver artikkel før noen generelle konklusjoner trekkes.

Artikkel 1 tar for seg flere aspekter av de britiske MRC retningslinjene for prosessevaluering, nærmere bestemt etterlevelse av anbefalinger for praksis, innhold i PMHC tjenestene samt deltagernes respons på og interaksjon med tjenestene. Både kvalitative og kvantitative data ble samlet inn og hoved analysene var deskriptiv og tematisk. Våre hovedfunn indikerte etterlevelse av flere viktige aspekter; å nå målgruppen, være lavterskel, ha relativt korte ventetider (median ventetid mellom første kontakt og behandlingsstart var 27 dager, IQR 18-39), ingen venteliste og hyppig bruk av selvhenvisning (33.3%). Noen aspekter var gjennomført mindre godt; liten bruk av veiledet selv-hjelp (mottatt av bare 1% av klientene), begrenset fokus på arbeidsdeltagelse (fra lav til noe grad av fokus for 70,8% av sykemeldte klienter), lite samarbeid med andre tjenester (ingen samarbeid for 85,3% av klientene) samt en rekke utfordringer for videre tjenesteutvikling. Disse kvantitative resultatene var i overenstemmelse med resultatene fra terapeutintervjuene. Sistnevnte gav mer innsikt i fremmende og hemmende faktorer for implementeringen. Resultatene viste også blant klientene en høy grad av fornøydhet med behandlingen, på tvers av klientgrupper og

alvorlighetsgrad ved oppstart (Mean = 3.93 (SD = .71, range 1-5).

I artikkel 2 ble aspekter knyttet til etterlevelse av behandlingen, operasjonalisert som CBT-

kompetanse/etterlevelse og allianse og deres rolle som behandlingsmekanismer i PMHC-tjenestene utforsket. Spesielt ble de psykometriske egenskapene til skjemaet Cognitive Therapy Adherence and Competence Scale (CTACS), benyttet til å vurdere behandlingstimene i PMHC undersøkt og dens sammenheng med utfall for klientene ved bruk av multilevel analyser. Psykometriske problemer med CTACS ble identifisert med hensyn til interrater-reliabilitet, divergerende og strukturell validitet.

(13)

11 Videre indikerte funnene at det gjennomsnittlige nivået av CBT-kompetanse var litt lavere enn det forhåndsdefinerte nivået av tilstrekkelighet (3.0). Samlet sett fant vi ingen bevis for en signifikant sammenheng mellom CBT-kompetanse og allianse på den ene siden og klientenes utfall på den andre siden.

I den siste artikkel 3 undersøkte vi spesifikke klient endringsmekanismer i PMHC-tjenestene. Ved å sette søkelys på en av hoved problematikkene i PMHC, sosial angst lidelse (SAD), ble temporale sammenhenger mellom sosial angst relatert unngåelsesadferd (AB) og sosiale angstkognisjoner (COG) samt deres innvirkning på generelle symptomer på angst undersøkt ved hjelp av structural equation modeling. Vi fant indikasjoner på at unngåelsesadferd predikerte kognisjoner, men ikke omvendt. Videre predikerte unngåelsesadferd, men ikke kognisjoner, klientens generelle angstnivå.

Effektene var fortsatt signifikante etter kontrollering for depresjon.

Oppsummert har denne avhandlingen gitt en omfattende prosessevaluering av

implementeringsfaktorer og virkningsmekanismer for PMHC-tjenestene. Resultatene stemmer overens med tidligere evalueringer noe som indikerer at PMHC-tjenestene klarer å følge viktige aspekter av programmet. Dette gjaldt særlig at de når ut til målgruppen, er lavterskel og korttids, samt har relativt kort ventetid. Generelt virker også klientene fornøyd til veldig fornøyd med PMHC- tjenestene. Imidlertid blir forbedringer angående noen viktige aspekter fremhevet; bruk av veiledet selvhjelp, samarbeid med andre tjenester, arbeidsfokus i behandlingen og CBT-kompetanse. Når det gjelder virkningsmekanismer i PMHC-tjenestene, ble ingen av behandlingsmekanismene undersøkt, CBT kompetanse og terapeutisk allianse, indikert som vesentlige bidragsytere til klient resultatene.

Samtidig ble psykometriske problemer med CTACS identifisert og kan ha bidratt til manglende innvirkning av disse mekanismene, men kan også peke på skalaens begrensede nytteverdi. Vi fant indikasjoner for signifikante klient endringsmekanismer i PMHC-tjenestene. Når klienter med symptomer relatert til SAD rapportere om endringer i deres unngåelsesadferd predikerte dette endringer i deres kognisjoner ved påfølgende anledning. Fokus på unngåelsesadferd for klienter i primær helse tjenesten som rapporterer symptomer på SAD kan således være avgjørende for den optimale effekten av relativt kortvarig CBT for denne gruppen.

Innsikt angående etterlevelse av PMHC retningslinjene, implementeringsutfordringer og

tilretteleggingsfaktorer er rapportert i denne avhandlingen, sammen med metodiske aspekter å vurdere når man undersøker mekanismer. Det er likevel behov for videre forskning på flere aspekter av PMHC-tjenestene før det vil være mulig å trygt bestemme at programmet tjener de tiltenkte funksjonene. Blant annet om mulige organisasjons og behandlingsendringer og effekter, og bedre forståelse av virkningsmekanismer, for eksempel mulige bivirkninger for klienter. På et bredere nivå

(14)

12 er det relativt ukjent i hvilken grad PMHC-tjenestene er kostnadseffektive og bidrar til å minske behandlingsgapet for angst og depresjon i den norske befolkningen og det berettiger videre undersøkelse. Forhåpentligvis kan denne avhandlingen inspirere og være nyttig for

beslutningstakere, norske kommuner, PMHC medarbeidere og andre land som ønsker å følge IAPT/PMHC tilnærmingene.

(15)

13

List of Papers

Paper 1

Lervik, L. V., Knapstad, M., & Smith, O. R. F. (2020). Process evaluation of Prompt Mental Health Care (PMHC): The Norwegian version of Improving Access to Psychological Therapies. BMC Health Services Research, 20(1), 437. https://doi.org/10.1186/s12913-020-05311-5

Paper 2

Lervik, L. V., Knapstad, M., Hoffart, A., & Smith, O. R. F. (2021). Psychometric properties of the Norwegian version of the Cognitive Therapy Adherence and Competence Scale (CTACS) and its associations with outcomes following treatment in IAPT Norway. Frontiers in Psychology, 12, 346.

https://doi.org/10.3389/fpsyg.2021.639225 Paper 3

Lervik, L. V., Hoffart, A., Knapstad, M., & Smith, O. R. F. (2021). Exploring the temporal associations between avoidance behavior and cognitions during the course of cognitive behavioral therapy for clients with symptoms of social anxiety disorder. Psychotherapy Research, Ahead of print, 1-14.

https://doi.org/10.1080/10503307.2021.1930243

(16)

14

List of Tables and Figures

Figure 1: Key aspects of Process Evaluation and Relationship amongst

them………. 26 Table 1: Overview of the Study Designs of the Three Papers……….. 35

(17)

15

1 Introduction

With the present thesis, the aim was to contribute to the literature through a process evaluation, thereby providing new knowledge regarding the implementation factors and mechanisms of impact of the Prompt Mental Health Care (PMHC) program, which is the Norwegian adaptation of the English Improving Access to Psychological Therapies (IAPT) program. In short, the IAPT and PMHC programs are both primary-care services developed and implemented to address the wide-ranging burden and treatment gap associated with anxiety and depression among the adult population. The data employed in this thesis are derived from the first randomized controlled trial (RCT) study of an IAPT-inspired service, which was conducted in the municipalities of Kristiansand and Sandnes in Norway from 2015–2017. In the following sections, I will first present some background information, discuss the goals, and provide a description of the IAPT/PMHC services and guidelines, together with previous evaluations of the services. Next, I will introduce the process evaluation model that served as a frame of reference for the research focus of this thesis. The subsequent sections of this

introductory chapter will cover the implementation factors, the mechanisms of impact, and the concept of treatment fidelity, together with the assessment methods and the assessment instrument (i.e., the Cognitive Therapy Adherence and Competence Scale [CTACS]) used in this thesis to evaluate the PMHC treatment sessions. Then, a short description of cognitive behavioral therapy (CBT), that is, the main treatment methodology used within IAPT and PMHC programs, will be presented, alongside the empirical evidence concerning the mechanisms of change in relation to CBT as well as important methodological considerations. The key knowledge gaps that the present thesis aims to address will also be highlighted throughout this introduction and then summarized in the final subsection.

Afterwards, in chapter two, the specific aims of the three papers included in this thesis will be presented.

1.1 Scope of the problem

1.1.1 Common mental health disorders (types, descriptions, prevalence)

Due to their prevalence, depression and anxiety are often considered to be among the common mental health disorders (CMDs), although this generic term fails to capture the conditions’

specificities (World Health Organization [WHO], 2017). In fact, depression is a relatively broad clinical category that is characterized by a persistent feeling of sadness, a loss of energy, and a lack of interest or pleasure in most activities (American Psychiatric Association [APA], 2013). Depression can also be classified as mild, moderate, or severe, depending on the amount and intensity of the presented symptoms. In 2015, more than 300 million people worldwide (4.4% of the global

population) were estimated to be living with depression (WHO, 2017). Anxiety, similar to depression,

(18)

16 is a broad clinical category that includes disorders ranging from simple phobias to highly disabling agoraphobia, panic disorder, generalized anxiety, and social anxiety. Anxiety is also considered a normal feeling when people are exposed to stress, and it may even be beneficial in some situations.

Fear is a related term, although it captures more the emotional response to an immediate threat, which causes a person to either fight or flee. What distinguishes these normal reactions from anxiety as a disorder are the exaggerated level of fear or anxiety experienced in the situation and the

consequences of that fear or anxiety with regard to the ability to function normally (APA, 2020).

Many people who suffer with an anxiety disorder also exhibit avoidance behavior toward certain situations or objects. In 2015, it was estimated that 3.6% of the global population (264 million people) suffered with anxiety (WHO, 2017). In Norway, around 20% of the adult population

experience a mental disorder each year, with anxiety and depression accounting for the majority of this proportion (Folkehelseinstituttet, 2018).

1.1.1.1 Definition of depression and anxiety in this thesis

Mental health disorders are often considered both on a continuum from the absence of symptoms to severe psychopathology as well as according to the specific thresholds for classification based on several criteria (L.A. Clark et al., 2017). Furthermore, a central component of diagnosis when it comes to mental health disorders is the degree to which a person is affected in terms of his or her

functioning during daily life. The most commonly used approaches for classifying the clinical levels of mental disorders are the WHO’s International Classification of Diseases (ICD) and the APA’s

Diagnostic and Statistical Manual of Mental Disorders (DSM). In the present thesis, the terms depression and anxiety are used to refer to both the sub-clinical and clinical symptom levels of the disorders because PMHC has the goal to be a low threshold intervention for sub-clinical as well as clinical levels of symptoms. Sub-clinical symptoms can be considered important targets both for alleviating distress and for preventing deterioration and other related consequences. For instance, there are indications that both the clinical and sub-clinical symptom levels of depression and anxiety are risk factors for disability pension awards (Knudsen et al., 2010). Another reason is that the PMHC services are intended to be both a preventive and a curative treatment approach. Thus, targeting clients with sub-clinical symptom levels may prove clinically relevant and prevent further negative developments (Cuijpers et al., 2004; Haller et al., 2014; Helmchen & Linden, 2000; Morgan et al., 2016; van Zoonen et al., 2014).

1.1.2 Consequences of depression and anxiety (QoL, social and work functioning, costs)

The consequences of living with anxiety and depression are relatively comprehensive, and the effects of the disorders are reflected at the individual, family, and societal levels (Trautmann et al., 2016).

Many facets of life are afflicted by depression and anxiety, including quality of live (QoL) (Olatunji et

(19)

17 al., 2007; Papakostas et al., 2004), sleep (Belleville et al., 2010; Buckner et al., 2008), and social and work functioning (Bültmann et al., 2006; Gärtner et al., 2010; Knudsen et al., 2013). Similar

consequences are seen in sub-clinical cases too (Fehm et al., 2008; Haller et al., 2014; Hoyer et al., 2002; Rucci et al., 2003). Moreover, in Norway, anxiety and depression are recognized as key reasons for accessing of welfare benefits, particularly sickness benefits and disability pensions (Kinge et al., 2017; Knudsen et al., 2013; Knudsen et al., 2012; Knudsen et al., 2010). Worldwide, the economic costs of mental disorders are tremendous, and they expected to increase in the coming years, largely due to population growth and aging (Patel et al., 2016; Trautmann et al., 2016). In Norway, mental health disorders have been estimated to be the costliest health conditions in terms of both

production losses and health-care expenditures (amounting to approximately 80 billion Norwegian kroner in 2013) (Kinge et al., 2017).

1.1.3 Treatment gap

Gross undertreatment has been noted in relation to both anxiety and depression (Collins et al., 2004). This discrepancy is often referred to as “a treatment gap. ”The gap” can be based on both the discrepancy between peoples self-reported need for treatment and their level of treatment received and also measured more objectively between the proportion who have a disease or ailment of a certain level and the proportion who receive treatment, and it is estimated to be more than 50%

worldwide (Alonso et al., 2018; Kohn et al., 2004; Kroenke et al., 2007; Thornicroft et al., 2017).

Higher treatment gaps and lower levels of appropriate care have been evidenced for low-income than for high-income countries (Alonso et al., 2018; Thornicroft et al., 2017). A high treatment gap has also been identified in Norway (Torvik et al., 2018). Indeed, in 2014, Norway was encouraged by the Organization for Economic Co-operation and Development (OECD, 2014) to address the limited provision of treatment for clients with mild and moderate anxiety and depression. Such challenges are now being addressed to a greater extent globally. For example, the WHO (2015) has established a treatment gap action program, while mental health is included among the United Nations

developmental goals. At the national level, England has addressed the treatment gap by developing the innovative IAPT program. More recently, Norway has followed the English example and

introduced the PMHC program.

1.2 The development of the IAPT and PMHC programs

In 2007, England’s National Health Service (NHS) introduced the IAPT program with the aim of providing better access to evidence-based treatments for clients suffering with depression and anxiety (D.M. Clark, 2011). The program has now been rolled out nationwide. Historically, several clinical and policy developments paved the way for the evolvement of the program. The two

(20)

18 contributions most commonly highlighted are, first, the National Institute of Clinical Excellence (NICE)’s publication in 2004 of clinical guidelines advocating CBT for the treatment of anxiety and depression and favoring a stepped-care approach. Second, the joint force of economists and clinical researchers publishing reports and articles, which argued that the cost of increasing people’s access to psychological therapies would ultimately pay for itself, due to the reduction in other costs for society, such as sick leave and medication costs (D.M. Clark, 2012; Layard, 2006). The IAPT program was initially tested and evaluated in two demonstration sites, namely Doncaster and Newham, before a decision was made to scale-up the program (D.M. Clark et al., 2009). Although the two demonstration sites differed somewhat in terms of their approaches, they shared the overarching aims of the IAPT program, that is, to offer low-threshold (i.e., no need for referrals from general practitioners [GPs]), free-of-charge, evidence-based psychological therapies for depression and anxiety, in accordance with the clinical guidelines provided by NICE (D.M Clark, 2011). Another important aspect of the program concerns the offering of both low- and high-intensity treatments in a stepped-care manner. A stepped-care approach means that clients start treatment at the lowest level and then receive higher levels of care if required. The provision of low-intensity treatment is considered an important way of narrowing the aforementioned treatment gap. Low-intensity treatments are considered to be less resource-intensive for clients as well as to require less interaction from a therapist (contact is often provided by phone, email, or online, or in groups), which means the latter often function more as a coach than as a traditional therapist (Bower &

Gilbody, 2005; Richards et al., 2011). Guided self-help and group-based psychoeducation approaches are considered to be lower-intensity treatments, whereas traditional face-to-face talking therapy is considered to be a high-intensity treatment (D.M. Clark, 2012). The education and training of more therapists is also a stated goal within the IAPT program. Finally, a built-in monitoring system is implemented across all the IAPT services, tracing each client’s level of anxiety and depression symptoms from session to session. From a research and evaluation perspective, continuous

monitoring is beneficial, and it forms a better basis for quality assurance and program improvement.

It is also considered a good way of rendering the services more transparent as well as helping clients and therapists track their progress toward recovery (NHS, 2018).

The IAPT program has inspired countries such as Australia, Ireland, and Canada to develop similar services (Bastiampillai et al., 2014; McDevitt-Petrovic et al., 2018; Mental Health Commission of Canada, 2018). The PMHC program is the Norwegian adaptation of the IAPT program, and it was initiated in 2012 by the Norwegian Ministry of Health and Care Services as a pilot project in 12 municipalities (Helsedirektoratet, 2013). Additionally, in the case of Norway, there were several international and national reports that formed the basis for the establishment of the PMHC service

(21)

19 (HelseOmsorg21, 2014; Meld. St. 16 (2010-2011); Meld. St. 34 (2012-2013); Meld. St. 47, 2008-2009;

Organisation for Economic Co-operation and Development [OECD], 2014). In different ways, these documents all promote mental health through early intervention, the integration of mental health services in community-based settings, and the strengthening of evidence-based research in municipalities. Today, approximately 55 Norwegian municipalities have established or intend to establish their own PMHC services. There are, however, several differences between the PMHC and IAPT programs, some of which will be pointed out in the following section, which presents a more detailed description of the Norwegian adaptation.

1.3 Description and goals of the PMHC program

During Norway’s initial piloting phase of the PMHC program, the municipalities applied for funding from the Norwegian Directorate of Health (NDH). If funding was granted, the PMHC program was established through a funding scheme that lasted for three years, with the possibility of extension of an additional fourth year. After this period, local government funding was required for the

continuation of the service. Currently, municipalities that want to establish a PMHC service still need to apply for funding from the NDH, although the funding scheme itself has gradually been adjusted.

As of this year (2021), municipalities that want to establish a PMHC service need to provide their own funding, albeit the CBT training continues to be available free of charge (Helsedirektoratet, 2019).

Guidelines on how to establish a PMHC service were provided by the NDH in 2013 (Helsedirektoratet, 2013). According to these guidelines, the PMHC program’s main target group is people suffering with anxiety and mild to moderate depression, while its main objectives are to reduce symptoms,

strengthen work ability, and be an effective service at the municipal level in terms of offering evidence-based treatment with short waiting times and without waiting lists. Further, the guidelines emphasize that the service needs to be low threshold, meaning that it should be free of charge for clients, without the need for a referral, be available for all within the target audience, and be run by personnel with the competence necessary to fulfill the purpose of the service. The inclusion criteria for accessing the service are being an inhabitant of the municipality, being ≥18 years of age, and having anxiety and/or mild to moderate symptoms of depression. As a result, the therapists do not provide the clients with a psychiatric diagnosis; rather, only an assessment of the symptom level and a clear problem description are required. In Norway, the age criterion has now been extended to include clients from the age of 16, although the 16–17 age group did not form part of the RCT study, which the current thesis is based on.

During treatment, the NDH guidelines emphasize the need for the regular evaluation of treatment progress through symptom monitoring (i.e., self-report questionnaires for anxiety and depression),

(22)

20 the importance of collaboration with other services (particular GPs), the need to integrate a work focus into treatment, and the need to ensure the user involvement of clients and dependents. The duration of treatment is intended to be relatively short, and the high-intensity face-to-face treatment no longer than 15 sessions.

According to the NDH guidelines, it is necessary for the team tasked with delivering the program to consist of at least four full-time equivalents, all with a minimum of three years’ college education in health and social studies, one clinical psychologist in at least a 50% position, and a part-time administrative position. All the therapists are required to complete an additional one-year training program in CBT under the auspices of the Norwegian Association for Cognitive Therapy. In Norway, all team members contribute at all levels of treatment. This flat treatment provision structure differs from the IAPT program, in which most services employ separate therapists providing low-intensity treatments (so-called psychological wellbeing practitioners [PWPs]) and high-intensity treatments (trainee or accredited CBT specialists) (NHS, 2018). Within the IAPT program, there is also a focus on providing weekly supervision by senior clinical practitioners in order to, among other things,

stimulate further CBT competence on the part of the therapists. In the case of the PMHC program, peer monitoring is emphasized, alongside the important supervisory role of the team psychologist.

CBT is the overarching therapy of choice in both the PMHC and IAPT programs, and both low- intensity (guided self-help and group-based psychoeducation) and high-intensity (face-to-face, individual) forms of treatments are offered. In addition to CBT, some IAPT services have started to offer other NICE-recommended therapy methods, including interpersonal therapy (IPT) for

depression. To date, such developments have not been seen in the PMHC program. Further details regarding the CBT treatment offered as part of the PMHC program are provided in subsection 1.3.1, while short descriptions of the CBT principles and mechanisms of change are presented in section 1.8.

The PMHC program is organized as a stepped-care service, although the guidelines also emphasize an alternative matched-care model. This implies that information obtained during the initial assessment and client preferences are decisive factors when it comes to the choice of treatment. Furthermore, it indicates that, different from the stepped-care model used in the IAPT program, the client does not necessarily start at the lowest treatment level (van Straten et al., 2006), which would typically be the case in stepped-care models. Still, to maximize the possibility of access to care for people in need, the NDH does recommend low-intensity treatments as the first choice whenever possible. However, low-intensity treatment is not recommended for some problem descriptions, such as post-traumatic stress disorder (PTSD) and social-anxiety-related issues (NHS, 2018). Notably, this recommendation

(23)

21 was not emphasized within the PMHC program. The evaluation of the services has been a focus from the very beginning in both the IAPT and PMHC programs. It is an important aspect of both initiatives, and its outcomes have provided valuable information and knowledge for the scale-up,

implementation, and continuation of the services. In section 1.4, evaluations of both services will be presented.

1.3.1 Training and treatment within the PMHC program

Both the IAPT and PMHC programs provide several different types of CBT treatments, ranging from low- to high-intensity treatment (Richards & Suckling, 2009; Smith et al., 2016). More specifically, guided self-help, computerized CBT programs, bibliotherapy, and psycho-educational groups are considered to be lower-intensity treatment options, while traditional face-to-face talking therapy is considered to be high intensity (Helsedirektoratet, 2013). In the IAPT program, the low-intensity treatments are mostly provided by PWPs who have been specifically trained to provide these low- intensity treatments (Richards et al., 2011). In the PMHC program, the multidisciplinary teams provide both low- and high-intensity therapies and receive training in all levels of treatment from the Norwegian Association for CBT. The Norwegian training program builds on the IAPT training program and consists of 13 seminars (delivered over the course of 26 days and 108 instruction hours), which are provided over a one-year period. Further, 12 of the seminars have an adjacent seminar day for supervision (96 hours). In addition, the candidates need to conduct approximately 10 sessions of

“buddy supervision.” The provided treatments utilize multiple specific CBT protocols that target both depression and a wide range of anxiety disorders, which are theoretically anchored in models such as the Clark and Wells treatment model for social anxiety disorder (D.M. Clark, 2005; Wells, 1997), the Clark and Salkovski treatment model for panic-disorder (D.M. Clark & Salkovskis, 1991), the Barlow and Craske model for panic disorder (Craske & Barlow, 2007), the Borkovec model for generalized anxiety disorder (Borkovec & Sharpless, 2004), the Wells metacognitive model for rumination and worry (Wells, 2006), and the Beck depression model (A.T. Beck, 1979). Aside from treatments for depression and anxiety, the PMHC therapists also receive training in delivering treatments for sleep problems (i.e., sleep restriction and stimulus control) and work-related CBT. The work-focused CBT training emphasizes the need to use specific work-focused CBT interventions to optimize the effect on return to work (Cullen et al., 2018). Common work-focused CBT aspects are developing work- related goals and thematizing work in every session and throughout the therapy, together with the provision of a checklist for work-focused CBT and a schema for assessing clients’ work situation (Berge & Repål, 2016).

(24)

22

1.4 Initial evaluations of the IAPT and PMHC programs

The initial evaluations of the IAPT program, which were conducted in 2006–2007, revealed promising results concerning the establishment of the services, large effect sizes in relation to the pre-post symptom reduction of anxiety and depression, and notable improvements in functional status at the post-treatment stage (D.M. Clark et al., 2009; Parry et al., 2011). The provision of low-intensity treatment, such as guided self-help, was highlighted as an important contributor to increasing access for more clients and retaining the throughput in treatment. The number of clients referred to the IAPT services was impressive, amounting to approximately 5500 people over a 13-month period for the first two demonstration sites, with 1900 finishing a course of treatment or receiving a minimum of two sessions, including the first assessment (D.M. Clark et al., 2009). Both the initial IAPT services reported good recovery rates close to the goal of a 50% rate. The median waiting time between referral and the first appointment was 25.2 days in Doncaster while it was 47 days in Newham, although it steadily increased during the evaluation period (Parry et al., 2011). However, Newham managed to curb this trend by changing the service model to a telephone assessment and using an administrator to coordinate appointments. The primary treatment type in Doncaster was CBT- oriented guided self-help, whereas Newham varied more between guided self-help and individual and group CBT (Parry et al., 2011). The average number of treatment contacts for a given client was five, by telephone or face-to-face, in Doncaster, and it was five hours in Newham. Further, self- referral was found to facilitate the reach of the services with regard to some underrepresented groups. The number of service staff varied during the evaluation period as well as between the services. Toward the end of the period, nearly 26 PWPs and 12 CBT therapists worked in the services (D.M. Clark et al., 2009). Notably, a considerable amount of turnover of the staff members were reported. Some difficulties regarding the establishment of the services were also underlined, such as collaboration with other services and users (Parry et al., 2011). Moreover, challenges in terms of evaluating the cost-effectiveness of the services were reported, with only the Doncaster IAPT service being indicated to likely be cost effective, although the estimates were associated with considerable uncertainty (Parry et al., 2011).

The results of the evaluations of the first 12 PMHC services, which were conducted in 2014–2016, indicated that the services were able to provide both low-threshold and short-term treatments. The median time from a client’s initial contact to their first assessment was 10 days (interquartile range [IQR] = 4–21), the median time between the first assessment and treatment was eight days (IQR = 3–

15), the median time between the initial contact and the first treatment session was 21 days (IQR = 11–40), and the median number of treatment sessions was five (IQR = 3-8) (Smith et al., 2016). The effect sizes with regard to the reduction of anxiety and depressive symptoms were high across the

(25)

23 pilot services, similar to the first two IAPT services, (Cohen’s d near -1.0). The recovery rates were indicated to be above the previously mentioned IAPT goal of a 50% recovery rate (Knapstad et al., 2018). Further, improvements were reported in the clients’ QoL as well as changes in their working life. In total, nearly 3200 clients were referred to the PMHC services, with around 1983 starting treatment. The first PMHC services provided more high-intensity than low-intensity treatments (mainly face-to-face treatment in 71.5% of cases, mainly group-based psychoeducation in 8.9% of cases, and mainly guided self-help in 16.9% of cases). The reductions in symptom levels were similar across all treatment types and durations, although they were somewhat less for the clients who received group-based psychoeducation when compared with the other two treatment types (Smith et al., 2016; Sæther et al., 2019). The notable challenges to implementation that were identified included the difficulty of reaching some underrepresented groups, such as clients above 67 years of age, with a low level of education, or from an immigrant background. The other identified

implementation challenges were the limited use of guided self-help, the conflicting experiences between therapists and clients of the work-focus in relation to the treatment, and difficulties concerning the continuation of services during the transition from central to local funding (Smith et al., 2016). These aspects could conceivably affect both access to and the effect of the PMHC program.

One major methodological limitation of both countries’ initial evaluations concerned the research design. More precisely, the evaluations were both based on single-group pre-post designs that used benchmarking for comparison. Benchmarking involves comparing study outcomes with results from other RCT studies conducted among similar target groups. With this approach, there is a risk of bias in terms of the selection of clients, as they might potentially differ from the sample under study (Knapstad, Lervik, et al., 2020). It is conceivable that the IAPT/PMHC samples were not sufficiently similar to the benchmark samples gathered from previous CBT trials. For example, the benchmark samples might have consisted of more severe clinical cases. To address concerns regarding the study design, the Norwegian Institute of Public Health took the initiative to evaluate the PMHC services by means of an RCT study. By randomly allocating study participants to different treatment conditions, differences between the study groups during follow-up may be attributed to the treatment(s) under investigation (Knapstad, Lervik, et al., 2020). The following subsection will present the results of the PMHC effectiveness trial.

1.4.1 The PMHC effectiveness trial

In the RCT study, the PMHC program was compared with treatment as usual (TAU) in two large Norwegian municipalities (ClinicalTrials.gov NCT03238872) (Knapstad, Lervik, et al., 2020). Prior to the introduction of the PMHC program, the treatment possibilities were limited for clients with mild

(26)

24 to moderate anxiety and depression in the primary-care services (Ose & Kaspersen, 2015). In the context of the RCT study, TAU was defined as treatment by any other service available within the municipality. Between October 2015 and September 2017, 681 clients were included in the trial. The results at the six-month follow-up point indicated significantly higher recovery rates for the PMHC group when compared with the TAU group (59% vs. 32%). The findings also indicated large to medium between-group effect size differences in favor of the PHMC group for the symptoms of depression (Cohen’s d = 0.88) and the symptoms of anxiety (Cohen’s d = 0.60), respectively

(Knapstad, Lervik, et al., 2020). Further, the PMHC program proved more effective than TAU in terms of improving most secondary outcomes (health-related QoL, positive mental wellbeing, functional status), albeit with one important exception regarding self-reported work participation. The treatment fidelity was also assessed for both individual sessions and group-based psychoeducation by means of the CTACS (Barber et al., 2003). The results indicated that treatments were provided within the sufficient range, although there was potential for improvement (cut-off score = >3.0, mean CTACS score for individual sessions = 2.8 [SD 0.7], and mean CTACS score for group-based psychoeducation = 3.5 [SD 0.3]) (Knapstad, Lervik, et al., 2020). All the reported treatment effects of the PMHC services were found to persist at the 12-month follow-up point (Sæther et al., 2020).

Additionally, the PMHC program was indicated to be effective in terms of treating a significant group of clients who were struggling with social and agoraphobic anxiety-related problems in both the short- and long term (Knapstad & Smith, 2021).

Aside from addressing the effectiveness of the PMHC program, the RCT study also aimed to conduct a process evaluation with a specific focus on the implementation factors and mechanisms of impact.

The present thesis is the result of this effort, and in the following section, I will introduce the process evaluation model that served as the theoretical starting point for Papers 1–3.

1.5 The MRC process evaluation model

According to the United Kingdom’s Medical Research Council (MRC) guidelines, a crucial aspect of testing complex interventions involves conducting a process evaluation (Moore et al., 2015).

Although the model was developed as a tool for complex public health interventions, it has been found to be highly relevant for other research domains, including health service evaluations (Moore et al., 2015). The authors emphasize that this type of evaluation represents an addition to effect evaluations, not a substitute. Furthermore, a process evaluation may be carried out during all the phases of the research process, starting from the early pilot phase through to the evaluation of effectiveness and the later scale-up phase (Moore et al., 2014). A complex intervention can be characterized as having multiple components that interact in order to produce change (Moore et al.,

(27)

25 2014). The PMHC services consist of several treatment types (e.g., guided self-help and individual face-to-face therapy), alongside multiple CBT elements (e.g., psychoeducation, exposure, and cognitive restructuring) and collaboration with existing services (e.g., GPs). As such, the PMHC services can be viewed as a complex intervention. The process evaluation perspective provided by the MRC underlines the importance of providing policymakers with information about how an intervention might be replicated as well as about the potential for reproducing effects. Prior to the present thesis, scientific reports from the Norwegian evaluations mostly focused on the effectiveness of the PMHC services in relation to the outcomes (Knapstad, Lervik, et al., 2020; Knapstad et al., 2018), and to a lesser extent, on aspects relevant to process evaluations (Smith et al., 2016). The implementation challenges identified during the early stages of the development of the PMHC services and the potential challenges connected to the RCT study require further investigation. The RCT study was not only conducted at a later development stage, but also at different sites than the evaluations of the first 12 services and in a different research context, which may give rise to further challenges. Such knowledge can aid in the interpretation of the results of the RCT study and also provide information regarding challenges and recommendations relevant to the continuation and establishment of similar services.

Within the MRC framework, a process evaluation is defined as a study that aims to determine how an intervention functions by examining three interacting components, namely implementation,

mechanisms of impact, and contextual factors (see Figure 1). Implementation is divided into two main aspects, that is, the implementation process (how delivery is achieved) and what is delivered (fidelity, dose, adaptation, and reach). The mechanisms of impact are conceptualized as the transitional mechanisms through which the intervention activities produce intended or unintended effects (Moore et al., 2014). The investigation of these mechanisms may include participants’

responses to and interactions with the intervention, the mediating factors (i.e., variables that statistically contribute to explaining the process through which two variables are related), and any unintended pathways and consequences.

(28)

26 Context is defined as “anything external to the intervention that may act as a barrier or facilitator to its implementation or its effects” (Moore et al., 2015, p. 2). Although, as shown in Figure 1, there is linear progression when conducting a process evaluation, the authors specified that there are feedback loops between the components, indicating the possibility to make adjustments. The model also highlights the importance of using both quantitative (e.g., self-report questionnaires) and qualitative (e.g., one-to-one interviews) methods depending on the research questions being addressed (Moore et al., 2014).

With this framework in mind, we aimed to address those aspects of relevance to the process evaluation of the PMHC program as whole, although we also wanted to focus on aspects with more specific relevance to CBT in general. This translated into the present thesis having three separate yet related research foci: 1) implementation, context, and outcomes of the PMHC service; 2) fidelity of the CBT treatment within the PMHC program; and 3) specific mechanisms of change in the symptoms

(29)

27 of anxiety and depression following CBT as part of the PMHC program. The backgrounds to the three research foci will be discussed in the following sections.

The remaining sections of this introductory chapter will present more details regarding

implementation factors, the concept of treatment fidelity, as operationalized in this thesis as CBT competence/adherence and alliance, together with the assessment methods and the scale used in the present thesis to measure treatment fidelity (i.e., the CTACS). With regard to the mechanisms of impact, it was considered important to examine the process variables and the associations with the treatment outcomes in relation to the PMHC, which is in accordance with the underlying CBT theory.

Therefore, this introduction provides a short summary of the principles of CBT, the mechanisms of change, and the empirical evidence, together with some methodological considerations. Lastly, a summary of the identified knowledge gaps will be presented.

1.6 Implementation factors, context, and outcome

This thesis builds on the initial evaluations of the PMHC and IAPT programs in order to further explore the issues and make comparisons. It was considered important to investigate both the implementation process and the content of the PMHC services. More specifically, and in accordance with implementation theory, policies formulated at a central level (e.g., the NDH) with the aim of influencing local practices (e.g., municipality) often encounter obstacles to their implementation (Offerdal, 2005). Two main approaches, which are referred to as the top-down and bottom-up approaches, can be utilized, and both are important in terms of achieving a comprehensive evaluation. In light of this, it was considered crucial to investigate both the degree of compliance with the objectives described in the NDH guidelines (top-down perspective) and the possible reasons for any deviation or local adjustments (bottom-up perspective).

Several aspects related to the implementation of the first 12 PMHC services were considered to have been implemented well, including the provision of low-threshold and short-term treatments with relatively short waiting times (Smith et al., 2016). However, other implementation aspects were found to be more challenging, such as reaching some underrepresented groups, the limited use of low-intensity treatments, the conflicting experiences between therapists and clients of the work- focus in relation to treatment, and difficulties concerning the continuation of services during the transition from central to local funding. A number of difficulties regarding the establishment of the services were also underlined in the initial IAPT evaluation, such as collaboration with other services and users. These findings revealed aspects that required further exploration in the context of the RCT study as well. Yet, given that the trial was carried out at a later stage of the PMHC program’s

(30)

28 development, at different sites, and in a different research context, it was expected that other implementation challenges might emerge.

1.7 Treatment fidelity

The first evaluation of the PMHC services did not include a direct measure of fidelity (Smith et al., 2016), which limited the knowledge derived regarding how well CBT had been implemented and complied with by the therapists. The issue of treatment fidelity is argued to be imperative when it comes to assessing how well an intervention is delivered as well as when interpreting the

effectiveness results (Perepletchikova, 2011). For example, if an intervention is not found to be effective, and if the level of treatment fidelity is considered low, the results can be attributed to poor implementation, rather than to the treatment not working. On the contrary, if the level of treatment fidelity is regarded as high, it appears more likely that the treatment is ineffective. Treatment fidelity is also important with regard to the quality improvement of services (Perepletchikova, 2011). Thus, in connection with the process evaluation of the two PMHC services included in the RCT study, it was considered important to assess treatment fidelity, together with other implementation aspects such as reach, dose, and adherence to the NDH guidelines.

A widely used conceptualization of treatment fidelity, which is sometimes referred to as treatment integrity, involves equating it with therapist adherence and competence (Perepletchikova, 2011).

Treatment differentiation is also often included under this umbrella term, although it is subsumed into adherence and considered to be most relevant when conducting comparative studies of psychological treatments (Fairburn & Cooper, 2011). Adherence concerns the degree to which therapists are delivering the theory-specified techniques or methods of the intervention (e.g., eliciting negative automatic thoughts, socializing to a model, and reviewing homework). Competence refers to the skill with which these techniques and methods are being delivered (Perepletchikova, 2011). However, despite the relatively clear theoretical distinction between the constructs of

adherence and competence, they are also considered to overlap in a hierarchical manner. It has been argued that adherence is a precondition for competence, while adherence alone does not imply competence (Waltz et al., 1993). One way of defining CBT competence is the framework created by Roth and Pilling (2007) in connection with the IAPT program. In their broad conceptualization, they identify five domains, namely common competences used in all psychological therapies (e.g., the ability to create a trusting relationship with a client), basic CBT competences used in both low- and high-intensity treatments (e.g., use of general CBT principles such as setting the agenda or reviewing homework), specific CBT techniques (i.e., the core techniques common to most CBT treatments, such as exposure techniques and Socratic questioning), problem-specific competences (i.e., the packages

(31)

29 developed to deliver high- and low-intensity treatments for specific problems such as depression, social anxiety, or panic disorder), and meta-competences (e.g., the ability to decide when and, why, it is indicated to deliver different treatment aspects) (Roth & Pilling, 2007). The authors acknowledge that due to the complexity of the framework, it cannot be used systematically to address adherence and competence; rather, they recommend it be used as a curriculum for training therapists (Roth &

Pilling, 2008). This framework is used in both the IAPT and PMHC training. For the assessment of adherence and competence, the authors recommend the use of two already existing measurement scales, namely the CTACS and the Cognitive Therapy Scale (CTS) (Roth & Pilling, 2008). Both scales will be discussed in more detail later in this chapter.

The therapeutic alliance, which is frequently referred to as the working- or helping alliance, can be defined as the overall collaborative facets of the client-therapist relationship (Flückiger et al., 2018;

Hatcher & Barends, 2006). In the present thesis, the term alliance is used when referring to this concept. Examples of the aspects of relevance to alliance include empathy, warmth, and agreement on goals and methods. Information concerning treatment fidelity can be employed both to address questions related to the implementation of treatment interventions and as an important aspect of the interpretation of the results of efficacy and effectiveness trials. This can also relate to the concept of alliance, although it has been the focus of relatively little CBT process research. Rather, alliance has been viewed from a clinical perspective as an important facilitator that is not sufficient to bring about therapeutic change on its own (Castonguay et al., 2010).

1.7.1 Standardized observer-rated measurement scales in the field of CBT

There are many scales available for assessing CBT competence. Some have been developed for specific trials, whereas others have been standardized. In a systematic review of methods for assessing competence scales in the field of CBT, two categories of standardized observer-rated measures were introduced, namely transdiagnostic and disorder-specific scales (Muse & McManus, 2013). The former category is assumed to be more general and so not aimed at specific disorders, whereas the latter is. It has been argued that the standardized scales have the potential to be effective methods for directly assessing therapist competence as well as for making comparisons across studies (Muse & McManus, 2013). The transdiagnostic scales (including the CTS and CTACS) have been presented as more useful in regular clinical settings than the disorder-specific scales, given the large degree of heterogeneity encountered in terms of diagnoses and treatment types (Muse &

McManus, 2013). The original CTS is an 11-item scale ranging from 0 (“poor”) to 6 (“excellent”) (Young & Beck, 1980). The CTS-R is a revised version of the CTS (Blackburn et al., 2001). It is a 12-item scale often used in the IAPT context (Branson et al., 2015; Liness et al., 2019a; Liness et al., 2019b).

Despite being extensively used and having shown some promising psychometric qualities, the

(32)

30 empirical evidence for the CTS is not well established, particularly in terms of its structural and predictive validity (Goldberg et al., 2020). The scale has also been criticized for only assessing competence (Barber et al., 2003). Revised versions such as the CTS-R have not been proven to be substantially better than the original scale (Kazantzis, Clayton, et al., 2018; Muse & McManus, 2013).

The CTACS is partly based on the CTS, and it was developed to provide overall scores for adherence and competence. The CTACS is the scale most commonly used for training and research in Norway, and it is also used in the education of PMHC therapists (T. Berge, personal communication, June 26, 2018). As mentioned above, the CTACS was used to evaluate treatment fidelity in the present thesis, and it will be discussed more fully in the methods chapter.

One major methodological challenge related to assessment is the fact that the constructs of competence/adherence and alliance, as with many constructs in the social sciences, are difficult to define and complex to measure (Ardito & Rabellino, 2011; Barber et al., 2007; Flückiger et al., 2018;

Kühne et al., 2020; Muse & McManus, 2013). This seems to be reflected in the great variety of measurement scales that have been used in the prior research (Ardito & Rabellino, 2011; Barber et al., 2007; Goldberg et al., 2020; Kühne et al., 2020; Muse & McManus, 2013; Schoenwald & Garland, 2013), both in terms of the actual measures and in terms of their modes of administration and assessment (i.e., audio vs. video; client, therapist, vs. expert rated). Moreover, efforts to systemically examine the measurement properties of the available instruments appear to be limited (Goldberg et al., 2020), while no studies have yet examined the underlying structure of the CTACS. However, examining the structure of the CTACS is highly relevant, as doing so could provide information on what constitutes competence in the field of CBT as well as whether some underlying factors are more important than others when it comes to predicting treatment outcomes. Some explanatory work has been done on the CTS (Affrunti & Creed, 2019; Goldberg et al., 2020; Vallis et al., 1986) and on a heavily modified version of the CTACS (Bjaastad et al., 2016). The former found both two factors (CBT skill and general therapy skill) and three factors (structure, CBT techniques, and therapeutic relationship skills) of relevance, while the latter identified two factors (CBT structure and session goals as well as process and relational skills).

1.8 CBT - mechanisms of change and empirical evidence

Currently, CBT is viewed as a collective term that includes several therapeutic models that focus on cognitive and behavioral therapeutic interventions (Berge & Repål, 2016; Craske, 2010). Several principles of CBT have been proposed. One general principle is that CBT is based on an ongoing formulation, as well as an individual conceptualization of clients’ problems, within a cognitive framework (also referred to as a case formulation) (J.S. Beck, 2011). Having a sound alliance is

Implementation factors and mechanisms of impact in the context of IAPT Norway: A process evaluation