Evaluating the implementation of the Empowering Coaching™ program in Norway

(1)

at the University of Bergen

Thesis for the degree of philosophiae doctor (PhD)

(YDOXDWLQJWKHLPSOHPHQWDWLRQRI WKH(PSRZHULQJ&RDFKLQJ

SURJUDPLQ1RUZD\

0DUJDUHW/6¡YLN

'DWHRIGHIHQFH$SULOWK

(2)

The material in this publication is protected by copyright law.

Year: 2017

Title: Evaluating the implementation of the Empowering Coaching™ program in Norway

Author: Margaret L. Søvik

Print: A T i Bjerch AS/University of Bergen

(3)

Scientific environment

The research presented in this thesis was carried out within the research group SIPA (Social Influence Processes on Adolescent Health), in the Department of Health Promotion and Development (HEMIL), at the Faculty of Psychology, University of Bergen. The study presented here employed data collected as part of an EU-funded project, called ‘Promoting Adolescents’ health through an intervention aimed at improving the quality of their participation in Physical Activity’ (PAPA). PAPA was a collaborative project between England, France, Greece, Norway and Spain, involving eight universities.

The Graduate School of Human Interaction and Growth (GHIG), at the Faculty of Psychology, University of Bergen, provided the doctoral training.

Professor Torill Larsen, from HEMIL, was my main supervisor. Hege Eikeland Tjomsland, Associate Professor at Hedmark University of Applied Sciences, and Professor Oddrun Samdal, also from HEMIL, were my co-supervisors.

(4)

Acknowledgments

There are so many people who have contributed to this thesis and made my time as a PhD candidate inspiring and fun. I sincerely appreciate all their contributions and support, and therefore my gratitude goes to:

- My main supervisor, Professor Torill Larsen, for believing in me and encouraging me to get involved in this project. I am so impressed by your knowledge of

implementation research, methods and health promotion, and grateful for your insightful and valuable input. Thank you for inspiring discussions, sharing your knowledge, always responding quickly, and for being supportive and positive, as well as motivating me. It has been great working with you in so many ways!

- My co-supervisor, Associate Professor Hege Eikeland Tjomsland, for inspiring discussions, as well as offering valuable and clear feedback, which have contributed to improvements in my work, and for always being so positive and motivating me.

- My other co-supervisor, Professor Oddrun Samdal, for valuable discussions and input in relation to the thesis and the papers, and for sometimes bringing other perspectives into the papers. You have also encouraged me to do my best. I am amazed by your knowledge and work capacity!

- Professor Bente Wold, head of the SIPA research group, for your valuable input on my thesis and the last paper. Being part of SIPA, as well as GHIG, has contributed to interesting discussions and knowledge on several topics and aspects relevant to both my teaching duties and my research. Additionally, the social happenings have been great.

- Aurélie van Hoye, co-author of my second paper, for clear and concise suggestions and for always providing feedback so quickly. It was a pleasure working with you.

(5)

- The PAPA Consortium (Prof. Isabel Balaguer, Prof. Jaume Cruz, Prof. Joan L.

Duda, Prof. Yngvar Ommundsen, Prof. Athanasios G. Papaioannou, Prof. Philippe Sarrazin, and Prof. Bente Wold), along with Prof. Lars Tore Ronglan (NIH) for his contributions to the training of the Norwegian coach educators, and the Norwegian Football Association (NFF). In addition, I would like to acknowledge the Norwegian coach educators and grassroots coaches for attending the PAPA project, and the European Commission's Seventh Framework Programme FP7/2007-2013 under the grant number 223600.

- Lise Augustson and Jørn Hetland for helping me figure out some basic statistics and take ‘control’ of the data set.

- The Faculty of Psychology, University of Bergen, and HEMIL for giving me this opportunity. The work environment at HEMIL has been so supportive and

stimulating. Special thanks to my colleagues Helga, Anne-Siri, Marte, Elisabeth Å., Annegreet, Ellen S., Frida and Olin. It has been great getting to know you! I will miss you all and your smiles, as well as our small talks in the kitchen area, during coffee breaks and lunches, along with the academic and private discussions and updates. I hope to stay in touch with you!

- The ‘Kappe-group’, in particular, Helga, Rouven, and Anne-Siri. I really appreciate being included in the group. You helped me to get started and complete my thesis, as I got ‘pushed’ by our deadlines, through your encouragement, input and support. Our meetings were also great social happenings.

- The guys who included me in ‘Tuesday soccer’, namely, Jørn, Mads, Sigurd, Tony, Samuel, Rune, Ådne, Olav and the rest of you who joined over the years. You cheered me up, made me laugh and made my challenges go away for an hour! It was so much fun and I wish I could have kept on playing with you guys.

- My dear friends who have not given up on me, even though my social life suffered the last year. Special thanks to: Linda for always being able to meet me for coffee or

(6)

dinner when I had the time, for great talks and for sharing ups and downs; Anne B.

for taking me hiking and trying to arrange ‘play-dates’ with our kids; Monica N. for being so positive and supportive and taking such good care of Mads when our afternoon logistics did not work; my friends in Haugesund – Elisabeth, Kathrine, Diana, Ingun, Anne and Birgitte – for always inviting me and my family to social happenings, surprising me with great family lunches and believing in me.

- My parents-in-law, Norma and Per, for always being supportive and positive, as well as helping with the kids. I really appreciate having you nearby.

- My parents, Inger and Martin, for always having encouraged me to do the things I really wanted to and given me the possibilities to develop in several ways. You are always positive about helping with the kids as well, which I deeply appreciate.

- Last, but not least, I want to thank my family. Rune, you have always let me follow my dreams and been positive about my desires for developing. You never see problems, only opportunities, and encourage me to do so, too. You are great at surprising me, by arranging babysitters and taking me out when I really need a break.

Thank you for being so supportive and patient, believing in me and taking such good care of our boys, Mads and Tobias. You all constantly remind me of the important things in life, cheer me up and make me happy. You are simply the best! To Mads and Tobias: I promise you that I will have more time with you, be more patient, and that I will be part of our future holidays!

(7)

Abstract

Leisure sport is considered an important health promoting arena, where coaches play a crucial role, as they can help foster athletes’ motivation for long-term sports participation and reduce dropout. Youth sports coaches are often voluntary parents with little coaching experience. Therefore, previous research has emphasized the need for coach education programs, which help develop coaches’ sports-related skills and knowledge on how to enhance players’ motivation for sustained sports participation.

However, there are inconsistent findings as to whether coaches use the knowledge acquired from coach education programs. Thus, a need for studying the pedagogical approach employed in these programs has been considered necessary, in addition to gaining an understanding of how a program is implemented by educators and responded to by the attending coaches. Therefore, the current thesis aimed to explore the implementation of the Empowering Coaching™ program in Norway, which was delivered to grassroots coaches (GCs) in youth soccer as a one-day workshop by trained coach educators (CEs). The program was developed within the PAPA project and builds on two theories of motivation; namely, self-determination and

achievement goals. It aims to develop GCs’ knowledge of and competence in enhancing players’ motivation for sustained sport participation through creating motivational climates. Such climates rely on GCs being autonomy and relatedness- supportive, as well as task-focused, thereby emphasizing development rather than results.

The thesis comprises three papers, which try to capture program implementation on two levels: CEs’ program delivery to GCs, and GCs’ reported use of the program.

Paper I applied qualitative and quantitative data to explore the CEs’ program implementation, which in turn are aligned with a convergent parallel mixed methods design. The aim was to appraise the extent to which the CEs implemented the program with fidelity and to gain knowledge on what adaptations were made. The

(8)

findings revealed that the program was delivered with moderate to high fidelity and that many positive adaptations were made, which mostly aligned with the program’s aims and theoretical foundation. The GCs were found to appreciate the program.

Paper II applied a convergent parallel mixed methods design, including qualitative and quantitative data to explore both the CEs’ quality of delivery and the GCs’

responsiveness to the program in depth. Through observational methods, the CEs were found to demonstrate high-quality delivery, employing an empowering delivery style (i.e., modeling program principles) as well as, facilitating active engagement among the GCs. The GCs’ responsiveness was positive, indicated in terms of reporting to be highly satisfied with the CEs’ delivery quality and actively participating in the workshop.

Paper III also employed a convergent parallel mixed methods approach, by applying self-reported questionnaire data to explore the GCs’ perceptions and use of the program six months after the workshop, while interviews were conducted in order to gain knowledge on perceived implementation barriers. The findings showed that, although the GCs had a positive perception of the program and found it easy to apply, only a few GCs often or always used program principles in their coaching practices.

This seemed to relate to barriers perceived in terms of both individual and contextual factors that seemed to influence their use of the program, such as: lack of time, collaboration challenges with co-coaches, as well as lack of support from co-coaches and club leadership. Additionally, a need for follow-up sessions with CEs was reported.

The findings from this thesis show that, although the Empowering Coaching™

program was well implemented by the CEs and met with positive responsiveness by the GCs, the GCs found it challenging to use the program principles in practice due to several barriers. The barriers were mainly related to contextual aspects, such as lack of support from co-coaches and club officials, and facilitation strategies in terms of follow-up by program providers. This indicates that program implementation ought to

(9)

be explored through several levels of participants and within an ecological perspective in order to gain a more complete picture of how a program is implemented, as well as knowledge on how to enhance future program

implementation. Furthermore, a one-day workshop does not seem comprehensive enough to change the GCs’ coaching behavior.

(10)

List of publications

The thesis is based on the following three papers:

Paper I: Søvik ML, Larsen T, Tjomsland HE, Samdal O. (2016). Evaluating the implementation of the Empowering Coaching™ programme: balancing fidelity and adaptation. Health Education,116(3), 238-258. doi: 10.1108/HE- 07-2014-0077.

Paper II: Søvik ML, van Hoye A, Larsen T, Tjomsland HE, Samdal O.

Implementation research: a mixed-method study exploring delivery quality of and participant responsiveness to the Empowering Coaching™ workshop.

To be resubmitted to Research for All.

Paper III: Søvik ML, Larsen T, Tjomsland HE, Samdal O, Wold B (in press). Barriers in implementing coach education in grassroots youth football: a convergent parallel mixed methods study. International Sport Coaching Journal.

(11)

1. Introduction

1.1 Background

The leisure sport setting has been emphasized as an important health promoting arena, as a large number of youth voluntarily engages in sports, which may lead them to enjoy positive health effects [1]. However, it is well known that youth often drop out of sports during adolescents [2-5] and that preventing dropout and increasing youth’s level of sustained physical activity can be challenging [6]. In this regard, youth sports coaches have been considered to play a vital role [7, 8], given that they, as significant others, can act as role models for athletes, as well as foster players’

motivation in terms of long-term sports participation [9], provided that the players have fun and enjoy the sport in which they participate [10]. Thus, competent coaches may create supportive team climates, which enhance the athletes’ learning [11, 12]

and positive experiences of sports [6, 8, 9, 13, 14]. Therefore, coaches may be perceived as important implementation agents within the field of health promotion [13, 15, 16]. As most youth coaches are volunteers, and often parents, with no formal training, competence or experience as coaches, coach education is a critical target for coach development [14, 17, 18]. Thus, the implementation of coach education programs is important to study in order to assess coach learning and program impact [19, 20]. Furthermore, to gain knowledge on how to contribute to enhanced

competence among youth sports coaches, there is also a need to explore how coaches perceive the content and delivery of coach education programs [14, 19, 21].

Therefore, the overall aim of the current thesis is to add new knowledge to the research on implementation from the field of leisure sports and coach education. The current thesis comprises an evaluation of the implementation of a community-based coach education program, which was implemented in Norway, as part of a

collaborative project in five European countries. The program is understood to be a prevention and health promotion intervention, such that implementation will, in the

(14)

current thesis, be described based on previous research on the implementation of prevention and health-promotion interventions.

1.2 Defining concepts

Within the field of implementation research, there is an inconsistent use of terms for some of the concepts applied [e.g., 22, 23-27]. In line with the terms applied by Durlak and DuPre [25] and Berkel and colleagues [22], the current thesis will apply the terms implementation when referring to the processes related to program delivery, and fidelity when referring to how the program was implemented in comparison with the intended program delivery. A broader description of the terms will be provided in chapter 2.

1.3 Why study program implementation?

As implementation of programs in real-world settings implies complex processes, which are often conducted by, and as an interaction between, several levels of participants [22], implementation seldom develops as intended [25, 28, 29].

Therefore, there is a need to study this process in order to understand whether a program was implemented as intended or not [24, 25, 28, 30-34]. Implementation, which is completely in accordance with program protocols (i.e., 100% fidelity), is not expected, as some modifications occur [25, 35]. Further, the necessary level of implementation fidelity in order to achieve the intended program effects has not yet been revealed, while there is an ongoing debate on what constitutes and how to measure the core elements of a program [25].

Knowledge of what has been implemented and the quality of delivery, as well as what aspects of a program were left out during implementation, may contribute to an understanding of a program’s effectiveness or lack thereof [25, 30, 31]. Thus, studying program implementation can provide information on how and why a program works in a real-life context and, in turn, an understanding of the program

(15)

outcomes [25, 36-40]. Therefore, studying program implementation may help avoid conducting a Type III error, which mistakenly implies that the program did not obtain the expected effect, when in reality this was due to poor implementation, rather than the quality of the program itself [25, 31]. Monitoring implementation may contribute to future improvements of programs, which are necessary to increase the likelihood of reaching intended outcomes [25, 28, 30, 33, 41]. Furthermore, monitoring and feedback on implementation efforts may also lead to better replications and sustainability of a program [29, 42].

Dane and Schneider [24] also emphasize that implementation may be challenged by contextual factors, such as training and barriers. Studying the quality of training of the participants who are expected to use the program can contribute to an

understanding of why a program achieves different outcomes when implemented in different contexts [30, 37]. Furthermore, due to the potential influence of the contexts (e.g., organizational capacity, support and resources) on program implementation, studying barriers for implementation in natural settings has been recommended [32, 43]. Successful implementation of evidence-based programs relies on a supportive context, competent and skilled providers (e.g., CEs), and receptive participants (i.e., the ones who receive training and are supposed to apply a program in practice) [44].

To capture the complex process of implementation knowledge on how to document implementation, is important, in terms of both when and which data to collect, as well as who can provide the relevant data [44, 45].

1.3.1 Why study implementation in coach education?

Previous research suggests that the effects of a coach education program on players’

motivation, perceptions of climate and continued participation in sports are mediated through changes in the coaches’ behavior [9]. However, there are inconsistent findings regarding the coaches’ use of program content after attending coach education [19]. Some suggests that coaches who have attended coach education programs apply more positive behaviors than those who have not [14]. Others suggest

(16)

that knowledge from coach education programs is not applied in practice because the knowledge has not been internalized [46, 47], especially on short-term education programs [14, 18, 19] or because programs are not implemented as intended [20].

Therefore, recent research has emphasized the importance of looking into the pedagogical approach applied when implementing coach education programs, together with the educators’ skills and qualities [21, 48, 49]: all factors that may influence effective quality implementation. Up to now, there has been a focus on the participants’ (e.g., coaches’) fidelity (delivery or use of a program) in their practical contexts. However, few studies have assessed the educators’ fidelity of

implementation and whether programs and delivery methods applied were valued by the attending coaches [14, 19]. These are all important aspects that may lead to increased understanding of how the implementation of coach education programs can be improved.

1.4 Description of the Empowering Coaching™ program

To understand the relevance of the theoretical perspectives of implementation and how the implementation of the Empowering Coaching™ program has been assessed, an elaboration of the program is required.

The Empowering Coaching™ program was developed within the EU-funded PAPA project in 2009/10 [50]. Researchers from the University of Birmingham, UK, developed the program, which was then translated and culturally adapted by researchers to be delivered to youth soccer coaches in France, Greece, Norway and Spain, in addition to England, in 2011. The PAPA project, which lasted from 2011 to 2014, entailed two phases: a) training of CEs by a PAPA team (i.e., in Norway, researchers from the Norwegian School of Sport Sciences) and b) training of GCs by CEs.

(17)

1.4.1 Purpose

The Empowering Coaching™ program was developed [51] as a response to the increased levels of inactivity and being overweight among European youth [1], as well as research suggesting that GCs can be important health promotion agents [13, 15, 16]. The program aims to enhance GCs’ competence with regard to creating a motivational (i.e., enjoyable, engaging and supportive) team climate within youth soccer in order to enhance the players’ motivation for and likelihood of long-term engagement in sport [51]. It aims to teach GCs specific strategies to apply in their coaching practices, to foster a more empowering and less disempowering coaching behavior, after having attended the Empowering Coaching™ workshop. An empowering behavior implies an autonomy- and relatedness-supportive, as well as task-focused approach to coaching, whereas a disempowering behavior reflects a controlling and ego-focused competence approach [51-53].

1.4.2 Content

The program’s theoretical foundation is based on self-determination theory (SDT) and achievement goals theory (AGT) [51]. There is an extensive body of literature supporting the tenets of SDT [e.g., 7, 14, 54-56] and AGT [e.g., 57] in the sports realm, due to their positive impact on the players’ intrinsic motivation for sport and their perceived well-being, through the satisfaction of their basic needs of autonomy, relatedness and competence.

The theoretical foundation and the program principles are presented in the first part of the workshop. From AGT [58], the Empowering Coaching™ program particularly draws on the distinction between task involvement and ego involvement related to the development of competence. Task-involving goals, which refer to an emphasis on positive development through learning and mastering tasks, are based on one’s perceptions of effort (i.e., doing one’s best) and self-referenced development. On the contrary, ego-involving goals, which refer to one’s attempts of demonstrating skills

(18)

by outperforming others (i.e., being superior with minimal effort), are appraised in reference to others’ performances. Hence, one’s perception of competence is related to a distinction between mastery and performance [59], which implies that one’s approach to competence influences how we think, feel and act while engaged in activities. Furthermore, within sports, the environment is considered to influence athletes’ perception of their own competence, through significant others’ (e.g., coaches) attitudes and behaviors. For instance, a task-involving climate seems to positively influence athletes’ enjoyment and satisfaction with sports, whereas an ego- involving climate may produce greater anxiety among the players and increase the number of athletes who drop out of their sport [60].

SDT [61] emphasizes the importance of satisfying an individual’s needs for autonomy, relatedness and competence in order to achieve intrinsic or quality motivation, as intrinsic motivation seems to positively influence one’s learning, performance and well-being. Autonomous engagement, feeling competent and having a socially supportive and empowering environment are important in order to be intrinsically motivated. The satisfaction of these needs can, however, be positively or negatively influenced by significant others, in that they may be controlling in terms of providing feedback that results in a feeling of incompetence and, hence, is not

perceived as supportive. The thwarting of needs may decrease an individual’s intrinsic motivation and, in turn, the enjoyment and interest in an activity. This may be of great importance in grassroots sports, as GCs’ behavior may influence athletes’

intrinsic motivation towards sustained sports participation.

The main focus of the program is on intrinsic versus extrinsic motivation, the ABC principles comprising Autonomy support (e.g., the players are ‘given a voice and a choice’), Belonging or relatedness support (e.g., all players feel respected and part of the team), and task- versus ego-oriented Competence (e.g., doing their best, not being the best). The rest of the workshop concerns how the GCs can develop a motivational climate through the application of seven strategies, which seek to enhance the

(19)

players’ intrinsic motivation, as well as their satisfaction regarding the basic psychological needs of autonomy, relatedness and competence. CLIMATE is an acronym for the seven strategies, which draw on aspects of both SDT and AGT, and focus on: Cooperative contribution, Learning emphasized, Intrinsic focus, Mastering the task, Authority with autonomy, Taking perspective, and Evaluation [51].

1.4.3 Coach educator training

Trying to ensure a consistent workshop delivery (i.e., content and delivery style) in the main trial, PAPA researchers from the Norwegian School of Sport Sciences provided the CEs with 34 hours of pre-implementation training. This comprised an introduction to the theoretical background of the Empowering Coaching™ program, CEs observing the PAPA researchers when delivering a ‘training’ workshop to pilot GCs, as well as CEs themselves delivering two training workshops to other pilot GCs. Furthermore, after having delivered their training workshops the CEs’ received individual tutoring from the PAPA researchers. Figure 1 illustrates the process of program implementation.

Figure 1. Overview of program delivery, including pre-implementation training.

1.4.4 Program delivery

The CEs were expected to deliver two one-day workshops each before the 2011 soccer season started. The workshop is considered a non-formal coach education approach, due to the time format and the fact that it does not result in a certification [19, 62]. In total, 17 workshops were implemented, as one CE delivered three workshops, while another delivered four. The ‘pre-packaged’ workshop was delivered using a delivery protocol, a PowerPoint slideshow and a workbook.

Individual and group tasks, in addition to video clips illustrating empowering and

researchers (pre- implementation

training)

coach educators (workshop implementation)

grassroots coaches (implementation

in practice)

players (expected main

effects)

(20)

disempowering coaching behaviors, were applied. The group tasks comprised discussions on relevant program topics and a game based on the ABC principles, which addressed Autonomy support, Belongingness (i.e., relatedness support) and Competence, through GCs discussing if exemplified coaching behaviors were perceived to be empowering or disempowering. The individual tasks comprised writing down notes concerning one’s own (i.e., GCs’) coaching philosophy, as well as specific plans for how to apply program principles in the next training session and match.

The delivery protocol was developed and applied in order to ensure that the

workshops were delivered as intended by the program developers. This implies that the CEs were expected to apply the program principles when conveying the

workshop, meaning that they should demonstrate an empowering behavior.

Furthermore, an interactive program delivery was emphasized, where the CEs were expected to encourage the GCs to provide their own examples, experiences and reflections concerning their coaching behavior and attitudes regarding the program principles and strategies [51]. Eight e-learning modules were released close to the end of the soccer season in Norway, due to translational procedures taking time, instead of at the beginning of the season.

(21)

2. Theoretical framework

Implementation is a complex process, which often includes several levels, dimensions and participants [23, 28, 45]. Adding to the complexity, there are several definitions relating to implementation, including how and what to measure in order to capture this process [e.g., 22, 24, 25, 28, 32, 41, 63]. To break down the complexity, this section will address what is meant by implementation, as well as how and what to study in implementation research, followed by an outline of different dimensions and factors related to the implementation process, including organizational factors, individual factors and program characteristics. This section ends with a presentation of a model based on Hasson [26], which shows the relationships between organizing factors, individual factors and program characteristics.

2.1 What is implementation?

During previous decades, implementation research was emphasized as an important aspect of effectiveness studies [25, 28, 30, 32, 41, 64], while several, somewhat inconsistent, terms and definitions have been suggested to describe implementation [e.g., 22, 24, 25, 28, 32, 63]. In general, implementation concerns the way in which a program is delivered (i.e., what it consists of when it is implemented), compared to how the program was designed to be delivered [22, 25, 31, 41]. This implies that the program content or components, the activities delivered and the methods applied need to be evaluated [31, 41, 44].

Dusenbury and colleagues [64] claim that no clear or unique definition exists for the multidimensional concept of implementation and encourage future studies to clearly define the terms applied, as well as be specific with regard to which dimensions are being studied. Implementation dimensions are also referred to as the ‘black box’ of implementation [26, 44]; Durlak and DuPre’s [25] conceptualization of

implementation comprises the following eight dimensions: fidelity (i.e., the extent to

(22)

which program components were delivered as intended), dosage (i.e., the amount of program delivered), the quality of delivery (i.e., how well a program was delivered), participant responsiveness (i.e., satisfaction and engagement among participants), program differentiation (i.e., the uniqueness of the program compared to similar programs), the monitoring of control or comparison groups, the program reach (i.e., rates and representativeness of participants), and adaptations (i.e., changes and modifications to the program).

Previous research has not reached any consensus about which, and how many, dimensions are necessary to measure when studying the implementation of prevention and health promotion programs [25, 28]. Most studies seem to measure only one of the dimensions [22, 23], usually fidelity or dosage [25, 64, 65]. However, the general recommendations found in the literature on implementation underlines the importance of studying several dimensions in order to gain comprehensive

knowledge of how a program is implemented [23, 25, 26, 28, 66]. In line with previous research suggesting relations between fidelity, quality of delivery, adaptations and participant responsiveness [22, 23, 26], these dimensions will be explored in the current thesis concerning CEs’ program delivery in depth.

Additionally, contextual factors related to GCs’ program use will be explored, as context has been considered to influence program implementation [25, 26, 67-70].

Program implementation usually involves several participants who influence how a program is implemented in a real-world setting [22]. These participants can also represent several levels including persons providing training (e.g., researchers), program providers (e.g., CEs) delivering or implementing the program, and

participants receiving the program (e.g., GCs) [22, 30]. Thus, ideally and as intended in the current thesis, implementation studies should involve measures of dimensions, which reflect the aspects that relate to several levels of participants.

(23)

2.2 Factors and dimensions related to program implementation

According to previous research [25, 32, 33, 71], successful program implementation is influenced by several contextual factors, which impact on an organization’s (e.g., soccer club’s) capacity to implement and maintain a program in practice.

Furthermore, successful implementation implies that a program is implemented with high quality [64]. In a systematic review [25], 23 contextual factors were considered as highly relevant for successful implementation. Some of these factors seem to interact within or across different implementation levels, as well as relate to different dimensions and stages of the implementation process [25, 32, 33, 72]. Hence, successful implementation needs to be considered in terms of an ecological approach [25, 73], including both individual and contextual factors [25, 32, 33, 66, 72].

Therefore, factors that appear to influence implementation will, in the following, be presented in relation to: a) an organizational level, b) an individual level, representing both program providers (e.g., CEs) and the participants receiving the program (e.g., GCs), and c) the characteristics of the program, including strategies to facilitate program implementation.

2.2.1 Organizational level

Context

Factors described as influential at the organizational level of an implementation process [25, 32, 33] seem to overlap with aspects, which are often reported as contextual factors influencing program implementation [26, 38, 68, 70, 74]. Hasson [26] operationalizes context as factors related to levels of politics, finances,

organizations, and groups of participants. Such factors are considered to influence program implementation, and hence, outcomes, as programs are always implemented within a given context [25, 26, 67-70].

(24)

Previous research suggests that an organization’s (e.g., soccer club’s) readiness to implement a program, along with the effectiveness with which a program is implemented, is influenced by the organization’s politics [25, 32]. This relates to whether or not the program seems to align, for instance, with a soccer club’s goals, needs and current practices. The same has been suggested in terms of an

organization’s culture (i.e., how things are done) [74] and history [75], which may affect why and how a program is implemented. This implies that the social norms and processes within a context, as well as the participants (e.g., CEs, GCs and club officials), will influence program implementation [69, 75]. Thus, if a program is implemented across multiple sites (e.g., different soccer clubs), the context is likely to have had an impact on how the program was implemented, if desired outcomes were achieved [31, 40, 68, 70, 75, 76], as well as how the different participants (e.g., GCs) perceived the program and its delivery [76].

Program adoption should be rooted at the leader level, as adoption often relates to a club’s politics and, thus, the effort that will be invested when implementing a program. Adoption may be reflected through strong and supportive leadership, active leadership participation in program training and implementation, and an emphasis on building a positive working climate [25, 32, 33]. These behaviors are indications of an organization’s capacity to implement a new program [25, 73], as supportive leadership is suggested to enhance participants’ sustained motivation towards engaging in program implementation. In this regard, prioritizing resources, such as time and money, in order to facilitate successful program implementation is important [25, 31-33].

Leadership’s willingness to meet and solve occurring challenges during program implementation is crucial. It may also increase staff (e.g., GCs’) stability, which seems to result in more effective program implementation than if there is a high staff turnover [32, 33]. Involving the staff who are expected to use a program when making decisions, as well as enhancing a shared vision regarding a program’s aims,

(25)

has also been considered important in terms of sustained engagement, motivation and staff willingness to change their practice [25, 32, 33, 77]. In addition to supportive leadership, implementation seems to be enhanced by having one or more program champions at the implementation site, preferably rooted at the level of administration [25, 32].

The contextual factors are, in the current thesis, important aspects related to program implementation at the GC level, as different factors may influence the GCs’ program use.

2.2.2 Individual level

At the individual level, implementation dimensions related to both providers (i.e., fidelity, adaptations and quality of delivery) and participants (i.e., participant responsiveness) may influence program implementation [22, 23, 26]. The providers are defined as the staff who implement a program [25], such as CEs, whereas participants are the persons receiving a program [22], who, in the current thesis, are the GCs. Provider characteristics that seem to be essential for successful

implementation concern the provider’s perception of the program [25, 32], which may influence their attitudes towards it and their motivation to fully implement it [32, 33]. The extent to which a program is implemented as intended is further related to the provider’s judgments about, and confidence in, having the skills required for implementing the program [25, 32, 33].

Fidelity

Fidelity, often also called adherence [23, 24, 26, 28], to the content delivered is, in the current thesis, referred to as the level at which the core program elements were delivered as planned by the program developers [24, 25, 28, 31] and as outlined in the delivery protocol. To assess fidelity of delivery, a specific understanding about the program is required [45] in order to gain detailed knowledge about which program elements were delivered as intended and which were not, instead of only appraising

(26)

the overall fidelity of program delivery [25, 31, 45, 78]. This knowledge may later help identify which program elements are critical in terms of implementing with high fidelity in order to achieve the desired outcomes [25, 28]. Previous reviews of the implementation of prevention and health promotion interventions found that high levels of fidelity seem to positively influence program outcomes [24, 25, 28, 30, 31], whereas low fidelity levels may result in poorer outcomes [24, 28, 31, 33]. As suggested by Hasson [26], this thesis refers to the fidelity of the CEs’ program delivery as a measure of coverage (i.e., number of key points delivered), as well as a description of frequency (i.e., whether the program was delivered as often as intended) and duration (i.e., whether the workshop lasted for the intended amount of time), in addition to measuring fidelity to the content of the core elements.

Adaptations

An aspect often related to fidelity is the level of adaptations that occur during program delivery [23, 24, 28, 35, 42, 72]. This is sometimes considered to be a separate dimension of implementation [22, 25]; this is also the case in the current thesis. Adaptations refer to positive or negative additions or modifications of program content, processes or methods related to the implementation of a program [22, 35], which commonly occur to some extent during program implementation [25, 29, 30, 35]. The adaptations most often reported are modifications to program content, procedures or dosage [35, 42], which seem to occur even when a program protocol is employed [35]. In the current thesis, adaptations are explored in terms of content added, skipped or modified by the CEs when delivering the workshop to the GCs, along with whether they are considered to be positive or negative adaptations.

Adaptations that align with a program’s underlying theoretical foundation are therefore considered to be positive adaptations. Negative adaptations are considered to be those that do not align with a program’s foundation or goals, whereas neutral adaptations neither align nor conflict with the goals and content [35, 45]. In school- based implementation research, correlations between high fidelity and positive

(27)

adaptations, together with low fidelity and negative adaptations, have been suggested.

Furthermore, the most experienced teachers are also found to make most positive adaptations [64]. Some researchers consider adaptations, in general, to result in poor program implementation, which lead to low levels of program effectiveness [42].

Others consider adaptations to local settings to enhance the possibilities of achieving high levels of adoption, quality program implementation [25, 28] and high levels of program effectiveness [79]. Thus, there are inconsistencies in the literature regarding the impact of adaptations on program outcomes, such that further research is needed to capture when, where and to what extent adaptations positively influence program outcomes, or otherwise [28, 30, 33, 35].

Quality of delivery

Quality of delivery seems to be defined in several ways [28, 45]; in general, however, it relates to how well a program is delivered [23, 25, 31, 32]. This is an important aspect of implementation, as only replicating program content is not an indication of successful implementation [30]. This implies that quality of delivery concerns other aspects than content delivery, such as the educator’s enthusiasm [22, 24], knowledge and understanding of the program theories. The latter may reflect the educator’s skills of guidance, responding to and discussing contributions from the participants in compliance with the program’s underlying theoretical assumptions and goals [24, 25, 28, 65], which comprise one of the aspects related to CEs’ program delivery in the current thesis.

An educator’s competence and skills at teaching, as well as employing the intended methods and procedures during program implementation, are considered critical to ensuring quality program delivery [22, 32, 43, 80]. The use of interactive methods and processes has particularly been emphasized, as this may facilitate participant engagement, which has been considered to increase learning and positively influence program outcomes [22, 25, 81, 82]. The facilitation of active engagement may also rely on the educator’s abilities to create safe environments where all participants feel

(28)

included [80, 83]. Thus, the CEs’ facilitation of active engagement and their creation of motivational climates will also be explored in the current thesis. Although studying delivery quality is recommended [64, 82], measuring this dimension is a difficult and time-consuming process, which ought to be studied through observational methods.

Therefore, this dimension has rarely been explored [45, 84]. However, a recent study by Pettigrew and colleagues [82] found that the delivery quality of a drug prevention intervention, which was delivered in a school setting, influenced the program outcomes in terms of less substance use when the intervention was well delivered, compared to when it was poorly implemented.

Participant responsiveness

Different aspects have been suggested to reflect participant responsiveness [24, 28, 83, 85]. Recently, the following two have particularly been emphasized: 1) behavioral aspects and 2) subjective aspects [22, 83, 86]. Active participation in the program and its activities [22, 24, 28, 30, 83, 85, 86], program attendance, home practice

completion and participants’ self-reported use of skills learned on the program have been suggested as reflecting the behavioral dimension of responsiveness [22, 83, 86].

The subjective dimension comprises measures of participants’ satisfaction with a program [22, 24, 30] related to whether the program skills were perceived to be useful, as well as how the group environment was perceived when the program was delivered [86].

There seems to be inconsistent findings regarding whether participant responsiveness, and what aspects thereof, influence outcomes [22, 86]. For instance, in their review, Berkel and colleagues [22] refer to several studies that found outcomes to be associated with attendance, home practice completion, active participation and satisfaction with program, whereas Schoenfelder and colleagues [86] found none of these associations. However, previous studies have indicated that research on participant responsiveness is rare [23, 31]. Thus, the current thesis will explore the GCs’ responsiveness to the Empowering Coaching™ program in terms of their active

(29)

participation in the workshop and their satisfaction with the program, as well as the CEs’ delivery quality.

2.2.3 Program characteristics

Complexity

In school-based research, program complexity is suggested to be related to whether or not a program is described in detail, as well as includes well-known methods and materials that are easy to use and perceived to be attractive by the providers [26, 33].

As such, an intervention’s complexity (i.e., the comprehensiveness of a program) seems to influence program implementation [23, 28]. That is, less complex programs are more likely to be fully implemented than complex programs, such that simpler programs are usually more easily adopted and result in more successful program outcomes [29]. Therefore, a program’s complexity may act as a moderator for fidelity [23, 26]. Furthermore, a program’s complexity has been considered to influence its flexibility and thus the likelihood of it being implemented with fidelity in real-world settings. This relates to the possibilities of adapting flexible programs to local needs and contexts, suggesting that flexible programs are more attractive for real-world implementation [25].

The program that has been evaluated in the current thesis is a pre-packaged program.

The program providers (i.e., CEs) were trained to implement the program according to a detailed delivery protocol, which they were expected to apply when delivering the program. Furthermore, the complexity of this program relates to the

implementation process, where the program is delivered at two levels (i.e., CEs and GCs) and expected to produce outcomes at a third level (i.e., players) [51].

Facilitation strategies

Program complexity also influences the need for facilitation strategies [23]. This implies that complex programs may require more supportive approaches to obtain high levels of fidelity and quality than more simple programs do. Hence, facilitation

(30)

strategies are suggested to moderate the level of fidelity with which a program is implemented [23, 26]. Several studies refer to the benefits of providing educators with pre-implementation training, detailed delivery manuals or guidelines, ongoing support or supervision, and feedback on their own delivery during program implementation [23, 25, 30-32, 72, 77]. These facilitation strategies may enhance program implementation and use [73], while the provision of educator training and delivery protocols may ensure a consistent program delivery among the educators.

Furthermore, it is suggested that effectiveness, fidelity, quality of delivery and sustained use are enhanced through sufficient facilitation strategies [23, 24, 31, 32, 77]. In particular, the combination of training and technical assistance has been considered to enhance implementation quality and learning outcomes [27].

Pre-implementation training, most often conducted in the form of face-to-face workshops [33], is recommended in order to facilitate active forms of learning for program providers so that the skills required for carrying out the program are more effectively developed. Additionally, the provider’s motivation, confidence and expectations related to program implementation should be focused on [25]. Training and technical assistance, in terms of different forms of follow-up support (e.g., booster sessions and telephone- or e-mail contact) for educators and participants, may help them deal with challenges and barriers that may occur throughout program implementation, as well as encourage sustained motivation, engagement and

commitment [25, 32, 33]. Feedback on program implementation involving low levels of fidelity may also enhance the level of fidelity in future program implementation [23]. The need for facilitation strategies seems to relate to the educators’ and participants’ experiences, implying that inexperienced educators have greater needs for training, technical assistance and feedback than more experienced ones [64].

In the current thesis, the CEs were supported by pre-implementation training, including practical delivery and tutorial feedback, whereas the GCs received a one-

(31)

day workshop and a workbook, as well as being expected to follow the e-learning modules that were released towards the end of the season.

2.3 Conceptual framework for program implementation

Recent research [22, 23, 26] has suggested associations between some of the implementation dimensions. Training and facilitation strategies may impact the educators’ quality of program delivery, for instance, in terms of a more enthusiastic program delivery. Quality of delivery may further affect the participants’

responsiveness to the program, as a program delivered with quality may enhance interest and engagement among the participants, along with their acceptance of or satisfaction with the program [22, 23]. Quality of delivery and participant

responsiveness may, in turn, positively influence the fidelity with which a program is implemented [23, 26]. Another relation between these dimensions is suggested by Berkel and colleagues [22], who consider quality of delivery as a mediator between fidelity and program outcomes, and participant responsiveness to mediate the effect of quality of delivery on program outcomes. To illustrate possible relations among different implementation dimensions and program outcomes, some conceptual frameworks and models for program implementation have been suggested [22, 23, 26].

The dimensions described in the previous section of the thesis may be illustrated as a model, aiming to enhance the understanding of the black box of implementation and how to measure relevant implementation dimensions [26]. Figure 2 is inspired by Hasson’s [26] modified conceptual framework for implementation fidelity, which builds upon and extends Carrol and colleagues’ [23] framework. However, as the implementation of the Empowering Coaching™ program involves two levels (i.e., CEs and GCs), through which the outcomes at the player level are expected to be mediated, the conceptual framework in Figure 2 has been extended to include the implementation levels of both CEs and GCs. Thus, a ‘double-looped’ model is

(32)

illustrated, based on the dimensions explored in the current thesis. Both Hasson [26]

and Carrol and colleagues [23] suggest relations among the implementation dimensions, such that they recommend studying all dimensions included in their frameworks. Ideally, all dimensions should have been explored at both levels, but this was not within the scope of the current thesis, due to the comprehensiveness of such an exploration.

Figure 2. A conceptual framework illustrating the implementation of the Empowering Coaching™ program at two levels, with possible moderators explored and described (based on Hasson [26]).

In Hasson’s [26] framework, the measure of implementation reflects to what extent the program providers deliver the program as intended through studying the fidelity of program delivery (e.g., at both CE and GC level). Furthermore, the following dimensions are suggested as possible moderators, which may interact with each other and, positively or negatively, influence the level of fidelity achieved: participant responsiveness, quality of delivery, intervention complexity, facilitation strategies and context [26]. The model illustrated in Figure 2 also includes a separate dimension of adaptations, based on Durlak and DuPre’s [25] and Berkel and colleagues’ [22]

(33)

conceptualization of implementation. Furthermore, in Figure 2, the possible

moderators have been divided into organizational dimensions, individual dimensions, and dimensions related to program characteristics.

As illustrated in Figure 2, the relationship between a program and its outcomes (e.g., at both GC and player level) is suggested to be influenced by how the program is implemented, in particular, by the extent to which program providers adhere to the content, activities and processes, which are expected to be implemented.

Furthermore, organizational, individual and program characteristic dimensions have been suggested to moderate the fidelity of program delivery [23, 26]. An analysis of which program components are most essential when seeking to achieve the expected outcomes are further emphasized once an assessment of the main outcomes has been conducted [23, 26, 84]. This aligns with an assessment of program differentiation [23-25], which may enhance knowledge on which are the most critical components to implement with fidelity in order to achieve the desired effect [23, 25, 26].

When applied to a study of the implementation of a coach education program, the conceptual framework for implementation can contribute to greater knowledge about the extent to which CEs adhere to the delivery protocol when implementing the program content (i.e., fidelity) and the GCs’ program use. Furthermore, participant responsiveness may reflect the participants’ (i.e., GCs’) active participation in the workshop, as well as their satisfaction with the program content and the program delivery. At level 2, participant responsiveness can reveal knowledge about how co- coaches and parents respond to GCs’ use of the program. Quality of delivery measures may reflect the CEs’ skills and competence in which they deliver the program, as well as their abilities to facilitate engagement among the participants during program delivery. An appraisal of facilitation strategies may reveal whether training of CEs, prior to program delivery, and their use of delivery protocols, was successful. Additionally, it may provide knowledge about relevant strategies to support the GCs’ use of the program in practice. The need for facilitation strategies

(34)

can be seen in relation to an evaluation of the program’s complexity, which also relates to the use of delivery protocols. In addition, the context’s influence on GCs’

program implementation can reveal factors, which seem to enhance or inhibit their use of the program. In the current thesis, the GC outcomes refer to what the GCs learned from the workshop. Thus, player outcomes of the Empowering Coaching™

program are supposed to be mediated through program implementation by both CEs and GCs (i.e., their program use), which, ideally, implies a need for exploring all these implementation dimensions at both the CE level and the GC level.

2.4 Summary and research objectives

To sum up, the importance of studying the implementation of health promotion and prevention programs has been emphasized in order to understand how and why a program works in real life. Since several factors, along with both educators and participants, seem to influence program implementation, several dimensions of implementation need to be studied to obtain a comprehensive picture of how a program is implemented. Therefore, the current thesis aims to add knowledge to the field of implementation research through exploring several dimensions related to the implementation of a coach education program. Furthermore, the thesis aims to examine the feasibility of combining measures, in terms of video recordings, qualitative interviews and questionnaire data, in order to highlight how the Empowering Coaching™ program unfolds in a practical setting. In addition, the generated knowledge may shed light on how to improve the implementation of the program in order to enhance positive program outcomes in future implementations.

The objectives for this thesis were:

1) To explore the CEs’ adherence to the program protocol and the adaptations made when implementing the program (Paper I).

2) To explore CEs’ quality of program delivery (Paper II).

(35)

3) To explore the participant responsiveness in terms of GCs’ satisfaction with the program content and program delivery, as well as their active participation in the workshop (Papers I and II).

4) To explore GCs’ use of the program and perceived implementation barriers when applying the program to their coaching practice (Paper III).

(36)

3. Materials and methods

Ideally, multiple or mixed methods may capture different aspects and dimensions relevant for an in-depth study of program implementation [22, 25, 35, 39, 40, 76, 77].

This approach may also capture the providers’ and participants’ perceptions of program implementation, which further validate data from one source against data from another source [25, 28, 35, 45]. Thus, the current thesis comprises mixed methods in order to explore the implementation of the Empowering Coaching™

program in the Norwegian arm of the PAPA project in depth.

In the following, the methodological approaches applied in the three studies comprising this thesis will be presented.

3.1 Case study design

Employing a case study design provides the possibility of exploring different aspects of a situation in order to gain a general impression of the situation through

triangulating multiple sources of data, such as observations, interviews and questionnaire data, as well as conducting different analyses and triangulating the results reported [87-89]. The triangulation of several data sources, which has been considered to strengthen the quality of the data collected in order to answer the research questions, has been reported as one of the main advantages when employing case studies [89]. In the current thesis, this implies the possibility of assessing the CEs’ program delivery from an observational perspective, as well as including measures of the participating GCs’ perceptions of the program and the CEs’ program delivery. Furthermore, a multiple case study approach also allows for a comparison between the different cases, (i.e., the seven CEs) [87, 89] in order to explore possible differences related to their program delivery.

In the current thesis, a case is defined as one CE (level 1), the GCs they trained (level 2) and who should use the program principles when coaching their players (level 3)

(37)

(see Figure 3). The current thesis has, however, only examined research questions related to the implementation process at levels 1 and 2, while the in-depth interviews at level 2 were only conducted for three out of the seven cases (i.e., 12 of the GCs).

Sampling more than one case provided the opportunity to both vertically and horizontally compare and contrast the implementation process across and within levels [89].

Figure 3. Case study design, illustrating levels and data sources applied at level 1 and 2 in the current thesis.

*Interviews conducted with the GCs included in three cases (3x4 GCs=12 GCs).

3.1.1 Mixed methods

According to Creswell [87], the methods applied when conducting research are guided by the study’s research questions. To gain in-depth knowledge on different aspects of program implementation, mixed methods were applied in the current thesis, as previous research has considered this approach to be sufficient in order to understand how a program is implemented in a real-world setting [90-92], as well as gain knowledge on coaches’ perceptions of a program’s content and delivery [93]. As suggested by Johnson and colleagues [91], this approach may provide knowledge that elaborates, illustrates and indicates convergence or contradictions to a greater extent than if applying only one method for studying each dimension.

Coach educator

(video recordings)

Grassroots coach (interview*, questionnaire)

Players

Players Level 1

Level 2

Level 3

(38)

Combining qualitative and quantitative data implies a reliance on different paradigms and, hence, different ontologies (the nature of reality), epistemologies (how reality is known) and methodologies [87, 94, 95]. Thus, a pragmatic approach [91, 95] was considered appropriate to the current thesis, as it allows for the collection of data from several sources based on what works to solve a problem [95]. By employing different methods or tools, as well as conducting different kinds of data analyses, mixed methods research can capture both contextual in-depth qualitative data (e.g., interaction data) and quantitative data illustrating a phenomenon through numbers, which represent a larger sample [94, 95].

The manner in which mixed methods are applied in the papers included in the current thesis aligns with a convergent parallel mixed methods design [95]. The qualitative and quantitative data were collected at the same time in order to highlight several aspects of a topic. The two different data approaches were given approximately the same weight, in that they were equally important in answering the research questions.

Furthermore, the qualitative and quantitative data were analyzed separately, then merged in the discussion and interpretation of the findings. In Papers I and II video recordings were analyzed both qualitatively and quantitatively, while the

questionnaire data were collected at the same time as the observational data. In Paper III, the interviews were conducted shortly after the collection of questionnaire data, but before the questionnaire data were analyzed. Thus, the manner in which the data were collected and analyzed aligns with the convergent parallel design, despite some more attention given to the qualitative data, especially in Paper III.

3.2 Study context

In Norway, the Empowering Coaching™ program was implemented in three regions:

a northern region, an eastern region and a western region. The Empowering Coaching™ workshop was delivered to 17 groups of participating GCs, with the number of GCs participating in each workshop ranging from two to 17. The

(39)

workshops were delivered in local soccer club houses in the three regions. Specific soccer clubs were sampled as clubs organizing the practicalities related to the workshop delivery (see section 3.3.2).

The context differed in each workshop, regarding both the persons involved and the location. When conveying the workshops, the CEs recorded their own delivery on video by locating the camera in a fixed position in order to film themselves and their PowerPoint slideshow. This was agreed upon within the Norwegian PAPA research teams. One of the reasons behind this decision was to make sure that members of the research teams did not influence the CEs’ program delivery, which would have been possible had they been present during the workshops. Another reason was more of a practical concern, that is, to avoid spending resources on traveling to the 17

workshops in three different regions. However, having the CEs record the workshops themselves resulted in some negative consequences. Some placed the camera in a poor position, such that the PowerPoint slideshow was not captured in the recordings.

The sound quality was sometimes poor (i.e., because of the distance from the camera to the CE), while sometimes none of the GCs was captured in the recordings at all.

3.3 Study participants and sampling procedures

Papers I and II were based on data from both CEs and GCs, whereas Paper III was based on data from GCs only. The sampling procedures applied in the Norwegian arm of the PAPA project followed the same criteria as in the other countries involved in the project. Some flexibility was allowed, however, in order to obtain the intended size of the GC sample [50].

3.3.1 Sampling of coach educators

In the Norwegian arm of the PAPA project, the recruitment of expert coaches to be trained as CEs was conducted in collaboration with the Norwegian Soccer (Football) Association (NFA) through purposeful sampling [96]. As the intervention was to be