1
Master’s Thesis
Learning from training with the use of technical tools
A thesis within Societal Safety, by Camilla Gjerde Ove
14.06.2013
2
Summary
This is a study of learning from training with the use of technical tools. This paper is written as part of the BRIDGE project. The aim of the BRIDGE project is to develop technical and organisational
solutions that can improve crisis and emergency management in the EU member states. The BRIDGE project consists of several Concept Cases. The focus in this thesis has been on a Concept Case called Training. A Norwegian company called Crisis Training AS (CTAS) is responsible for the Concept Case Training. CTAS have created a training solution called FRITS which consist of three technological tools for use during training. Two of the tools in this package have been studied, AKKA and MeTracker. The purpose of the thesis was to examine why training solutions assisted by technical tools increase the learning potential from training for personnel within the professional emergency network. And the conditions that need to be in place to achieve learning effects from training activities. Learning is not something that easily can be measured, it was therefore necessary to find some indicators for learning. Content, commitment and context was chosen as indicators. The content says something about what is learned. The commitment says something about how the learning happens and the context describes the conditions which make it possible to learn. In order to study the effects technical tools, for use during training, have on learning, two main sources was chosen; the military at Rena and the BRIDGE project. Links can be drawn from the technology at Rena to AKKA, one of the tools in FRITS. Both the technological solutions at Rena and AKKA offer opportunities to collect and store large amounts of data from exercises. This can be used further for learning purposes. It was found that the technology can influence the content of what is learned through detailed planning, collection of data from exercises and storing of data from previous exercises. The technology can also influence the other indicators for the individuals learning through activities surrounding the use of technology, for instance through preparation meetings, discussions and involvement in activities. The technology can increase the learning potential, but other conditions must also be in place. This is conditions the technology can facilitate, but the technology alone cannot provide the conditions.
Focus on the individuals learning and allowing for involvement and contextual features in combination with the technology can provide good conditions for learning from training for the emergency response personnel.
3
Preface
This thesis marks the end of my master study within Societal Safety. I was included in the
comprehensive research project, BRIDGE. Writing this thesis has been an exciting process. I have been in contact with many resourceful people and gotten insight in the innovative world of research.
I have attended several meetings and conferences, travelled to Austria and within Norway.
I am grateful for being included in the project, not many get to spend their time planning exercises, viewing innovative technology and meeting resourceful people from all over Europe while writing their thesis. Writing this thesis has also been demanding, gaining insights in the complex world of research and understanding all aspects of the BRIDGE project has been challenging. One of the products studied in this thesis was still under development this have also been a challenge.
I would like to thank everyone who has contributed in this thesis. A special thanks to Morten
Wenstad and Jan Erik Holen from Crisis Training AS for including me in their work, providing me with information regarding their products and for providing me with valuable feedback on the paper. I would also like to thank my faculty supervisor Ove Njå for providing me with academic support, and helpful comments. By encouraging me and asking critical me questions he has made me reflect on important aspects during the process of writing this thesis.
And last I would like to thank Magnus, my husband for providing me with useful feedback and support.
4
Table of content
Summary ...2
Preface ...3
List of tables ...5
1. Introduction ...6
1.1 Background ...6
1.2 Objectives ...6
1.3 BRIDGE/CTAS ...7
1.3.1 The BRIDGE Project ...7
1.3.2 Concept Case Training ...9
2. Theoretical framework ... 17
2.1 Learning ... 17
2.2 Training terminology ... 20
2.3 Training methodology ... 21
3. Research design ... 26
3.1 Research method ... 26
3.2 Data collection ... 27
3.3 Data analysis ... 29
3.4 Validity and reliability ... 30
4. Analysis and results ... 32
4.1 Rena ... 32
4.2 BRIDGE/ CTAS ... 37
4.2.1 Training in Hell ... 37
4.2.2 Demo 2... 48
5. Discussion ... 56
5.1 Rena ... 56
5.2 BRIDGE/CTAS ... 59
5.2.1 Training in Hell ... 59
5.2.2 Demo 2... 62
6. Conclusion ... 68
References ... 69
APPENDIX ... 71
Appendix 1. Rena: Oppfølgingsspørsmål ... 71
Appendix 2. Øvelse Hell: Observasjonsmoment ... 72
5
Appendix 3. Øvelse Hell: Intervjuguide ... 73
Appendix 4. Demo 2: Intervjuguide ... 74
List of tables
Table 1: List of interviews... 29Table 2: Elements of learning ... 30
Table 3: Results from Rena... 36
Table 4: Characteristics of the observers ... 39
Table 5: AKKA's contribution during the exercise in Hell ... 46
Table 6: AKKA's contribution on indicators of learning ... 47
Table 7: Comparison of MeTracker with the non-technological alternative ... 51
Table 8: Comparison of AKKA with the non-technological alternative... 55
6
1. Introduction 1.1 Background
On a regular basis society gets hit by a crisis, hurricane Sandy is an example off this. How the government handles the crisis; before, during and after the hurricane will have an impact on the consequences. Crisis management is extremely important because it influences the immediate as well as the long term effects of a social system (Rosenthal, Boin, & Comfort, 2001). Many lives were probably saved during Sandy, because the government, among other things, warned people in advance and arranged for evacuation.
Preparedness training has been recognized as an important tool in the preparation of a crisis, in most industrialised countries training is mandated by legislation and executive rules. Training for handling crisis has therefore become a natural part of most institutions in the society. It is an opportunity to test the preparedness plan and the exercise participants get the ability to practice in a controlled environment, so that they are capable to act in a better way if something unexpected should happen.
Although training is fairly common, the focus is usually on the training activity itself; the planning of an exercise, necessary tools and effects, in addition to testing the skills of the participants. Learning is often something that is taken for granted. It is therefore interesting to study the learning effect training can have. How can we learn from training? What conditions needs to be in place to increase the possibility of learning?
All sorts of equipment has been developed to support the learning outcome of an exercise, for instance dummies for medical training, weapons for shooting practice, simulations for flight training and so on. These are all tools that are made to improve participant’s individual skills. Tools for organising and controlling a training scenario are also under development in the BRIDGE project. It is assumed that these types of tools will have a positive effect on learning from training activities.
1.2 Objectives
To ensure that learning happens during training new methods might be needed. One way of doing this can be to adopt new technology to support all the phases when designing a training solution another way might be to use technology to gather observations during exercises. These are examples of technological solutions that are under development in the BRIDGE Project1. The purpose of the thesis is to examine technical tools and solutions that are created for use during training and study
1 The BRIDGE Project is a research project which seeks to develop technical and organisational solutions that can improve crisis and emergency management within the European Union.
7 why these tools can influence the learning potential. Based on this the following main problem has been developed:
Why can training solutions supported by technical tools increase the learning potential from training for personnel within the professional emergency network? What conditions must be in place to achieve learning effects from training activities?
In order to answer the problem formulated above it is necessary to define learning and find a way learning can be measured. In addition it will be necessary to use theory on how training should be conducted to achieve the best learning results. It will also be necessary to study the technical solutions in BRIDGE and the effects they have or can have.
1.3 BRIDGE/CTAS
1.3.1 The BRIDGE Project
This thesis is written as a part of the research project BRIDGE. The aim of the project is to increase the safety of citizens by developing technical and organisational solutions that can improve crisis and emergency management in the EU member states. One of the main focus areas in the project is on how cooperation among different agencies and organisations can be made more effective.
This is a large project which consists of different cross disciplinary academics, technology researchers and developers, domain experts and end-user representatives. The companies involved are from different European countries, and meetings are held on a regular basis. Actors from Norway are SINTEF, Crisis Training AS and RAKOS (BRIDGE, 2012).
An important collaborating partner in this thesis is Crisis Training AS (CTAS) which develops optimal learning and training methodology supported by technical tools for use during training (CTAS, 2012).
This company is involved in the BRIDGE project through the development of training solutions.
Another collaborating partner is RAKOS (Regional Centre for Emergency Medical Research and Development). RAKOS is a centre of competence at the Stavanger University Hospital. They are working for better coordination and collaboration between the community health system, with the General Physicians, the A&E departments at the hospitals and the ambulance services (including cars, helicopters and boats). RAKOS is also involved in the BRIDGE project, with an end-user perspective of the tools.
The figure underneath illustrate the extent of the BRIDGE project. The project consists of several Concept Cases which are under development. Through the project, several tools for use under
8 emergencies will be developed in addition to training solutions. There is a wide spectre of tools which are being developed in the project, for instance a Master table, an electronic Triage tool, a rescue application and a training methodology tool. The Concepts Cases are presented at different demonstrations, a total number of four demonstrations will be held during the time of the project, two demonstrations have already been carried out. The first one was held in September 2012. The purpose of this demonstration was to see how fire-fighters could include some of the BRIDGE concepts in their work under realistic conditions. The second demonstration (demo 2) was held in April 2013, this was a demonstration of the bridge concepts which allowed for feedback from a selection of end-users. At the demonstration a situation of a large scale emergency was presented and the focus was on visualisation and interaction. The third demonstration will be held in
September 2013 and is a full-scale exercise which focuses upon multi-agencies collaboration. Demo 2 and demo 3 will have the same scenario, which is a terrorist attack with shooting and explosions in Risavika harbour, in Sola. The fourth demonstration is a large-scale emergency scenario involving a chemical disaster at a virtual facility called “ExploChemco”; this will be an integrated exercise which will present the final result of the BRIDGE project.
Figure 1: The extent of the BRIDGE project (Illustration: BRIDGE)
9 1.3.2 Concept Case Training
There are several concept cases within the BRIDGE project. The Concept Case Training is in focus in this thesis. This solution has gotten the name FRITS which stand for First Responders Integrated Training System. CTAS is responsible for this concept case. FRITS consists of several tools, were the final version of FRITS will function as a tool-box allowing for scenario-based training in a fully
controlled environment. The purpose of FRITS is to develop learning and training methodology which is supported by integrated sub-systems to improve the quality of emergency response and crisis management between agencies and at different levels of authority. Tools in FRITS can be used on different types of training this include; live-training2, virtual training 3and constructive training4. CTAS has developed a learning and training process which functions as the basis of FRITS. The process consists of five phases necessary when designing a training solution; Analyse, Plan, Execute, Evaluate and Lessons learned. See figure 2. Information regarding the phases in the learning and training process is based upon information provided by CTAS.
The following example will be used to elaborate the phases in the learning and training model.
Interaction and communication amongst the emergency services have gotten a lot of attention in Norway since the terrorist attack on the 22nd of July 2011. This can be a starting point for planning an exercise. In this example one assumes that it has been decided that the emergency service are going to train on an emergency scenario. A committee of planners has been formed with representatives from the medical emergency services, the police force, the fire department, the civil defence and volunteer organisations. They are responsible for planning this exercise based on the overall goal to train on interaction and communication.
2 Live-training is training conducted in real time in the field where participants use real equipment and do their tasks like they would in a normal situation
3 Virtual training is way of training which involves real people operating in a virtually simulated environment.
Games can be an example of this.
4 Constructive Training is a form of virtual training which allows for simulated people to operate in a simulated environment. Real people provide inputs to the scenario, but are not involved in deciding the outcome. This enables staff to perform their operational tasks in an unconstrained exercise environment.
10
Figure 2: CTAS Learning and Training Methodology (CTAS)
With this as a starting point one should begin the initial analysis. The first thing the planners should review is relevant emergency plans. The different departments most likely have different plans these should be considered and important aspects of the plans can be selected for testing during the exercise. Further the planners should gather and discuss experiences from their day by day work and previous exercises. The intent while using tools in FRITS is that these experiences already have been collected and are ready to be retrieved when planning new exercises. The different departments can therefor retrieve documentation from previous exercises and use this as basis for further discussions.
By doing this one can ensure that important learning points are considered when planning this exercise. After this the planners should conduct a training need analysis. The purpose of this point is to reveal necessary training areas. In this process the exercise participants skills and abilities should be considered as well as other areas were improvement is needed. Based on the steps above the planners should be able to form training objectives. At this point a distinction should be made; is the purpose to learn something or test the process. Based on this distinction different procedures should be followed. If the purpose of the exercise is to learn something the exercise participants should be involved in a greater extent than if the purpose is to test plans and procedures. In this example it was decided that the purpose of the exercise was to learn something. This means that the exercise participants should be involved. Information letters to the exercise participants should be formulated and arrangements for preparing activities should be considered. The objectives of the exercise can
11 for instance be to improve communication procedures and interactions amongst the different
agencies, to request assistance from the civil defence and to make contact with a nearby municipality. These objectives can be broken down into smaller elements and questions for use during the exercise. Once the objectives are formulated and agreed upon the planners should consider the need for supporting elements. It can be necessary with specific equipment or assistance from other agencies. This can for instance be necessary emergency vehicles, evacuation support by the civil defence, different props, like dummies and vehicles in addition to markers and observers.
The purpose of this point is to clarify the need for resources on an early stage and consider the cost attached to involving certain resources in the exercise. In the analysing phase it is also necessary to define the process for evaluation. The planners need to consider the evaluation process in the analysing phase because then the conditions necessary for the evaluation can be planned for. For instance if detailed information regarding certain areas during the exercise is necessary then one can ensure that these areas are properly documented by placing cameras and assigning observers to this areas.
The next phase is planning, in this phase the planners should consider the exercise at a more detailed level. First one should define a training scenario on the basis of what was discovered in the analysing phase. In this case the scenario must fit with the goals of all the involved agencies. A relevant
scenario can be an explosion in the city centre with an unknown origin. This scenario will require communication and interaction amongst the agencies and create uncertainties since they do not know if the explosion was a result of a criminal act or an accident. An explosion in the city centre will also lead to mass causalities which facilitates the involvement of the civil defence. At this point the planners should also determine type of exercise, virtual, constructive or live, or a combination of them. In this example it was decided that a live exercise should be conducted. After these elements have been agreed upon an execution plan should be formulated. This plan should describe what happens, where and when. The next activity in the planning phase is securing resources, at this point the planners know what kind of resources they will require during the exercise. It is therefore important to make arrangements with observers, markers, exercise participants and agencies. This scenario will require multiple markers and observers. To increase their performance during the exercise it will be necessary to arrange for preparing activities. Making arrangements early in the process can ensure enough resources and allow for preparation. Based on the results from the training need analysis preparing activities for the exercise participants may also be necessary. For better communication amongst the agencies it might be necessary to discuss communication procedures before the exercise. This allows for preparation and involvement which can improve the participants’ performance during the exercise.
12 The next phase is Execute, in this phase the training scenario will be played out. Before the exercise can be played out it will be necessary to establish the training area. At this point equipment will be placed at the scene this can for instance be a damaged car, markers, a fire source and elements which represents a broken building. In addition it can be necessary to place recording equipment and ensure that there is a wireless internet connection at the scene. It can also be necessary to assign observers to specific areas. After the area is established the exercise is initiated on the basis of the execution plan and the scenario is played out. Materials for use during the evaluation phase will be collected during the exercise using different methods depending on type of exercise and defined evaluation process. After the exercise is completed the area needs to be cleared, equipment need to be removed and markers needs to be transferred.
After the exercise has been completed and the area has been cleared the evaluation phase begins. At this point data collected from the exercise should be sorted and analysed. There might be gaps in the data foundation or uncertainties that needs to be clarified this can be done with the observers assistance. After this initial analysis one should evaluate the data against training objectives. How was the performance of the exercise participants in relation to the objectives for the exercise? This provides the foundation for the evaluation where one discuss both positive and negative findings from the exercise. The evaluation process will vary depending on type of exercise and type of goals established for the exercise. In this example an evaluation meeting was held after the exercise where results from the exercise were presented. Different processes are required depending on if the goals are related to one specific agency or if they are related to cooperation amongst agencies. Since this was an exercise where interaction was important all the agencies was gathered and the goals of the exercise was discussed against the results provided by the observers. The final step in this phase is certification. This is an optional activity which could be used after individuals have completed the exercise as a part of a larger training program. The emergency workers might be required to attend a certain amount of exercises during a year. Certifications might be necessary to prove attendance during the exercise.
The last phase in this model is lessons learned. The purpose of this phase is to transfer experiences from the exercise to the organisation, for instance changes in daily operation or changes in plans. The results from the example might suggest that the performance of the exercise participants was good, but not in accordance to the plans. This can point to weaknesses in the plan and facilitate changes in them.
This learning and training process function as the basis for the training solution FRITS. To ensure that training is conducted in a proper way technical tools have been developed. FRITS is a package
13 consisting of three tools; MeTracker, AKKA and a virtual training tool. The first two phases Analyse and Plan can be conducted using the MeTracker program. The planners in the example above could use MeTracker to guide them through the phases and documentation as basis for decision making in the two phases can be implemented in the program. The Executing and Evaluation phase can be conducted using AKKA. AKKA can be used to gather information in the form of written text, pictures and video recordings. This can be used further in the evaluation phase where the results are
presented to the exercise participants. Lessons learned are supported by both MeTracker and AKKA.
Results from AKKA can be processed and implemented in MeTracker for further storing. The virtual training tool supports this package by offering virtual training solutions. Currently CTAS is testing a virtual training tool called VBS2. The three tools will be elaborated in the following.
MeTracker is short for Methodology Tracker and is a program that CTAS is developing. The program is called MeTracker because it gives the user information about their progress in their step by step planning, execution and evaluation of an exercise. MeTracker tracks the training process by using the five phases of learning and training methodology; analyse, plan, execute, evaluate and lessons learned. After one phase has been completed the progress will be visible for all the planners. When using the program the planners will know how much work is left. The MeTracker program will be the focal point of interaction throughout the entire process to collect and store relevant information in one database. The program is used as a planning and management tool by guiding the user through the different phases. The end- product after completing the two first phases is documentation for the execution of the exercise. This includes among others, training objectives, a description of the scenario, information relevant to managing the exercise and information about the exercise participants. Relevant information can be implemented in AKKA. During the exercise observations are gathered using AKKA. The observations can then automatically be generated to a report supporting the evaluation process. Experiences and feedback collected with AKKA can further be stored in MeTracker. Gradually a large foundation of experiences from a number of previous exercises can be built up. This also includes data implemented from the first phases, as well as training objectives. Relevant data from earlier exercises can then be gathered and used as input when designing a new exercise. This can simplify the process of designing a training scenario since data implemented in previous exercises may be relevant for new exercises. The program allows for different templates to be generated, for instance an information letter with exercises goals to the participants or a standardised evaluation report. In the example above could an information letter be generated to the participants using the templates in MeTracker, since the purpose of the exercise was to learn. The structure of MeTracker can be seen in the figure below.
14
Figure 3: Structure of MeTracker (Illustration: CTAS)
AKKA is a tool mostly used in live-training, the tool allows for distributed observation and analysis in real time. AKKA is a tool which can gather and handle large amounts of information. The AKKA solution consists of a web portal where the objectives for a specific task, for instance an exercise are defined. The tools contribution to the FRITS solution is as mentioned above in the Execution and Evaluation phase. Based on the objectives created through the process in MeTracker reporting templates consisting of observation questions are created and uploaded to the web- portal. This web-portal will automatically synchronise with connecting Pad’s (android clients). Observers and instructors access the observation questions through their Pad by logging in with a unique ID linked to their specific role during the exercise. Observers in the example above could use AKKA to register their observations during the exercise. In addition to the observations questions other
documentation may also be uploaded in the web-portal, for instance plans and procedures. This can be used by the observers to check procedures and provide feedback on the behaviour of the exercise participants. Everything uploaded in the web-portal is available on the Pad. AKKA has a fairly
standard layout with common icons like save and refresh. Writing the observations in the program is
15 fairly easy, questions are answered with simple Yes/No check boxes, and boxes for written
observations and comments are available below. It is also possible to include a photo in the observation by taking a picture with the Pad. The structure of AKKA is illustrated below. Typically observers will carry a Pad and use the Pad to register observations, take pictures, video recordings and write comments. These recordings are sent directly to the web portal, this allows for viewing and analysing of data in real time. The personnel in the field can also receive information through the web portal this can contribute to situational awareness. The collection process is automatic since observations are registered electronically. Processing the data can be done during the exercise and right after. Usually one will be able to process the data in an hour. This allows for data to be presented during the evaluation meeting right after the exercise. Both positive and negative feedback are registered, which allows for thorough feedback (Saab, 2013). By implementing experiences from AKKA in MeTracker one can ensure that data is taken into consideration when planning new exercises. AKKA was created by Saab.
Figure 4: Structure of AKKA (Illustration: CTAS)
CTAS has tested the Virtual Game Engine, Virtual Battlespace 2 (VBS2). VBS2 is a program that allows for virtual training and simulations. Both individual and collective training can be conducted, and the program can simulate real-world systems. CTAS is using this program as a part of the training solution in FRITS. They are creating real life scenarios by modelling the infrastructure of real places like Risavika Harbour and Sola Airport. Different scenarios can be executed and the trainees get the ability to train in familiar surroundings. The purpose is to make scenarios as realistic as possible. The tool can be used as preparation for an exercise, for instance as preparation for a full-scale exercise.
16 Simulations of the scenario could be created and the exercise participants could use the tool to play their real life part. Two simulation pictures from Sola airport created with VBS2 are shown
underneath (Figure 5 and 6). The program allows for operating in a first person perspective, several actors can operate different virtual figures in the scenario and perform their task simultaneously. In addition to this observers can view the scenario from a bird eye perspective and inject event variations and resources. Some figures can be controlled by virtual intelligence while others are controlled by trainees. The program is developed by Bohemia interactive simulations (Bohemia, 2011). This tool as a part of the FRITS solution is still under development, it was therefore not possible to study this program while in use. For that reason in combination with time limitations it was decided that this tool should not be studied any further.
Figure 6: Simulation in VBS2 (CTAS)
The FRITS solution offer technological solution which can ensure that the steps in the learning and training model are followed. MeTracker can be used in the Analysing and Planning phase and AKKA can be used to gather information during the exercise which again can be used in the Evaluation phase. Important lessons revealed though the collected data can be implemented in MeTracker, thus a circle of learning and training is created. The intent is that information gathered from the process of designing a training scenario, during the scenario and after the scenario will ensure a better data foundation. This can contribute to more targeted training and ensure that previous experiences are taken into consideration. Virtual training supports this solution by offering simulations which allows for practice in a virtual environment.
Figure 5: Simulation in VBS2 (CTAS)
17
2. Theoretical framework
In this chapter a selection of theories will be described. The major goal in this paper is to show links or connections between training, technology and learning. The main problem consists of two comprehensive terms; learning and training. A definition of learning is necessary because the objective of this thesis is to study why technical tools in use during training can increase the learning potential. To be able to answer the main problem learning needs to be operationalized. A model for learning in emergency response work will be presented and this model will be used to study learning from training activities while using technology.
A description of training concepts will be presented in addition to theory on training methodology.
The concepts are described because some of the concepts may have different meaning depending situation and context and some concepts are defined to clarify their meaning. Both theories on learning and theories on training methodology will be used to study the conditions which need to be in place to achieve learning effects from training activities.
2.1 Learning
Learning is a difficult term because it can have a wide spectre of meaning. There are many theories on learning, and individuals may learn in different ways. Learning is something that happens all the time; one learns how to deal with different situations while it happens. Norms in a group, language, behaviour, how to preform different tasks are some examples. Some authors talk about single- and double circuit learning, where the single circuit learning process is characterised by a change in behaviour as a result of a mistake or a poor result. In this process some learning has taken place, the change in behaviour represents learning. The double circuit learning process is more complex, as a result of a mistake or poor results over time one starts to question why the actions don’t lead to the desired outcome. One looks beyond the behaviour and at the goals. Is it possible that the goal behind the action is not the goal one wants to achieve? The main difference between single circuit and double circuit learning is the reflection beyond own actions (Jacobsen & Thorsvik, 2007). Learning can by this definition be seen as change in behaviour.
Braut and Njå (2010)suggest a wider definition of learning which also involves confirmation of existing knowledge and achieving a deeper comprehension of emergency response. They define learning as; “processes related to establishing new knowledge aiming to implement changes to, gaining deeper comprehension of and/or confirming the basis for, current apprehensions and practices” (Braut & Njå, 2010, p. 10). This definition implies that learning is not just change in behaviour it also includes processes that confirm existing knowledge and processes of achieving a better understanding.
18 Two views of learning has gradually been developed; an individual cognitive approach and a
sociocultural approach. The traditional view is the individual cognitive approach, were individual mental abilities are in focus. This view sees learning as something that the individual must acquire and implies that learning is the activity where the brain is filled with new information or new skills.
The knowledge is seen as something which is assembled in our minds, and the knowledge belongs to the individual. The sociocultural approach views learning as practice. In this definition the focus has shifted from having knowledge to knowing. Followers of this view says that learning cannot be considered separately from the context were it takes place. The learner is seen as an active participator in certain kinds of activities rather than someone who is accumulating private skills (Sfard, 1998). Both views of learning have been criticised, the individual cognitive approach for not considering important social aspects of learning and the sociocultural approach for not considering individual abilities. For that reason Sfard (1998) suggests a combination of the two approaches.
Sommer, Braut, and Njå (2013) have developed a model for learning in emergency response work.
The model was developed on the basis of a combination of the individual cognitive approach and the sociocultural approach to learning. Both individual mental abilities and social aspects are included in this model. In this model the wide definition of learning as change, confirmation and/ or deeper comprehension is used. The model consists of four parts, see figure 7. The starting point for
understanding learning is the person, the persons learning is influenced by three conditions; content, commitment and context.
Content; for learning to happen there have to be something to learn, like the use of equipment, specific skills, certain behaviour or how to understand or interpret situations. This is abilities or knowledge that the individual are supposed to learn. The content can be both practical and theoretical knowledge.
Commitment; mental or physical activity by the individual is crucial for learning to happen. In addition to involvement in work related activities through active participation and interaction. They need to experience that a change is necessary and want to learn.
Context; certain conditions needs to be in place to allow learning to happen. Learning happens in an environment and through interaction with other people. Contextual factors like interpersonal relationships, cultural elements, social climate and practice will therefore influence the learning outcome. To ensure sharing of experiences and stories of failure one is dependent on an environment characterised by trust and openness.
19
Figure 7: Learning in emergency response work (Sommer et al., 2013)
The second part of this model is decision-making and response. This part represents the individuals’
performance in an emergency situation. This involves both real events and training situations. The emergency workers behaviour and response is a result of decisions they make which again influence the outcome of the situation. The next part of the model is reflection. Reflection has an important role when it comes to learning, for learning to happen one need to reflect on own performance and abilities. To be able to learn one need to actively interpret situations and experience how the behaviour results in positive or negative outcomes. Exchange of information amongst individuals is also important.
The final part of the model is change, conformation and/or comprehension. A need for change can be revealed as a result of reflections and discussions after an incident. The changes can for instance be in structures, behaviours or working methods. If an incident is handled well there might not be a need for change. This does not mean that learning does not happen. Experiences from the incident can confirm that the proper behaviour was performed. This confirmation is an important element in learning. Even if incidents do not lead to change or provide conformation some learning can happen in the form of comprehension of existing knowledge. Comprehensive knowledge about response work may in many situations be crucial for a successful response. For instance comprehensive
knowledge about response work can make the emergency workers handle unknown situations better Commitment
Content
Context
Decision-making and Response PERSON
Change, Confirmation
And/ or Comprehension
Reflection
20 since they may have experiences with similar situation and knowledge which can support their actions (Sommer et al., 2013).
Learning can be operationalized through the elements content, commitment and context. The content describes what is learned, for instance skills or knowledge. How learning happens is described by the commitment, for instance bodily activity and interaction. The context describe the conditions which makes it possible for learning, for instance through relationships or by using existing knowledge within the organisation (Sommer et al., 2013).
2.2 Training terminology
In this section training terminology will be described, and certain terms will be defined. More detailed theory on training will be described in the next chapter, Training methodology.
The words; training, exercise and simulations are used intermittent by different authors and have different meaning depending on how one use the words. For instance the word simulation is often used instead of training in many European countries and exercise can be seen as a form of training.
In this thesis Perry’s (2004) definitions on training and exercise will be used, where training is seen as a more overall term used on educational activities and an exercise is seen as an operational test.
Thus an exercise is only a part of the training. The term simulation can be used in different contexts and describe different aspects, all sorts of exercise scenarios can be seen as a simulations, in addition to virtual games, virtual illustrations and so on. A common feature of simulations is that they have a source in reality; they imitate part of the reality (Borodzicz, 2005).
Exercise scenarios are created so that one can practice in a safe environment, it can be viewed as a constructed opportunity to test a plan. One gets the ability to see how protocols and equipment work and can use this to evaluate the plan. The plan can be seen as a product of the planning process and is therefore the connection between the disaster planning and the disaster response. When preparing a plan one conduct an analysis of needs that arise, actions which could be undertaken and necessary resources to support the necessary actions. Disaster exercises can serve a variety of functions. One of the most critical functions is detecting difficulties in executing the strategies and tactics proposed by the plan in addition to detecting operational failures or weaknesses (Perry, 2004). There are several ways to conduct a training session which require more or less resources.
Exercise scenarios can be divided into three main types; Table-top exercises, Functional exercises and Full-scale exercises. The following descriptions of these is based on Peterson and Perry (1999).
A table top exercise is an exercise typically done at the table where the participants are presented with a simulated event. The participants usually have different roles and are supposed to provide a
21 solution to the problem that arises. Typically they provide a verbal response where they describe actions they would initiate as response to different demands. Controllers monitor the participant’s response and in some situations they inject event variations or crisis into the exercise process.
A functional exercise has a higher level of complexity than the table-top exercise. In a functional exercise one selects parts of a disaster response plan and conducts a test. This type of exercise is usually done in real-time in the field with operational personnel executing their functions using appropriate equipment. This type of exercise can be seen as a limited full-scale exercise.
A full-scale exercise is the most complex form of exercise. It involves testing of a major part of the disaster response plan. The exercise is conducted in real-time in the field where multiple response agencies participate. The exercise has a high level of realism and requires many resources; a full staff of controllers, props and actors.
2.3 Training methodology
A training session can be conducted in many ways, the amount of resources necessary and the degree of reality varies depending on the desired learning outcome. Despite the large variety of training scenario’s some recommendations for training can still be made. Vogel-Walcutt, Fiorella and Malone (2013) has conducted a study of training strategies for military training using grounded theory, they have developed a framework of pre-training strategies, during-training strategies and post-training strategies. This chapter is based upon their classification of training phases. Vogel- Walcutt et al. (2013) have a more traditional way of learning where the abilities of the teachers are in focus and the trainees are supposed to learn from the teachers efforts. This view of learning differs from the view by Sommer et al. (2013) which focuses more on the individuals learning abilities. The strategies by Vogel-Walcutt et al. (2013) can still be used to say something about useful
considerations in the pre- training, during training and post- training phase. These considerations can be used to support elements of the individuals learning.
Pre-training phase
A training session is designed by trainers. Their competence and abilities will have a large influence on the outcome of the training session. It is therefore important that the trainers have a basic understanding of what the participants do on a regular basis and that they are in a position to relate the training to the participants needs (Bennett, 2010). The trainers can be a group of people with different backgrounds who work together in designing an exercise as in the example used when describing the learning and training process. Considerations regarding their abilities might be necessary before the group is formed.
22 Early in the pre-training phase one should determine what the necessary training areas are. In order to do that it is necessary to identify relevant plans to be tested (Bennett, 2010). Further it is
important to determine the prior knowledge of the trainees so that the exercise can be designed on the basis of their needs. An analysis of needs and expectations from both the users and the
stakeholders is necessary because both parts need to be included on an early stage. Involvement of both parts will make them take the exercise more seriously and help put the exercise on the agenda (t Hart, 2002).
One of the first things that should be done when planning an exercise is defining goals, the goals will provide the framework for the exercise. The goals should be clear, specific and realistic. It can be useful to include the exercise participants when determining goals because they might have different views on necessary training objectives than the people who are planning the exercise (Vogel-Walcutt et al., 2013). By including the participants in the process of determining goals they are involved in the process which can influence the exercise participants’ commitment in addition to influence on the content. Influence on learning points can be crucial when it comes to learning because if the participants’ don’t find the training areas as relevant the probability of learning decreases (Sommer et al., 2013). One should define both overall goals and more specific goals. On the basis of these goals one will be able to develop specific observation questions which can be used during training.
For instance can a list of observation questions be implemented in AKKA and used by the observers in preparation for and during the exercise. Further planning of the exercise should revolve around a way to achieve the goals. Focus on the goals prior to the exercise can help motivate the exercise participants to concentrate on the most relevant material during training and select appropriate learning strategies that will assist in achieving those goals (Vogel-Walcutt et al., 2013). The elements mentioned above can be considered and determined in MeTracker’s Analyse phase.
After the goals are set, one should define an incident which fits with the training goals. This corresponds with the planning phase in MeTracker. Based on the chosen incident, one should consider relevant agencies to include in the exercise (Bennett, 2010). The incident should not be predictable; pre- planned exercises with only a limited range of scenarios can easily appear uninteresting to the participants and lead to an automated process (t Hart, 2002). The exercise scenario represents part of the context were the learning happens. A training scenario should appear relevant and interesting to the participants, if the scenario is regarded as too unlikely then the exercise might lose some of its credibility, and if the exercise is too familiar one risks that the participants lose interest and perform their tasks without reflection. There is a fine line between the two, the exercise should not be too tough, but it should not be too friendly either (t Hart, 2002).
23 Another important step is to choose instructors. An instructor has an important role in controlling the exercise and in giving feedback to the participants. It is therefore important that the instructor is experienced and dedicated in helping the participants achieve the goals (Bennett, 2010). In the pre- training phase experienced instructors can be used to activate the participant’s prior knowledge before they are being presented with new materials. More importantly they can help the exercise participants in integrating their existing knowledge with newly presented material (Vogel-Walcutt et al., 2013). One should also spend some time on finding observers and markers. The observers skills in noticing critical events will influence what is discussed during the evaluation which again can have an effect on what is learned (Løvik, 2010). Securing resources is also an element in MeTracker’s planning phase.
The next step should be to describe and define key terms of training material; this will provide the exercise participants with background information on important facts and concepts. When
presenting key terms in the pre-training phase the participants get the ability to integrate new knowledge with existing knowledge before the exercise. This means that they can focus upon the more complex relationship between those facts and concepts during training, instead of spending time on figuring out what the terms means. The tasks in the pre-training phase are meant to help the participants focus upon relevant materials, activate existing knowledge and provide them with relevant background information. The learning outcome from these tasks will vary depending on the level of expertise the exercise participants have. Experienced participants will probably have a deeper understanding of the training situation and might not need the same amount of information as the inexperienced need. It is therefore important to adjust the tasks in the pre-training phase to prior knowledge of the participants (Vogel-Walcutt et al., 2013). This corresponds to the competence improvement activities in the planning phase in MeTracker.
During- training
The during- training strategies by Vogel-Walcutt et al. (2013) has a strong focus on the instructors abilities. How they should act during training in order to teach the participants something. This can be used by the planners to arrange for this type of conditions however it will be just important to consider the trainees abilities and motivation for learning, like contextual elements, involvement in activities and influence on learning points.
In the executing phase of an exercise there are several factors which should be taken into
consideration. Vogel-Walcutt et al. (2013) use three sub-categories to describe different strategies for use during training; information presentation, providing guidance and practice. To be able to provide the necessary support during training one is dependent on experienced instructors to follow
24 and guide the participants. The instructor needs to know about different training strategies and have experience in using them. They should provide feedback during the exercise, this can be done through time-outs, where they identify crucial decisions or events and discuss actions or thoughts. It is also necessary with observers; an observer should take notes during the exercise and be prepared to offer honest and frank evaluation of the exercise. An observer can also provide feedback and inform the instructor about crucial events (Bennett, 2010). AKKA is a tool which allows for this type of feedback. AKKA provides the instructors and observers with tangible observation questions which they will be able to answer during the exercise. The questions offer a way of standardising feedback from the observers and can make sure that observations are made in accordance with specific goals.
The tool also allows for feedback during the exercise, information regarding the scenario can be sent to the participants through AKKA.
An important factor the instructor needs to consider is how the training materials are presented.
Different methods like combining oral and visual information, personalizing the information, using signals and animations can help the exercise participants understanding of the exercise goals and tasks. AKKA can be used to gather different data from the exercise, for instance pictures and videos.
This can be of assistance when preparing data for presentation. Guidance is proven to be another important factor when it comes to the learning potential from training sessions. By providing, examples, hints, immediate feedback and explanatory feedback one can guide the participants through the exercise (Vogel-Walcutt et al., 2013).
Allowing for the participants to practice different skills on an individual basis can help them prepare for a larger exercise. In addition to this it can be useful to allow for several smaller training sessions over time (Vogel-Walcutt et al., 2013).
Post-training
It is important to give all the phases in the training session sufficient amount of time, the analysis and evaluation is equally important as the exercise. The results of the exercise should reflect the training goals. After the exercise a debriefing should be conducted, this can be a brief oral review right after the exercise followed by a review meeting a couple of days later. The time and date of the review meeting should be clearly expressed, and all the exercise participants should be invited to attend the meeting. When deciding on a date one should allow for sufficient amount of time so that the
participants get the ability to reflect on the exercise. All the participants should get the opportunity to ask questions, and strategies, tactics and decisions that were made during the exercise should be
25 explained. Written reviews by the observers and instructors are important in the evaluation of the exercise. They should therefore provide open, honest and fair feedback from the exercise. Their written reviews can be used as a checklist for the discussion during the debriefing meeting. If AKKA is used during the exercise can the list of observation questions function as a check-list and data collected with AKKA can be presented to the participants. The purpose of the meeting is to allow for honest assessment of everyone involved in the exercise not to make individuals or groups look bad.
Leaders of the review should make it clear that the actions discussed during the meetings should not be taken personally (Bennett, 2010). The instructor should provide a summary of the participant’s performance, it is important to include corrective and explanatory feedback, in addition to
suggestions for performance improvement. Both good and bad decisions should be discussed, and the discussion should follow the exercise time-line. It can be useful to ask the participants to reflect on their own actions and suggest areas that need improving. The participants will most likely have noticed some of the same areas of improvement as the instructor, and they will more easily accept the information if they are the ones who are suggesting it. It can also be useful to assess the participants through testing. Through testing the participants get practice in receiving information from the long-term memory which strengthens the materials memory trace. This is useful because it is the same retrieval practice that is necessary when solving real-life problems (Vogel-Walcutt et al., 2013).
26
3. Research design 3.1 Research method
In this thesis a study of technical tools for use during training has been conducted. The main
objective was to study the effects these tools have on learning. The study has been conducted using two main sources; the military at Rena and the BRIDGE project. Results from this study are based upon observations, interviews and literature study.
The main problem of this thesis was as follows;
Why can training solutions supported by technical tools increase the learning potential from training for personnel within the professional emergency network? What conditions must be in place to achieve learning effects from training activities?
This problem is twofold; first it seeks to answer why learning increases with the use of technological solutions. Second it seeks to describe necessary conditions which need to be in place to achieve learning effects.
On the basis of the research questions a qualitative research method has been selected. The reason for this was the nature of the research questions which seeks to provide in-depth data regarding the technology’s potential and effects on learning. Since most of the products are still in the developing phase, experiences from individuals and in-depth interviews with them was seen as more useful than large amounts of quantitative data.
The technological solutions within the BRIDGE project are under development. To be able to say something about experiences regarding technological solutions during training a study at Rena was conducted. They have years of experience with technological solutions for use during training and experiences from Rena can say something about what kind of results one could expect from other technical solutions.
One of products, MeTracker, which was studied in the BRIDGE project, was not ready to be tested.
This was a challenge when working on this thesis. Since the product was incomplete, actual results on learning could not be studied. The focus when studying this product was on the potential the solution could have. During demo 2 experiences and feedback regarding both MeTracker and AKKA were recorded, followed by interviews with some end-users afterwards. Results regarding this product express how the product can work based on information provided by the developer, CTAS.
27 The AKKA tool was tested during an exercise in the Hell tunnel in Trondheim. This exercise was followed and provided important feedback regarding how this tool can support training activities.
This exercise was an opportunity to study the effects the technology can have on learning. A
challenge when observing this exercise was that the technology only had a small role in the exercise and the whole planning process was not followed.
3.2 Data collection
The data collection process started with a visit at Rena military camp. Two different training solutions were presented; one solution for live- training and one for constructive training. Informal interviews were conducted with the guide and individuals from each of the departments. During the visit arrangements regarding follow-up interviews were made. Two follow-up interviews were conducted over mail with representatives from each of the departments, one of them was with the leader of the constructive training department and this was the same individual who also was interviewed during the visit at Rena. The other one had the role as an instructor in the live- training department. These sources were chosen because they had in-depth information and experience with the training solutions. Interviews with individuals who have trained with the different solutions could have been beneficial for a more versatile description of the solutions at Rena. One could expect that the leaders and individuals who work with the solutions in their everyday work is more positive to the training solutions than the individuals actually train in the solution. Because of time limitations interviews with trainees was not conducted.
Results from a report from the Norwegian Defence University College (Krigsskolen) were also used.
This report was developed on the basis of a large study of the constructive training solution at Rena and is based upon observation during training and interviews with approximately 100 individuals.
The BRIDGE project has been followed since November 2012. Experiences have been gathered at two conferences, one in Klagenfurt and one in Stavanger (demo 2), and during an exercise in the Hell tunnel in Trondheim. Information was collected through conversations with representatives from the project, through two different BRIDGE online databases and through observations and interviews.
Two semi- structured interviews were conducted with end user representatives after demo 2. These two was chosen because they attended the presentations and were experienced individuals who could say something about how the technology could function in their organisations. More
interviews could have ensured a wider perspective on the solutions. However opinions and feedback from the end- users made during demo 2 was recorded this contributed to feedback on several aspect relevant to this thesis and provided a more versatile picture than the interviews alone.
28 Planning meetings for the exercise in Risavika Harbour has also been followed. In the planning committee, representatives from the police force, the fire department, the medical emergency service, the civil defence and volunteer organisations has been present. This group is planning the exercise where the BRIDGE tools will be tested in September, and representatives from this group were also present at Demo 2. The two interviews from demo 2 were with individuals from this group.
As a part of the BRIDGE project the exercise in the Hell tunnel in Trondheim was followed.
Observations were made during an educational preparation meeting on the day before the exercise, during the exercise and during the following evaluation meeting. The focus of observation was the observers, how they received the information regarding the technology and how they used the technology. A list of specific observation areas had been developed in advance. This list can be found in the appendix. During the exercise two observers were observed while using the technology and observations was also made of observers without the tool. Semi- structured interviews was
conducted with four observers over the phone. Two of the interviews were with individuals, and the last one was a group interview with two observers. The informants were chosen because of their unique experience with use of technology during the exercise. One of the informants used the technology during the exercise, one of them had the ability to use the technology and the last two attended preparing activities but did not have the ability to use the technology. One observer who used the technology during the exercise was not available for interview. This individual could have provided more insights on how the technology functioned during the exercise. The interviews were of informal nature, an interview guide had been developed in advance, however this was not strictly followed. The intent during the interviews was to get in-depth answers regarding their experience.
For list of interviews see the table 1.
29 Unstructured Semi structured
Rena
Guide Questions face- face
Leader of constructive training division
Questions face- face Interview via mail
Leader of live-training division Questions face-face Instructor in Excon (live-
training)
Interview via mail
BRIDGE: Hell
Observer Interview via
telephone
Observer Interview via
telephone
Two observers Group interview via
telephone BRIDGE: Demo 2
Participant at demo 2 Interview face- face
Participant at demo 2 Interview face- face
Table 1: List of interviews
The basis for the semi- structured interviews was interview guides which had been developed in advance. The interview guides can be found in the appendix (note that observation lists and interview guides are in Norwegian). During the interview they were asked to elaborate certain opinions and follow-up questions were asked.
3.3 Data analysis
The main problem in this thesis was to explain why training solutions assisted by technical tools can improve the learning potential for personnel within the emergency network. And to describe the conditions that need to be in place to achieve learning effects from training activities. In order to do that it was necessary to find some indicators which could be used to measure learning. Learning is a continuous process, it was not possible to measure change in behaviour or other exact results on learning during this study. In order to do that it would be necessary with multiple studies where changes from one situation to another were studied. This was not an option. The model by Sommer et al. (2013) has been used as basis for studying learning, and the elements content, commitment
30 and context was used as indicators for the individuals learning. These elements were further broken down into concrete elements. In a training situation elements are related to the content conditions which allows for a better understanding of the situation; like concrete information regarding the individuals specific skills and behaviour, how things are connected and explanations regarding the chain of events. Elements related to the commitment are conditions which allows for involvement in mental and physical activities, active participation and interaction. Elements related to the context in are conditions that allows for interpersonal relationships, sharing of experiences and open
discussions. The technology’s influence on learning during training is analysed based upon the following table.
Content
Allowing for feedback regarding the individuals specific skills Allowing for feedback regarding the individuals behaviour Allowing for explanatory feedback on how things are connected Allowing for feedback through multiple sources
Commitment
Allowing for involvement in preparing activities
Allowing for mental and physical activities before, during and after training Allowing for interactions and discussions before, during and after training Allowing for active participation before, during and after training
Context
Allowing for interpersonal relationships
Allowing for sharing of experience and stories of failure
Allowing for open discussions regarding the use of results from the technology
Allowing for an open and trusting environment
Table 2: Elements of learning
The focus when studying the technology was on the differences the technology provided in relation to the non-technological alternative. These differences were studied further against the three indicators for learning.
3.4 Validity and reliability
The results from the study at Rena may be influence by the selection of informants. A more differentiated picture of the solution could have been described had the trainees also been interviewed. This is a limitation with this paper. The informants at Rena were chosen because they had long experience with the solution and could provide in-depth information regarding how the solutions work. This is information participants would not be able to provide in the same extent. The follow-up interviews with the individuals from Rena were sent via mail, this can have influence the
31 answers they provided since it requires more efforts of the informants to formulate written answers.
It was therefore decided to limit the follow-up questions to nine questions. By sending questions over mail one also lose the ability to ask for elaborations, however the informants provided detailed answers and they were open for answering more questions over mail.
The results from demo 2 are to some degree of hypothetical nature. The solution is under
development and actual effects on learning could not be studied. How the solution work is described based upon information on how the solution can, and is planned to, work. A study of the complete solution will most likely provide different results than what has been discovered in this paper. The results can only say something about the potential the technology can have, not how it actual work.
Two interviews were conducted after demo 2, to ensure the validity of the responses more interviews could have been conducted. The comments made by the end-users during the
demonstration were recorded and a summary of their comments can be found in the results chapter.
During the presentation of AKKA and MeTracker they made several comments, it is therefore assumed that the combination of feedback during the demonstration with interviews after the demonstration provides an adequate picture of their opinions of the technological solutions.
32
4. Analysis and results
In this chapter results from studies at Rena, Hell and demo 2 will be presented and analysed. The results will be analysed on the basis of on the main problem and theories on learning and training.
4.1 Rena
A study of Rena was conducted because they have fully developed training solutions assisted by different technical tools. Experiences from Rena are important because it says something about how technological solutions can work and what effects the technology has.
The military at Rena have a solution for live training and different solutions for virtual or constructive training. The live-training solution consists of technological equipment for use during and after training. They train under realistic conditions in a large forest within the military camp. Up to 800 people can train at the same time. Equipment they normally would use, like wests, helmets, rifles, armoured vehicles and rocket launchers are altered or replaced so that a scenario can be executed without the use of sharp bullets. The helmets and wests are equipped with a GPS tracker and sensors which receive signals through Bluetooth. The rifles are real; a laser with a receiver is attached on the tip of the rifle. The rocket launchers are replications, but the size and weight is the same as a real one. The armoured vehicles are also real and they are also equipped with sensors. The exercise is monitored by observers and instructors in the field, in addition to remote monitoring by operators in a control room (referred to as Excon). As many as nine people can operate in Excon at the same time;
they follow the exercise through a computer program called ExPERT, which shows the position of the exercise participants and the armoured vehicles on a map. The operators in Excon can also inject event variations. The computer also shows injured people in the scenario. In addition to following the participants on the computer screen they can also listen to radio communication. The operators in Excon communicate with instructors and observers in the field; they inform the operators in Excon about critical events, when doing so the operators in Excon can take screen shots of the situation which can be used in during action reviews (DAR) or after action reviews (AAR). The time of the screen shot is also noted automatically in the radio log so that it can easily be retrieved for AAR. After the exercise they have an AAR where they use information gathered during the exercise. The
technology allows for skill training on an individual basis as well as strategic training on higher levels.
The training solution assisted by technical tools facilitates feedback and analysis which would not be possible without the technology. First of all it allows them to view recordings from the exercise this can show/explain what the situation was at a given time. Recordings from the radio enable them to learn communication procedures and understand each other and military terminology better.
