www.nes2015.no
Lillehammer, November 1.- 4.
Proceedings of NES2015
Nordic Ergonomics Society 47th Annual Conference 01 – 04 November 2015, Lillehammer, Norway
Editors:
Knut Inge Fostervold Svein Åge Kjøs Johnsen Leif Rydstedt
Reidulf G. Watten
Publisher:
NEHF (Norwegian society for Ergonomics and Human Factors) c/o Lyskultur
Strandveien 55 1366 Lysaker WWW.ergonom.no [email protected]
© NEHF - 2015
ISBN 978-82-995747-5-4
The Impact of Trainer on Training Transferability
Salman Nazira, Kjell Ivar Øvergårda, Davide Mancab
aTraining and Assessment Research Group, Department of Maritime Technology and Innovation, Buskerud and Vestfold University College, Postboks 4, 3199 Borre, Norway
bPSE-Lab, Department of Chemical Engineering “Giulio Natta”, Politecnico di Milano – Piazza Leonardo da Vinci 32, 20133 Milano, Italy
This paper seeks the answer of: “The Impact of Trainer on Training Transferability”. Since last two decades, the use of advance and immersive training simulators have became a common practice in various domains. With advancement in technology it is possible to conduct training without a trainer We currently know very little about the effects of training without a trainer. This research challenge was tested in an experiment to find the impact on the acquisition of skills of trainees with and without a trainer. To test this effect we used The Plant Simulator (Nazir & Manca, 2014). Findings indicate that persons trained with a trainer produce fewer errors and requires less help, while those trained in the augmented virtual reality simulator is more efficient and uses less time.
Keywords: Training, Trainer, Assessment, Simulator, Augmented Virtual Reality
1. Introduction
The impact of training on the profitability and safety of processes are shown both empirically and theoretically in various studies (Nazir et al., 2015). It is now well known that investment in training enables the individuals, teams, organizations and industries to achieve their targets in more comprehensive manners (Naderpour, Nazir & Lu, 2015;
Salas et al., 2006). Various questions, for instance training for hazards and safety (Baker et al., 2006), training effectiveness, rate of return of training (Kluge, 2014, Kluge, Nazir, Manca, 2015), impact of training on team work (Salas et al., 2006) and the like, have been recently answered or at least under continuous investigations. New technological advancements have brought new challenges to the operators (Lützhöft & Lundh, 2009).
The evolution and revolution of industrial processes have made the roles and goals of operators challenging and delicate at the same time. Therefore, digging deeper on the topics of operator training is an investment, which researchers are trying to make.
Complex socio-technical systems such as the chemical industry involve higher levels of automation, overload of information reaching the operators, safety critical scenarios, distant and different working environments of field and control room operators, and hazardous nature of processes (Nazir et al., 2012; 2014). Various training methods are used to instil the operators with necessary skills to cope with demanding daily
operations. The authors are interested in investigating the effectiveness of various training methods on the performance of operators.
Research Question. The aim of this experiment was to identify the effects of training without a trainer. To do this we conducted an experiment comparing traditional trainer-based training with a novel trainer-free training situation using a simulator.
C2: Organization, work and social environment 2
NES2015 C2-5
2. Methods
An experiment was designed and conducted for a specific procedure of a polymerization plant, i.e. catalyst-injector switch, where the operator has to follow a sequence of actions in a precise and timely way. The procedure for catalyst injector switch involves a total of 29 actions (only first 9 actions were selected for this experiment). A successful catalyst switch can be ensured only by following accurately such a sequence. Any errors in following the sequence can result in triggering an abnormal situation that may lead to a significant impact on the process operation and to relevant economic losses. Failure of catalyst switch is enough to bring losses of millions of euros as it requires complex maintenance activities based on the shutdown of the plant (and following startup when the malfunction is fixed) with consequent production losses and need for production rescheduling. Therefore, other than analysing the effect of training methods in IVEs on the performance of operators, this study focuses the attention on the effectiveness of training procedures for catalyst injectors switch in the production of PP.
Two groups of participants (N = 24) were trained within an immersive
environment. One group (call it human trainer group) was trained with the support and help of a trainer, where the trainer demonstrated all the actions to the trainees in the training phase (see Figure 1).
Augmented Virtual Reality (AVR) was used for the training of the second group (call it AVR group). The augmented feature in the immersive simulator enabled the trainee to follow a set of sequence of actions. We have devised the following methodology for automated training
- the valve that need to be operated is in yellow colour (see Figure 2)
- once the valve is operated (closed or opened) the colour will change to green (see Figure 3)
- at the same instant the valve that need to be operated right after the previous one will appear in yellow
- the above sequence will be followed for all the procedures
Figure 1 – Training session for human trainer
group Figure 2 – Training session of augmented virtual
reality group. The augmented yellow colour of the valve indicates that the valve shall be operated
Once the training phase is finished the participants are evaluated for the performance of the task. With reference to the catalyst injectors switch procedure, the performance assessment is based on the following dependent variables, which are named as Operator Performance Indicators.
Total Score: The total score obtained by the trainee. It is evaluated by a dedicated algorithm and is based on the number and nature of errors conducted and the number of helps requested.
Number of helps requested: If and when the help(s) was requested by participants during the course of experiment; they were recorded by the assessment software.
Number of errors: Each time the participant failed to perform the correct action the software registered it as an error.
Total time taken: As the name of this performance indicator suggests, it is the total time taken by participant to complete all the actions.
3. Results
Analyses of difference were done (for each of the performance indicator stated above) with the non-parametric Mann-Whitney U-test. The Probability of Superiority was used as an effect size measurement. The probability of superiority (PS; Ruscio & Gera, 2013) of is a measure indicating the probability that a randomly chosen value from one group is larger than a randomly chosen value from another group. If the groups were identical the PS would be 0.5 (giving a 50% chance that a value from Group A is bigger than a value from group B). If the confidence interval (CI) of PS does not contain .50 that means that we have a significant difference at p = 1 - (CI / 100). Please see Grissom and Kim (2012) for information on how to calculate PS and CI for PS.
As can be seen from Table 1 there is a clear tendency that the trainer group did fewer errors and received a higher total score than the augmented reality group. The effect sizes were of medium and small size on Errors and total score respectively. The data also indicate that the Augmented reality-trained participants where faster than the Trainer-group and that they required less help. However, these differences were not significant.
Figure 3 – A participant from AVR is in
training phase Figure 4 – A participant is conducting the final performance assessment after having gone through the training phase
C2: Organization, work and social environment 2
NES2015 C2-7
Table 1: Comparison of the Trainer Group and the Augmented Reality-group
Groups
Trainer AVR
Measurement M (SD) M (SD) p PS [95% CI]
Errors 0.42 [0.9] 1.17 [1.03] 0.046* .29 [.11, .48]
Time 400.8 [164.1] 322.6 [122.1] 0.326 .60 [.41, .84]
Help 0.5 [1.168] 1.58 [2.35] 0.092 .32 [.13, .52]
Total Score 87.1 [27.4] 65.1 [29.6] 0.031* .75 [.54, .92]
Note: AVR = Augmented Virtual Reality, M = Means, SD = Standard Deviations, p = p-value for Mann-Whitney U-test, PS = Probability of Superiority, CI = Confidence interval. The table shows mean and standard deviations for the two groups. p-values are for the non-parametric Mann-Whitney U-test measuring the difference between the two groups. PS shows the exact probability that a randomly chosen value from the Trainer-group is bigger than the Augmented Reality-group. If the confidence interval contains .50 then the difference is not statistically significant.
4. Discussion
Evaluating the performance of operators in a dynamic, time dependent task is a challenge in itself. To overcome the same, an algorithm (Manca et al., 2014) was developed and validated which allowed a real time performance assessment of trainees while doing the catalyst-injector switch (a procedure which is performed bi-annually and where errors costs about millions of euros). The with trainer group showed higher accuracy, precision, identification skill and lower help requirements than those of AVR group. On the other hand the AVR trainer group showed better speed when compared with the former.
A possible explanation of this effect is that while the AVR group got more training in doing the actual movements, they did not get additional system information.
On the other hand, the Trainer group would get more information about the system through the trainers presentation and hence they were more successful in creating a working mental model of the work task and the requirements. However, the trainer group did not get additional training on the performance of the movements. Nevertheless, there is a need of further experiments with similar research questions (possibly with more participants) that can increase the understanding of the role of trainer on overall training effectiveness.
5. Conclusion
The results show that the training lead by trainer produced more precise set of actions by trainee, whereas, training with augmented virtual reality resulted in higher efficiency in terms of response time. This study paves a way for further experiments and research in understanding the role and impact of trainer on the learning outcomes.
References
Baker, D. P., Day, R., & Salas, E. (2006). Teamwork as an essential component of high‐
reliability organizations. Health services research, 41(4p2), 1576-1598.
Grissom, R. J., & Kim, J. J. (2012). Effect sizes for research: Univariate and multivariate applications. Routledge.
Kluge, A. (2014). The acquisition of knowledge and skills for taskwork and teamwork to control complex technical systems: A cognitive and macroergonomics perspective.
Springer.
Kluge, A., Nazir, S., & Manca, D. (2014). Advanced applications in process control and training needs of field and control room operators. IIE Transactions on Occupational Ergonomics and Human Factors, 3-4, 121-136.
Lützhöft, M., & Lundh, M. (2009). Maritime Application of Control Systems. T.
Ivergård, Hunt, B. (eds.). Handbook of Control Room Design and Ergonomics-A Perspective for the Future. CRC Press.
Manca, D., Nazir, S., Colombo, S., & Kluge, A. (2014). Procedure for automated assessment of industrial operators. Chemical Engineering Transactions, 36, 391-396.
Naderpour, M., Nazir, S., & Lu, J. (2015). The role of situation awareness in accidents of large-scale technological systems. Process Safety and Environmental Protection, 97, 13-24. doi: http://dx.doi.org/10.1016/j.psep.2015.06.002
Nazir, S., & Manca, D. (2014). How a plant simulator can improve industrial safety.
Process Safety Progress, 34, 3, 237–243, DOI: 10.1002/prs.11714
Nazir, S., Colombo, S., & Manca, D. (2012). The role of situation awareness for the operators of process industry. Chemical Engineering Transactions, 26, 303-308.
Nazir, S., Kluge, A., & Manca, D. (2014). Automation in Process Industry: Cure or Curse? How can Training Improve Operator’s Performance. Computer Aided Chemical Engineering 33, 889-894.
Nazir, S., Sorensen, L.J., Øvergård, K.I., & Manca, D. (2015). Impact of training methods on Distributed Situation Awareness of industrial operators. Safety Science 73, 136- 145.
Ruscio, J. & Gera, B. L. (2013). Generalizations and extensions of the probability of superiority effect size estimator. Multivariate Behavioral Research, 48, 208-219.
Salas, E., Cooke, N. J., & Rosen, M. A. (2008). On teams, teamwork, and team performance: Discoveries and developments. Human Factors, 50(3), 540-547.
Chemical Engineering Transactions, 26, 303-308.
C2: Organization, work and social environment 2
NES2015 C2-9
