R ESEARCH D ESIGN

The aim of this dissertation is to investigate the effectiveness of the leadership development program at the RNoNA that takes place during the first year of their education at RNoNA.

The leadership development program at the RNoNA represents, as described in chapter 8, a systematic approach to developing and transforming the cadets to become more effective leaders so they can apply Auftragstaktik to cope with the demands of 4^th Generation Warfare.

This indicates a quasi-experimental design, and I found it appropriate to use The One-Group Pretest-Posttest Design (Cook & Campbell, 1979). In such a design, all the cadets are observed, and, following a pretest (O1) measurement, all of them are exposed to the leadership development program (the treatment X). The effect of the leadership development program would then be estimated simply by examining the average difference between the posttest measurement, O_2, and the pretest measurement, O1. This is a widely used design, but unfortunately it has several weaknesses that must be considered. According to Cook and

Campbell (1979) the most important ones are history, statistical regression, maturation, and testing.

One of the unique features of the leadership development program at the RNoNA is that it is carried out in real life settings where the respondents work together, performing their normal duties. This increases the validity of this study, especially its external validity, because the experimental situation reflects the natural setting or situation to which I want to generalize.

Furthermore, they are also representative because this study covers the whole population of cadets within each cohort of cadets, and because this empirical study covers four cohorts of cadets, it is reasonable to assume that they are representative for those officers commissioned from the RNoNA (Frankfort-Nacmias & Nachmias, 1996).

It would have been unethical (and impossible) to assign individuals to either a control group

—cadets that did not participate in the leadership development program—or to an experimental group—those that participated in the program. Although the chosen design is a one-group pretest-posttest design it contains many of the characteristics of a classic experimental design because the teams are composed the same way each time, and they are exposed to the same treatment under similar conditions where data is collected. This contributes to increasing the reliability of treatment implementation and the random heterogeneity of respondents which reduces this effect on statistical conclusion validity (Cook & Campbell, 1979). Concerning the latter issue the population at hand could be considered to be homogenous while the RNoNA focuses on variation to maximize the leadership development effect. Furthermore, the dependent variable can be adjusted equally for the teams within each cohort. All the teams in Cohort 2002, for example, participated in exercise Magellan, giving them all the same treatment, while this exercise was not in the curriculum for Cohort 2001. The cadets are in the similar environment, the treatment environment, most of the time that they are awake. During exercise Magellan, they are in the environment 24 hours per day for 10 to 11 weeks. This significantly reduces the threat of history, since they are in fact partly physically isolated. It also contributes to reducing random irrelevances in the experimental setting and thereby increases the statistical conclusion validity (Cook & Campbell, 1979). These are factors which normally can threaten a quasi-experimental design with existing leader and work teams because they usually operate in and out of different environments, exposing them to a range of variables that can threaten the design and the predictive validity of the study. Because the

development program is an intensive program of one year duration, the difference between the posttest and pretest would most likely not be a result of any historical events intervening during this period.

As previously discussed the aim of the leadership development program is to expand each cadet’s role-taking ability, hence maturation. This indicates that an effective program would result in a difference between pretest and posttest. Then if the expected and obtained scores differ, this might be either because the posttest was affected by the leadership development program or knowledge gained at the first testing altered testing on the subsequent testing.

The RNoNA, as discussed in chapter 8 and can be seen from appendix A, provides the cadets with feedback on their team and leadership behavior, using the SPGR, to enhance their role-taking abilities. Maturation is a direct goal of the RNoNA’s leadership development program, so an ineffective program would not lead to any maturation. By applying their own scores (their identities when it comes to leadership behavior), and peer ratings (their reputations as result of their leadership behavior), the classical maturation threat to this design is limited by data from multiple sources. Consequently, this indicates that if the obtained scores differ, it is most likely because of a change in their peers’

orientations of team and leadership behavior, resulting in a different score and not because of maturation as result of any effects other than the leadership development program.

Although research on personality does indicate a weak maturation effect as a result of aging—for example, people tend to become more agreeable and less extroverted with age—

Costa et al. (2000) showed changes in the five domains of 1 to 2 T-score points in nonclinical sample of adults over a 6- to 9- year period, indicating no significant changes. It may, therefore, be concluded that any significant change over a one year period would most likely be a result of the leadership development program.

The threat of statistical regression is partly reduced because the leadership development program at the RNoNA is given to all the cadets, who are randomly distributed within the teams, and not to a special group of them—a group of underachievers, for example. As is illustrated in Figure 10.4, several measurements using the SPGR will also be taken during the intervention in order to study social interaction within each team. This is important since they stay together in teams throughout the program, and so, because they have to function as a team over a long time span, the groups become real teams for each cadet. They will, as result of this, quickly develop a history, and at the same time, they will also have a future

together. In fact, their results and military grades, which become part of their permanent record, depend on how well they function as teams and leaders⁴⁵. In a traditional classic experiment, by contrast, the teams would be ad hoc groups composed of people who neither had a history of working together nor are supposed to do so in the future. Another important aspect of the leadership development program at the RNoNA is that it is designed in such way that the cadets are working with novel outcome behavior because they are progressively exposed to a higher level of novelty, ambiguity, and complexity, measured by the SPGR at the pretest and posttest. This strengthens the internal validity of the study (Cook &

Campbell, 1979). Furthermore the cadets were not provided any feedback on the pretest measurement, except from the SPGR, which is used by the RNoNA during the developmental process. There were no changes concerning the instrumentation, questions, or scoring system between the pretest and posttest to reduce the threats of internal validity to the chosen design. This permits more closely approximating the result of the leadership development program at the RNoNA after the exposure. The effect of the leadership development could then be estimated by examining the average difference between posttests and pretests.