The purpose of this evaluation is not to evaluate 22 unique research schools, ra-ther it is to investigate to what extent the national scheme has contributed to in-crease the quality of the Norwegian researcher education by meeting the following objectives, defined by the RCN:
• Increased completion rates (alternatively: reduced drop-outs)
• Reduced completion time
• Increased internationalization
• Strengthening of national cooperation within scientific fields, including more (small) national institutions in larger research networks
Scientific field Applications Grants Success rate
Technology 33 2 6 %
Social Sciences 14 3 21 %
Medicine and Health Sciences 18 8 44 %
Mathematics and Natural Sciences 17 7 41 %
Humanities 7 2 29 %
Total 89 22 25 %
• Raising the activity and the capacity of the cooperative environments, e.g.
through increased supply of courses and better access to research networks and guidance
Another main objective of the scheme is to promote the recruitment of PhD stu-dents. This goal, however, given the terms of reference for this evaluation, will not be subject to our evaluation. Rather, the current evaluation was specifically asked to respond to the following four questions:
1. Is the completion rate (i.e. absence of drop-out) higher among PhD students affiliated with national research schools, compared to PhD students that have not been affiliated with any research schools?
2. Is the time-to-degree of completed PhDs, i.e., completion time, lower among PhD students at the national research schools, compared to PhD students that have not been affiliated with any research schools?
3. To what extent have the national research schools contributed to tionalization, by for example travel grants, support to stays abroad, interna-tional course instructors, supervisors, etc.?
4. Have the research schools contributed to strengthening national coopera-tion within scientific fields, professions and thematic areas; including more (small) national institutions, and in what ways?
In addition to investigating the success of the research school scheme on these re-sults-oriented tasks, the design of the national research school scheme was to be evaluated. More specifically, the evaluation addresses these questions:
1. Evaluate whether the national research school scheme is organized in an appropriate way to achieve the goals that were set for the scheme (number 1-4 above)
2. Evaluate whether the resources and funding made available to the schools is appropriate
3. Consider various aspects that may explain the research schools’ goal achievement, such as:
a. Scientific and administrative leadership b. Cooperation between the partner institutions
c. Number of members (size of the research school, i.e., number of partner institutions and/or number of PhD students affiliated with the school) d. Procedures for recruiting PhD candidates
e. Work on internationalization
f. Involvement of supervisors and other scientific personnel at the partici-pating institutions
The methodological design developed by NIFU to answer the questions above are outlined in the next sections. Below we define two key measures in the evaluation
that may be understood and measured in several ways: completion rate and com-pletion time.
The completion rate measures how many of the PhD students that completes their doctoral training, that is, submit their thesis and get their doctoral disserta-tion approved. High compledisserta-tion rates imply that the number of unfinished PhDs – or drop outs – is low.
There are several measurement problems when estimating the completion rates of PhD students. Former studies have often used a ten-year threshold to cal-culate completion rates. That is, PhD students that have not handed in their thesis for approval within ten years after they started the PhD education are considered drop-outs (and considered likely never to complete). This definition cannot be used to evaluate the research school scheme as most students in the scheme have started their doctoral work only two to six years ago. The first five schools were established in 2008, but most students at these schools were admitted in later years. The same applies for students at the schools established in 2012. Only for PhD students that began their research training before 2013/2014 would it be possible to complete their PhDs within estimated time to degree (four or three years) by late 2017, which is our last observation point.
In a NIFU report from 2012 (Kyvik & Olsen, 2012) found that the completion curve flats out after 5-6 years, so that a reasonable choice of completion cut-off could be six years. Six years is also the choice made by the Norwegian Ministry of Knowledge (see e.g. the Ministry’s annual ‘Condition reports for Higher Educa-tion’). In this report the available data has restricted us to use a cut-off of five years.
Given the admission time for most students at the national research schools, a cut-off of the more preferable six years (or ideally ten years), would have excluded too many students for meaningful comparisons to be made. A cut-off of four years would have included even more students, but the comparison of completion rates would then be pointless since practically no one completes their degree within four years.
The completion time is far more unambiguous, since the selection of PhD stu-dents is fixed: all those PhD stustu-dents that have complete their PhDs, regardless of when they were admitted to a PhD programme or how long time that they have used. The disadvantage of this method, is that in many schools few PhD students have completed their PhDs, so that the completion time may be somewhat artifi-cially low as more PhD students will complete at a later time, thus increasing the average completion time. The advantage of this method, is that if we use this defi-nition while comparing PhD students from the research schools with other PhD students (see section 1.3.1), this artefact will be the same in both groups, thus making them comparable.