Summarising the policy shift in evaluation and assessment in higher education institutions

The massification of higher education and the size of HE systems appear to partly explain the need for greater formality of management (Westerheijden, Stensaker, et al., 2007: 2-3). Linked to the prevailing public policy climate of reform and regulation outlined in the previous chapter, quality assurance is considered by the authors as ―here to stay‖. Defining the purpose and form of QA are, however, not easily rectified not least because the purposes of HE are so diffuse and even though the central focus should be on student learning, the potential outputs are so difficult to identify, (Westerheijden, Stensaker, et al., 2007: 4).

Kogan notes that ―there has always been evaluation in higher education‖, based on certification and validation of knowledge and its producers (2004: 3).

However, the advent of massification, increased competition and ―political suspicion‖ of profession power has increased the importance of, as well as changing the nature of, evaluation (2004: 4). Evaluation developed as an

―instrument of public policy‖, highlighted further under NPM (Henkel, 2004:

86). Quoting Neave‘s (1998) idea of the evaluative state, Kogan goes on to recognise the continuing shift towards evaluation for ―policy adhesion‖, as well as a posteriori focus on product control rather than process investigation, with a purpose of steering HE more closely towards ―national priorities‖. This was also problematic as it was combined with ex ante / posteriori financing (Neave, 2004). Kogan sees this is as a collision with the traditional technology of HE, where academics set the agenda and quality criteria (2004: 6). The underlying problem is one of intention and decision for evaluation structures, that is, whether they should be ―purgative or developmental‖ (2004: 8).

Vedung refers to the evolution of a ―special evaluation tradition‖ within higher education⁹⁶, whereby ―professionals themselves carry out the evaluations against their own professional, mostly unwritten and tacit, quality norms in self-evaluations and against quality norms of their peers‖ (2003: 42). The underlying emphasis of such models is ―dialogue, discussion and deliberation‖, rather than goal-attainment or effects per se; an ―exercise in professionalism‖ rather than

―scientific exercise‖ (2003: 64). At the same time he recognises that a democratisation of the evaluation process has increased the role of the

―ordinary‖ stakeholder. There is a tension between the democratic focus of evaluation theorists favouring greater participation and the increasingly consumer oriented approaches more closely associated with New Public Management reform, as outlined in the previous chapter. The former is inclusion

96While Vedung applies this work to Sweden many of the observations are drawn more generally from and apply to the wider evaluation field.

97

and improvement focused where the latter focuses more on accountability and outcomes. Vedung recognises that in complex fields such as education there has been a principal of the public sector being ―profession-driven‖ (2003: 64-5).

This is tempered slightly by adopting peer-review processes to ensure some degree of parity with other public sector arrangements. Public policy changes and greater demands for accountability as well as impact assessments create a tension for these processes, as will be seen in subsequent sections of this chapter.

Reichert (2007) outlines six ―pre-conditions‖ for effective QA, split between individual and institutional responsibility. She argues that individuals must trust that evaluation will offer some benefit, which will involve exposing weaknesses and using ―time and effort‖ to rectify them. At the same time, institutions must build their autonomy, develop strong leadership capable of addressing change, and provide the necessary resources for change and development. This is supported by her recognition that a key limitation in quality enhancement raised by HEIs has not always been the ―nature‖ of the QA initiative but the resources available for follow up (Reichert, 2007: 6). Problems it seems are already evident to HEIs, but either ignored or shelved. Westerheijden, Hulpiau and Waeytens (2007) recognised systematic variations in the general QA model noting there to be distinct phases in the design and development of QA schemes.

These phases are thought to be linked to the social and policy context which affect the hierarchical development of these processes (2007: 298). Aside from external issues like HE policy, political climate, economy and demography, the authors also note the importance of ―internal dynamics‖, which result from ―the learning effects that result from the actors playing their part in subsequent rounds of quality assurance‖ (ibid.). This can be positive learning when academic staff who have developed capacity for self-evaluation become more engaged in contributing to the improvement of the institutional quality culture.

The authors also recognised negative learning, which took place when staff

―learn to play the tricks‖ of QA without it affecting the ―internal life‖ of the quality of teaching and research, also known as ―window dressing‖. There are further perceptions of underlying problems with QA at the micro level. Harvey and Newton consider the ―contention‖ to be how quality can be improved by

―asking an amorphous group of academics to identify their strengths and weaknesses‖ (2007: 226). The authors suggest that these activities, along with the arrival of external ―raiding parties‖ passing ―summary judgement‖, might lead to policy compliance, regulation or control without affecting quality per se.

The authors consider this a bureaucratic process removed from the basic activities of education and research. Henkel (2002) also reflects that changing the structures within Higher Education in England as a result of legislation have not resulted in a significant decrease of ambiguity within organisations. It is to this topic that I turn to next.

98

The structure of HEIs and accountability

Henkel agrees that these demands are clearly part of the wider public sector progression towards new public management, whereby HEIs have been

―required to increase their efficiency and to subscribe to various forms of quality assurance‖ (2002: 29). The author reflects, however, that this creates a challenge for HEIs, as they must balance commerciality with academic standards and an increased tension arises between ―mediating‖ central policy and maintaining and strengthening their institutional autonomy (2002: 30). Henkel noted that in England this was reinforced by a greater degree of direct intervention from the state as well as the requirement to improve management structures.

As part of these wider reforms, Henkel notes how organisational structures within HEIs have come under increasing attention, especially with regard to the long standing notion that decision making is based upon collegiality and community. The author recognises that this perception may have been overemphasised, and notes that there has often been difficulty in resolving conflicts that are endemic within generally loosely coupled systems (2002: 30).

It is interesting though that she further states, drawing on the work of Bargh et al., that accountability has not in of itself been merely about improving and tightening structures, but rather that accountability appears to be viewed with regard to societal interests including those of the state. It might appear, though, that the latter have become synonymous with central demands.

Within these accountability focused systems, QA plays an increasingly more important role. Henkel‘s research notes that the policies linked to QA are linked to the allocation of resources, which in turn is linked to organisational reputation (2002: 33 - 34). Henkel suggests that as a result ―academics are under constant scrutiny by senior managers‖ within a system of ―growing insecurity‖, whereby academics feel they must meet the needs of administrators rather than the other way around (ibid: 34). This is of course a nuanced position, and Henkel recognises that despite all the changes within the system, leaders continue to insist that it is the academic at the base level that drives ―institutional success‖.

Becher and Kogan (1992: 169) offer three, ―potentially conflicting‖, modes of accountability: public contractual/managerial, professional and consumerist⁹⁷. They note that all these modes influenced higher education from the 1980s.

Brennan and Shah recognise that these are related to internal decision making in HEIs, coming to the heart of value structures across groups as well as those held by individuals (2000: 33). The public contractual mode is about performance related to collective policy models, whereas professional accountability focuses more on the intrinsic quality of a particular subject related to the values within

97 These modes also appear similar to Bleiklie‘s (1998) conception of universities as government agencies, cultural institutions or corporate enterprises which later developed into 4 expectations and are outlined further down (2004).

99

that academic field. The consumerist mode is more greatly focused on responding to market demands. These three modes offer a useful framework against which interview responses can be considered. They are, however, anticipated to exhibit a degree of overlap.

The development of HEIs has led to four major expectations or templates of how they should be organised via: academic quality, collegial coordination, social responsibility and business enterprise (Bleiklie, 2004). The fourth template is a departure from more traditional ideas of Higher Education, where the HEI is considered a ―producer‖ of ―quality‖ services. Bleiklie remarks however that whilst ―quality and ‗quality assurance‘ are emphasised as fundamental goals, the most important expectation… is the efficiency with which it produces useful services… to the benefit of the users of its services‖

(Bleiklie, 2004: 14). This became increasingly more noticeable in both Norway and England from the 1990s, where despite different emphases of policy there was a central ―concern‖ about costs of HE and greater interest in the ―product‖

(2004: 18).

As was noted above, in recent times the shift in England has been from higher education run by state bureaucracy to greater autonomy for HEIs in an attempt to produce a more flexible, deregulated governance driven system ―responsive to contextual (societal) demands‖ (Westerheijden, 2007: 75). At the same time, Westerheijden notes that both sides of the ―North Sea‖ were increasingly interested in value for money, rate of return to society and economy, opening for market mechanisms and consumer choice. Change should be improvement focused and instrumental, rather than incremental, and HEIs held accountable (2007: 76). Images of quality higher education were thus redefined.

Questions are raised, then, as to how much change is visible as a result of legislation to change the management style and decision structures across HEIs.

Despite the development of more hierarchical structures Henkel claims that there is still a ―high degree of organisational complexity and ambiguity‖, and this appears to be moderated by age and culture, or how ―traditional‖ the HEI under investigation is (2002: 35). Such categories are of course difficult to define, but Henkel (2000, 2002) suggests that academics are less likely to express themselves as managers within more traditional universities. In addition, ambivalence was a common feeling amongst academics concerning being a manager. Henkel‘s findings could, perhaps, be considered also to illuminate attitudes to decision structures and processes, in this case concerning what the focus of an evaluation should be, how should it be decided and who should be involved in that process. There are of course many other factors that will influence such attitudes and perceptions, including prior experience of those involved, both inside and outside of the organisation of which they are members.

The most important point is though that the introduction of such principles into the academic arena will be tempered by institutional values and traditions, which will often become a source of greater ambiguity, despite attempting to bring

100

greater clarification (2002: 37). The development of quality assurance systems, as well as external organisations, designed to control and oversee implementation and progress, is described by Dahler-Larsen as part of the institutionalisation of the quality wave (2008: 67). The author notes a shift of focus towards the quality of way organisations act rather than what they offer, with greater interest in environmental acceptance than resolving instrumental tasks (ibid.). These perspectives will be outlined in more depth in the next chapter.