V ALIDITY AND RELIABILITY

In order for the study to exert any influence on theory building or the field of practice, the findings must contain a certain level of credibility and trustworthiness. The insights and the inferences transmitted must be perceived as credible to scholars and practitioners within the actual field of enquiry. Moreover, the findings and inference must be transferable to a larger social grouping than those under investigation. The issue of reliability and validity must therefore be addressed at the earliest possible stage, especially in the construction of the research design.

112 5.8.1 Reliability

Reliability refers in essence to the extent to which the findings of a study can be replicated (Yin, 1994). It must be likely that a subsequent research project would end up with similar findings, given the identical research questions and the similar use of procedures, tools and techniques. However, most research is not replicated by other scholars or students in practical terms.

This ideal must therefore be realized through other means. The reliability criterion then boils down to the extent to which other researchers would be able to reach agreement - that the findings and the inferences make sense. It is argued that the reliability of a qualitative research design can be assessed through dependability and consistency. Dependability is ensured through the development of an audit trail in the study that provides both stability of the data and makes it possible to control and assess the tracks of the stream (Lincoln & Guba, 1985). An audit trail has been developed through the computer system N6 aimed at documenting the researcher’s steps and movements through data collection, analysis and interpretation. The data sources, i.e. the verbatim transcriptions from the interviews, are carefully ordered in the computer system, so that others can easily retrieve each data entity. The text units are organized so that it is easy to follow the interview dialogue systematically.

Furthermore, the researcher’s annotations and interview memos are connected to the text unit to which they belong. By using a simple function key, a reviewer can inspect the evolvement of the researcher’s interpretative work during the interview dialogue. The text units are also carefully marked with codes that easily link them to the informants and the research sites they are drawn from. In this respect, text units can be controlled and compared systematically with the interpretations made in the case memo from each of the research sites. Besides, qualitative scholars emphasize that it must be possible to assess and confirm the extent to which the interpretations of the researcher are grounded in events rather in the inquirer’s personal constructions. In the study reported in this thesis, building a systematic audit trail of the reporting procedures has been an important vehicle in the efforts of ensuring reliability in terms of a transparent body of data that can easily be trailed by external parties. Audit procedures are therefore the fundamental operations through which external reviewers can ensure the credibility of the study.

Consistency is sought to be achieved through triangulation of methods and replication of observations. Issues, themes and whole interviews are replicated in the same uniqueness of context, in order to prevent

113

‘superstitious’ or biased inferences. The critical incidents and patterns in particular have undergone crosschecking and replication.

5.8.2 Internal validity

Internal validity concerns the internal logic and consistency of the research, and refers to the extent to which the findings and inferences are based on valid interpretations and comparisons (Punch, 1998). Internal validity can be enhanced by triangulation, where multiple data collection methods and analysis procedures can strengthen both reliability and validity.

Triangulation involves crosschecking findings by using different sources of data. Reliability and internal validity considerations were imputed to the design all from the early stage, through the proactive use of multiple data sources in concert. For example, critical events emerging during interviews were, as far as possible, crosschecked against secondary documents collected from the research site. Such documents were for example minutes of meetings, protocols, plan documents, time schedules, annual reports, project reports and student evaluation reports. A considerable body of secondary data documents was therefore gathered during the interview processes, which were closely related to the topics on the agenda during the interviews.

Another practice that may promote internal validity of the study is through contesting data and interpretations of interviews against the people and groups to which the interpretations refer. For example, themes referring to collaboration among teachers were taken up in interviews with middle managers and principals. These themes were then taken up in team interviews and were also crosschecked against document evidence. The internal validity was also sought to be increased through the replication of interviews, in terms of follow up interviews of the same participants focusing on the same themes and issues as in the first session. In this way, the criteria of credibility, consistency and truthfulness of interpretations and findings were sought to be addressed. The interplay between the techniques employed is sought to provide internal validity to the study.

5.8.3 External validity

External validity refers in principle to the potential for generalization based on the findings of the study. Are the inferences transferable to other settings and contexts? The researcher has clearly laid out the details of the research

114

context, data collection and analysis, so that external reviewers and readers may judge the issue. Three of the chapters of the thesis lay out the characteristics of the context under investigation, i.e. the educational field as well as a detailed description of the research sites of the main study. This enables the reviewer to make independent judgments about the extent to which interpretations and findings are bound to the specific context under investigation.

Moreover, what are the boundaries and limitations of the transferability?

Some of the findings will no doubt refer to context specific contingencies;

for example, the context bound variables inherent in the distributed training curriculum found in the Norwegian context. Triangulation is also recommended as a tactic in the pursuit of external validity. The main case study is conducted through two sub-cases that are close to most similar cases, which in itself represents a replication method. Another triangulation strategy has been to conduct the interplay between in-depth case study investigation and analysis of large-scale secondary data. In this way, a correspondence between field-specific evidence and case –specific evidence has been established.