S TUDY 1: THE QUANTITATIVE APPROACH

5.2.1 Study design

Randomised controlled trials are considered the gold standard method for evaluating quality improvement, service delivery and healthcare interventions to yield reliable

evidence between cause and effects.¹⁰¹ The stepped wedge RCT design was used in the unidirectional crossover delivery of the WHO SSC intervention.²⁰ The stepped wedge is a pragmatic study design which can reconcile the need for robust evaluations of service delivery and patient interventions such as the SSC.⁸⁷ This is particularly evident with interventions expected to do more good than harm. The design requires randomisation of clusters to different sequences (Figure 4). These sequences dictate the order (or timing) in which each cluster will switch to the intervention condition, described as the step.¹⁰² Implementing complex interventions, such as the SSC, requires sufficient recourses available.⁸⁰ Thus, the stepped wedge cluster RCT fulfils a dual role, serving as a scientific tool that incorporates a fair way to determine the order of rollouts under logistic constraints.⁸⁷ Moreover, to avoid contamination of the SSC from the intervention to the control group, randomising organisations or healthcare professionals in clusters was performed as recommended, rather than at the individual patient level.^{86 87 102}

There are methodological complexities involved in using the SW cluster RCT design, which may increase complexity of reporting, such as potential confounding with time, possibility of contamination within a cluster and time varying treatment effects.¹⁰² However, implementing the SSC by use of the stepped wedge cluster RCT was considered the best suited RCT design for this intervention. The Consolidated Standards of Reporting Trials (CONSORT) has been updated and published in 2018 to include an extension for the stepped wedge cluster randomised trials.¹⁰² As the initial implementation study of the SSC intervention was conducted in 2009-2010, the study was reported in compliance with the CONSORT guideline at that time

(Appendix 10.5).

5.2.2 Validity

Validity is a quality criterion referring to the degree to which inferences made in a study are accurate, well-founded and in measurement: the degree to which an

instrument measures what it is intended to measure.⁹⁰ The researchers involved in the primary data collection, analysis and handling of the original study²⁰ were also engaged in this follow-up study. Access to data, assessment of the identified data set

(in terms of the appropriateness for the research question), adequacy of data quality and technical usability of the data were, therefore, endorsed by the research group. By performing the secondary analysis together with the original research group, the samples and variables previously measured were well known, and so the risk of detecting deficiency in the data set was reduced. The following four types of validity will be discussed in relation to Study 1: internal, statistical conclusion, construct and external validity.

Internal validity

The internal validity concerns the validity of inferences that, given the existence of an empirical relationship, it is the independent variable: here meaning the actual

utilisation of the SSC that caused the outcome, rather than other factors.⁹⁰ The secondary analyses were based on data previously collected, and detailed descriptions of the data collection method, quality controls and definition of SSC compliance cut- off have previously been reported.^{20 73 91}

The timing of SAP administration was retrieved from the patients’ medical records and registered as categorical data. To reduce the risk of threat to internal validity, the corresponding categories were agreed upon in the research team prior to the data collection. In case of several SAP administrations, we classified provision of SAP according to the time point of the first dosage. Also, for SAP infusions > 500mL, provision of SAP was sorted according to the time-point at the end of the infusion. In contrast to SAP injections, or the short time infusions <100mL, the former might endure for 30-60 minutes. To further ensure the validity of case classification of SAP provision, ambiguities were discussed among HVW and ASH.

Another threat to internal validity is that routinely collected data might be hampered by random errors or inaccuracies in data quality. However, using the routine data registered in the daily clinical practice reflects the “real world”, and there were no changes in how the perioperative data were recorded during the study period. Also, the healthcare personnel who registered the perioperative care processes were employed in the specific surgical units constituting the surgical clusters. Thus, they

were part of the same cluster before and after the SSC introduction. The random errors were, therefore, likely to be equally present before and after the SSC

intervention. The healthcare personnel were also blinded as to which measures were of interest to the study. This applied to process data as well as the patient outcomes.

To avoid the bias in detecting positive SSC effects, the intervention arm of the study equalled intention to treat.

Statistical conclusion validity

Statistical conclusion validity concerns the validity of the inferences, where the inferences are that there truly is an empirical relationship, or correlation, between the cause and effect.⁹⁰ Statistical power was based on the sample size calculation from the original study.²⁰ As we performed a secondary analysis, there was no possibility of increasing sample size to avoid risk of committing a type II error (accepting the null hypothesis when it is false). Yet when the relationship between variables (effect size) is considered strong (as assumed with recommendations in perioperative guidelines and patient outcomes), effects can be detected statistically significant even with small research samples.⁹⁰ Given the available process measures, the statistical analyses were performed as appropriate for the different variables. A limitation to the study was lack of patients’ core temperature as a parameter. Also, we had no

available measures for important items such as preoperative risk assessment or team briefing. Such variables, although difficult to measure, might have influenced the statistically confirmed relationship.

Construct validity

Construct validity involves inferences from the particulars of the study to the higher- order constructs that they are intended to represent.⁹⁰ One threat to construct validity is the effect on the dependent variable resulting from the healthcare personnel’s awareness that they are participants under study, known as the Hawthorne effect. To reduce risk of information bias, all clinical personnel participating in the SSC performance were not informed as to which study outcomes were measured. Yet the risk of healthcare personnel crossing over from intervention to control was also

present, in particular for the junior anaesthetists who did rotations between the surgical units. However, the substantial decrease in complications found in the study based on the original dataset indicate that such bias did not affect the study

significantly.²⁰

External validity

External validity concerns inferences about the extent to which relationships observed in a study hold true over variations in people, conditions, and settings, as well as over variations in treatments and outcome.⁹⁰ Even though data were included from only one hospital, the surgical specialities involved represent heterogeneity, which increases the external validity of the study. In addition, the included perioperative care processes measures and the use of the WHO SSC relate to universal

recommendations of evidence based guidelines of perioperative patient safety, which also supports external validity of the study results.^{10 12}

5.2.3 Reliability

Reliability refers to stability of measures when repeatedly used.⁹⁰ The quality performance of the SSC’s utilisation, involving registration of items listed in the SSC and describing cut-off limit of SSC fidelity, were quality assessed as previously described.^{20 73 91} Categorisation of measures involving SAP provision, were quality assessed by categories previously agreed upon, and ambiguous cases were assessed, together with clinical expertise, to reach consensus.