6 Discussion
6.4 Methodological considerations
6.4.2 Validity relating to study design
The results in the present thesis are based on quantitative designs. Studies I and II had a longitudinal design, while the cross-cultural Study III had a cross-sectional design. Both internal and external validity, of course, depend on valid interpretations of test scores and their implications (construct validity) as well as on statistical validity.
6.4.2.1 Internal validity
Internal validity is the validity of the inference from an observed covariation to a causal interpretation (Kleven, 2008; Shadish et al., 2002). Internal validity is important whenever an inference is made to the effect that something influences something else. The data used in the present thesis were correlational and so represent a sufficient logical basis for prediction (when they are longitudinal, as in Skoleklar), although the correlational design precludes firm conclusions (Kleven, 2008).
Experimental designs have been recommended to eliminate possible threats to internal validity (Shadish et al., 2002). Kleven (2008) has suggested that although studies do not have an experimental design and cannot state causality, researchers should carefully discuss possible causal relationships between variables. However, inferring possible causation from correlation requires an approach that considers theory, design, data replication, and causal assumptions, only some of which are empirically verifiable (Kline, 2016).
The analytical models used in the three Studies in this thesis were specified by theory and previous findings (Kline, 2016), which is important because, for example, a path model may be wrong even if it
Discussion
fits the data very well (at least in a non-experimental design) (Kleven, 2008). Thus, even if the design of this thesis could not state causality, the studies could give support to theory and previous evidence. Moreover, Study II had a longitudinal design and investigated indirect effects, controlled for prior skills and covariates, and included several mediators.
For example, it found that self-regulation in ECEC predicted mathematical skills in first grade (controlling for ECEC mathematics), which in turn predicted fifth-grade mathematical achievement.
According to Hayes (2013), including correlated mediators in the model makes it possible to disentangle spurious associations from potential causal associations.
In regression analyses and path analyses, controlling for other relevant variables is important to get closer to a possible causal interpretation.
Thus, omitted variables may be a potential for bias. Depending on the correlations between measured and unmeasured variables, estimates of direct effects can be too high or too low (Kline, 2016). In the path analysis for reading comprehensions (Study II), for example, the predictors explained 35% (R2 = .35) of the variance, suggesting the critical importance of other (omitted) variables, such as listening comprehension, rapid-naming speed, and intelligence. Thus, other variables that were not included in the models may have biased the results.
Theoretically, the observed effect of gender and SES on children`s skills (Studies I and III) are assumed to not work in the opposite direction.
However, the present thesis could not and did not aim to uncover the causal mechanism underlying the effect of, for example, maternal education on child outcomes. Instead, it aimed to investigate the role of maternal education in predicting children`s academic skills in the transition from ECEC to first grade. This thesis did not have measures of parenting or teacher behavior or any proximal processes. As such, it could not examine any possible mediating mechanism. Instead, the results were discussed in light of theory and previous evidence.
Discussion
Internal validity is always local. Any possible causal conclusions drawn are limited to the particular context studied (Kleven, 2008). Whether such possible conclusions can be generalized or transferred to other contexts is a matter of external validity, which is discussed below.
6.4.2.2 External validity
External validity concerns the validity of inferences made from the context of a study to a wider context or other contexts (Kleven, 2008;
Shadish et al., 2002). Generalization over situations, groups, and persons is dependent mainly on similarities and differences between the situations or persons studied and the situations or persons with regard to which inferences are drawn.
The present thesis is based on a convenience-sample approach. Data in both samples were collected in rural counties, which were not nationally representative. For example, the Norwegian sample had few children with immigrant status, especially in the fifth-grade data. In the U.S.
sample, all children had attended preschool (because they were recruited through preschools), whereas only 46% of children in Oregon aged three or four years were enrolled in a preschool in 2016 (Early Care and Education Profiles, 2018).
Some general advice with regard to generalization has been given (Cronbach, 1975; Kleven, 2008). First, it is suggested that results (knowledge claims) should be considered context-bound. This is in line with the framework of the Bioecological Model of Development. In fact, children’s developmental outcomes vary systematically based on context characteristics, meaning that results must be discussed in relation to their context. In the present thesis, an effort has been made to heed this advice.
Second, generalization should be considered as a working hypothesis rather than as a conclusion. The third suggestion is to study the same phenomenon in other contexts to see whether the same results are obtained there. This is fundamental to the present thesis. Not much
Discussion
quantitative research into young children’s development has been conducted in the Norwegian cultural context, and it is important to investigate whether results found in this context align with prior evidence using other samples from other cultural contexts. For example, prior international studies have found early self-regulation to be foundational for later academic achievement. Study II investigated this association in a Norwegian educational and social context. Finally, it is important to pay attention to exceptions as well as to results confirming “the rule,” as exceptions may indicate context-specific conditions. For example, Study III did not find maternal education to significantly predict children’s self-regulation in the Norwegian sample, contrary to most prior findings using U.S. samples.