4.4 Method
4.4.1 Structured Literature Review: How It Was Done
In SLR, it is first important to define a protocol that is going to be used in the search phase. This includes defining which terms to use and where to find the information. Then, a lot of articles will then be collected using this protocol. If there are too many articles, it may be preferable to add more specificity in the search. Then, the filtering begins. This is done by removing one by one until only the relevant results are left. In the end, the classification of quality and inclusion criteria is done on the left results.
Defining the search protocol
To perform the search, there were chosen three key terms; physics, computational thinking, andsecondary school. The main idea here is to find articles that overlap with physics, CT, or programming, and all kinds of education that introduce physics for the first time. It was also preferable to include a fourth element,teacher created tasks, in this thesis, but as it was hard to find any results, including this, it was left out intentionally as it returned too few results. Figure4.2 attempts to explain what that was trying to be achieved.
Figure 4.2: Our keywords and their relation in the search.
Based on these key terms, it was essential to find synonyms for each of them.
This is done in table4.2. It was also chosen only to find results that were published
between January 2010 and January 2020. The search was only performed on abstracts, titles, and author keywords.
Group 1 Group 2 Group 3
Term 1 physics computational thinking introductory Term 2 algorithmic thinking beginner
Table 4.2: Terms and synonyms (groups) used in query. Table structure inspired by Kofod-Petersen (2015).
More specifically, the query is defined like a boolean search:
”physics”AND(”computational thinking”OR”algorithmic thinking”
OR ”programming” OR ”coding”) AND (”introductory” OR
”beginner”OR”secondary school”OR”high school”OR”pre-university”
OR ”STEM education” OR ”K-12”)
By using quotes around all terms (””), it was possible to avoid that search engines will stem the search terms. I.e. “beginner” will not match with
“beginning” and “begin” if quotes are added.
Retrieving the literature
For this SLR, five different databases were used: ACM Digital Library19, IEEE Xplore Digital Library20, Science Direct21, Scopus22, and Web og Science23.
The search in each database required different syntax to achieve the same query. For reproducibility, a list of links has been provided to each of the databases specifying all the criteria that were defined in the search protocol, in the table below. The table also contains the number of results that were returned as of January 2020.
Database Query Results
ACM Digital Library Link to query 22 IEEE Xplore Digital Library Link to query 12
Science Direct Link to query 32
Scopus Link to query 136
Web of Science Link to query 83
Total 285
4.4 Method Selection of primary studies
After retrieving the literature using the protocol specified earlier, there was received 285 matches in total. Thus, as only studies from five separate databases were retrieved, many of the studies were duplicated. After removing duplicates, only 212 unique matches were left.
A lot of the studies were not relevant. It was, therefore, possible to remove the most obvious ones. This resulted in excluding 130 studies, reducing down to 82 potentially relevant studies.
Study quality assessment
In this step, a more thorough study of the studies needed to be done. All of them had some level of common ground with the field that was searched for, but only the studies that were thematically relevant should be included. To achieve that, it was needed to create a few criteria that defined what to look for in each study.
The ones that did not match any of the criteria, or matched the excluding ones, were excluded.
The criteria, defined in table 4.3, were divided into quality criteria (QC), inclusion criteria (IC), and exclusion criteria (EC). The inclusion criteria were then again divided into primary and secondary groups, where the primary group contained the studies that were spot on the topic, and the second group contained the studies that had some important elements that were preferred.
Criteria ID Criteria
IC1 Learning introductory physics using CT or programming IC2 Focuses on learning physics (less focus on CT or programming) IC4 Uses a introductory physics course in their evaluation
EC1 Focuses on learning CT or programming (not physics) EC2 Focuses on higher level physics courses
EC3 Does not contain CT or programming
EC4 Learning introductory physics using other approaches QC1 (t24=0.5) Is there a clear statement of the aim of the research?
QC2 (t=1.0) Is the study results reproducible?
QC3 (t=0.5) Does the study compare their approach with other approaches?
Table 4.3: Inclusion and quality criteria. Inspired by Kofod-Petersen (2015).
First, the results left went through all the inclusion and exclusion criteria. It started with a different set of criteria initially, but after seeing that the criteria were too vague, they got refined step by step, as well as more exclusion criteria, were added. It is possible to see the ending criteria in Table 4.3. However, this resulted in having nine studies left.
At last, it needed to evaluate them on quality. This was done by giving quality points based on the specific quality criteria. If a quality criterion was not filled, they got 0 points. If it matched, they got 1 point. If they partly matched, they
24Points that are needed to pass the quality criterion threshold (t).
got 0.5 points. A threshold was then set for each criterion that decided which studies that should pass or not, based on their points on that given criterion. See the t-values at the bottom of Table 4.3. Thus, after evaluating all the studies, they all passed the quality test.
Final studies
The studies that were included in the end result can be found in Table 4.4.
Title Reference Year
C2STEM: a System for Synergistic Learning of Physics and Computational Thinking A hybrid approach for using programming exercises
in introductory physics
Orban et al.
(2018)
2018 A novel approach for using programming exercises in
electromagnetism coursework
Orban (2017) 2017 Developing students’ creativity by Physics lessons Marciuc and
Miron (2017)
2017 Using GeoGebra and Vpython software for teaching
motion in a uniform gravitational field
Marciuc et al.
(2016)
2016 The effect of computer science on physics learning in
a computational science environment
Taub et al.
(2015)
2015 Computational problems in introductory physics:
Lessons from a bead on a wire
Bensky and Table 4.4: The studies that were included after the final selection phase in the SLR.