The latter question gave the pupils the chance to add any thoughts or opinions they had not been able to express through the interview so far, and it was also a way of ending the interview.
Each interview lasted for 20-30 minutes. All the pupil interviews took place at the school. They were carried out in three separate meeting rooms provided by the teacher and during the pupils’ English lessons. The process of interviewing began during the last week of the CLIL project and was finished after the project had ended. All the interviews were
audiotaped and only brief notes consisting of keywords were written. This allowed the interviewer to be attentive to each participant’s body language and response in order to ask quick follow-up questions and register any comprehension difficulties. There was some concern whether the interview participants would have sufficient memory of the project and the information the interviewer was requesting. For example, when the participants were asked about the texts, they had difficulties recollecting all of them. Another limitation was that the participants were not always able to provide in-depth reasons for their responses.
Thus the data included some short answers, such as It was fun, or It was different without any further explanations.
4.7 Questionnaires
Questionnaires have the advantage that the process of collecting the data takes fairly little time. However, by using questionnaires, it is not possible to ‘probe deeply into respondents’
beliefs, attitudes, and inner experiences’ (Borg et al. 2003: 222), which differs from the interview situation, where the interviewer can seek a deeper understanding of statements.
The design of questionnaires is structured and more standardized than other methods.
The items can be closed or open in form. Closed-form items allow for the participant to answer by choosing between prespecified options, similar to doing a multiple choice test (Borg et al. 2003: 227). Open-form items are questions where the answer is written freely and in the participant’s own words. The data collected through the latter form makes for more time-consuming analysis, and it demands more writing. In addition, closed-form items are easier to analyse and quantify (Borg et al. 2003: 228). They produce numeric data, which makes it easier to compare the participants’ answers.
48 4.7.1 The pre-project questionnaire
The pupils were given a questionnaire at the beginning of the project in the last week of October and a second questionnaire in March after the project had ended. The questionnaire was in English. The permission slips were required before the pupils could answer the first questionnaire. Due to the items in the questionnaire, the teacher chose not to inform the pupils too much, and did not start the project properly until the questionnaire was completed by the pupils. All the 29 pupils in the English class signed the permission slip and agreed to
participate in the study. The questionnaire was anonymous. Initially on the sheet, the researcher’s identity was revealed, in addition to an instruction where the pupils were asked not to write their names and to answer the items as honestly as possible. In order to cover a broad range of topics and a large number of questions, the questionnaire consisted mainly of closed-form items, namely 20 closed-form Likert-scale items and one open-form question (see Appendix 2a). Answering open-form items demands more writing and is time
consuming. The open-form question required the pupils to write two sentences about: What do you think it is going to be like to work on this Second World War project in English?
The majority of the closed-form items asked the participants to choose to which extent they agreed or disagreed with the item. According to Borg et al. (2003: 228), a questionnaire measuring attitudes using an attitude scale needs to use at least ten items in order to ‘obtain a reliable assessment of an individual’s attitude’. The attitude scale consisted of five options in a range from ‘strongly agree’ to ‘strongly disagree’, a common scale design. The number of items was substantial, containing 19 attitudinal Likert-scale items and one item where the participants were to range six activities on a scale from 1 to 6 according to how well they learn by using them. This item proved confusing for the pupils to answer as intended and was excluded from the data presentation.
The items in the pre-project questionnaire were chosen in relation to the research questions and were designed to cover a number of topic areas, namely ‘expectations’,
perceived ‘benefits’ and ‘enjoyment’, in addition to ‘interest’, ‘beliefs’ and ‘attitudes’. Some examples of the items are:
I think it will be difficult to work on this project.
I look forward to using English in order to learn about History.
I enjoy reading English texts.
49 4.7.2 The post-project questionnaire
The second questionnaire was designed to be comparable with the first one and was also in English. It consisted of 20 closed-form Likert-scale items, and two open-ended questions requiring the pupils to write one sentence to each question (see Appendix 2b). The items explored the pupils’ experiences, attitudes and beliefs about content-based learning in a second language compared to learning in Norwegian. It questioned the pupils about how they enjoyed the various activities. In addition, items were added as a consequence of knowledge gained through the observations and interviews, for example: The project lasted for too long, The texts were too difficult, There were too many texts to read and We watched too many films.
The items, or statements, were constructed to elicit responses in five categories, namely ‘enjoyment’, ‘interest’, ‘motivation’, ‘perceived benefits’ and ‘challenges’. Examples of items in the ‘enjoyment’ category were: I enjoyed working on this project, I enjoyed presenting my chosen work to others and I would like to have more projects like this.
Examples of items to reveal the pupils’ interest in the topic and the materials were: I have become more interested in World War II after working on this project and The texts in the project were more interesting than the texts in the textbook. However, it should be noted that the project did include texts from English textbooks, but not the class’s textbook. In terms of motivation, pupils were asked whether having the opportunity to choose between several optional tasks was motivating. As for possible ‘benefits’ and ‘challenges’, the pupils were asked to express their development with regard to language and content, e.g. I learned a lot of new English words by working on this project and I learned a lot about the topic by working on this project. In order to report what the pupils considered as challenging in terms of
language or materials, they were presented with several items, e.g. I would have learned more if the project had been in Norwegian and The texts were too difficult.
The two open-ended questions were placed at the end of the questionnaire. The
intention was to collect data about the pupils’ opinions on learning language to explore a topic and how they had generally experienced the project. The open-ended questions had a greater opportunity to provide more in-depth information about all the participants’ experiences of the CLIL project, in their own words. The formulation of the first question may have been too difficult for some pupils and therefore some did not answer it.
The researcher handed out the questionnaire and explained the importance of the pupils’ participation, as well the procedures for answering it. The researcher encouraged the
50 pupils to raise their hands and ask when faced with any problems. Results showed that even though none of the pupils asked for assistance, it did not mean they were not faced with difficulties. 22 of the 29 pupils were present. The seven pupils not present were later handed the questionnaire by the teacher, who made sure they completed it.