The following text will account for the preparation of the instruments used through pilot studies, procedures for gaining permission and the data collection.
3.4.1 Pilot studies
Three pilot studies were conducted, two in the fall of 2003, and one in January 2004. The intention behind the pilot studies was to see if information, questions and statements seemed clear and relevant, and to find out how much time was used doing the procedure.
Pilot 1:
The main focus here was to get response on the translation of the Q-samples. I asked three preschool teachers to try it out. I wanted to find out if the statements felt relevant and meaningful, or if any statements seemed unclear.
All the statements were on a sheet of paper and they were asked to write the category letter (A to E – from least to most) on each statement. One thought it was too difficult and did not do it. The two others thought it was difficult also, but did it and used from 1 ½ to 2 hours on it. There were some remarks to a few words that were used. With an exception of one or two statements, they all seemed relevant.
Pilot 2:
This time I tried out the following translated documents:
1. Orientation and invitation to teachers 2. Instruction concerning the instruments
3. Questionnaire (I combined the Teachers Self-Efficacy Scale and demographic questions in one folder)
4. Q-sort guide
5. Q-sample 1 + answer sheet
6. Q-sample 2 + answer sheet 7. Q-sample 3 + answer sheet
I used bright colors (red, blue and yellow) on the Q-samples, both answer sheets and statement cards. The answer sheets (Appendix II, III, and IV) were titled to the theme according to Q-samples 1, 2 and 3, and divided into 25 squares each. Five squares were used to the five categories from A to E ( Q 1:
From least to most characteristic of my beliefs about discipline and behavior management; Q 2: From ‘Those practices that are least essential and/or characteristic of my teaching’ to ‘Those practices that are most essential and/or characteristic of my teaching’; Q 3: From least to most characteristic of my belief system). The remaining squares were to illustrate where the statement cards were to be put. The statement cards were cut into squares to fit the squares on the answer sheet. Tape with glue on both sides was taped to each of these 20 squares on the answer sheet. When the informant had decided where she wanted to put the statement cards, she had to take away the top paper from the double-sided tape and attach the card to the answer sheet under the category that best expressed her point of view.
I tried all of these things out on two people with different educational backgrounds. The orientation and instructions worked well, and so did the questionnaire. They enjoyed doing the Q sorting, and liked the bright colors.
They only had some minor comments on some phrases. They used from 1 to 1
½ hours on all of it. The double-sided tape worked, but vas a bit difficult for those with very short nails. (I had to abandon the use of ‘stickers’ for the same reason and also because they did not come in the correct size.)
Pilot 3:
I had made some minor changes in the instructions and also changed one word in the questionnaire (translations issue discussed with second translator).
This time I tried it all out on four people, the same three from the first pilot study and one new person that had both teacher and preschool-teacher education but worked in first and second grade in school. This time I used a new type of tape that seemed easier to get hold of with short nails.
The feedback was that the orientation and instructions were clear and easy to follow. The tape worked well for all but one who had very short nails. Two used 45 minutes and two used 1 hour. There were no problems with the questionnaire. All four liked the bright colors and really enjoyed doing the Q sorting. Even the person who thought it was too difficult in the first pilot, said it was fun this time. I thought it would be especially important to get her impression of the instruments, because she had quite a bit performance anxiety. Only two of the six people I have asked had some problems not being able to put more than four statements under category E (most characteristic…) There were also comments on the need to reflect on several of the Q-sort statements to have a closer look at/become more conscious of their own opinions.
The experience from these three small pilot studies assured me that the information, instructions and instruments would work as intended.
3.4.2 Data collection
All the instruments, the instructions and a return envelope with prepaid postage and return address were gathered in an envelope where information about the study was pasted on the front. Each set had a code-number. Each daycare center and school received an envelope with the instruments etc for each teacher that had agreed to participate. When each teacher had finished, he or she was to put it all in the return envelope and post it. This was to ensure anonymity and the return envelopes were only opened by me.
Data collection was done in the period between March and June 2004. Among the 74 participants (29.1%) who had said they would be willing to be interviewed (an item in the questionnaire) six teachers were selected as described previously and contacted. They all agreed to schedule a time for me to interview them and to use a tape-recorder. During the interview, which lasted from forty-five minutes to one hour, they could look at the questionnaire and Q-sorts they had done. The interviews were conducted in the beginning of June before summer vacation started. They were later transcribed and rechecked with the recorded version before analysis.
After the data-collection was finished a thank you letter was sent to all the participating daycare centers and schools. The letter also contained information about the completed data collection and the two organizations that each received a prize. On behalf of all the participants, one daycare center and one school among the participating organizations were chosen to receive a book-check each with the amount of 500 Norwegian “kroner” as a “thank you” from me for participating in the study.