D EVELOPMENT OF SURVEY INSTRUMENT

A questionnaire was developed and used to collect the survey data. Items in the questionnaire measured concepts drawn from management theory and the information systems literature. Each construct was measured using mul-tiple items. The items, with the exception of some demographic data such as gender and educational background, were measured on 7-point Likert scales.

The definition and operationalization of each measure is described in greater detail in section 5.4.

To ensure that the instrument was valid for use, a draft instrument was quali-tatively and quantiquali-tatively pre-tested as suggested by Straub (1989). Instru-ment validation was done through three operations — pretest, technical vali-dation, and pilot test. Then a full-scale survey was conducted.

80

5.3.1 Pretest

In the pretest, the draft questionnaire was subject to a qualitative testing of validities. Personal interviews were conducted with a number of participants in order to locate and correct weaknesses in the questionnaire instrument.

Interviewees included: an academic expert in methodology, an academic expert in IT outsourcing, an academic expert in language and business cul-ture, two practitioner experts in IT outsourcing, and two transplants affected by IT outsourcing. In addition a technical expert in QuestBack was con-sulted. The selection of interviewees was designed to get maximum feedback from various experts. The total number of interviewees in the pretest was eight.

Interviews were both formal face-to-face, on-site meetings with practitio-ners, and more informal (and repeated) discussions with academic and tech-nical experts. The duration of each practitioner interview was approximately 45 minutes to an hour. At the beginning of each interview, respondents were informed they were participating in a pre-test, and, in addition to responding to the questions, respondents were also asked to comment verbally on the conceptual clarity and word usage in the questionnaire. On the average, re-spondents took about 15 – 20 minutes to complete the questionnaire. The remaining time was used to discuss each respondent’s reaction to the ques-tionnaire. Overall, respondents had little trouble understanding the question-naire.

One respondent explicitly commented that the questionnaire was too long, but others, when asked to comment on the questionnaire’s length, did not express fatigue or lack of interest. One respondent suggested that a change from English to Norwegian language would ease respondents’ completion of the questionnaire. “At least the researcher should avoid the use of difficult academic terms,” the respondent suggested. The respondent pinpointed the use of some advanced business English terms, and he had to use a dictionary a couple of times to make sure he had understood the questions. Since the vendor company is a global outsourcer, they had English as their formal business language. Discussing this problem with the chief human resource officer of the vendor company, he told me the employees were used to an-swering internal questionnaires in English. And thus the researcher con-cluded to continue with the English version of the questionnaire. One re-spondent also reported a need for better positioning of questions. Another respondent underlined the difference between business cultures in the two outsourcing parties. The client company had a value-based management and the vendor culture was based on top-down management, he said.

As the intention was to collect data through a web-based tool, several of the respondents suggested putting some effort into writing an introduction email.

81

Included in the introduction email was: a short presentation of the research project, a statement of support from the outsourcer’s top management and the Works Council, and an assurance of confidentiality and anonymity. The last point was important since the level of analysis was at an individual level.

The pretest also led to the development of two screening questions at the beginning of the questionnaire to make sure respondents were delivering service back to their prior employer. This was done to make secure respon-dents were in the transplant target group. This screening was also explained to the respondents in the short introduction to the survey. Finally, an aca-demic expert in language and business culture reviewed the questionnaire.

The language expert, fluent both in Norwegian and English, checked the questionnaire for consistency, flow, imprecise word use, and obvious mis-takes.

The intention of the pretest was to ensure clarity and readability of the ques-tionnaire and to ascertain that the theory-based items tapped issues of con-cern to the respondents (content validity and reliability). Clarification of constructs and the means of the operationalization were also undertaken (construct validity). Content validity was addressed by encouraging partici-pants to single out pointless questions and suggest new areas of inquiry.

5.3.2 Technical validation and pilot test

To further validate the instrument, a pilot survey was carried out. The pilot test disconfirmed measurement problems in the instrument. Besides their use in validation, pilot tests are also desirable because they provide a testing ground for final administration of the instrument. A small department, geo-graphically separated from others, was selected as the site for the pilot study.

The questionnaire was sent to thirteen employees, with a data collection period of one week. Eight employees responded, which gave a response rate of 61.5% for the pilot study.

In the introduction the respondents were told they were participating in a pilot test of the questionnaire. In addition to responding to the questions, they were asked to comment verbally on the conceptual clarity and word usage in the questionnaire. A separate textbox for this purpose was available at the end of the questionnaire and they were also invited to make a tele-phone call. Three respondents made a tele-phone call to the researcher and five respondents filled out the text box at the end of the questionnaire to give their comments. Respondents agreed upon the relevance of the research. In unison, they suggested using Norwegian in the final questionnaire. They were used to speaking technical English. As they did encounter some diffi-cult business or management terms, they had to use a dictionary to make

82

sure they understood all of the questions. Thus the average time to fill out the questionnaire was around 20 minutes.

The purpose of technical instrument validation is to validate construct valid-ity and reliabilvalid-ity. If measures vary little across methods of measuring those variables, they can be said to be independent of the methods used to gather the data and to demonstrate high construct validity. This research has adapted existing measures from the literature to ensure strong construct validity. Reliability is a statement about the stability of individual measures across replications from the same source of information (multi items scale). If enough respondents are inconsistent in their answers to these items, the items will contain abnormally high measurement error and hence, unreliable measures. Appendix D shows that Cronbach alphas in this study were not very different from alphas in other studies.

5.3.3 Revisions to construct definitions and measurements

Two pointless questions were removed from the questionnaire. These items were pinpointed from several respondents in the pretest and the pilot study.

Items were removed from scales with high alpha. In addition, three item scales with low alpha were extended with one item each. Complementary core competence scale was extended with one new item from Van der He-jden (2001). Flexibility scale was extended with one item from Rokkan &

Haugland (2002), and the solidarity scale was extended with one new item from the same authors. A careful introduction was given to each set of ques-tions for positioning purposes. This was suggested by academic experts, but also as an answer to the respondents’ difficulties with reading some of the questions.

The questionnaire was translated to Norwegian, based on unison suggestions from respondents in the pilot study and from one academic expert. The most frequent method of accomplishing the adaptation, and verifying the equiva-lence of new scale with the original language version, is the procedure of translation, back-translation, and verification (Brislin, 1970). Back-translation is typically used to verify semantic equivalence of a translated measure to the original scale. The initial translation from English to Norwe-gian was done by the researcher. A draft version of the questionnaire, with both English items and translated Norwegian items, was emailed to two aca-demic experts fluent in Norwegian and English. Some corrections were sug-gested by the reviewers, and these were implemented into the final version of questionnaire.

Although validity of the adapted scale depends fundamentally on semantic equivalence, back translation always involves subjective evaluations. A study by Mallinckrodt and Wang (2004) developed a “dual-language

split-83

half” quantitative method of verification to supplement back-translation judgments. Correlations between the “dual-language split-halves” were not statistically different from correlations between all-English sets of the same items. There were also no differences in retest reliabilities. And thus, this research used the following procedure in translating and verifying semantic equivalences of the adapted measures. First, the researcher prepared a bilin-gual first draft translation. Second, two bilinbilin-gual academic experts verified the equivalence of translation and original version item-by-item. They were knowledgeable about the constructs and they were working independently.

They also verified equivalence of the translated scale instructions. If items or portions of instructions were initially declared not equivalent, this was dis-cussed until all items were declared equivalent. The quantitative verification phase was dropped since prior research found no such problems.