Listeners and their intelligibility data

So far, the generation of stimuli and the organization of stimuli into stimulus sets have been discussed. In the intelligibility experiment, a total of 103 native Norwegians listened to these sentences and reproduced them in writing so that the impacts of the manipulations could be measured by the experimenter. All listeners reported normal hearing. None had experience with N2 speech on a level judged as extraordinary. The listeners were of both sexes and from all parts of Norway. This section describes the setup for how the listeners listened to the various stimulus sets, and for how the resulting data was further processed by the

experimenter. This setup was fairly complicated, and the reader is encouraged to read the present section thoroughly as information provided here is crucial in order to understand the data analyses throughout this chapter.

Different listeners perceive foreign accented utterances with differing ease for many reasons (e.g. Gass & Varonis, 1984). For instance, different listeners will have different familiarity with listening to foreign accented speech, and they may also have different familiarity with foreign accented speech originating from different L1s. An extreme way of removing the factor listener would be to use the same listener group across all stimulus sets. The same listeners would then have to listen to 6 different manipulations of the same sentence.

However, at the same time, listeners would become more and more influenced by learning the sentences as the number of times they heard the sentences increased. Naturally, this stepwise increased effect of learning would interfere with the measurement of the manipulation effect.

In the literature, this problem has been solved by using different listener groups for different stimulus sets. When there are different sentences within a stimulus set and different sentences across the different stimulus sets, this means that each listener is presented with the same sentence only once. Maassen & Povel (1985) investigated the intelligibility of deaf peoples’

speech and used different listener groups for different stimulus sets. However, they also pointed out that differences between the listener groups could have influenced the results, because one group might consist of listeners for whom deaf speech was more intelligible than for listeners in another group. Even if one takes care to assemble in the same group listeners who are as similar as possible in terms of for instance experience with N2 speech, different listeners may still perceive somewhat differently. In other words, the approach with different listener groups for different stimulus sets removes sentence learning as a factor, but retains the listener as a factor.

Both approaches thus have advantages and disadvantages. For this reason, the present perception experiment was carried out using a combination of these two approaches as explained in the following.

In the present experiment, some listeners listened through only one stimulus set (single-session listeners), whereas others listened to first one stimulus set and then, immediately after, another stimulus set (double-session listeners). The data from the double-session listeners’

first listening session was therefore free from learning effects. The data from the double-session listeners’ second listening double-session may be influenced by learning effects. However, because the double-session listeners have listened to no more than two stimulus sets (not 6, which was the total number of stimulus sets), the learning factor was binary (present or non-present in the data) and could therefore be investigated as a factor in multifactor statistical tests.

The number of listeners was not equal among the stimulus sets. Also, the ratio between the number of intelligibility data influenced by learning effects (from second listening sessions) and the number of intelligibility data free from learning effects (from first listening sessions) was not equal between the stimulus sets. The discussion from now on will focus on intelligibility data rather than on the listeners. It is necessary to define three different groups of intelligibility data according to their function when intelligibility is compared between two stimulus sets. Intelligibility has been measured using each of these three data groups. The data groups are explained in the following.

The All Data group:

This group comprises all the intelligibility data, regardless of the unequal number of data between the stimulus sets, and regardless of the unequal ratio between data affected/not affected by learning across the stimulus sets.

Let us take the measurement of the intelligibility score difference between the original and the duration manipulated stimulus sets as an example. Across L1 groups, the comparison with All Data comprised 1765 intelligibility scores for the original stimuli and 1596 intelligibility scores for the duration manipulated stimuli. For the intelligibility data from the original stimuli, 678 were affected by learning and 1087 were free from learning effects. For the intelligibility data from the duration manipulated stimuli, 756 were affected by learning and 840 were free of learning effects. As explained earlier in this section, the skewed influence of learning effects across the stimulus sets was unproblematic because the learning effect was defined as a factor in multifactor statistical tests.

The Paired Data group:

All subjects who listened to the close-original stimuli listened to two stimulus sets. Moreover, these subjects were organized in the following way. Every subject who listened to the

close-original duration (COD) stimuli listened to two stimulus sets, with the other being the duration manipulated (D) stimuli. Similarly, every subject who listened to the close-original intonation (COI) stimuli listened to two stimulus sets, with the other being the intonation manipulated (I) stimuli. Note that the Paired Data comprised all the COD and COI data, but only a subset of the D and I data. (In other words, there are more D and I data than those paired with the COD and COI data).

For the subjects who provided the Paired Data, half listened to the close-original stimuli in their first session and the manipulated stimuli in their second session, and the other half listened to the manipulated stimuli in their first session and the close-original stimuli in their second session. For the Paired Data, there is thus an equal number of data with and without learning effects, both for the comparisons between the COD and D stimuli and for the comparisons between the COI and I stimuli.

There are Paired Data only for the COD/D comparison and the COI/I comparison. The setup for the Paired Data is further clarified by the illustration in Table 4.2.

Table 4.2: Comparison of intelligibility data between COD and D stimuli and between COI and I stimuli with the Paired Data.

COD COI D I

Listener 1-20

Listener 21-40

Listener 1-20

Listener 21-40

The illustration shows that the same subjects (listeners 1-20) have listened to both COD and D stimuli. Another group of subjects (listeners 21-40) have listened to COI and I stimuli. The Paired Data enables within-listener comparisons of data across the stimulus sets, thus eliminating the listener factor.

The Paired Data is a subgroup of All Data. In other words, the All Data group is comprised of data that were also analyzed as Paired Data.

The Rest Data group:

As explained above, Paired Data is a subgroup of All Data. This means that when an effect is found when investigating All Data and when investigating Paired Data, the effect for All Data could in fact be due to effects present only within the Paired Data. In order to find out whether

effects found for All Data are due to effects only within the Paired Data, intelligibility will also be investigated for a third data group called the Rest Data. The Rest Data equals All Data minus Paired Data.

The relation between the All Data, Paired Data and Rest Data groups is illustrated in Figure 4.1.

Figure 4.1: The three groups of intelligibility data.

To sum up, the data groups are as follows: The All Data group comprises all the data from each of the compared stimulus sets, regardless of the skewed number of data between the sets and the skewed influence of learning effects between the sets. The Paired Data group comprises only those data that originated from the same listeners across the COD and D conditions and across the COI and I conditions. The Rest Data are the All Data minus the Paired Data, and were used to investigate the reliability of effects found for All Data (i.e.

whether effects found for All Data were actually due to effects present just in the subgroup called Paired Data). The advantage of the Paired Data is that all listener effects are eliminated, as opposed to merely reduced in approaches using homogeneous listener groups.

Note that there are Paired Data, and hence also Rest Data, only for the COD/D and COI/I comparisons.