A Momentary Lapse of Affect: Continuous Change of Emotions During Listening to Different Structural Parts of Pop-Songs – an Eye Tracking and Pupillometry Study
Shoaib Nabil
Department of Psychology, Faculty of Social Science, University of Oslo PSY4092 – Master’s Thesis
Professor Bruno Laeng Fall 2021
Abstract
Music has been always playing an important role in the human cognition system and its impact on regulating the affective states of the human mind is gaining successive importance in the research of cognitive neuroscience. A large portion of mainstream music falls under the
‘umbrella’ term pop-music. We investigated how different structures (intro, verse, chorus, outro, etc.) of pop-music changes listeners affective states during purposeful listening. In this exploratory research, we implemented James A. Russell’s circumplex model of affect (1980) using gaze position (as voluntary self-report index), and pupillometry (as involuntary
physiological index) as main measurement tools. We used two experimental designs, first, with the experimenter-selected songs (N = 32) and then as a follow-up with participant- selected songs (N = 10). In the experimenter-selected songs – we observed a gradual decrease (highest during intro, lowest during outro) in pupil diameter (mm) during listening periods.
This may be interpreted within the adaptive gain theory of Ashton-Jones and Cohen (2005).
Moreover, sudden decrease of pupil diameter after the intro part suggested an anticipation effect, as proposed by David Huron (2008). In the self-selected songs, changes of pupil diameter, accompanied with changes of pleasure and arousal affective states during the chorus part, supported the emotional impact of a meaningful song part in cognition.
Furthermore, participants felt more pleasant and aroused listening to self-selected songs compared with experimenter-selected songs. All in all, this explanatory study indicates a successful application of the Russell’s circumplex model along with eye-tracking measures in the setting of continuous report of affective states. Moreover, it indicates the importance of using self-selected music when studying affective responses to music.
Keywords: pop-songs, song structure, circumplex model of Russell, pupillometry
Acknowledgement
I am feeling delighted while writing this acknowledgement. It was my true pleasure to have a supervisor like Professor Bruno Laeng. I felt most fortunate to have his unconditional
support, mentorship, and absolute guidance all over this journey, which made it possible for me to complete the thesis.
I am grateful to my parents, my siblings, and my friends for their support and company during this time.
Finally, my gratitude to all the true musicians over the world to make our existence a little bit more bearable and meaningful.
Table of Contents
1. Introduction ... 6
1.1 What is Music? ... 6
1.2 Pop-music: it’s emergence and functionality in contemporary culture ... 7
1.3 Recipe for pop-music: It’s structural basis ... 8
Intro ... 10
Verse ... 11
Pre-chorus ... 11
Chorus ... 12
Post-chorus ... 13
Bridge ... 13
Outro ... 14
Instrumental solo ... 14
1.4 Music and emotion ... 14
1.5 Application of Russell’s circumplex model of affect in music cognition ... 16
First argument and its defense ... 18
Second argument and its defense ... 18
Third argument and its defense ... 19
Fourth argument and its defense ... 19
1.6 Present study ... 20
Attention related processing ... 23
Emotion and aesthetics related processing ... 23
Neurophysiological related processing ... 23
2. Methodology ... 25
2.1 Participants ... 25
2.2 Apparatus and Materials ... 25
2.3 Experiment: Experimenter-selected ... 27
2.3.1 Design: Experimenter-selected ... 27
2.3.2 Procedure: Experimenter-selected ... 30
2.3.3 Results: Experimenter-selected ... 32
2.3.3.1 Mean fixation time percentage (%) in the Arousal and Valence dimension of Circumplex display ... 33
2.3.3.2 Mean fixation time % for ‘common’ song-parts in quadrants ... 35
Intro ... 35
Verse ... 36
Chorus ... 36
Outro ... 37
2.3.3.3 Mean pupil diameter in quadrants of the circumplex during ‘total listening’ vs ‘chorus listening’ ... 38
Total listening ... 38
Chorus listening ... 38
2.3.3.4 Mean pupil diameter for common song parts ... 39
2.3.3.5 Mean pupil diameter for each song during different segments ... 40
All out of love – Air supply ... 41
The thrill is gone – B.B. King ... 41
Take me home, country roads – John Denver ... 41
Knockin’ on heaven’s door – Bob Dylan ... 41
Imagine – John Lennon ... 42
Smells like teen spirit – Nirvana ... 42
2.3.4 Discussion: Experimenter-selected ... 43
2.4 Experiment: Self-selected ... 45
2.4.1 Design: Self-selected songs ... 46
2.4.2 Procedure: Self-selected ... 46
2.4.3 Results: Self-selected ... 46
2.4.3.1 Fixation time percentage ... 47
2.4.3.2 Mean Pupil ... 48
Total-listening time ... 48
Chorus-time ... 49
2.4.3.3 UP vs EP – During total listening & chorus-time... 50
2.4.4 Discussion: Self-selected ... 51
3. General discussion ... 52
4. Limitations ... 52
5. Future implications ... 53
6. Conclusion ... 53
7. Declaration of contribution ... 53
8. References ... 54
Appendices ... 74
1. Introduction 1.1 What is Music?
Music is a complex human creation which reflects some of our highest cognitive capabilities. It is complex not necessarily because of its structure, but because humans, as a species, do use, re-uses, implement its practicability in their social-personal life (Schulkin &
Raglan, 2014). Music could be considered as an artistic form of expression just like other forms of art but the concept or the definition of music are a matter of strong debate (Kania, 2014). Some refer to music also as a language or a form of communication sharing features of speech (Ross et al., 2007), also because it is clearly related to it at the level of brain function (Patel, 2007), although concrete understanding of music in the linguistic term is yet to be achieved. Just as modern language comprises of the five main components which are phonemes, morphemes, lexemes, syntax, and context, music has its own distinctive features as well, and it ranges from – note, tone, contour, timbre, melody, pitch, and many more.
Music has also similarities with other cognitive domains, like mathematics (we do not know if mathematics itself is considered a language, there is no unitary consensus regarding this matter), in a sense where music can be mass-produced using its notational ‘recipe’ by implementing artificial intelligence (Frid et al., 2020), seems achievable. However, not all achievable things are in our desires. Our desires seem to be regulated by our emotions, our affects, and we can consider this as a two-way process.
Any sort of music, with its specified or unspecified notation, chord
progressions, harmony, isochronous (or not) beats, has consequences on our emotions (Grewe et al., 2007). These consequences can be short-term or long-term, not only in regions of the brain during/after listening to it (Blood & Zatorre, 2001). An important aspect is the
spontaneous anticipation of specific musical events (Huron, 2008, chap. 7 & 12). For these
reasons, music is a relevant research object of investigation within the field of Cognitive Psychology.
There is no question that music can elicit emotions. It seems possible to observe or experience the full spectrum of emotions during the music listening experience. For
psychological research, what could be the standard measure enabling us to know when and what part/piece of music evokes specific emotions? These are the important questions music cognition psychologist has been investigated for many decades. However, before excavating into these processes, we must draw a clear line about what type of music we are considering for discussion. It is pop-music.
1.2 Pop-music: it’s emergence and functionality in contemporary culture
Pop-music is a large ‘umbrella’ term– consisting of different genres or subgenres from jazz/metal to hip-hop/country. This cultural stream, with its continuously changing form and in an unpredictable manner can interact with different institutions of the societal system (Kuspit, 1976); but this will not influence us to comprehend the true emotional impact of pop music upon the general population.
There is a large number of debates about the emotions and its functionality in our social and cultural life. The discussion has gone through questioning its functionality on the basic rationality/logic both in favor or against for its applicability in physical or social background (e.g., Dewey, 1895; Hebb, 1949; Mandler, 1984; Barrett & Campos, 1987;
Ekman, 1992; Johnson-Laird & Oatley, 1992; Lazarus, 1991; Levenson, 1994; Oatley &
Jenkins, 1992; Plutchik, 1980; Tooby & Cosmides, 1990). We know that each aspect of human existence produces/generates emotions where its functionality is rooted in to its true or artificial necessities (Keltner & Gross, 2010).
Furthermore, there is an intimate connection between music and human emotions (Robinson, 1994; Lonsdale & North, 2011; Huron, 2012; Swaminathan & Schellenberg,
2015). Thus, it is meaningful to know how historically this has emerged and gone through different tendencies in our modern cultural and social environment. Popular culture has been producing different commodities and, reasonably, music is one of the important one. Popular music functions as a leading cultural statement (Serra`, 2012).
Technological advancement throughout the history has shaped popular-music culture, music performance, listening habits of the consumer of music and music industry (Webster, 2002; Hui et al., 2013; Scott, 2009, p. 147 – 168). In an important book - How Music Works (2012) - musician and writer David Byrne discussed two pioneering technological periods – Analog and Digital which have shaped enormously the way a musician creates music, the way it has been distributed, and the way it will be consumed. The year 1878 could be the approximate time of the advent of recorded music. During that period, the advent of phonograph boxes in different parlor/pubs was very sophisticated at that time. After then the gramophone (1988), the invention of the amplifier (1907), the electric guitar (1931), the first applications of the stereo sound system (1940) in a movie, emergence of advance
technologies on that time, i.e., magnetic tape/cassettes (1958), portable devices like Walkman (1979), the first compact disc/CD (1980), the revolution of mp3 (1993) and in the near past the iPod (2001). Considering and stepping on those cornerstones, the music making
market shifted its motto to ‘stimulate the concert-going experience in the consumer’s living room’ (Cameron, 1995) and music happened to be on the way of becoming more and more available to general people.
1.3 Recipe for pop-music: It’s structural basis
What is the recipe for creating a pop-song? There is no simple answer or formula. But we can imply that it follows rather explicitly a set of rules and set of structure. We cannot imagine a folk song as having the same format as a rock song. There is always some notational difference, some variance in scale, few tonal/melodic discrepancies, difference
on timbre/texture, and contrast between meters. Scholars have indicated that, typically pop- songs do not deviate from the 32 bar or verse-chorus form or ternary or strophic forms (Covach, 2005; Frei-Hauenschild, 2015). The 12-bar blues help us to understand the
formation of jazz as well (Steedman, 1984, p. 52 – 77). Yes, there is vast range of genres that we cannot even mentioned here, and perhaps meta-genres in the production line as well, but simply following these song structure/forms can help us to dissect the popular song genre. In sum, all the pop-song seems to follow a common template (Sloan & Harding, 2019, p. 44 - 53).
Walter Everett in his book The Foundations of Rock: From "Blue Suede Shoes" to
"Suite: Judy Blue Eyes" (2008), has noted that verse and chorus are the two most important foundational-block of a popular song. He distinguished these two by implying that,
“whenever the music of verse comes again after chorus, typically it gives way to a new set of lyrics, however the chorus remains always the same”. Chorus is the part where the audience can directly interact with it by singing or at least murmuring at their best. According to several scholars, the chorus or hook in a melodic song, is the most memorable catch phrase or melody which is repeated in a song (Kuroff, 1982; Bennett, 1983; Shaw, 1982). The other parts of the popular song structure are also important in their own way but in a sense less memorable and less grabbing for the listener’s attention.
Let us discuss in details all these popular song structural parts in the following sections (Everett, 1999, p. 15; Everett, 2008, p. 146; Burns, 1987; Sloan & Charlie, 2019, p.
51; Davidson & Heartwood, 1996, p. 6; Whitesell, 2008, p. 151; Watson, 2003, p. 87 – 90;
Keys, 2018, p. 109; Summach, 2011; Bob Dylan, 2009; Elton John, 2017; Nirvana, 2009;
Dolly Parton, 2017; John Lee Hooker Official, 2016; Rihanna, 2009; Metallica, 2009;
Middleton, 1999; Covach & Boone, 2005; Burns, 1987; Hurst & Delson, 1980, p. 58).
Intro
As the title states, Intro is the introduction, the path that leads towards a musical piece. Intro does the work of setting up the mood for the songs. From
a simple piano/guitar chord, to sudden/lazy up-down beat of the drums makes it sound to be groovy in texture, or even artificial sounds of the synthesizer, the humming of the singer or the back vocal. Intro can be a hint about what is going to happen next. The purpose of into is to lure the listener into the song. It is the short breathtaking part of a song, but
not the only one. Breathtaking in a metaphorical sense that, when an intro builds up the suspense about what is going to be happens, it creates anticipation, whereas when the intro part ends, the downbeat drops in and gives a relief to the listener, who starts to breathe again. Relief means, that the listener does not have to wait to know what is going to happen next. Intro can be as brief as one second in a song. Elton John’s famous track Rocket
man (1972) has barely an intro unless we consider a single stroke of a piano chord -
em7. A question should arise from this phenomenon, as most of us have been consumers of pop-songs for a long time, why do we anticipate with such clichés? It might have some evolutionary explanation as discussed by David Huron (2006) in his book Sweet Anticipation.
We generally tend to anticipate future events (in a song) based on (overused) habitual techniques in the formation part of a song or notably in all music. According to his
observations, our body and perception/sensations likes to ‘resolve’ surprises. Whenever any kind of tension or a moment of uncertainty occurs, even if it occurs through
our auditory senses, the body reacts all the time, so as to neutralize the prediction error (or in other domains the ‘threat’ of an unknown future). Clearly the possibility of anything wrong happening while listening to music is infinitesimally small, but our biological system is already geared towards avoiding taking risks and this aptitude may show up in other (less risky) domains, including art and music. In the context of music listening, when some sort of
tension builds during the brief/moderate intro, our perceptual/cognitive system
becomes alert and, when suddenly the pressure drops during the next part of the song, a sense of relief happens that has a clear positive connotation. That relief is due to a response from the reward system, which works as a motivation system for all types of future events.
Verse
Coming after the intro part, in a pop-song the verse serves as a poetic/lyrical description of the songs. Song's context/theme/narrative/emotions
are being represented/depicted in different verses, the progress of consecutive verses can be seen as like chapters of a book, where the story gradually progresses, but it does not occur always in the straight line. In a typical song there is always more than one verse, the total number of verses could range between 3 to 6. Verse’s – often poetic paragraphs, containing lyrics, can also be written in different design form, e.g., an AABB or ABAB rhythm scheme format. Interestingly, different verses can have distinct sets of lyrics, though the format of rhythmic scheme can be changed and there can be connectedness between the verses of a same piece of song. Another way of understanding verse is to consider it as a unit that elaborates the tonic – or the home key of a musical piece.
Pre-chorus
It exists sometimes, in certain type of pop-songs. For example, the grunge genre band Nirvana’s famous track Smells like teen spirit (1991) from the ‘Nevermind’ album has a pre-chorus part just after the verses and before the chorus. Although rather optional, the pre- chorus works as a connector between verse and chorus with intermediary material, normally using subdominant harmonies. However, it is not always restricted to this function.
Sometimes, when verse and chorus implement a similar harmonic structure, the pre-
chorus brings out a unique harmonic pattern than the other two. Because of this characteristic, pre-chorus is also known as build-up, channel, or transitional bridge.
Chorus
Also known as refrain, though chorus is central part of a song, following the verse/pre-chorus. The text of the chorus and the melody is typically repeated several times during a song. Chorus is the part that contains what is called the hook of a song or sometimes the title/key lyrics line, which keep recurring in a song. In fact, chorus is the only significant element of the structure of a song that is being repeated musically and lyrically at least for a single time in a song, i.e., Dolly Parton’s famous track “…and I will always love you
(1973)”. A lucid definition of hook was given by Monaco and Riordan (1980) as, “a musical or lyrical phrase that stands out and is easily remembered”. So, hook can be the chorus part of those songs, but it is not explicitly true for all cases. An important paper in this manner, titled A typology of ‘hooks’ in popular records (1987) by Gary Burns, investigated the types and classifications of hook in pop-songs. According to Burns, the hook can be found
in textual elements and non-textual elements of a modern musical piece.
Rhythm/melody/harmony individually or together can act as hook, which can be considered as a textual element. A song named Boom Boom (1968) by John Lee Hooker from his ‘Urban Blues’ Album can be a clear example of hook as a textual element. In this song, we hear a strong drum-bass 4 beat along with bass & electric guitar playing with note-pause-note-pause sequence, maintained until the end of the track (except on the solo/chorus). However, the hook as a non-textual element can have a varied range. When a music performer is
performing music, this will vary – and the different performances of a same song can even produce/alter the hook. If we consider looking upon the technological sides of this topic, then we shall see sound engineering techniques (channel balancing, noise cancelling,
mixing), recording studio facilities, even converting a piece from mono to stereo can affect the presence/emergence of hook in music. A hook can be formed through the instrumentation of a song, even the tempo of a song piece can function as a hook as well. It would not
be wrong to confirm that the chorus is constant for a song, but the hook is not; although sometimes in the live performance, it is frequently noticeable that performers intentionally alter actual chorus, i.e., Bob Dylan’s knockin’ on heaven's door (1973) from the album ‘Pat Garrett and Billy the Kid’, Bob Dylan performed the chorus a bit differently in the live version at MTV’S 1995 unplugged concert.
Post-chorus
Is another part of the song structure that comes right after the chorus. Not every pop- song has a pre-chorus in it. This section of a song has similarity to the chorus to the naked ears, but a close analysis of post-chorus can reveal the distinctiveness between these
consecutive parts. If we listen to pop artist Rihanna’s famous track – Umbrella (2007) from the album ‘Good Girl Gone Bad’ we will observe an example of post-chorus. Although this example is implicit, not a clear one, it is still countable for analysis. Another useful
terminology that comes along with post-chorus is called hybrid. In hybrids, post-
chorus maintains the hook from the song tracks, while in the meantime interjects few extra features.
Bridge
This part of a song typically comes right after the chorus part. In a book by Davidson and Heartwood titled Song writing for beginners (1996), bridge is defined as “a
device that used dot break up the repetition pattern of the song and keep
the listeners' attention.... In a bridge the pattern of the words and music change”. Bridge is not a ‘must-have’ part, and its presence varies song to song. A decent example of bridge can be found in a song titled Memory Remains (1997) from the 7th studio album – ‘Reload’ - of heavy metal band Metallica’s. In this track the band used the back-vocal of English singer Marianne Faithfull (1946 – Present). Her iconic back vocal, oddly humming (ta-da-da- da-da-da-da-da-da-da-da-ta-da-da) was used as a bridge.
Outro
It simply is the conclusion of a song. A song can end in diverse ways. Maybe slowing down of tempo (ritardando), gradual decreasing volume (fading) while subtly echoing the chorus or something else. There are several ways of saying goodbye, so as many ways of ending a song/musical piece. In general, during the outro, composer/songwriter does not include anything new in this part. Many times, a pop-song decides to perform its outro by fading out, while reiterating a brief segment of the pop-song repeatedly.
These are the major parts making the structure of pop-songs. However, there are a few other parts that can be, like – elision, instrumental solo and/or ad lib. We will briefly touch the definition of instrumental solo as because it is one of the interests of this text.
Instrumental solo
The purpose of the instrumental solo or solo is to give a particular
signature, specifically found in the rock/metal/jazz genres popular songs. Another vital role of instrumental solo is to display an instrumentalist and highlight their talent/abilities as an expert or virtuoso musician. Modern solo part of pop-songs is typically comprising
guitar playing.
1.4 Music and emotion
Music has a prevalent effect on our everyday mundane life. Whether we are indoors or outdoors, we are often surrounded by music. In an average week, the general American population listens (mainly passively) to 18 hours of music during their daily activities (Motion Picture Association of America, Inc, 2007). If an average person sleeps 8 hours per day, then people spend about 15% of their daily time listening (but not necessarily
attentively) to music. That is a considerable numerical value for a single person. Music has a vital impact on people's affect, mood, and/or emotions. Music can alter our emotions.
Consciously or not, people use music to manipulate their moods and emotions (Juslin et al., 2008; Juslin & Västfjäll, 2008; Scherer & Zentner, 2001).
Person to person differences on musical choices have connections with personal traits and value system (e.g., Delsing et al., 2008; Rentfrow & Gosling, 2003; Zweigenhaft, 2008).
People can also use music as a medium for self-expression (e.g., North & Hargreaves, 1999;
Rentfrow & Gosling, 2006; Rentfrow et al., 2009).
A puzzling question is whether music is a way to communicate emotion to the listener as the music composer/performer is trying to convey (Juslin, 2000; Thompson & Robitaille, 1992). Keeping in mind that there exist composers and musicians who do not think music intends or does convey emotions, e.g., Igor Stravinsky (1982 – 1971), several self-report studies (Barrett et al., 2010; Zentner et al., 2008) confirm that nevertheless music (including Stravinsky’s) evokes certain mood, emotions and affect in its consumers. As converging evidence, using more objective measurements than just self-reports, several experiment using some physiological index (Krumhansl, 1997; Salimpoor et al., 2009) or
neuroimaging displaying brain activities (Blood & Zatorre, 2001; Menon & Levitin, 2005) revealing that music does produce different ranges of emotions and these can range from positive to negative and from exciting to unexciting.
The main question is whether music induces emotions all over the emotional
spectrum? In a study titled - An Experience Sampling Study of Emotional Reactions to Music:
Listener, Music, and Situation (2008), Patrik N. Juslin and colleagues, monitored naturally- occurring music listening habits during everyday chores. They found that positive emotions such as happiness/recollection of cherished events of past occurred most frequently rather than negative emotions such as anger/boredom/anxiety. While people were experiencing different music listening, the changes of emotional states did not occur randomly. Also, they are interconnected with the developmental stages’ individuals go through (Christenson &
Roberts, 1998; Holbrook & Schindler, 1989; Janssen et al., 2007), several aspects of social perceptions (Lonsdale & North, 2011; Rentfrow & Gosling, 2003) including the building of their own identity (i.e., representing their faith). Listening to specific types of music, individuals seem standing out for themselves, and the music may be giving them strength to identify themselves as individuals of a specific group of people. In sum, we are willing to be perceived by the community/society as who we are also on the basis of our musical preferences (North & Hargreaves, 1999; Rentfrow & Gosling, 2003).
1.5 Application of Russell’s circumplex model of affect in music cognition
One of the founding fathers of experimental psychology, Wilhelm Wundt, once wrote that “people are never in a state entirely free from feeling” (1897/1998, p.92). All our mental states are infused with affect (Spencer, 1855; Sully, 1892). We have pointed out a
few important aspects of regulation of emotions in music listening experience, but we should also address the range of emotions and how does these emotions differentiate/assimilate with each other. There are three main critical approaches to describe participants’ conscious experiences while listening to music: discrete emotion theory, dimension model, and eclectic approaches (Scherer, 2004). We use in the present project as our theoretical framework, the dimension model of James A. Russell of Boston University (2000 – present).
Russell’s important contribution has been in developing the ‘circumplex model’ of affect. In the journal of Personality and Social Psychology, James A. Russell published an influential article titled A Circumplex Model of Affect (1980), which showed some factor
analytic support for a fundamental dichotomist organization of affect. Specifically, he provided evidence that affective dimensions of human cognition are interrelated in a highly systematic fashion, rather than being autonomous of each other. The valence/pleasure
dimension denotes positivity and negativity of affect, and the arousal dimension indicates the degree of intensity of affect ranging from high to low (Cowie & Cornelius, 2003; Pereira,
2000; Schröder, 2004). Thus, Russell proposed a ‘circumplex’ model where the
interrelations of this affective concepts fall within a metaphorical circular space (see Figure 1) essential defined by two orthogonal axes: 1) the pleasure-displeasure (known as valence), or the dimension of experience which refers to the hedonic mode. 2) the experience of activation or intensity, also described as a sense of mobilization/energy (Barette & Russell, 1999). Interestingly, similar models based on the pleasure/activation coordinates of affect had already been proposed by Wilhelm Wundt (1912/1924) and later by several other
psychologists, like Harold H. Schlosberg (1941), Larsen and Diener (1992) and Reisenzein (1994).
Figure 1
A circumplex model of affect. Adapted from Barrett and Russell (1998
Russell’s bipolar dimensional (within a two-dimensional space) structure of affect has also generated academic controversies and critique. A similar type of structure was proposed by Watson and Tellegen (1985), who described an affective structure regarding two
dimensions of valence (Positive and negative affect) that tacitly convey activation, in contrast to distinguishing pleasantness and activation. However, Russell’s model is
unique compared to others in various degrees. His circumplex model of affect has been used in different domains, i.e., it was used to study the ratings of valence and arousal of emotional faces by children and adults who were diagnosed with autism spectrum disorder (Tseng et al., 2014). The applicability of this model was also implemented into the research field of
transportation to successfully describe the changes of motorcycle riders' affective state at the intersection of roads (Samuel et al., 2019).
This model was particularly criticized in its dichotomic aspect of bipolarity, but it has managed well against the critiques as follows -
First argument and its defense
A first question arises about the actual independence of the positive and negative affective states, that according to the geometrical positioning of the positive and negative affect model’s diagram. The affective states are exactly 90° apart or orthogonal within the circumplex structure, implying a 0% correlation between them (Barrett & Russell, 1999) or complete independence.
Second argument and its defense
Another hurdle is in terms of empirical independence. Many researchers have reported small-scale correlation between positivity and negativity of valence, which contradicts the definition of this model. However, many critical researchers have reported quantitative correlation between the pleasant and unpleasant affect, though this is significantly less than ‘–
1’. This correlation may be due to various reasons, such as – time-span sampled (Diener &
Emmons, 1984), the role of systematic and random error of the measurement (Barrett &
Russell, 1998; Green et al., 1993), unsuccessfully specification of the precise model of
semantically bipolar opposite (Feldman Barrett & Russell, 1998) and the types of formatting the response (Russell & Carroll, 1999).
Third argument and its defense
Third important argument against bipolarity is in the form of neurophysiological independence. At the neurophysiological level, bipolarity in the affective state does not seem to be observed (Cacioppo et al., 1997). However, it is possible to counter this statement by considering that the underlying physiological structure is not required to be isomorphic with the structure of consciousness (affect is the consciously available essential feelings of pleasure-displeasure and activation). Furthermore, Cacioppo et al. (1997) admitted that even when the neural processes of affect are not independent, there is a chance that
bipolarity emerges in the forming of conscious affective feelings.
Fourth argument and its defense
Another controversy against Russell’s circumplex model is in terms of
Circumplex structure (affective items spread say evenly around the circumference of the space) versus Simple structure (where affective items were falling into tight clusters, with gaps between the clusters) (Barrett & Russell, 1999). Affective structure seems compatible with the circumplex structure rather than a simple structure. The framework of affective space fails to fulfill the threshold criteria for simple structure according to empirical indices and Structural Equation Modeling – SEM (Breckler, 1990; Russell & Barrett, 1999).
More details on the activation dimension (i.e., arousal) will help us comprehend the circumplex model better. Historically, the definition of activation dimension has
created confusion. Some researchers mistakenly related self-report and the direct index of bodily arousal. However, in their paper (1999), Barrett and Russell suggested that
activation/arousal and valence/pleasure – are dimensions of conscious experience that have
clear neurophysiological correlates (e.g., Lang et al., 1997). However, they argue that there is still a lot to comprehend about this relationship.
1.6 Present study
So far, we have discussed about significance of music in everyday human life, we narrowed down our focus towards pop-music and its structures, and we have agreed to the fact of continuously changing affective states of mind while experiencing music. For a structural and organized understanding of affective states/affect, we introduced the well- established classical circumplex model of affect by James A. Russell. In the present thesis, we applied this model by attempting to observe and measure the change of this dimensions of affect continuously and in relation to the various parts (i.e., intro, verse, pre- chorus, chorus, solo, bridge, outro) of pop-music structure in a listening experience. We hypothesized that, while listening to various parts of pop-music, affective states of mind will be changing in terms of both valence and arousal level. In particular, the chorus should be the most anticipated part of the pop-song. That is, listeners should experience positive valence and probably high arousal, as well during the part.
Monitoring change of affect during music listening experience can be achieved at some level though self-report questionnaires. But this approach is both simplistic and problematic. In fact, reporting verbally the moment to moment change in affect does not seem practically possible. Therefore, we used a systematic and objective approach by use of the combined eye-tracking technique and pupillometry. There is a significant different between tracking the gaze and pupillometry. While gaze tracking (number of fixations, saccades, blinks, etc.) monitors the self-report of the subject (Tai et al., 2006; Livingstone &
Isaacowitz, 2015; Cassioli et al., 2021) pupillometry measures the dilation/constriction of the pupil which is not under the strategic control of the participants (Laeng et al., 2012).
Moreover, pupillometry is not only applicable to the arousal dimension of the emotional
stimuli (e.g., Aboyoun & Dabbs, 1998; Bernick et al., 1971; Hamel, 1974; Hess et al., 1965;
Peavler & McLaughlin, 1967) but also reveals other fundamental cognitive mechanisms, such as – cognitive load, mental calculation, intensive aspect of attention, intensity of the sentence processing system, etc. (Beatty & Kahneman, 1966; Hess & Polt, 1964; Ahern & Beatty, 1979; Just & Carpenter, 1993).
This current thesis project is strongly based on the original study by Laeng et al., (2021) study titled: Defrosting’ music chills with naltrexone: The role of endogenous opioids for the intensity of musical pleasure. In their experiment, Laeng and colleagues investigated the effects of an opioid antagonist (naltrexone) versus placebo on participants while they were listened to either their self-selected music or that of other participants (as control condition). They continuously measured the affective states using a self-report scale
presented on the screen as a ‘thermometer’, where participants shifted their eyes in this visual analog scale, so to report continuously experiencing a specific ‘hedonic’ experiences.
Importantly, in the meantime, given the use of an eye-tracker for the above subjective reports, they also obtained objective measurements of each participant’s pupil diameter.
Furthermore, the above study continued the investigation initiated in a previous study also by Laeng and colleagues (2016) titled: Music Chills: The eye pupil as a mirror to
music’s soul. In this experiment, they objectively monitored music induced emotional ‘chill’
responses by the degree of pupil dilation Music listening can induce strong affective positive feelings in the listeners and this strong ‘feelings’ seems to reach a ‘peck’ momentum at any specific moment most commonly while listening to favorite musical piece. In these scenarios, an intense ‘peak’ is a profound culmination of pleasure and euphoria, a personal feeling of merging into the musical moment (Benzon, 2002; Sloboda, 1991). Interestingly, this perceived ‘peak’ momentum commonly has a physiological component that refers to as a felling ‘chills’ at any momentum during the musical listening experience (Harrison & Loui,
2014; Huron, 2008; Konečni, 2008; Panksepp, 1995; Scherer et al., 2001). Moreover, ‘chills’
can include several bodily responses such as– the gooseflesh type of skin tickling sensation, moistness of eyes, shivering/feeling cold, piloerection, dry mouth, palpitation, sighing, stomach feelings, etc. (Gabrielsson, 2011; Goldstein, 1980; Lowis, 1998).
Pioneered by Hess and Polt (1960), the method of recording changes in pupil diameter has shown to be a practical index of mental states and, in particular, arousal. At about the same time, Kahneman and Beatty (1966) achieved positive results using pupil diameter as a measuring tool of cognitive workload (either short-term memory load or load on attentional capacity). Since then, investigating distinct aspects of cognition via
pupillometry has become increasingly common (Laeng et al., 2012). Changes of pupil
diameter are automatic and involuntary responses, so it can be used as a reliable measurement tool for observing spontaneous changes in affective states while listening to the various parts of a musical piece.
In addition, there is a limited number of studies available which addresses specifically the circumplex model, in relation to music listening. Only few studies (including the two above from Oslo) have used pupillometry during music listening.
It is possible that different patterns of pupil changes can give clues to changes in consciousness. Interestingly, during dichotic music listening experience participants shift their focus of conscious attention towards a particular musical piece that can be captured by examining the pupillary time series (Kang & Wheatley, 2015).
There are three theoretically novel reasons (attentional processing, aesthetic processing, and neurophysiological processing) motivating the observation of pupillary response in relation to changes of affect during music listening (Laeng et al., 2016): 1) attention, 2) aesthetic judgment, and, 3) neurophysiology.
Attention related processing
The pupil dilation is observable with every increase of attention by all types of mental processes (Goldwater, 1972), most interestingly, pupil dilation provides a window on changes in mental effort or the intensity of mental processing or onto the moment-to-moment uptake of available processing resources (Kahneman 1973, 2011; Kahneman et al., 1969). In
addition, dilations of the pupil may provide the best and easiest physiological available index of mental effort/cognitive workload (Kahneman, 1973, 2011). In other words, pupillometry can precisely measure the load on attentional resources as well as the amount of focusing of consciousness at a given time (Hoeks & Levelt, 1993; Kang & Wheatley, 2015).
Emotion and aesthetics related processing
The sensitivity of the eye pupil to emotional engagement has a strong empirical basis – especially in relation to the arousal dimension of affective experiences (Hess & Polt, 1960; Bradley et al., 2008). Indeed, there is a close relation between pupil size and emotional arousal (Granholm & Steinhauer, 2004). An important study in this manner, involved
imagining sexually stimulating scenarios (to the point of orgasm) in women, which
concomitantly yielded pupillary dilations equal to 30% - 40% size of non-excited condition (Whipple et al., 1992). Since pupil dilation is an involuntary physiological response and cannot be manipulated directly in any simple way (Laeng & Sulutvedt, 2014), it provides a unique parameter for measuring arousal level. Clearly this applies to several aesthetic experience, as shown by an eye tracking study that involved interpreting film narrative and film music revealed the effectiveness of pupillometry (Wallengren & Strukelj, 2015).
Neurophysiological related processing
When precisely implicated by music, distinct modulatory systems of the
brain are highly related and possibly casually involved in this response process (see Koelsch, 2015; Koelsch et al, 2015). One functional MRI (Magnetic Resonance
Imaging) study revealed that, while participants were listening to music of their own choosing, they experienced musical chills comparing to randomly selected pieces of
music. Those chill experiences were associated with greater blood flow in different regions of the brain such as – ventral striatum, dorsomedial midbrain, amygdala,
hippocampus, cortex, and orbitofrontal prefrontal cortex (Blood & Zatorre, 2001). The same researchers repeated the same conditions with same participants in a parallel experiment with PET (Positron Emission Tomography) which showed endogenous dopamine release in the striatum region of the brain during chills experience (Salimpoor et al., 2011). Not only did the studies related chills to the dopaminergic system, but also to increase of NE (norepinephrine) while listening to music. These findings suggested an interaction with generalized arousal control system in the brain, comprising NE, serotonin, or prolactin (Huron, 2008) for governing emotions. Moreover, hormonal system based on mesolimbic and mesocortical dopamine (DA) (Ikemoto & Panskeep, 1999) and on the endogenous opiate system (Goldstein, 1980).
Specifically, in the first experiment of the present study, we asked our participants to listen to 6 pre-selected pop-songs (the genre’s range from soft rock, blues, country, folk rock, soft rock/pop, and grudge). While listening to songs, participants were sitting in front of the computer screen equipped with stationary eye-tracking devices with it. During
listening period, they were shown unanimated two-dimensional (with level of arousal and valence) display of the circumplex of affect on the computer screen. They were specifically instructed to look at the emotion they were experiencing during the listening time. In a follow-up experiment, 10 of the participants from the previous experiment were requested to suggest 3 songs of their own choice among their favorites. These self-selected songs
constituted the stimuli of the second, follow-up experiment.
In both setting (experimenter-selected = 6 song; self-selected = 3 songs) the eye tracker captured participant’s subjectively rating their affective state by looking at the Russell’s two-dimensional model space and worked as an effective physiological index of their arousal state.
2. Methodology 2.1 Participants
A total number of 32 participants volunteered to participate in the first experiment (experimenter-selected). All the participants were residing in Norway at the time and students in different disciplines. Their age ranged from 22 to 37 years; mean age was 28 (SD = 3.21).
Among them 11 were females.
After finishing the first experiment, approximately 3 and a half months later, we requested 10 participants among the previous 32 participants to participate in a second experiment (this time with self-selected pop-songs). In this follow-up experiment, the participant’s age ranged from 22 to 33, and mean age was 28.6 (SD = 3.382). Among 4 of them were females.
All participantssigned a consent form before taking part in the experiments. All had either normal or corrected to normal (by contact lenses) eye vision. Moreover,
in each experiment, all participants completed the whole session and there were no major technical difficulties. Lastly, participants were compensated with a 200 NOK gift card for their valuable contribution for each of the experiments.
2.2 Apparatus and Materials
We used RED – Remote Eye Tracking device for the eye-tracking and pupillometry.
This device is a stationary eye-tracker built by SMI – SensoMotoric Instruments, a German based company that was specialized in producing high-tech eye-tracking devices. The RED can operate within the distance between 0.5 meter to 1.5 meter. It has an infrared light
sensitive video camera with the source of 2 infrared lights (on the left and right side of the RED). The RED can detect changes in eye position as small as 0.004 millimeter. Movements of gaze, fixations and different pupillary responses were collected using the I-View Software (SMI) at a sampling rate of 60 Hz. All the experimental stimuli were presented using the Experiment Center (SMI) on a 22-inch liquid crystal display (LCD) monitor (P2210, Dell., Round Rock, US) with a resolution of 1680 × 1050 and a refresh rate of 60 Hz. Participants had to put their chin on a standard mounted chin-rest, which was fixed, only the height of the chin-rest is adjustable while it was necessary. We used a chin-rest to maintain a standard distance of 70 cm of the eyes from the stimulus monitor. Participants listened to songs from the Creative A50 audio speakers.
In the first experiment (experimenter-selected) with 32 participants, we used pop- songs in English that ranged from late 60’s to early 90’s of the last century (see - Table 1 for details). After listening to the 6 experimenter-selected songs, participants were requested to fill out The Goldsmiths Musical Sophistication Index, v1.0 – questionnaire (Müllensiefen et al., 2014).
In the second experiment (self-selected), ten of the original participants were initially requested to provide the names of 3 of their most favorite songs as well as the YouTube link of their favorite version of each song (when applicable). Each of these three obtained song files were used for that participant only. In total, the experiment used 30 different songs from the 10 participants ranging from the year 1984 to 2021 and comprising different languages (Bengali, English, Hindi, South Korean) and genres. After completing the experiment,
participants were requested to fill out The MUSEBAQ: A Modular Tool for Music Research to Assess Musicianship, Musical Capacity, Music Preferences and Motivations for Music Use questionnaire (Chin et al., 2016).
Table 1
List of (experimenter-selected) songs in the experimenter-selected design and regarding information
Artist Song Name Album Genre Year Duration
(msec)1 Air
Supply
All out of love Lost in love
Soft rock
1980 2290002 B. B.
King
The thrill is gone Completely well
Blues 1969 3190003 John
Denver
Take me home, country roads
Poems, prayers and promises
Country 1974 1880004 Bob
Dylan
Knockin’
on heaven’s door
Pat Garrett and Billy the Kid
Folk rock
1973 1440005 John
Lennon
Imagine Imagine Soft
rock/pop
1971 1830006 Nirvana Smells like teen
spirit
Nevermind Grunge 1991 2780007
2.3 Experiment: Experimenter-selected 2.3.1 Design: Experimenter-selected
We used 6 experimenter-selected songs (see Table 1) in this experiment. As well as we divided the 32 participants into 4 different groups (8+8+8+8). For this purpose, we used an independent measure/between-groups experimental design with 4 groups and we named it as – A1, A2, B1, B2. Each group consists of 8 participants. Here, the A and B defines
respectively two vertically flipped circumplex images according to the positioning of the polarities of the arousal dimension (see Figure 3). The numbers ‘1’ and ‘2’ states the
1 We Considered the ending point of each song at the time when there was no trace of sound.
2 From Air Supply (2009)
3 From B. B. King (2018)
4 From John Denver (2013)
5 From Bob Dylan (2019)
6 From Johnlennon (2018)
7 From Nirvana (2009)
sequence (original and reversed) of the songs (see Table 2). The purpose of these groupings of blocks was to counterbalance possible effects of songs’ sequence.
The reason for inverting the position of the valence labels is that a previous study (Brisson et. al., 2013) showed that, the measured pupil size can be significantly over and/or underestimated depending on gaze-positions and in different ways and extents for different types of eye trackers. We conducted a bias-detection experiment (n = 12) prior to the following experiments so as to estimate the presence of bias and its measurement error (see Figure 2). We requested participants to follow an animated disk moving along an invisible circle on screen, stretching from near the top to the bottom border of the screen. According to the outcomes of this test (see Appendices) we decided to invert the vertical coordinate of the circumplex image since there was a significant tendency to pupil size overestimation in the lowest position of the screen. Hence, by flipping one coordinate we counterbalanced positions and minimized the effect of this bias across all participants.
Therefore, for the conditions A1 and A2, the valence dimension was displayed vertically (where the pleasant part is at the top and unpleasant part is at the bottom), arousal dimension was always displayed horizontally (see Figure 3) with the exciting pole displayed on the right side of the circle. Moreover, for the conditions B1 and B2, the valence dimension was again displayed vertically (but inverting the polarities of valence – i.e., unpleasant at the top, and pleasant at the bottom). The arousal dimension was displayed as same in condition A1 and A2 (see Figure 3). Hence, we counter-balanced the positions of the end poles of the vertical valence coordinate. Furthermore, we used a slightly different phrasing of affective states from the original Russell’s 1980 model so that to make it more comprehensible to both native speakers and not.
Figure 2
a. A graphical representing from the output of ANOVA showing standard errors while measuring pupil size in different angles (°) using RED.
b. A graphical representation from the output of Fisher’s PLSD showing interaction of angles (°) which had significant effects
In the unchanging screen image, the intensity of Color (RGB: 0 - 250), was set according to the following, red = 150, green = 150, blue = 150. These background values were used also for the Instructions slides.
For the presentation sequence of the songs, we used fixed sequence for the groups A1 and B1 and, A2 and B2 (see Table 2). This is also counterbalancing against possible effects of sequence. The initial ordering of A1 and B1 was a random selection.
Table 2
The ordering of the pop-songs in different conditions for Experiment: Experimenter Selected
Order of the Pop-songs
Serial Condition - A1 & B1 Condition - A2 & B2
1 Air Supply - All out of love Nirvana – Smells like teen spirit 2 B. B. King – The thrill is gone John Lennon – Imagine
3 John Denver – Take me home, country roads
Bob Dylan – Knockin’ on heaven’s door
4 Bob Dylan – Knockin’ on heaven’s door
John Denver – Take me home, country roads
5 John Lennon – Imagine B. B. King – The thrill is gone 6 Nirvana – Smells like teen spirit Air Supply - All out of love
Figure 3
Circumplex displays for the condition A1, A1 & B1, B2 in Experiment 1. In (a) valence is in vertical position with pleasant is at the top and unpleasant is at the bottom. Whereas, in (b) unpleasant is at the top and pleasant is at the bottom, maintaining the axial positions of the arousal dimensions as same as (a).
2.3.2 Procedure: Experimenter-selected
All participants were welcomed in the Cognitive Laboratory of the Psychology department of University of Oslo. First, they were given some brief explanation about the experiment they are going to participate. Then, they sat in front of the SMI RED which is attached to the stimulus monitor. Participants had to position their head in a fixed chin-rest equipment.
At the beginning of the experiment, participants had to go through the Calibration and Validation procedure, so the eye-tracking machine could register the gaze movements.
During this 4-point calibration, we accepted only when error values were less than or equal to 0.5 mm. In rare cases, we accepted values high as 0.9 mm or less. See Figure 4 for
experimental timeline.
• After the calibration and validation part, a slide appears on the computer screen with written instructions (for detailed instructions, see Appendices), describing the steps of
the experiment. This was not time-limited. When participants finished reading and ensured that they comprehended the task, they could move to the next slide by pressing the spacebar of the keyboard.
• In the next phase, we showed the circumplex display (for conditions - A1/A2/B1/B2;
we used the respective image) to the participants to get familiar with various labels of affects. This part was time-limited (25000 msec).
• The next slide included 2nd part of the instruction, where we described the axis of the circumplex model showed in the previous slide (for detailed instructions, see
Appendices). Participants could move to the next slide when they finished reading and had comprehended the task by pressing the spacebar of the keyboard.
• In the next part, we showed a real time eye-tracking video from the experimenter’s viewpoint; this is, showing a person’s gaze positions while subjectively reporting her affective states of mind, by looking at the simplified circumplex model while listening to a musical piece (Hotel California by Eagles8, not included as one of the test songs).
The purpose of this video was to facilitate understanding of the role of gaze
movements in subjective ratings and these were tracked during the experiment. We limited the timing of this part with the duration of 28000 msec.
• Next slide was a practice trial. We used the actual stimuli image of the circumplex model for each respective experimental group (either A1, A2 or B1, B2), along with the previous example song (Hotel California, that we ran for 60000 msec before initiating the experiment).
• Each subsequent trial, presented each of the 6 experimenter-selected always showing the circumplex model variant of the specific conditions (A1/A2/B1/B2). Only from
8 From The Eagles - Topic (2018)
this phase of the experiment, we recorded and measured eye-tracking data.
Participants could take a short-break after finished listening a song and before starting Figure 4
Experimental timeline (for both Experiments – experimenter-selected & participant selected)
a new songs. During each pause, we programmed the eye-tracker to not record any data.
• After finishing the experimental task, participants were asked to fill the up The Goldsmiths Musical Sophistication Index, v1.0 – questionnaire. Later they were thanked for their valuable time for contributing to this study and were given a gift card as a compensation.
2.3.3 Results: Experimenter-selected
We defined area of interests (AOI) corresponding to the four (invisible) quadrants created by the intersection of the valence and arousal coordinates to use for analysis of gaze position (i.e., emotional ratings) and the computation of pupil diameters within each quadrant. We filtered-out the fixations and dwell time (fixation time percentage) outside the circumplex display. AOIs were hand drawn over the implicit quadrants within the circle of
Figure 5
Illustration of the Area of interests - AOI in color corresponding to each quadrant. Nota Bene: These AOIs were not visible to participants during the experiment.
Russell’s model: Top left, Top right, Bottom left and bottom right. We did rearrange the data (for the First Experiment – condition A (1,2) & B (1,2) according this AOI (see Figure 5).
We also extracted pupillometry data from this part of the experiment. After testing 32 participants with the experimenter-selected pop-songs, we analyzed separately the mean pupil diameter (mm) and the fixation time percentage/dwell time percentage.
2.3.3.1 Mean fixation time percentage (%) in the Arousal and Valence dimension of Circumplex display
A repeated measure ANOVA (α = 0.05) was performed to observe the single and combined effect of Arousal dimension (Unexciting, Exciting) and Valence dimension (Unpleasant, Pleasant) of Russell’s model. The analysis revealed a non-significant effect of
Arousal, F (1,31) = 3.6, p < 0.07. Furthermore, the result for the effect of valence dimension was significant, F (1,31) = 51.2, p < 0.001. Figure 6 shows participants spend most of their fixation time either in the unexciting-pleasant quadrant (mean = 28.2, SE = 2.2) or exciting- pleasant quadrant (mean = 28.4, SE = 2) or in other words, within the pleasant half of the circumplex display. However, about a quarter of the total song time, participants directed gaze within the unpleasant half of the display.
We computed the percentage of time which participants spent in different quadrans of the circumplex display. The procedure allowed to estimate the main intersection of the two core coordinates of affect, as subjectively experience: Valence and Arousal.
Figure 6
Experimenter selected: Mean fixation time % for the four quadrants of Russell’s circumplex model. The bars represent the standard error (SE)
2.3.3.2 Mean fixation time % for ‘common’ song-parts in quadrants
Repeated measure ANOVA was conducted among common song parts – Intro, Verse, Chorus and Outro to observe the single and combined effect of the Arousal and Valence dimension of the circumplex display, are as follows –
Intro
A within-subject repeated measure ANOVA (α = 0.05) revealed significant effect of valence dimension on the mean fixation time percentage, F (1,31) = 39.2, p < 0.0001. The effect of arousal dimension was not significant.
Mean fixation time % in the Unexciting-Unpleasant (UU) quadrant was 9.2 (SD = 6.7, SE = 1.2), in the Unexciting-Pleasant (UP) quadrant was 26 (SD = 9.8, SE = 1.7), in the Exciting-Unpleasant (EU) quadrant was 10.6 (SD = 9.1, SE = 1.6), and in the Exciting- Pleasant (EP) was 28.5 (SD = 12.7, SE = 2.2). See Figure 7.a.
Figure 7.a
Mean fixation time percentage in quadrants during – intro. The bar represents standard errors (SE)
Verse
A within-subject repeated measure ANOVA (α = 0.05) revealed significant effect of arousal dimension on the mean fixation time percentage, F (1,31) = 11.4, p = 0.002. The effect of valence dimension was also significant, F (1,31) = 47.2, p < 0.0001
Mean fixation time % in the Unexciting-Unpleasant (UU) quadrant was 13.7 (SD = 10, SE = 1.8), in the Unexciting-Pleasant (UP) quadrant was 32.7 (SD = 14.3, SE = 2.5), in the Exciting-Unpleasant (EU) quadrant was 8.1 (SD = 6.8, SE = 1.2), and in the Exciting- Pleasant (EP) was 25.2 (SD = 11.2, SE = 2). See Figure 7.b.
Figure 7.b
Mean fixation time percentage in quadrants during – verse. The bar represents standard errors (SE)
Chorus
A within-subject repeated measure ANOVA (α = 0.05) revealed significant effect of valence dimension on the mean fixation time percentage, F (1,31) = 58.3, p = 0.0001. The effect of arousal*valence dimension was also significant, F (1,31) = 4.7, p = 0.04.
Mean fixation time % in the Unexciting-Unpleasant (UU) quadrant was 14.5 (SD = 10.5, SE = 1.8), in the Unexciting-Pleasant (UP) quadrant was 28.3 (SD = 15.4, SE = 2.7), in
Figure 7.c
Mean fixation time percentage in quadrants during – chorus. The bar represents standard errors (SE)
the Exciting-Unpleasant (EU) quadrant was 7.1 (SD = 6.2, SE = 1.1), and in the Exciting- Pleasant (EP) was 30.6 (SD = 13.4, SE = 2.4). See Figure 7.c.
Outro
Figure 7.d
Mean fixation time percentage in quadrants during – outro. The bar represents standard errors (SE)
A within-subject repeated measure ANOVA (α = 0.05) revealed significant effect of arousal dimension on the mean fixation time percentage, F (1,31) = 8.8, p = 0.006. The effect of valence dimension was also significant, F (1,31) = 27.6, p < 0.0001
Mean fixation time % in the Unexciting-Unpleasant (UU) quadrant was 17 (SD = 13.6, SE = 2.4), in the Unexciting-Pleasant (UP) quadrant was 28.5 (SD = 16, SE = 2.9), in the Exciting-Unpleasant (EU) quadrant was 7.6 (SD = 6.3, SE = 1.1), and in the Exciting- Pleasant (EP) was 23.3 (SD = 13.5, SE = 2.4). See Figure 7.d.
2.3.3.3 Mean pupil diameter in quadrants of the circumplex during ‘total listening’ vs
‘chorus listening’
Total listening
A within-subject repeated measure ANOVA (α = 0.05) revealed significant effect of Arousal (exciting, unexciting) on the pupil diameter, F (1,31) = 7.101, p < 0.0121. The effect of Valence (pleasant, unpleasant) was not significant on pupil diameter.
See, Figure 8 a., representing change of pupil diameter in different quadrants of the circumplex display. Mean pupil size (mm) in the Unexciting-Unpleasant (UU) quadrant was 4.1 (SD = 0.6, SE = 0.1), in the Unexciting-Pleasant (UP) quadrant was 4.1 (SD = 0.6, SE = 0.1), in the Exciting-Unpleasant (EU) quadrant was 4.1 (SD = 0.6, SE = 0.1), and in the Exciting-Pleasant (EP) was 4.1 (SD = 0.6, SE = 0.1).
Chorus listening
A within-subject repeated measure ANOVA (α = 0.05) revealed non-significant effect of both Arousal (exciting, unexciting) and Valence (pleasant, unpleasant) dimension on pupil diameter.
See, Figure 8 b., representing change of pupil diameter in different quadrants of the circumplex display. Mean pupil size (mm) in the Unexciting-Unpleasant (UU) quadrant was
4.0 (SD = 0.6, SE = 0.1), in the Unexciting-Pleasant (UP) quadrant was 4.0 (SD = 0.5, SE = 0.1), in the Exciting-Unpleasant (EU) quadrant was 4.0 (SD = 0.5, SE = 0.1), and in the Exciting-Pleasant (EP) quadrant was 4.0 (SD = 0.5, SE = 0.1).
2.3.3.4 Mean pupil diameter for common song parts
An analysis of within-subject repeated measure ANOVA (α = 0.05) revealed a significant effect of different song segments (Intro, Verse, Chorus and, Outro) on subject’s pupil size, F (3,31) = 86.6, p < 0.0001. During the overall Intro for all the participants and songs
Figure 8
Representation of the mean pupil diameter (mm) in each quadrant of the circumplex display during two events. a. representing mean pupil size (mm) during total listening time. b.
representing mean pupil size (mm) during chorus listening time.
The bars represent the standard error (SE) in both a. and b.
– the pupil size was 4.4 (mm), and it decreased gradually as follows, during Verse = 4.1 (mm), during Chorus = 4.0 (mm) and, during Outro = 3.9 (mm) (see Figure 9).
We considered the – Intro, Verse, Chorus and Outro parts of the pop-song structure for analysis, because these four parts were common in all songs we selected. Moreover, during the
segmentation of different song parts (see Appendices for list of songs with segmented parts and relevant durations), it was found each of the songs have at least - one intro, two verse, two chorus (except for the song The thrill is gone – B.B. King, the first guitar-solo was selected as chorus) and one outro (except for the song The thrill is gone – B.B. King; the second guitar- solo was used as the outro). The thrill is gone is a famous popular blues song (follows a 12- bar-blues structure) by B.B. King, and it is not unusual for this song type of not having any chorus or typical outro part (Doll, 2011).
Figure 9
A box plot diagram representing mean pupil diameter (mm) for all common song parts averaged together for all songs
2.3.3.5 Mean pupil diameter for each song during different segments The analysis on each song are as follows –
All out of love – Air supply
A within-subject repeated measure ANOVA (α = 0.05) revealed significant effect of different song parts (Intro, Verse 1, Chorus 1, Verse 2, Chorus 2, Bridge, Chorus 3 and Outro) on the pupil size, F (7,31) = 19.1, p < 0.0001. See, Figure 10a. representing change of pupil diameter in different parts. Mean pupil during Intro (4.5 mm) gradually decreased till the Outro (4.0 mm).
The thrill is gone – B.B. King
A within-subject repeated measure ANOVA (α = 0.05) revealed significant effect of different song parts (Intro, Verse 1, Verse 2, Guitar solo 1, Verse 3, Verse 4, Guitar solo 2) on the pupil size, F (6,31) = 22.3, p < 0.0001. See Figure 10b. representing change of pupil diameter in different parts. Mean pupil during Intro (4.3 mm) gradually decreased till the Solo-2 (3.9 mm).
Take me home, country roads – John Denver
A within-subject repeated measure ANOVA (α = 0.05) revealed significant effect of different song parts (Intro, Verse 1, Chorus 1, Verse 2, Chorus 2, Bridge, Chorus 3, Outro)on the pupil size, F (7,31) = 28.6, p < 0.0001. See Figure 10c. representing change of pupil diameter in different parts. Mean pupil during Intro (4.5 mm) gradually decreased till the Outro (4.0 mm).
Knockin’ on heaven’s door – Bob Dylan
A within-subject repeated measure ANOVA (α = 0.05) revealed significant effect of different song parts (Intro, Verse 1, Chorus 1, Verse 2, Chorus 2, Outro)on the pupil size, F (5,31) = 26.5, p < 0.0001. See Figure 10d. representing change of pupil diameter in different parts. Mean pupil during Intro (4.3 mm) gradually decreased till the Outro (4.0 mm).
Imagine – John Lennon
A within-subject repeated measure ANOVA (α = 0.05) revealed significant effect of different song parts (Intro, Verse 1, Pre-chorus 1, Verse 2, Pre-chorus 2, Chorus 1, Verse 3, Pre-chorus 3, Chorus 2, Outro)on the pupil size, F (9,31) = 28.7, p < 0.0001. See Figure 10e.
representing change of pupil diameter in different parts. Mean pupil during Intro (4.5 mm) gradually decreased till the Outro (4.0 mm).
Figure 10
Mean Pupil diameter (mm) for different segments of each experimenter selected songs. The bar for each song (a., b., c., d., e., f.) represents standard error (SE)
Smells like teen spirit – Nirvana
A within-subject repeated measure ANOVA (α = 0.05) revealed significant effect of different song parts (Into 1, Intro 2, Intro 3, Verse 1, Pre-chorus 1, Chorus 1, Verse 2, Pre- chorus 2, Chorus 2, Guitar solo, Intro 4, Verse 3, Pre-chorus 3, Chorus -Outro)on the pupil
