Exploring Movement-Based Rhythmic Interaction with Senior Citizens

(1)

Exploring Movement-Based Rhythmic Interaction with Senior Citizens

Rune B. Rosseland

Thesis submitted for the degree of Philosophiae Doctor (Ph.D.)

Department of Informatics

Faculty of Mathematics and Natural Sciences University of Oslo

June 2018

(2)

© Rune B. Rosseland, 2018

Series of dissertations submitted to the

Faculty of Mathematics and Natural Sciences, University of Oslo No. 1991

ISSN 1501-7710

reproduced or transmitted, in any form or by any means, without permission.

Cover: Hanne Baadsgaard Utigard.

Print production: Reprosentralen, University of Oslo.

(3)

ABSTRACT

This thesis reports on a concept-based interaction design research project focusing on movement-based interaction with music. The project aimed to design rhythmic interactions with music to promote and motivate increased physical activity and social interaction among senior citizens in general and seniors with Alzheimer’s disease in particular. Employing a Research through Design approach, the project used sketching and prototyping in software and hardware to explore and produce knowledge about a specific but generic conceptual idea. The idea, formulated as the conductor concept, describes an interactive music player that adapts and synchronizes the beat of (self-selected) music to the tempo of repetitive bodily movements in the same way that an orchestra synchronizes to the tempo of the conductor’s gestures. The project sought to understand the important relationships and essential characteristics of the conceptual idea and to explore how it could be designed to create meaningful and aesthetically pleasing interactive experiences.

In an explorative and iterative process of seeing-moving-seeing, an interactive research prototype, RepMoves, was designed. Beyond developing and exploring the prototype in the lab, various versions of it were also brought out into the field to test, explore, and open up discussions about the interactive experience with senior citizens, caretakers, and other informants.

The contribution of this thesis is to discuss and critically examine how Research through Design was used to produce design-relevant knowledge about the conceptual idea and its potential use in the context(s) of social activity groups for senior citizens (with and without Alzheimer’s disease). It opens up a design space that is informed by the conceptual idea; the designed artifact; the interactive experience; the contextual exploration of the prototype with senior citizens, caretakers, and other informants, and; a wide range of existing research relevant to the inquiry. The project contributes a specific synthesis of knowledge pertinent to interaction design practitioners and researchers interested in designing movement-based interaction with music. It also offers reflections on how the artifact was used to engage senior citizens, caretakers, and other informants in explorations and discussions about a future-oriented design concept.

(4)

(5)

ACKNOWLEDGMENTS

First and foremost, I would like to give a warm and heartfelt thank you to my main supervisor Alma Leora Culén who has spent considerable time and effort to guide, critique, and encourage me throughout the last four years. I would also like to give a big thank you to my co-supervisor Anders-Petter Andersson and all my colleagues in the DESIGN group for valuable input, support, and discussions. I am also grateful to all the senior participants, their caretakers, and other informants who have provided invaluable input and feedback on the work presented here. Finally, I would like to thank my family and friends who have supported me throughout this process.

(6)

(7)

PART I

––––––––––––––––––––––––––––––––––––––––––––––––––

1 Introduction

––––––––––––––––––––––––––––––––––––––––––––––––––

1.1 Music and movement

[M]usic can be thought of as a type of legal performance-enhancing drug.

Costas I. Karageorghis & David-Lee Priest [1, p.47]

Music is widely used to motivate, pace, and sustain different forms of exercises and physical activity practices. For many people, music is an essential part of their exercise regime. We listen to music while jogging, cycling, walking, rowing, lifting weights, and so on, and the music affects our performance and experience in various ways.

Research has shown that music increases arousal, elevates mood, influences emotions, improves endurance, reduces perceived effort, distracts from pain and fatigue, helps us keep a steady pace, and makes a boring and repetitive activity more engaging and enjoyable [1]–[3]. "When listening to music, people run farther, bike longer and swim faster than usual–often without realizing it" [2]. In this thesis, I refer to these effects generally as the ‘motivational effects of music.’

On a neurological level, there are direct connections between auditory neurons and motor neurons, and studies have shown that listening to music produces activity in regions of the brain responsible for coordinating movement [2], [4]–[6]. As such, human beings seem to be hard-wired to associate music with bodily movement.

In fact, the human brain may have evolved with the expectation that, wherever there is music, there is movement—although this idea emerges more from the imaginative minds of speculating evolutionary psychologists than from experimental evidence. Before the invention of reed flutes and other musical instruments, our ancestors likely produced the earliest forms of music by singing, screaming, chanting or otherwise using their vocal cords, as well as by physically interacting with their own bodies, other people and the environment. A fast tempo would have likely required fast movements: quick clapping or foot stamping, perhaps. Deep, loud sounds would have demanded great energy and force—belting a note or beating the ground or a rock. In its conception, music was likely an extension of the human body. Maybe the brain remembers it that way [2].

Karageorghis and Priest [1] identify four factors that influence how music affects us during exercise and repetitive physical activity: Rhythm response, musicality, cultural impact, and association.

(12)

Rhythm response refers to the effects of musical rhythm, especially tempo (speed of music as measured in beats per min [bpm]). Musicality refers to the pitch-related elements of music such as harmony and melody. Cultural impact concerns the pervasiveness of the music within society or a sub-cultural group. Finally, association refers to the extra-musical associations that may be evoked. [1, p. 45]

Rhythm response and musicality are considered internal factors because they are inherently linked to the musical content. Cultural impact and association are external factors as they refer to phenomena external to the music itself. The four factors are hierarchically related, implying that rhythm response is the most influential and important factor, followed by musicality, cultural impact, and finally, association being the least important.

These four factors are, in different ways, all relevant to the interaction design research project reported in this thesis: the RepMoves project. In fact, their relevance to the project is reflected in their hierarchical influence on human movement. The project explored the basic conceptual idea of using tempo (rhythm response) as an interactive link between repetitive movement and musical beat to motivate physical activity and social interaction. In other words, the project explores the design of an interactive music system that dynamically synchronizes the beat of any (self-selected) music to the tempo of human movement. Human beings have a natural inclination to adapt their movement pace to the rhythm of music, and this project explores the idea that the music can also adjust its tempo to match a person’s (or several people’s combined) movement tempo. In this thesis, this conceptual idea is referred to as ‘the conductor concept,’ which builds on the metaphor of an orchestra conductor who controls the tempo of the music played by the orchestra through repetitive hand gestures. This concept is explained in more detail in chapter 2 (section 2.2.2). The underlying assumption was that by ensuring that the beat of the music coincides with the tempo of repetitive movement, the system would support and magnify the motivational effects of music. The goal was to produce an aesthetic and meaningful interactive experience that leans on the power of musical rhythm to motivate, pace, and sustain human movement. However, it is important to underline that this thesis focuses on the design and exploration of the interactive system/experience rather than empirical assessment of the motivational effect of such a system. Based on evidence from the literature, it takes the motivational effects of music as given, and asks how this can be taken advantage of in the design of interactive technologies aiming to promote and motivate physical activity. Furthermore, this thesis explores these ideas in the context of designing welfare technology for senior citizens. It asks how the conductor concept can be used to promote and motivate increased physical activity and social interaction among senior citizens. The next section gives some background for this inquiry.

1.2 Problem setting

As a result of increasing lifespan expectancy and decreasing birthrates in many nations, the world is facing an increasingly aging population. This is leading to a

(13)

growing interest in keeping aging citizens healthy, active, and independent. In particular, there is a demand for new technologies that can contribute to improve and support the health and welfare of senior citizens and to allow seniors more autonomy and independence in their daily lives. In Norway, governments, institutions, and organizations are calling for the design and development of innovative technologies that can help senior citizens prevent, reduce, and manage age-related health challenges, and to reduce their dependence on care services [7]–[10]. This is rooted in the realization that current models of elderly care are economically unsustainable as an increasing portion of the population is reaching retirement age. Governments and public institutions see the potential for new health and welfare technologies to improve public health care services while reducing the cost of those services on a per capita basis.

So far, these efforts have produced a multitude of devices and systems for monitoring and managing health challenges related to old age, such as ‘smart’ pill dispensers, fall detection and alarm systems, various forms of personal health monitoring systems, and a wide range of other devices and systems. These are all potentially useful technologies that can contribute to improve, extend, and even save lives, and it is not my intention to diminish their importance or usefulness. However, in various ways, such technologies may contribute to construe or mediate [11], [12], the recipient user as old, ill, disabled, or otherwise in need of help and care. This raises some important questions regarding our collective view and treatment of elderly people. For example, how does it affect a person to be told that ‘you are no longer capable of remembering to take your medication, and no-one has the time to help you so here’s a device that will remind you of this fact several times a day’? Or worse: ‘some of the pills you have been prescribed will make you disoriented and forgetful, so here’s a device that will remind you to take those pills.’ Technology design is not morally and ethically neutral, and it is crucially important to consider how the technologies we design may shape the everyday reality of their intended users [13], [14]. The interaction design research project presented in this thesis did not address specific problems or age-related health challenges, but focused on an imagined future where seniors, either individually or in groups, engage in musically augmented physical activity practices – not because they ‘should be more physically active,’ but because they enjoy the experience. The project sought to understand how we can use music to design interactive experiences that mediate aesthetically enjoyable and emotionally meaningful physical activity practices. Taking a salutogenic [15] approach to health promotion, the project sought to understand how interaction design can be used to promote positive potentials for good health as opposed to avoiding negative potentials for disease and disability. By taking departure in the fact that physical activity is an important and effective modifier of risk of developing age-related health challenges [16]–[19], and combining it with the universal power of music to induce, motivate, pace, and sustain repetitive movement [1]–[3], the research aimed to produce knowledge on how we can design interactive music technologies to create physical activity practices that are intrinsically motivated rather than externally prescribed [20], [21].

(14)

Accordingly, this thesis is a qualitative and phenomenologically based study of interactive technology design oriented towards the health and welfare of senior citizens. The project explored the conductor concept through the design, development, and contextual exploration of an interactive prototype named RepMoves (short for Repetitive Moves). As such, it addresses two distinct, but tightly interwoven themes. First, it focuses on the process of designing the prototype as a way to better understand the conceptual idea and to make the conductor concept accessible for experiential exploration. Second, it focuses on how the research prototype was used to open up conversations with senior citizens, caretakers, and other stakeholders about the experience it provides and its potential to promote and motivate physical activity and social engagement among senior citizens. Throughout this process, the prototype was iteratively developed through an ongoing exploration of different design choices.

These choices were based on: (i) my own qualitative and embodied judgments about the emerging interactive experience, and (ii) on feedback and insights gained from sessions with participating seniors, caretakers, researchers, and other informants in real-world use contexts. The process did not follow a pre-defined and structured research methodology but evolved relatively organically through a pragmatic [61, 112, 125] and phenomenological [54-58] engagement with design materials, the emerging artifact, and its use in specific contexts. The initial design idea was guided by curiosity regarding the imagined interactive coupling represented by the conductor concept, and its potential for producing rich and aesthetically pleasing interactive experiences: What if the music you listen to automatically synchronizes its beat and tempo to the tempo of your bodily movements? How would it feel? How would it affect your movements? How could such a system be designed to produce rich and aesthetically pleasing experiences? How could such a system be designed to involve senior citizens in experiential explorations of the generic and conceptual idea?

Furthermore, the work presented in this thesis was informed and influenced by research and theories from a wide range of research fields, such as gerontology (e.g.

[25]–[28]), gerontechnology (e.g. [29]–[33]), health promotion (e.g. [15], [34]–[36]), neurology of music and movement (e.g.[5], [6], [37]–[39]), music therapy (e.g. [40]–

[47]), sports medicine (e.g. [1], [3], [17], [48]–[51]), psychology of human motivation (e.g. [20], [21], [52], [53]), phenomenology (e.g. [54], [55]), post-phenomenology (e.g.

[11]–[14], [56]–[58]), pragmatist philosophy and aesthetics (e.g. [59]–[61]), somaesthetics (e.g. [62]–[67]), movement-based interaction (e.g. [68]–[81]), and exergaming (e.g. [82]–[94]), to name the most central.

1.3 Structure of this thesis

This is a compilation thesis based on four published and one submitted research papers. The thesis is divided into two parts. Part I, (the "kappa" in Norwegian) provides an introduction and overview of the five papers and tie them together in a narrative structure that tells the story of my Ph.D. project. Part II contains the five included research papers. The remainder of this thesis is structured as follows:

(15)

Chapter 2 introduces theoretical perspectives, concepts, and frameworks within interaction design research and philosophy of science and technology that are relevant to the research presented in this thesis.

Chapter 3 presents Research through Design (RtD) as the chosen research methodology, emphasizing the design and use of prototypes, artifacts, and products as an approach to knowledge production in interaction design research.

Chapter 4 describes early fieldwork and a literature review focusing on the involvement of seniors in the design of welfare technology for seniors. The chapter concludes with a discussion of how the findings from this initial research led to the formation of the RepMoves research project. The chapter is based on the two first research papers included in this thesis.

Chapter 5 presents the RepMoves research project. It documents and describes how the project evolved from a conceptual idea into a functional prototype, and how the prototype was used to engage senior citizens and other informants and stakeholders in contextual explorations of the interactive experience it provides. The chapter is based on the last three included papers.

Chapter 6 discusses how the design and contextual exploration of the prototype produced design-relevant knowledge about the conceptual idea. It draws on a variety of existing theories, frameworks, concepts, and approaches to practice-oriented interaction design research to frame the discussions. The chapter is structured according to four research questions that reflect the evolution of the research inquiry.

The next section introduces the four research questions.

1.4 Research questions

I used the design process intentionally to raise questions about the interactive concept and its use in specific contexts while simultaneously constructing and exploring material and contextual answers to those questions. As such, the project was more focused on problem setting than problem-solving, where the conceptual idea of synchronizing musical beat to the tempo of human movement took center stage, and where the research questions evolved in parallel with the evolving problem setting.

Here, the questions are sequentially organized according to the evolution of the research project.

RQ1: How can prototyping be used to build a material and experiential understanding of tempo as an interactive coupling between music and movement?

This research question implicitly recognizes that a functional system is a necessary prerequisite for the experience to take place. The question is directed towards

(16)

concerned with exploring and understanding the interactive logic and mechanics required to realize the conceptual idea and focuses on the concrete details of constructing the interactive system. Furthermore, an understanding of the interactive logic is not complete without a good understanding of the resulting interactive experience. Accordingly, this research question addresses the dialectic process of going back and forth between making changes to the emerging system and experientially evaluating the effects of those changes by actively interacting with it. This research question is mainly addressed in the third paper included in this thesis, presented in section 5.1. Also, section 5.4 covers the latter part of the design process which is not included in paper 3.

RQ2: How can the RepMoves prototype be designed to facilitate interpersonal entrainment between two people (with Alzheimer’s disease) engaged in co-located repetitive movement activities?

This research question emerged as I started to reflect on how the interactive concept could be used to facilitate and promote social interaction among senior citizens.

Moving to the rhythm of music is an inherently social activity, and we derive pleasure from moving in sync with other people. Hence, this research question explores how the conductor concept can be designed and used to facilitate interpersonal entrainment (a gradual adjustment towards a shared tempo) and synchrony of bodily movements. Furthermore, the question is explored in the context of an activity group for elderly people with mild Alzheimer’s disease or dementia. The results from this study are documented in paper 4, which is presented in section 5.3.

RQ3: How can RepMoves be optimized with respect to the characteristics of a research product (inquiry-driven, finish, fit, independent) to support its use in an activity group for Alzheimer patients?

The study of interpersonal entrainment with Alzheimer patients led to discussions and speculations with the caretakers regarding how the system could be modified and further developed to become a regular part of the group’s activity program. Inspired by the ideas we came up with, the RepMoves prototype underwent a significant redesign process where other design elements such as visual narratives (photos and videos), a scoring system, and an RFID-based token-system were implemented. As a result, the focus of the inquiry changed from an open-ended exploration of the generic interactive concept (and experience) towards a more specific and targeted design intended for a specific use context. This research question is addressed in paper 5, presented in section 5.5.

RQ4: What kind of knowledge is generated in the process of designing a deliberately open and flexible research prototype and exploring it in specific contexts of use?

The first three research questions focused on specific parts of the design/research process to discuss how they generated design-relevant knowledge about the conductor concept and its use with senior citizens in specific real-world use contexts. The fourth

(17)

research question, on the other hand, opens up for higher-level reflection and discussion of what kind of knowledge the RepMoves project generated, and how it relates to existing knowledge within Interaction Design (IxD) and Research through Design (RtD). It also reflects on how the project produced knowledge about how to involve senior citizens in general, and seniors with Alzheimer’s disease in particular, in future-oriented design explorations.

1.5 Contributions

This thesis contributes to the emerging field of Research through Design (RtD).

Coming from a computer science (CS) tradition, it intersects with the fields of Human- Computer Interaction (HCI) and Interaction Design (IxD). As such, it is located within the third wave [22] or paradigm [23], [24] of HCI, which emphasize the phenomenologically, culturally, and socially situated nature of technology use.

Furthermore, it is a practice-based study of interactive technology design that focuses on exploring and proposing potential and alternative futures.

The overarching contribution of this thesis to the field of RtD is to describe and document the approach taken in one specific RtD project, and to discuss how, and what kind of knowledge was produced in the process. The contributions exist in the space between the conceptual idea, the designed artifact, the interactive experience, and the contextual exploration of the prototype with senior citizens, caretakers and a range of other informants. In addition, the contribution is informed and framed by existing frameworks, concept, and theories within interaction design research, as well as a variety of relevant research fields such as gerontology, gerontechnology, and music therapy, among others. In sum, the contribution of the thesis is to elucidate how, and what kind of knowledge was produced through the design and exploration of an interactive system.

Concept: On a conceptual level, the thesis contributes to the repertoire of interaction design practitioners and researchers interested in designing musical interactions. It gives detailed descriptions of how the concept was implemented in a functional artifact, and how it was adapted and refined to achieve the desired interactive experience. It opens up a design space where music can be used to promote and motivate physical activity and social interaction.

Experience: The thesis describes and discusses how the interactive experience evolved as a result of intentional design changes in the emerging artifact. It grounds design choices in qualitative judgments about the phenomenological experience of interacting with the system, and it describes how the experience evolved as a result of those changes.

Artifact: The thesis also contributes a functional artifact that, explicitly and

(18)

(https://github.com/rrsslnd/repmoves), thereby enabling other designers and researchers to build upon and extend the work presented here, or to take it in completely different directions, depending on their interests.

Contextual use with seniors with Alzheimer’s disease: The thesis also contributes with a discussion of how the interactive concept/experience was designed and used to engage senior citizens, with and without Alzheimer’s disease, in explorations and discussions about the concept/experience. It reflects on observations, comments, and discussions with participating seniors, caretakers, and other informants regarding potential uses of the concept/experience for the promotion of physical activity and social interaction among Alzheimer patients. This includes the addition of other design elements, such as the use of image sequences and video narratives to create a more compelling and meaningful interactive experience.

Furthermore, the thesis also contributes some reflections and practical guidelines for working with Alzheimer patients to explore future-oriented interaction design concepts.

Methodology: Finally, on a methodological level, the thesis contributes to ongoing discussions about Research through Design, and prototyping in particular, as an approach to knowledge production in interaction design research. It describes and documents the process of (i) constructing and refining the prototype, and (ii) its use in real-world use contexts to produce knowledge about the underlying conceptual idea and its potential as a tool for promoting increased physical activity and social interaction among senior citizens (with and without Alzheimer’s disease). Thus, the thesis describes how knowledge was produced and opens up the research approach to constructive discussion and critique by the interaction design research community.

1.6 Limitations

This thesis is limited by the scope of the research project and the nature of the conceptual idea. It is a qualitative inquiry, and it does not claim to present objective and universally valid knowledge. However, there is an important distinction to be made between the subject of the design project (the conceptual idea), and the subject of the inquiry (how design is used to produce knowledge about the conceptual idea).

Rather than attempting to uncover some fundamental truth about the world, the thesis takes departure in a fundamental relation between human movement and music (rhythm and tempo) and creates a particular design space to explore this relation.

Hence, the contribution of the thesis is not primarily concerned with the epistemological status of the interactive concept, but rather to describe and discuss how, and what kind of knowledge was produced through design. As mentioned in section 1.2, the work presented in this thesis was informed and influenced by a diverse range of related research fields and theories. Although it touches upon some of these research fields, it is outside the scope of this thesis to give a comprehensive account of how they relate to and informed the work presented here.

(19)

––––––––––––––––––––––––––––––––––––––––––––––––––

2 Theoretical Concepts and Perspectives

––––––––––––––––––––––––––––––––––––––––––––––––––

[A] great deal of design theory tends to be generative and suggestive, rather than verifiable through falsification. This seems self-evident in the case of ‘manifestos’, but also of more grounded generalisations from particular design examples. The problem here is not just that theory underspecifies design, so that practitioners will be faced by innumerable decisions whatever theory they use, but that theory is underspecified by design, in the sense that many aspects of a successful design will not be captured by a given theory.

William Gaver [95, pp. 943–944]

This chapter presents the main theoretical frameworks, concepts, and perspectives that have informed and guided the work presented in this thesis. This includes a somewhat mixed list of theories and perspectives that relate to the project in different ways, and at various levels of abstraction. It starts by presenting some fundamental differences between scientific and designerly approaches to reality, knowledge production, and theory. Next, it presents selected frameworks and theories originating within interaction design research practice, including the interaction design research triangle [96], concept-driven interaction design research [97], strong concepts [98] and generic design thinking [99]. This is followed by more abstract concepts and perspectives borrowed from phenomenology and post-phenomenology to frame the knowledge contribution of this thesis. These include: embodied and hermeneutic human-technology relations [58], multistability [57], [100], and technological mediation [11], [12]. Each theory or concept is followed by a discussion of its relevance to the RepMoves project.

2.1 Science and design

There are tensions within the HCI and interaction design research communities regarding the fundamental differences between science and design, and the implications this has for interaction design research [22], [30]–[39]. Science with a big ‘S’ is a never-ending knowledge-building project where the ultimate goal is the discovery of truth. It is the pursuit of objective and universal knowledge about the present or historical world [95], [104]. Design, on the other hand, is a forward-looking endeavor. It is in pursuit of the new and non-existent, and it actively interferes with the world in an attempt to change it and to create preferred and possible futures.

Science is analytic, attempting to deconstruct and pick apart phenomena to understand, explain, and predict reality. Design is synthetic, attempting to combine, shape and form existing reality into a new, preferred one. Thus, science and design approach reality in fundamentally different ways. It is acknowledged that this is an

(20)

the distinction is relevant to current debates about the nature and legitimacy of interaction design research in HCI. It raises important questions about the rigor, relevance, and accountability of interaction design research [101], [102], [105], [111].

By what measures can we claim to know that the outcome of the research is true? Is it possible to make claims about the objectivity, universality, and validity of knowledge contributions? Is it even desirable? How can research contributions have relevance for interaction design practice? How can design methods be used in the pursuit of new knowledge? How should IxD research contributions be documented, evaluated and critiqued?

According to Gaver [101, p. 147], science and design operate according to, and are defined by, different and largely incommensurable forms of accountability:

Science is defined by epistemological accountability, in which the essential requirement is to be able to explain and defend the basis of one’s claimed knowledge. Design, in contrast, works with aesthetic accountability, where “aesthetic” refers to how satisfactory the composition of multiple design features are (as opposed to how ‘beautiful’ it might be). The requirement here is to be able to explain and defend—or, more typically, to demonstrate—

that one’s design works.

The ability to explain and defend scientific research is inextricably linked to methodological rigor. Scientists are expected to adhere to tested and approved methodological protocols and to describe how methods were used to produce the given result. Thus, other researchers can repeat the procedure to see if it produces the same results. Reliability and objectivity are ensured through repeated and independent replication of results.

In design, on the other hand, there is no guarantee that two designers applying the same design methodology to the same design problem will produce the same result.

Indeed, it is highly unlikely, as well as undesirable. Reducing the design process to a procedural how-to recipe would negate the creative power of design to produce new, inspired, and unexpected designs in response to underdetermined [105], messy [112], and wicked [113] design problems. Aesthetic accountability values an integration of formal, functional, material, social, cultural, emotional, and moral concerns, among others, into a compositional whole – an artifact, or ‘ultimate particular’ [114] – that works in its intended use context. Accordingly, design involves judgments of quality based on personal and professional sensibilities and interpretations regarding what works, and what is good. Design does not seek objective, generalizable, and replicable truths, but good designs that work well, and that generate new ideas, reflections, and visions of potential and possible futures. Accordingly, design research emphasizes and foregrounds "designerly ways of knowing" [115], [116].

Theory is the primary vehicle for progress and advancement of knowledge in scientific disciplines. It provides an ontological framework of entities and causal mechanisms that allows explanation and prediction of phenomena in the world [101]. Theory forms the basis for the development of hypotheses, which are operationalized into experiments, allowing empirical assessment. Data from experiments are analyzed

(21)

according to the theoretical assumptions manifested in the hypotheses, thereby contributing to extend, refine, or clarify the underlying theoretical model.

The role of theory in interaction design is ambivalent and debated [30], [32], [40], [46]–[51]. On the one hand, interaction design researchers are expected to publish their research in academic conferences and journals, where epistemological accountability is the ultimate yardstick of acceptance and excellence. Accordingly, interaction design researchers hoping to publish their research must conform to the epistemological demands of the relevant academic and scientific communities. HCI researchers have drawn on a multitude of theories and methods from disparate scientific disciplines, from cognitive psychology, ecological psychology, engineering, anthropology, ethnomethodology and ethnography, behavioral science, activity theory, grounded theory, among others [103]. On the other hand, interaction design practitioners, operating according to aesthetic accountabilities, have trouble relating to and adopting research output framed by theories from ‘external’ scientific disciplines [103], [111]. Stolterman [105, p. 56] argues that research aiming to contribute to interaction design practice must be rooted in a fundamental understanding of the nature of "design as a unique human activity of inquiry and action," and that scientific disciplines may not provide the most suitable approach to design complexity. Similarly, Buchanan [123, p. 17] points to, and reflects on this central dilemma of design research:

What is the nature of a discipline that brings together knowledge from so many other disciplines and integrates it for the creation of successful products that have impact on human life and serve human beings in the accomplishment of their individual and collective goals?

Those involved in design research are easily drawn into research in other fields. Indeed, it is tempting to evaluate design research by its contributions to other fields. In design research, however, the central challenge is to understand how designers may move into other fields for productive work and then return with results that bear on the problems of design practice.

Design knowledge, it seems to me, lies in our grasp of the principles and methods of design that allow this activity to take place and lead to effective products. The alternative, common among some design theorists and researchers, is to believe that design must ultimately be reduced to one or another of the other disciplines—i.e. cognitive science, engineering, fine art, anthropology, marketing, and so forth.

Since the 1990s, HCI and interaction design research have increasingly been influenced by theories, methods, and approaches from design practice. For example, Rittel and Weber’s [113] notion of wicked problems and Donald Schön’s [112]

understanding of design as a reflective conversation with the materials of a ‘messy’

design situation have had a significant influence on interaction design research.

According to Buchanan [124, p. 16]:

Design problems are ‘indeterminate’ and ‘wicked’ because design has no subject matter of its own apart from what a designer conceives it to be. The subject matter of design is potentially universal in scope, because design thinking may be applied to any area of human experience. But in the process of application, the designer must discover or invent a particular subject out of the problems or issues of specific circumstances.

(22)

This understanding of design problems as inherently ‘wicked’ emphasizes the fact that design is not primarily focused on problem-solving, but problem setting. According to Schön [125, p. 2]:

[I]t is clear that a "problem space" is not given with the presentation of design task: the designer constructs the design world within which he sets the dimensions of his problem space and invents the moves by which he attempts to find solutions.

Schön conceptualizes design as an iterative re-framing of the problem space through a process of seeing-moving-seeing: First, the designer ‘sees’ the problem space or design situation, reflecting on and evaluating how the situation differs from a desired or ideal state. This ‘seeing’ is not understood literally as ‘visually seeing,’ but as a more or less tacit understanding and sensory appreciation for ‘what’s there.’ Furthermore, this

‘seeing’ implies normative judgments of quality with regards to “what’s bad and needs fixing, or what’s good and needs to be preserved or developed” [125, p. 10]. Based on such normative judgments of quality, the designer conceives and makes a ‘design move’ – a material and intentional intervention or ‘change experiment’ aimed at improving the situation. Finally, in the second act of ‘seeing,’ he evaluates and reflects on the outcome of the design move. Did it have the intended effect? How did it change the situation? What are the (inevitable) unintended consequences? Are the unintended consequences desirable, tolerable, or undesirable? The answers to these questions determine whether the move is accepted and incorporated into the evolving artifact and problem setting, or discarded, to be replaced by a new design move. Thus, designers manage design complexity incrementally through a reflective ‘conversation’

with the materials of the design situation. Furthermore, the evolving sketches and artifacts become the designer’s increasingly refined arguments and proposals for what constitutes good design, given the parameters of the evolving design space.

2.2 Interaction design theory

2.2.1 The interaction design research triangle

Fällman [96] developed the interaction design research triangle as a model to frame, guide, discuss, and reflect upon our approaches to interaction design research. The model is built on the recognition that there are three fundamental forms of interaction design research activities (design practice, design studies, and design exploration) where each activity "has its own purpose and intended outcome and [where] the rigor and relevance have to be defined and measured in relation to what the intention and outcome of the activity is" [102, p. 268]. Each corner of the triangle (see Figure 1) represents one of the three IxD research practices. It recognizes that research activities normally do not fit neatly into one of the three practices, but exist somewhere in the space between two, or even all three, research practices. Thus, the model creates a two-dimensional space for plotting the position of a particular research activity. I find the model helpful in framing and discussing the different research activities of my Ph.D. project, and the contributions of this thesis. In what follows, I give brief

(23)

descriptions of each of the three IxD research practices and discuss how they relate to my research project.

Figure 1. The interaction design research triangle of design practice, design studies, and design exploration [96, p. 5].

Design practice

Design practice as a research activity is very similar to the work of professional interaction design practitioners outside of academia. It is a generative and synthetic practice where the researcher becomes an integral part of a multidisciplinary design team working on a real project with real clients, not primarily as a researcher or observer, but as a designer. This involves taking part in the actual hands-on design work of sketching, constructing and building artifacts and prototypes; dealing with time and resource constraints; communicating and negotiating with a variety of stakeholders, including clients, managers, engineers, sales and marketing, potential users, and so on [96]. Thus, the researcher can build an appreciation and understanding of the tacit knowledge and competencies that are involved in professional design practice. However, unlike a regular designer, interaction design researchers must approach the process "with an explicit design research question in mind, or with the clear intent of forming such a question from their activities" [96, p.

6]. The research question does not have to align with the direction and goal of the design project but can be formulated to focus on particular issues or themes that are relevant from a research perspective.

Design exploration

Similar to design practice, design exploration is synthetic and proactive, involving the researcher in a reflective, hands-on process of designing and constructing prototypes,

(24)

requirements of a client or user, design exploration revolves around the researcher’s own research interests, where "the most important question is: What if?" [96, p. 7].

Design exploration intends to experiment, question and provoke critical reflection on the current state of the world, and to imagine possible, alternative, and preferred futures. "[D]esign exploration is a way to comment on a phenomenon by bringing forth an artifact that often in itself, without overhead explanation, becomes a statement or a contribution to an ongoing societal discussion" [96, p. 8]. Thus, the artifact is not primarily concerned with functionality and utilitarian purposes, but with larger, more complex issues of human and social ideals, values, and notions of the good life [126]. Nevertheless, design exploration can also be used in more traditionally oriented research where design is used as a driving force in the research process, but where the research interests are more aligned with scientific approaches to knowledge production: "[T]his is the case when the kind of knowledge and user experience sought is the kind that cannot be obtained if design–the bringing forth of an artifact such as a research prototype–is not a vital part of the research process" [96, p. 8].

Design studies

Design studies are the type of interaction design research activities "that most closely resembles traditional academic disciplines" [96, p. 9], where the goal is to build upon and contribute to a cumulative body of knowledge. This requires an analytical engagement with design theory, methods, history, and philosophy, as well as theories and approaches from a variety of other disciplines. It also involves presenting and publishing research outcomes in academic conferences and journals. "[U]nlike design practice, [design studies] seeks the general rather than the particular, aims to describe and understand rather than create and change, and because of that often appears as distancing to its character rather than involving" [96, p. 9].

Trajectories, loops, and dimensions

The triangular mapping of the three activity areas makes it possible to position a particular research activity within the model. However, as Fallman [96] points out, it is not so much the positioning of a particular activity within the model that makes it useful, but the way in which it enables reflection and discussion about how interaction design researchers move in between the three activity areas. To that end, the model also includes the concepts of trajectories, loops, and dimensions. Trajectories are intentional or unwanted drifting between research activities. They enable discussions about the perspective and direction of a particular research activity, how the outcome of the activity may feed into another activity, and "what kind of quality measures, guarantors, and stakeholders we will face when moving in between different activity areas" [96, p. 11]. Loops are trajectories without start and endpoints, signifying an ability to freely move back and forth between two, and in some cases all three, activity areas. Thus, activities in different activity areas feed into each other, iteratively driving the research forward. Finally, dimensions infuse the model with meaning by creating conceptual continuums and tensions between the activity areas.

(25)

Relevance to the RepMoves project

This thesis is built on the journey of the RepMoves project, which, first and foremost, was an explorative project. It explored the conceptual idea of rhythmic interaction with music through the design of a research prototype. The artifact was designed to make the imagined user experience available for further experiential exploration and discussion. However, the underlying purpose of the exploration was to understand how the concept could potentially be used to inspire and promote increased physical activity and social interaction among senior citizens. As such, there was an implicit trajectory from design exploration towards design practice, with its focus on designing a functionally coherent and marketable product. To be clear, it was not the intention to develop the prototype into a commercial product during this project. Nevertheless, the ultimate reason for exploring the idea was to better understand how it could potentially form the basis for, or become a part of, a future product aiming to engage senior citizens in aesthetically pleasing and socially engaging physical activity practices. In parallel with the design and exploration of the prototype and its underlying idea, I engaged with a wide array of literature and research on interaction design theory, methods, and practice, some of which are presented in this chapter.

Accordingly, the project was not guided and framed by an overarching theoretical framework but took inspiration and guidance from a variety of sources. I also consulted and drew on research and theories from other research fields, such as gerontology, gerontechnology, and music therapy, to inform the design process and to understand the potential of the interactive concept to promote health and wellbeing in senior citizens. This is presented in more detail in chapter 5.

In the following sections, I describe some of the central theoretical concepts and perspectives that have guided and informed this interaction design research project.

Each section gives a brief introduction to the theory in question, followed by a discussion of its relevance to the research project.

2.2.2 Concept-driven IxD research

Concept-driven interaction design research [97] is an approach to knowledge production in HCI that employs design to explore theoretical ideas and concepts. It is an explorative and future-oriented approach that attempts to manifest theoretical ideas in concrete designs to promote theoretical thinking and reflection and to probe, provoke, refine, and improve our understanding of the concepts in question. As a research approach, it aims to contribute to the intellectual and conceptual development of the field rather than to address specific contextual needs of users.

Accordingly, concept designs are based on and start from existing theories and concepts within the field rather than empirical data from a particular use-context.

Artifacts are developed and optimized to communicate the conceptual ideas they represent, requiring them to be strong conceptually, but also well-rounded designs that work as compositional wholes. It is both a design-centered and a theory-oriented process where "the design and realization of concrete and particular concept designs

(26)

developed interaction design theories" [97, p. 113]. Thus, the practice of doing design becomes an act of theorizing, where practice and theory are entirely inseparable from each other, and where the outcome is an artifact that materializes and operationalizes theoretical ideas. By changing people’s understanding of what is possible and desirable, good concept designs expand existing design spaces or open up entirely new ones. They are intermediate, mid-range theories "that are both intellectually interesting and applicable to interaction design" [97, p. 112].

Although the conceptual underpinnings of the RepMoves project were not explicitly and intentionally developed according to the principles of concept-driven interaction design, the project resembled and was inspired by this approach to theory development. In this project, I formulated a conceptual idea for how interactive technology could be used to create aesthetically engaging and meaningful interactive couplings between movement and music. The conceptual idea emerged from the recognition that repetitive movement and musical beat structures share fundamental, temporal characteristics that can be described and quantified in terms of tempo. This opens up the possibility of using tempo as an interactive link, or coupling, between movement tempo and music tempo. In other words, it makes it possible to create a digital music player that automatically and dynamically adjusts the tempo of the music in such a way that the beat synchronizes with the movement tempo.

Throughout the project, this concept has been vicariously named ‘rhythmic interaction (with music)’ and ‘the conductor concept.’ Rhythmic interaction has previously been used to describe various ways of using rhythm in the design of interactive interfaces, such as interactions with mobile devices [127]; playing a mobile tennis game [128]; selecting songs from a music library [129], and; an emerging principle in experiential design [130], among others. As such, the term lacks precision and does not directly communicate the idea explored in this project. ‘The conductor concept’ is a simple metaphor meant to illustrate the interactive relationship manifested in the conceptual idea: An orchestra conductor directs and controls the tempo of the music played by the orchestra through repetitive hand/arm gestures, but the conductor’s gestures are also reciprocally influenced by the rhythm of the music played by the musicians. There is an interactive body-music dialogue between the musicians and the conductor that establishes the tempo of the music. Throughout the rest of this thesis, and the included papers, rhythmic interaction and the conductor concept are used interchangeably to signify the basic idea of using tempo as an interactive coupling between movement and music. Finally, RepMoves is the name of the research prototype that was designed and refined throughout the project to explore the conceptual idea and the experience it provides.

The RepMoves project did not explore the conductor concept as an isolated theoretical construct but sought to understand how it could be used to inspire and promote increased physical activity in senior citizens by making the movement activity itself more aesthetically engaging and meaningful. This intention was inspired and

(27)

informed by a synthesis of different insights related to the subject of designing welfare technology for senior citizens, which can be summed up in the following line of reasoning:

1. Inactivity and sedentary lifestyles are important risk factors for a wide range of age-related health challenges, and even small increases in daily physical activity can have beneficial effects on health and wellbeing [16]–[19].

2. Senior citizens are, generally, aware of the importance of regular exercise and physical activity. However, many find it difficult to adhere to exercise programs over time [17], [19], [48].

3. Exercise is, by definition, repetitive. The monotony of aerobic exercise activities "is regarded as one of the key demotivating factors to exercise adherence" [90, p. 409].

4. Music is widely used to energize, motivate, and pace repetitive exercise activities, and people derive pleasure from moving in sync with the beat of the music [1], [2], [43], [131]. However, depending on musical preferences, the tempo of a particular piece of music may not necessarily fit with the preferred pace of the exercise activity.

5. The conductor concept could be used to dynamically adjust the tempo of the music to match the movement pace, thereby ensuring that the beat of the music serves to guide and support the user’s preferred movement pace.

6. Digital sound technology makes it possible to dynamically change the tempo of a piece of music to synchronize to an external tempo.

7. New sensor technologies make it possible to capture the tempo of a wide range of repetitive movement activities.

Thus, an initial design space is created where points:

• 1-3 outline a wicked problem regarding seniors’ motivation for exercise and physical activity;

• 4 and 5 presents a conceptual idea for how the problem can be addressed through interaction design, and;

• 6 and 7 presents technological opportunities that make it possible to realize the envisioned concept in a functional artifact.

This design space was explored in what can be described as a reflective conversation with materials, resulting in the construction and iterative refinement of the RepMoves prototype. Furthermore, the evolving prototype was exposed to, explored, and tested by senior citizens and other stakeholders in a variety of settings. The concept was not formulated with any specific use practice or exercise context in mind but in recognition of the potential applicability of the concept to a wide range of physical activity practices. Accordingly, the RepMoves project did not focus on designing a specific product to be used in a particular context, but on designing a research prototype that enabled experiential exploration of the ideas represented by the

(28)

open and basic implementation of the concept to an increasingly more defined and refined artifact that pointed towards a potential, future product.

2.2.3 Strong and generic concepts

Building on the concept-driven approach formulated by Stolterman and Wiberg [97], Höök and Löwgren [98] proposed strong concepts as a form of intermediate-level design knowledge that resides at an abstraction level ‘above’ specific design instances, but

‘below’ grand theories (Figure 2).

Figure 2. Höök and Löwgren’s [98, p. 2] visualization of strong concepts as an intermediate-level knowledge form existing on an abstraction level between specific designs and general theories.

[A strong concept] is generative and carries a core design idea, cutting across particular use situations and even application domains; concerned with interactive behavior, not static appearance; is a design element and a part of an artifact and, at the same time, speaks of a use practice and behavior over time. [98, p. 1]

Similarly, Wiberg and Stolterman [99] build on the concept-driven approach by introducing generic design thinking as a way to assess the knowledge contribution of research prototypes. Generic design thinking establishes an intermediate, conceptual connection between ultimate particulars [114] and more general ideas. "A generic design in HCI can be seen as a design concept that captures some essential qualities of a large number of particular designs, i.e., it defines a class or design space of interactive systems" [99, p. 535]. By grouping and comparing a large number of designs, the authors argue that it becomes possible to examine and assess the generic and invariant qualities of different classes of systems. Furthermore, it enables an assessment of the novelty and knowledge contribution of new designs, which can lead to the identification of new design spaces and new classes of systems.

(29)

The conductor concept was conceived of as a generic or ‘universal’ interactive coupling that could be used to augment a variety of repetitive movement activities with synchronized music. The process of building the initial proof-of-concept prototype expanded and refined my understanding of the generic nature of the concept. To me, the definition of a strong concept, as quoted above, resonates with my work and succinctly describes some of the central qualities of the conductor concept. It is generative and carries the core design idea of using interactive technology to synchronize musical beat to the tempo of human movement. Due to its generic nature, the concept has the potential to be used in different use contexts and application domains, and the work presented in this thesis explores RepMoves in a variety of contexts and application domains. The concept is not concerned with static appearance, but with specific interactive relations between movement and music. The concept is a design element and part of an artifact, and it addresses use practice over time in at least two different ways: First, the concept is fundamentally temporal as it is directly linked to the temporal advancement of an interactive experience. Second, the concept was explored as a potential way to promote increased physical activity in senior citizens over time.

Nevertheless, there are some differences between my approach to the ‘strong’ and

‘generic’ nature of the conductor concept and these authors’ notion of strong concepts and generic design thinking. Höök and Löwgren [98] argue that a strong concept needs to be horizontally and vertically grounded. Horizontal grounding implies comparing a particular concept design to other similar designs to examine and delineate the nature of the strong concept. This also resembles Wiberg and Stolterman’s [99] argument for grouping and comparing different classes of systems.

Vertical grounding demonstrates how the strong concept, as an intermediate level knowledge form, connects specific designs or ultimate particulars to more general, overarching and abstracted theories. To some extent, this thesis and the included papers compare and contrast the conductor concept and RepMoves prototype to other designs, as well as connect it to more philosophical and abstract theories.

However, my focus and interpretation of the conductor concept as ‘strong’ and

‘generic’ is, first and foremost, tied to how the interactive coupling can be applied and used in a variety of designs and use contexts.

2.3 Concepts and perspectives from phenomenology and post- phenomenology

2.3.1 Embodiment

The phenomenology of Maurice Merleau-Ponty is concerned with ideas of perception, embodiment, and having a world through our senses [55], [132]. He argues that, in order to know and think about the world, we first have to perceive it,

Exploring Movement-Based Rhythmic Interaction with Senior Citizens