Proceedings of the International Conference on New Inferfaces for Musical Expression

(1)

Proceedings of the International Conference on New Inferfaces for Musical Expression

NIME 2011

I M

E

2 0 1 1

O

S

L

O

(2)

(3)

Conference on New Interfaces for Musical Expression

30 May – 1 June 2011 Oslo, Norway

Edited by:

Alexander Refsum Jensenius Anders Tveit

Rolf Inge Godøy

Dan Overholt

(4)

All copyrights remain with the authors.

Copies may be ordered from:

Department of Musicology P.O. Box 1017 Blindern N-0315 Oslo, Norway Web sites

www.nime2011.org www.nime.org Cover design Thomas Kjellberg

ISSN 2220-4792 (Print) ISSN 2220-4806 (Online) ISSN 2220-4814 (USB)

(5)

(6)

(7)

On behalf of the University of Oslo, the Norwegian Academy of Music and our partners and sponsors, we are proud to present the 11th edition of NIME.

In its 11th year NIME has become an important conference series, the meeting point of researchers, de- velopers and artists from all over the world. Even though participants come from widely different back- grounds, they share a mutual interest in groundbreak- ing music and technology.

Since the start of the conference series, the word NIME has started to take on meaning in itself, inde- pendent of the annual conferences. Even outside the core group of annual conference participants, people start to know that NIME is somehow related to exciting musical and artistic research and practice. Still, though, NIME is mainly used as a noun, e.g. “bring your favourite NIMEs to the jam session tonight.” Perhaps it is now time to start using it also as a verb: to nime.

What’s in a word? We are often asked by people what NIME actually means. There is an official answer, but we rather like the idea that the four letter acronym can take on new meanings:

• N = New, Novel, …

• I = Interfaces, Instruments, …

• M = Musical, Multimedial, …

• E = Expression, Exploration, …

Whatever the meaning of the letters, the underlying idea is the hunt for new understanding, development and artistic exploration of devices in music. This type of exploration in (and on the borders between) science and art is not a problem for people in the NIME community. Outside the NIME community, however, our experience is that the worlds of science and art are often separate. We believe that the conference and the community can make a difference, and show that science and art need each other to prosper.

The NIME conference has over the last decade grown from a workshop at CHI in 2001, to have more than 500 submissions in 2011. This record number of submissions has made it possible to set up a large and varied program that we hope will be inspiring for eve- ryone being present. Despite the large submission number, we decided to keep NIME as an “intimate”

conference, a conference where it is possible to at- tend everything. We have stuck with the idea of single- track presentations, even though this means that the acceptance rate for oral presentation was as low as 15%. Keeping with the NIME spirit, though, we think that large poster and demo sessions are probably the best way of seeing, testing, and exploring various new instruments/interfaces in practice.

Programming concerts for a conference like NIME is challenging. Here we have tried to find a balance between novel instruments, performance maturity, and artistic expression. We were happy to see that many picked up on our challenge of submitting combined

“paper + performance” proposals. These submissions were treated as two separate submissions at first,

performance. Keeping up to international standards on both a scientific paper and a performance is not an easy task. But we see that many people in the community are up for it, and we highly encourage this type of double submissions also for future conferences.

There will be three keynote lectures, all of which will approach the topic of NIME from different angles.

Tellef Kvifte’s lecture will bring in historical and or- ganological perspectives through a discussion of digital instruments in the 19th century. David Rokeby will talk about his exploration of using the body in interactive art, something which is currently very popular in the NIME community. Sergi Jordà will talk about his instruments, and possibilities/challenges in working between science, art and industry. We hope these lectures will be inspiring and help to draw some longer lines in between the shorter and more fast-paced presentations at the conference.

We are also happy to present a series of pre-NIME events. As usual, there is a large set of tutorials and workshops held by local and international NIME participants. This year there is also a symposium called

“Technology and Aesthetics,” produced by NOTAM. The Art.on.Wires Media festival is a 5 day long feast of hacking, lectures and concerts, produced by the newly established Art.on.Wires socity. Finally, there is an exhibition on Sonic Interaction Design produced by COST Action IC0601 and BEK.

A number of organisations and individuals have contributed to making this conference a reality. There is only one thing to say: thank you!

All in all, we hope that NIME 2011 will represent another milestone in the development of the NIME community, and be of inspiration to all of you who partici- pate. Happy NIMEing.

Alexander Refsum Jensenius & Kjell Tore Innervik University of Oslo & Norwegian Academy of Music

(8)

(9)

Institutions University of Oslo

Department of Musicology Norwegian Academy of Music NOTAM

Norwegian Centre for Technology in Music and Arts BEK

Bergen Center for Electronic Arts NTNU

Norwegian University of Science and Technology Simula Research Laboratory Norwegian Museum of Science, Technology and Medicine COST Action IC0601

Chairs

Conference chairs:

Alexander Refsum Jensenius Kjell Tore Innervik

Paper chairs:

Alexander Refsum Jensenius Rolf Inge Godøy

Music chairs:

Kjell Tore Innervik Ivar Frounberg Demonstration chair:

Dan Overholt SID exhibition curators:

Trond Lossius Frauke Behrendt Symposium chairs:

Notto J. W. Thelle Jøran Rudi Rune Molvær Art.on.Wires chair:

Alexander Eichhorn

Alexander Eichhorn Ivar Frounberg Rolf Inge Godøy Trond Lossius Dan Overholt Jøran Rudi Jim Tørresen

Steering Committee Frédéric Bevilacqua Tina Blaine

Sidney Fels Michael Lyons Sile O’Modhrain Yoichi Nagashima Joe Paradiso Carol Parkinson Norbert Schnell Eric Singer Atau Tanaka

Local organization Arjun Chandra

Anette Pauline Forsbakk Kyrre Glette

Yngve Hafting Mats Høvin Thomas Kjellberg Cato Langnes Kristian Nymoen Otto Christian Pay Ståle A. Skogstad Renate Hauge Sund Siren Tjøtta Anders Tveit Knut Vik Arve Voldsund

Anne Cathrine Wesnes Ellen Wingerei

Alison Bullock Aarsten

Symposium Asbjørn Blokkum Flø Cato Langnes Rune Molvær Jøran Rudi Henrik Sundt Notto J. W. Thelle Hans Wilmers

Jason Geistweidt

SID exhibition Dag Andreassen Daniel Arfib

Maria Grazia Ballerano Frauke Behrendt Anne Marthe Dyvi Espen Egeland Elisabeth Gmeiner Thomas Hermann Trond Lossius Monique Mossefinn Alessandra Paccamiccio Inge de Prins

Matteo Razzanelli Davide Rocchesso Aranzazu Sanchez Henning Sandsdalen Lars Ove Toft Marieke Verbiesen Frode Weium

All names are in alphabetical order.

(10)

(11)

Sarah Fdili Alaoui Jesse Allison Anders Andersson Frauke Behrendt Ross Bencina Edgar Berdahl Andreas Bergsland Eirik Birkeland Tina Blaine Ben Bogart Sinan Bokesoy Bert Bongers Brennon Bortz Mathieu Bosi Nicolas Bouillot Øyvind Brandtsegg Roberto Bresin Nick Bryan-Kinns Gaspard Bucher Eivind Buene Ivica Bukvic Jamie Bullock Sinan Bökesoy Niels Böttcher Baptiste Caramiaux Alvaro Cassinelli Parag Chordia Mats Claesson Michael Cohen Graham Coleman Nick Collins Langdon Crawford Alain Crevoisier Nicolas D’Alessandro Palle Dahlstedt Roger Dannenberg Scott Deal

Smilen Dimitrov Paul Doornbusch Luke Dubois Alexander Eichhorn Trond Engum Georg Essl Sidney Fels Robin Fencott Rebecca Fiebrink Wolfgang Fohl Federico Fontana Angelo Fraietta Alexandre Francois Adrian Freed Jason Freeman Anders Friberg Henrik Frisk Ivar Frounberg Ichiro Fujinaga Andrew Cavan Fyans

Rolf Inge Godøy Lars Graugaard Tobias Grosshauser Sylvain le Groux Carlos Guedes Michael Gurevich Bjørnar Habbestad Aristotelis Hadjakos Morten Halle Tor Halmrast Keith Hamel

Kjetil Falkenberg Hansen Mark Havryliv

Andrew Hawryshkewich Tomás Henriques Saburo Hirano Hannes Hoelzl Risto Holopainen Mark David Hosale Bill Hsu

Mats Høvin Kjell Tore Innervik Javier Jaimovich Jordi Janer

Alexander Refsum Jensenius Andrew Johnston

Sergi Jorda Haruhiro Katayose Peter Kirn

Benjamin Knapp Juraj Kojs Mariusz Kozak Tellef Kvifte Johnathan F. Lee Paul Lehrman George E. Lewis Takuro Mizuta Lippit Trond Lossius Michael Lyons John Maccallum Matthieu Macret Thor Magnusson Joseph Malloch

Adnan Marquez-Borbon Mark Marshall

Kjetil Svalastog Matheussen James Maxwell

Eduardo Miranda Thomas B. Moeslund Katherine Moriwaki Florian Floyd Mueller Yoichi Nagashima Luiz Naveda Kia Ng

Per Anders Nilsson Kristian Nymoen

Jyri Pakarinen Brett Park Philippe Pasquier Jean-Marc Pelletier Nils Peters

Toiviainen Petri Timothy Place Patrick Pogscheba Anthony Rowe Robert Rowe Jøran Rudi Even Ruud Joel Ryan Jan Schacher Margaret Schedel Norbert Schnell Erwin Schoonderwaldt Diemo Schwarz Richard Scott Stefania Serafin Greg Shear Stephen Sinclair Ståle A. Skogstad Scott Smallwood Stefan Smulovitz Hugo Solis Jorge Solis Andrew Sorensen Hans Tammen Peter Tornquist Giuseppe Torre Dan Trueman George Tzanetakis Jim Tørresen Owen Vallis Giovanna Varni Bill Verplank Anders Vinjar Gualtiero Volpe Carl Haakon Waadeland Marcelo M. Wanderley Ge Wang

Rob Waring Hans Wilmers Lonce Wyse Björn Wöldecke Anna Xambo Matthew Yee-King Tomoko Yonezawa Takegawa Yoshinari Mark Zadel

Michael Zbyszynski Tone Åse

(12)

(13)

The NIME call for participation was published 1 Sep- tember 2010, and is republished below. A total of 502 submissions were made in all categories of the call.

All submissions in the paper and performance tracks were subject to a single-blind peer review process by a group of international experts. Submissions in the performance + paper track were at first evaluated as two separate submissions, and re-evaluated as a combined submission in the final selection process.

Submissions for the SID exhibition, installations and tutorials were selected by groups of curators.

Paper track

Of 204 submissions in the different paper categories, the following have been selected:

• Oral presentation: 33

• Poster presentation: 80

• Demonstrations: 16

Other tracks

The following numbers of submissions have been selected from the other tracks (submission number in parentheses):

• Concerts: 35 (134)

• Installations 3 (33)

• Tutorials 19 (26)

• SID exhibition: 12 (102)

Call for participation

We invite you to be part of the International Confer- ence on New Interfaces for Musical Expression. The core purpose of the NIME conference is to present the latest results in design, development, performance and analysis of/for/with new interfaces and instruments for musical use. In 2011 we will put an extra emphasis on performance aspects related to NIME, something which will also be addressed in a symposium, workshops and master classes in the days leading up to the conference.

We invite for the following types of submissions (see below for details):

• Paper (oral/poster/demo)

• Performance

• Performance Plus Paper

• Exhibition works

• Installation

• Workshop

• Paper/performance/installation/workshop submission: 31 January 2011 (22:00 CET)

• Review notification: 18 March 2011

• Final paper deadline: 26 April 2011

For any further information/questions/comments/

suggestions, please contact the organizing committee.

Topics

• Novel controllers and interfaces for musical ex-

pression

• Novel musical instruments

• Augmented/hyper instruments

• Novel controllers for collaborative performance

• Interfaces for dance and physical expression

• Interactive game music

• Robotic music

• Interactive sound and multimedia installations

• Interactive sonification

• Sensor and actuator technologies

• Haptic and force feedback devices

• Interface protocols and data formats

• Motion, gesture and music

• Perceptual and cognitive issues

• Interactivity design and software tools

• Sonic interaction design

• NIME intersecting with game design

• Musical mapping strategies

• Performance analysis

• Performance rendering and generative algorithms

• Machine learning in performance systems

• Experiences with novel interfaces in live performance and composition

• Surveys of past work and stimulating ideas for future research

• Historical studies in twentieth-century instrument design

• Experiences with novel interfaces in education and entertainment

• Reports on student projects in the framework of NIME related courses

• Artistic, cultural, and social impact of NIME technology

• Biological and bio-inspired systems

• Mobile music technologies

• Musical human-computer interaction

• Multimodal expressive interfaces

• Practice-based research approaches/

methodologies/criticism

Call for papers

We welcome submissions of original research on all above mentioned (and other) topics related to development and artistic use of new interfaces for musical expression. There are three different paper submission categories:

• Full paper (up to 6 pages in proceedings, longer oral presentation, optional demo)

(14)

Please use this template when preparing your manu- script. Submitted papers will be subject to a single- blind peer review process by an international expert committee. All accepted papers will be published in the conference proceedings, under an ISSN/ISBN reference, and will be available online after the conference.

Call for performances

We welcome submission of pieces for three different types of performance venues:

• Concert hall performance

• Club performance

• Foyer “stunt” performance

Any type of NIME performance pieces are welcome, but we would particularly like to encourage the use of motion capture techniques in performance. For this we can make available several different types of motion capture systems (Qualisys, XSens, Optitrack, Mega).

Network pieces and mobile music pieces are also en- couraged. Within reasonable limits, we may be able to provide musicians to perform pieces. Typical NIME performance pieces last for 5-15 minutes, but shorter and longer performance proposals may also be taken into consideration.

Submitted proposals will be subject to a peer review process by an international expert committee.

Documentation of the performances will be available online after the conference.

Call for performance plus paper

To support more cross-disciplinary work between scientific and artistic research, we highly encourage submission of performance pieces related to papers. Here the scientific presentation may be the basis for the artistic presentation, or vice versa.

Submissions within this category will have to be done for both the piece and the paper, with a clear note that paper and piece belongs together. Evaluation will be done on the combined quality of both submissions.

Call for exhibition works

In connection with NIME 2011 an exhibition on sonic interaction design will be curated in collaboration with the EU COST IC0601 Action on Sonic Interaction Design

(SID). For the exhibition we are looking for works using sonic interaction within arts, music and design as well as examples of sonification for research and artistic purposes. The exhibition will take place at the Norwe- gian Museum of Science, Technology and Medicine and run for three months over the summer 2011. We also aim to include works in public spaces to be presented at various locations in Oslo (possibly) for a shorter du- ration in parallel with NIME.

submissions within this category (5 November). More information about the exhibition, including pictures of the venue, can be found at the SID web site. Any further enquiries concerning the exhibition should be addressed to the curators: [email protected].

Call for installations

In addition to the SID exhibition, we also call for installations to be presented during the NIME conference only. These may be foyer location installations or room-based installations in connection to the conference venues.

Submitted proposals will be subject to a peer review process by an international expert committee. Docu- mentation of the installations will be available online after the conference.

Call for tutorials and workshops

We call for short (3 hours) or long (6 hours) workshops and tutorials. These can be targeted towards specialist techniques, platforms, hardware, software or peda- gogical topics for the advancement of fellow NIME-ers and people with experience related to the topic. They can also be targeted towards visitors to the NIME community, novices/newbies, interested student participants, people from other fields, and members of the public getting to know the potential of NIME.

Tutorial proposers should clearly indicate the audience and assumed knowledge of their intended participants to help us market to the appropriate audience. Workshops and tutorials can relate to, but are not limited to, the topics of the conference. This is a good opportunity to explore a specialised interest or interdisciplinary topic in depth with greater time for discourse, debate, collaboration.

Admission to workshops and tutorials will be charged separately from the main conference. Propos- er(s) are responsible for publishing any workshop proceedings (if desired) and should engage in the promo- tion of their event amongst own networks. Workshops may be cancelled or combined if there is inadequate participation.

(15)

NIME 2010: University of Technology Sydney, Sydney, Australia NIME 2009: Carnegie Mellon University, Pittsburgh, USA NIME 2008: Casa Paganini, Genoa, Italy

NIME 2007: New York University, New York, USA NIME 2006: IRCAM Centre Pompidou, Paris, France

NIME 2005: University of British Columbia, Vancouver, Canada NIME 2004: Shizuoka University of Art and Culture, Hamamatsu, Japan NIME 2003: McGill University, Montreal, Canada

NIME 2002: Media Lab Europe, Dublin, Ireland NIME 2001: CHI 2001, Seattle, USA

(16)

(17)

All lectures and oral presentations are in Auditorium 1 at the University Library (Georg Sverdrups hus) at the University of Oslo. The poster and demo sessions are in the seminar rooms on the 3rd floor in the same building. Please consult the program book for additional information.

Keynote lectures

Keynote Lecture 1: Musical Instrument User Interfaces: the Digital Background of the Analog Revolution. . . 1 Tellef Kvifte

Keynote Lecture 2: Adventures in Phy-gital Space . . . 2 David Rokeby

Keynote Lecture 3: Digital Lutherie and Multithreaded Musical Performance: Artistic, Scientific and Commercial Perspectives . . . 3 Sergi Jordà

Paper session A — Monday 30 May 11:00–12:30

The Overtone Fiddle: an Actuated Acoustic Instrument . . . 4 Dan Overholt

A Low-Cost, Low-Latency Multi-Touch Table with Haptic Feedback for Musical Applications . . . 8 Colby Leider, Matthew Montag, Stefan Sullivan and Scott Dickey

The Electromagnetically Sustained Rhodes Piano . . . 14 Greg Shear and Matthew Wright

Gamelan Elektrika: An Electronic Balinese Gamelan . . . 18 Laurel Pardue, Christine Southworth, Andrew Boch, Matt Boch and Alex Rigopulos

Sonicstrument: A Musical Interface with Stereotypical Acoustic Transducers . . . 24 Jeong-Seob Lee and Woon Seung Yeo

Poster session B — Monday 30 May 13:30–14:30

Solar Sound Arts: Creating Instruments and Devices Powered by Photovoltaic Technologies . . . 28 Scott Smallwood

An Approach to Collaborative Music Composition . . . 32 Niklas Klügel, Marc René Frieß and Georg Groh

A Reference Architecture and Score Representation for Popular Music Human-Computer Music Performance Systems . . . 36 Nicolas Gold and Roger Dannenberg

V’OCT (Ritual): An Interactive Vocal Work for Bodycoder System and 8 Channel Spatialization . . . 40 Mark Bokowiec

First Person Shooters as Collaborative Multiprocess Instruments . . . 44 Florent Berthaut, Haruhiro Katayose, Hironori Wakama, Naoyuki Totani and Yuichi Sato

Studying Interdependencies in Music Performance: An Interactive Tool . . . 48 Tilo Hähnel and Axel Berndt

1city 1001vibrations: development of a interactive sound installation with robotic instrument performance . . . 52 Sinan Bokesoy and Patrick Adler

The medium is the message: Composing instruments and performing mappings . . . 56 Tim Murray-Browne, Di Mainstone, Nick Bryan-Kinns and Mark D. Plumbley

Clothesline as a Metaphor for a Musical Interface . . . 60 Seunghun Kim, Luke Keunhyung Kim, Songhee Jeong and Woon Seung Yeo

EGGS in action . . . 64 Pietro Polotti and Maurizio Goina

A Reverberation Instrument Based on Perceptual Mapping . . . 68 Berit Janssen

Vibrotactile Feedback-Assisted Performance. . . 72 Lauren Hayes

(18)

Ryan McGee, Yuan-Yi Fan and Reza Ali

Vibration, Volts and Sonic Art: A practice and theory of electromechanical sound . . . 84 Jon Pigott

Automatic Rhythmic Performance in Max/MSP: the kin.rhythmicator . . . 88 George Sioros and Carlos Guedes

Towards a Voltage-Controlled Computer — Control and Interaction Beyond an Embedded System . . . 92 Andre Goncalves

Polyhymnia: An automatic piano performance system with statistical modeling of polyphonic expression and musical symbol

interpretation . . . 96 Tae Hun Kim, Satoru Fukayama, Takuya Nishimoto and Shigeki Sagayama

Multitouch Interface for Audio Mixing . . . 100 Juan Pablo Carrascal and Sergi Jorda

Cognitive Architecture in Mobile Music Interactions . . . 104 Nate Derbinsky and Georg Essl

The Self-Supervising Machine . . . 108 Benjamin D. Smith and Guy E. Garnett

Beatscape, a mixed virtual-physical environment for musical ensembles . . . 112 Aaron Albin, Sertan Senturk, Akito Van Troyer, Brian Blosser, Oliver Jan and Gil Weinberg

MoodifierLive: Interactive and collaborative expressive music performance on mobile devices . . . 116 Marco Fabiani, Gaël Dubus and Roberto Bresin

A Physically Based Sound Space for Procedural Agents . . . 120 Benjamin Schroeder, Marc Ainger and Richard Parent

Acquisition and study of blowing pressure profiles in recorder playing . . . 124 Francisco Garcia, Leny Vinceslas, Esteban Maestre and Josep Tubau

Experiences from video-controlled sound installations . . . 128 Anders Friberg and Anna Källblad

ROOM#81 — Agent-Based Instrument for Experiencing Architectural and Vocal Cues . . . 132 Nicolas d’Alessandro, Roberto Calderon and Stefanie Müller

Demo session C — Monday 30 May 13:30–14:30

Kinetic Particles Synthesizer Using Multi-Touch Screen Interface of Mobile Devices . . . 136 Yasuo Kuhara and Daiki Kobayashi

The Sound Flinger: A Haptic Spatializer . . . 138 Christopher Carlson, Eli Marschner and Hunter Mccurry

Daft Datum – an Interface for Producing Music Through Foot-Based Interaction . . . 140 Ravi Kondapalli and Benzhen Sung

Strike on Stage: a percussion and media performance . . . 142 Charles Martin and Chi-Hsia Lai

Paper session D — Monday 30 May 14:30–15:30

Gestural Embodiment of Environmental Sounds: an Experimental Study . . . 144 Baptiste Caramiaux, Patrick Susini, Tommaso Bianco, Frédéric Bevilacqua, Olivier Houix, Norbert Schnell and Nicolas Misdariis Listening to Your Brain: Implicit Interaction in Collaborative Music Performances . . . 149

Sebastian Mealla, Aleksander Valjamae, Mathieu Bosi and Sergi Jorda

Examining How Musicians Create Augmented Musical Instruments . . . 155 Dan Newton and Mark Marshall

Paper session E — Monday 30 May 16:00–17:00

Tahakum: A Multi-Purpose Audio Control Framework . . . 161 Zachary Seldess and Toshiro Yamada

(19)

Edgar Berdahl and Wendy Ju

Paper session F — Tuesday 31 May 09:00–10:50

Two Turntables and a Mobile Phone . . . 179 Nicholas J. Bryan and Ge Wang

MadPad: A Crowdsourcing System for Audiovisual Sampling . . . 185 Nick Kruge and Ge Wang

The Visual in Mobile Music Performance . . . 191 Patrick O’Keefe and Georg Essl

Designing for the iPad: Magic Fiddle . . . 197 Ge Wang, Jieun Oh and Tom Lieber

MobileMuse: Integral Music Control Goes Mobile . . . 203 Benjamin Knapp and Brennon Bortz

Tangible Performance Management of Grid-based Laptop Orchestras . . . 207 Stephen Beck, Chris Branton, Sharath Maddineni, Brygg Ullmer and Shantenu Jha

Poster session G — Tuesday 31 May 13:30–14:30

Audio Arduino — an ALSA (Advanced Linux Sound Architecture) audio driver for FTDI-based Arduinos . . . 211 Smilen Dimitrov and Stefania Serafin

Musical control of a pipe based on acoustic resonance . . . 217 Seunghun Kim and Woon Seung Yeo

Play Fluency in Music Improvisation Games for Novices . . . 220 Anne-Marie Hansen, Hans Jørgen Andersen and Pirkko Raudaskoski

The Bass Sleeve: A Real-time Multimedia Gestural Controller for Augmented Electric Bass Performance . . . 224 Izzi Ramkissoon

The KarmetiK NotomotoN: A New Breed of Musical Robot for Teaching and Performance . . . 228 Ajay Kapur, Michael Darling, James Murphy, Jordan Hochenbaum, Dimitri Diakopoulos and Trimpin

The Manipuller: Strings Manipulation and Multi-Dimensional Force Sensing . . . 232 Adrian Barenca Aliaga and Giuseppe Torre

Mapping Objects with the Surface Editor . . . 236 Alain Crevoisier and Cécile Picard-Limpens

Adding Z-Depth and Pressure Expressivity to Tangible Tabletop Surfaces . . . 240 Jordan Hochenbaum and Ajay Kapur

Hex Player—A Virtual Musical Controller . . . 244 Andrew Milne, Anna Xambó, Robin Laney, David B. Sharp, Anthony Prechtl and Simon Holland

Rhythm Performance from a Spectral Point of View . . . 248 Carl Haakon Waadeland

Nuvolet : 3D gesture-driven collaborative audio mosaicing . . . 252 Josep M Comajuncosas, Enric Guaus, Alex Barrachina and John O’Connell

Effective and expressive movements in a French-Canadian fiddler’s performance . . . 256 Erwin Schoonderwaldt and Alexander Refsum Jensenius

Flowspace – A Hybrid Ecosystem . . . 260 Daniel Bisig, Jan Schacher and Martin Neukom

Implementing a Finite Difference-Based Real-time Sound Synthesizer using GPUs . . . 264 Marc Sosnick and William Hsu

An Artificial Intelligence Architecture for Musical Expressiveness that Learns by Imitation . . . 268 Axel Tidemann

TweetDreams: Making music with the audience and the world using real-time Twitter data . . . 272 Luke Dahl, Jorge Herrera and Carr Wilkerson

(20)

Andrew Johnston

Intuitive Real-Time Control of Spectral Model Synthesis. . . 284 Phillip Popp and Matthew Wright

BeatJockey: A new tool for enhancing DJ skills . . . 288 Pablo Molina, Martin Haro and Sergi Jordà

Traces – Body, Motion and Sound . . . 292 Jan Schacher and Angela Stoecklin

MoodMixer: EEG-based Collaborative Sonification . . . 296 Grace Leslie and Tim Mullen

OSC Implementation and Evaluation of the Xsens MVN suit . . . 300 Ståle A. Skogstad, Kristian Nymoen, Yago de Quay and Alexander Refsum Jensenius

The effect of visualizing audio targets in a musical listening and performance task . . . 304 Lonce Wyse, Norikazu Mitani and Suranga Nanayakkara

Composability for Musical Gesture Signal Processing using new OSC-based Object and Functional Programming Extensions to

Max/MSP . . . 308 Freed Adrian, John Maccallum and Andrew Schmeder

SoundSaber — A Motion Capture Instrument . . . 312 Kristian Nymoen, Ståle A. Skogstad and Alexander Refsum Jensenius

A modulation matrix for complex parameter sets . . . 316 Øyvind Brandtsegg, Sigurd Saue and Thom Johansen

Demo session H — Tuesday 31 May 13:30–14:30

Sound Low Fun. . . 320 Yu-Chung Tseng, Che-Wei Liu, Tzu-Heng Chi and Hui-Yu Wang

Autonomous New Media Artefacts (AutoNMA) . . . 322 Edgar Berdahl and Chris Chafe

Creating Musical Expression using Kinect . . . 324 Min-Joon Yoo, Jin-Wook Beak and In-Kwon Lee

Making grains tangible: microtouch for microsound . . . 326 Staas de Jong

Sound Selection by Gestures . . . 329 Baptiste Caramiaux, Frederic Bevilacqua and Norbert Schnell

Paper session I — Tuesday 31 May 14:30–15:30

An Open Source Interface based on Biological Neural Networks for Interactive Music Performance . . . 331 Hernán KerlleÃevich, Manuel Eguia and Pablo Riera

Recognition Of Multivariate Temporal Musical Gestures Using N-Dimensional Dynamic Time Warping . . . 337 Nicholas Gillian, R. Benjamin Knapp and Sile O’Modhrain

A Machine Learning Toolbox For Musician Computer Interaction . . . 343 Nicholas Gillian, R. Benjamin Knapp and Sile O’Modhrain

Paper session J — Tuesday 31 May 16:00–17:00

Music and Technology in Death and the Powers . . . 349 Elena Jessop, Peter Torpey and Benjamin Bloomberg

Design and Evaluation of a Hybrid Reality Performance . . . 355 Victor Zappi, Dario Mazzanti, Andrea Brogni and Darwin Caldwell

InkSplorer : Exploring Musical Ideas on Paper and Computer . . . 361 Jérémie Garcia, Theophanis Tsandilas, Carlos Agon and Wendy Mackay

(21)

Pedro Lopes, Alfredo Ferreira and Joao Madeiras Pereira

Designing Digital Musical Interactions in Experimental Contexts . . . 373 Adnan Marquez-Borbon, Michael Gurevich, A. Cavan Fyans and Paul Stapleton

Crackle: A mobile multitouch topology for exploratory sound interaction . . . 377 Jonathan Reus

A principled approach to developing new languages for live coding . . . 381 Samuel Aaron, Alan F. Blackwell, Richard Hoadley and Tim Regan

Integra Live: a new graphical user interface for live electronic music . . . 387 Jamie Bullock, Daniel Beattie and Jerome Turner

Paper session L — Wednesday 1 June 11:00–12:30

Robust and Reliable Fabric, Piezoresistive Multitouch Sensing Surfaces for Musical Controllers . . . 393 Jung-Sim Roh, Yotam Mann, Adrian Freed and David Wessel

Examining the Effects of Embedded Vibrotactile Feedback on the Feel of a Digital Musical Instrument . . . 399 Mark Marshall and Marcelo Wanderley

HIDUINO: A firmware for building driverless USB-MIDI devices using the Arduino microcontroller . . . 405 Dimitri Diakopoulos and Ajay Kapur

Latency improvement in sensor wireless transmission using IEEE 802.15.4 . . . 409 Emmanuel Flety and Côme Maestracci

The Snyderphonics Manta, a Novel USB Touch Controller . . . 413 Jeff Snyder

Poster session M — Wednesday 1 June 13:30–14:30

On Movement, Structure and Abstraction in Generative Audiovisual Improvisation . . . 417 William Hsu

Creating Interactive Multimedia Works with Bio-data . . . 421 Claudia Robles Angel

TresnaNet: musical generation based on network protocols. . . 425 Paula Ustarroz

Designing a Music Performance Space for Persons with Intellectual Learning Disabilities . . . 429 Matti Luhtala, Tiina Kymäläinen and Johan Plomp

Raja — A Multidisciplinary Artistic Performance . . . 433 Tom Ahola, Teemu Ahmaniemi, Koray Tahiroglu, Fabio Belloni and Ville Ranki

Eobody3: A ready-to-use pre-mapped & multi-protocol sensor interface . . . 437 Emmanuelle Gallin and Marc Sirguy

Eye Tapping: How to Beat Out an Accurate Rhythm using Eye Movements . . . 441 Rasmus Bååth, Thomas Strandberg and Christian Balkenius

MelodyMorph: A Reconfigurable Musical Instrument . . . 445 Eric Rosenbaum

Flo)(ps: Between Habitual and Explorative Action-Sound Relationships . . . 448 Karmen Franinovic

Wekinating 000000Swan: Using Machine Learning to Create and Control Complex Artistic Systems . . . 453 Margaret Schedel, Rebecca Fiebrink and Phoenix Perry

MTCF: A framework for designing and coding musical tabletop applications directly in Pure Data . . . 457 Carles F. Julià, Daniel Gallardo and Sergi Jordà

Physical modelling enabling enaction: an example . . . 461 David Pirrò and Gerhard Eckel

SoundGrasp: A Gestural Interface for the Performance of Live Music . . . 465 Thomas Mitchell and Imogen Heap

Minding the (Transatlantic) Gap: An Internet-Enabled Acoustic Brain-Computer Music Interface . . . 469 Tim Mullen, Richard Warp and Adam Jansch

(22)

Cumhur Erkut, Antti Jylhä and Reha Di¸sçio˘glu

A Hair Ribbon Deflection Model for Low-Intrusiveness Measurement of Bow Force in Violin Performance . . . 481 Marco Marchini, Panos Papiotis, Alfonso Perez and Esteban Maestre

Random Access Remixing on the iPad . . . 487 Jonathan Forsyth, Aron Glennon and Juan Bello

Designing the EP trio: Instrument identities, control and performance practice in an electronic chamber music ensemble . . . 491 Erika Donald, Ben Duinker and Eliot Britton

Perceptions of Skill in Performances with Acoustic and Electronic Instruments . . . 495 Cavan Fyans and Michael Gurevich

Cognitive Issues in Computer Music Programming . . . 499 Hiroki Nishino

Seaboard: a new piano keyboard-related interface combining discrete and continuous control . . . 503 Roland Lamb and Andrew Robertson

Music Interfaces for Novice Users: Composing Music on a Public Display with Hand Gestures . . . 507 Gilbert Beyer and Max Meier

Expanding the role of the instrument . . . 511 Birgitta Cappelen and Anders-Petter Andersson

Wireless Digital/Analog Sensors for Music and Dance Performances . . . 515 Todor Todoroff

Real-time control and creative convolution — exchanging techniques between distinct genres . . . 519 Trond Engum

The Six Fantasies Machine – an instrument modelling phrases from Paul Lansky’s Six Fantasies . . . 523 Andreas Bergsland

Demo session N — Wednesday 1 June 13:30–14:30

Gliss: An Intuitive Sequencer for the iPhone and iPad . . . 527 Jan Trützschler von Falkenstein

Quadrofeelia — A New Instrument for Sliding into Notes . . . 529 Jiffer Harriman, Locky Casey, Linden Melvin and Mike Repper

SQUEEZY: Extending a Multi-touch Screen with Force Sensing Objects for Controlling Articulatory Synthesis . . . 531 Johnty Wang, Nicolas D’Alessandro, Sidney Fels and Bob Pritchard

SWAF: Towards a Web Application Framework for Composition and Documentation of Soundscape . . . 533 Souhwan Choe and Kyogu Lee

Playing the "MO" — Gestural Control and Re-Embodiment of Recorded Sound and Music . . . 535 Norbert Schnell, Frederic Bevilacqua, Nicolas Rasamimana, Julien Blois, Fabrice Guedy and Emmanuel Flety

(LAND)MOVES . . . 537 Bruno Zamborlin, Marco Liuni and Giorgio Partesana

Can Haptics make New Music? — Fader and Plank Demos . . . 539 Bill Verplank and Francesco Georg

(23)

(24)

(25)

Keynote Lecture 1: Musical Instrument User Interfaces:

the Digital Background of the Analog Revolution

Tellef Kvifte

University of Oslo Department of Musicology

[email protected]

ABSTRACT

In this keynote lecture, examples from the development of new user interfaces on free reed instruments and woodwinds in the 19th century are used as a starting point for dis- cussing user interfaces as part of a wider technological, aesthetic and cultural context. The concepts of analog/digital are used to characterize not only the underlying technology, but also aspects of musical parameters and user interfaces, like “discrete” (scale steps; keys) and “continuously variable”

(glissandi/vibrato; slides/sliders).

The free reed instruments – the most common of these being the harmonica, accordion and harmonium – were developed with a large number of different user interfaces.

Many of these are now obsolete, but many are still surviv- ing. In my lecture I will argue that the control of digital pitch-classes (scale steps) is a central focus in many of these instruments, and also in other instruments developed in this time period and onwards.

It is further argued that there has been a development from a digital pitch-class-oriented culture to a preoccupa- tion with control of analog musical qualities – especially timbre – in the last part of the 20th century. This has been in parallell to changes in the dominating media and technologies for production and distribution of music, and, obviously, in instrument and user interface design.

Thus, musical instrument interfaces, aesthetic preferences, and dominating production technologies can be seen as a system of mutually dependent elements in this development from digital to analog.

Biography

Tellef Kvifte is full professor at Department of Musicology at the University of Oslo. His research interest spans from Norwegian hardanger fiddle music through theory of rhythm to theoretical organology and music technology, and has published internationally in all these areas. His most recent research concerns perspectives on the co-development of music, music tehcnology, notation and concepts of sounds.

Kvifte occasionally also appears on the professional World music scene as a musician, and performs tin whistles, hardanger fiddles, saxophones, laptops and a variety of other instruments with confidence. He worked professionally as a television producer before taking up his academic career, and is still a noted record producer.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

NIME’11,30 May–1 June 2011, Oslo, Norway.

Copyright remains with the author(s).

Figure 1: Tellef Kvifte (Photo: Tom Hatlestad)

(26)

Keynote Lecture 2: Adventures in Phy-gital Space

David Rokeby

http://www.davidrokeby.com [email protected]

ABSTRACT

David Rokeby spent the 10 years from 1981 to 1991 ges- turing in mid-air and throwing his body against the virtual while creatingVery Nervous System, an interactive installation which tracks body movement with video cameras and turns the movement into music and/or sound. Develop- ing this work and exhibiting it around the world gave him a wealth of opportunities to experience and observe what happens when we place our bodies at the conjunction of physical and digital spaces. Since the early nineties, he has often returned to this exploration of “phy-gital” experience in a range of video and sound installations, considering this hybrid space as one of the fundamental features of life in a digital culture.

In his presentation, Rokeby will explore characteristics of the experience of phy-gital space, reflecting in particular on how these features affect interactive performance and interactive performers. Then he will present a variety of projects which expand the notion of interactive performance into publicly accessible interactive installations.

Main thrusts of this examination will include the effect of phy-gital space on the interactor’s mind and body, virtuos- ity in the context of interactive interfaces, and the interface as audience.

Biography

David Rokeby’s early work Very Nervous System (1982- 1991) was a pioneering work of interactive art, translating physical gestures into real-time interactive sound environ- ments. It was presented at the Venice Biennale in 1986, and was awarded a Prix Ars Electronica Award of Distinction in 1991. Several of his works have addressed issues of digital surveillance, includingTaken (2002), andSorting Daemon (2003). Other works engage in a critical examination of the differences between human and artificial intelligence. The Giver of Names(1991-) andn-cha(n)t (2001) are artificial subjective entities, provoked by objects or spoken words in their immediate environment to formulate sentences and speak them aloud. David Rokeby has exhibited and lectured extensively in the Americas, Europe and Asia. His awards include a Governor General’s Award in Visual and Media Arts (2002), a Prix Ars Electronica Golden Nica for Interactive Art (2002), and a British Academy of Film and Television Arts “BAFTA” award in Interactive art (2000).

Figure 1: David Rokeby

(27)

Keynote Lecture 3: Digital Lutherie and Multithreaded Musical Performance: Artistic, Scientific and Commercial

Perspectives

Sergi Jordà

Universitat Pompeu Fabra Music Technology Group

[email protected]

ABSTRACT

In 1982, I was studying for a B.Sc. in fundamental physics, when I first ever saw a computer and soon discovered the magic of computer programming. It was such a revelation that some weeks later I had decided to give up saxophone practice and free jazz in order to become a computer music improviser! Since then, I have pursued the complexity, del- icacy and futility of real-time, multidimensional performer–

instrument–interaction from different and complementary perspectives.

Initially I did this from a freer and purely aesthetically driven artistic/performer and freelance perspective, then trying to systematize and expand this empirical knowledge from a more scientific/academic point of view as a researcher at the Music Technology Group in Barcelona (1999–present), and more recently, also from an industrial/commercial perspective, manufacturing and selling new electronic musical instruments at Reactable Systems¹ (2009–present).

Art, research and business seem three quite distinct activities, and yet I do not really experience it that way, perhaps because as I understand it,digital lutheriecannot work properly without any of these three legs. It has to inevitably start from music, from musical needs and realities, without walking blind or reinventing the wheel at every new step, and at last, without forgetting the potential user.

In this keynote lecture, I will give an overview of my jour- ney from these three angles, with a special focus on what I call “multithreaded musical instruments,” the term that could define my main activities and research area for the last 15 years.

Biography

Sergi Jord`a holds a B.S. in Fundamental Physics and a Ph.D. in Computer Science and Digital Communication. He is a researcher in the Music Technology Group of Universitat Pompeu Fabra in Barcelona, and a lecturer in the same university, where he teaches computer music, HCI, and interactive media arts. He has written many articles, books, given workshops and lectured though Europe, Asia and America, always trying to bridge HCI, music performance and interactive media arts. He has received several international

1http://www.reactable.com

awards, including the prestigious Ars Electronica’s Golden Nica in 2008. He is currently best known as one of the inven- tors of the Reactable, a tabletop musical instrument that in 2007 accomplished mass popularity after being integrated in Icelandic artist Bj¨ork’s Volta world Tour.

Figure 1: Sergi Jord`a

(28)

The Overtone Fiddle: an Actuated Acoustic Instrument

Dan Overholt

Department of Architecture, Design and Media Technology Aalborg University, Denmark

Niels Jernes Vej 14, 3-107 [email protected]

ABSTRACT

The Overtone Fiddle is a new violin-family instrument that incorporates electronic sensors, integrated DSP, and physical actuation of the acoustic body. An embedded tactile sound transducer creates extra vibrations in the body of the Overtone Fiddle, allowing performer control and sensation via both traditional violin techniques, as well as extended playing techniques that incorporate shared man/machine control of the resulting sound. A magnetic pickup system is mounted to the end of the fiddle’s fingerboard in order to detect the signals from the vibrating strings, deliberately not capturing vibrations from the full body of the instrument. This focused sensing approach allows less restrained use of DSP-generated feedback signals, as there is very little direct leakage from the actuator embedded in the body of the instrument back to the pickup.

Keywords

Actuated Musical Instruments, Hybrid Instruments, Active Acoustics, Electronic Violin

1. INTRODUCTION

The Overtone Fiddle follows upon the development of the author’s prior Overtone Violin [6], with a change of focus towards another area of investigation. Whereas the Overtone Violin is entirely electronic (there is no use of a resonating acoustic body), the Overtone Fiddle described here integrates a full acoustic chamber. It receives resonant stimulation directly from both the strings on the instrument, and from an internally mounted tactile sound transducer, which is controlled via DSP running on an attached iPod Touch®. The physical design of the instrument accommodates this by incorporating space to mount the iPod locally, as can be seen in Figure 1.

The instrument is essentially a ‘pochette’ [13] type of violin design, with a standard violin length and a 2” external width.

Luthier Don Rickert, of Adventurous Muse [9] made plans and constructed the instrument requested for this project. The internal cavity is 1.75” wide in order to accommodate a specific tactile sound transducer. The overhanging top and back, in addition to adding to the vibrating surface, and thus, sonority of the instrument, also provide mounting surfaces for external components such as batteries, pickup preamps, and so forth.

The entire back of the instrument is made of maple, and screwed onto the instrument sound chamber (main body) in order to allow easier access to internal elements and wiring.

2. SENSOR & ACTUATOR DESIGN

The Overtone Fiddle uses a tightly focused sensing approach to capture string vibrations – several designs were attempted, only the chosen method is described herein.

Figure 1. The Overtone Fiddle – first prototype.

2.1 Magnetic Pickup

While an optical pickup system similar to that used with the Overtone Violin could have been designed for the Overtone Fiddle as well, it was deemed unnecessary, as a commercially available magnetic pickup system was found that captures the movements of the strings themselves directly. This system is called REBO [8], and it functions in the same manner as an electromagnetic guitar pickup. It is mounted to the end of the instrument’s fingerboard (see Figure 2, left).

Figure 2. Left, the REBO pickup system mounted on the Overtone Fiddle, and right, the internally mounted tactile sound transducer (not shown here, a similar transducer is

located in the instrument’s second acoustic body).

2.2 Tactile Sound Transducers

Signals can be injected directly into the main acoustic body of the instrument via a voice-coil type of tactile sound transducer (see Figure 2, right), as well as into an optional second acoustic body that hangs below the instrument (see Figure 3). This lower resonating body is made of very thin carbon fiber and balsa wood – materials that would not be strong enough to support the full string tension of strings on the main body – thus allowing extremely efficient transfer of acoustic energy from another embedded tactile transducer, to the structural elements of the box. The box itself is quite a simple design for this prototype. It was designed to dimensions allowing it to function as a proper Helmholtz-resonator (internal shape and porting), in order to maximize the volume of the resulting sound. As such, it is actually capable of producing louder tones than the main body of the instrument.

NIME’11, 30 May–1 June 2011, Oslo, Norway.

(29)

Figure 3. The Overtone Fiddle with carbon fiber / balsa wood second acoustic body mounted underneath.

Designed as a 5.6" x 5.6" x 1.2" box, this second body has a total internal volume of roughly 37.6 cubic inches. To relate this to the spherical shape of a traditional Helmholtz resonator, solving equation 1 below provides the radius of a sphere with an equivalent internal volume. Then, to arrive at the proper size of the soundhole, this radius is divided by four, resulting in a prototype design with a 0.519” radius circular soundhole.

Equation 1. Solving for the radius of an equivalent spherical Helmholtz resonator, given the internal volume of

the prototype second body.

The second body of the Overtone Fiddle is also driven with DSP-generated feedback signals, usually based on the sound from the strings (indeed, any audio signal the performer desires is possible, if suitably programmed). Many types of responsive software can be programmed to run on the iPod touch mounted on the fiddle. Sounds and effects can be responsive to any motions sensed by the accelerometers and gyroscopes in the iPod Touch, with parameters controlled by both real-time analysis of incoming sound from the strings, and gestural movements of the performer.

The tactile transducers inside both the top and bottom resonating bodies are driven independently by a 11.1volt Li-Ion battery-powered Class-T stereo audio amplifier. As mentioned, the main body of the instrument is designed to accommodate this, by providing space for mounting the battery, amplifier, and associated circuitry. This makes the entire instrument self- contained (not including the bow and its corresponding electronic circuit).

2.3 Bow Design

The bow used with the Overtone Fiddle is custom made by the E.W. Incredibow company, from a simple carbon fiber rod that is lighter (almost half the weight of a wood bow) and longer than a normal violin bow. This is helpful in order to accommodate the added mass of a small battery-powered sensor circuit based on the CUI32 [5], along with a wireless 802.11g radio module [11], and an absolute orientation sensor [2]. A simple BASIC-language program was written on the CUI32 using StickOS [12], which is the default operating system for the CUI32. It receives the stream of orientation vectors from the sensor, and translates them into Open Sound Control (OSC) protocol, in order to send them to the iPod Touch. The orientation sensor reports the direction in which the bow is pointing using Euler angles or quaternions, sending updates at 300Hz. This is accomplished by its sensor fusion algorithm, which combines data from internal accelerometers, gyroscopes, and magnetometers.

The WiFly module is configured to broadcast its own

‘AdHoc’ 802.11 base station, which is then chosen as the network to join in the ‘WiFi settings’ of the iPod Touch. This allows the CUI32 to send UDP and/or OSC and communicate

easily with the iPod Touch. One of the strengths of this approach is that the orientation of the bow can be compared to the violin/iPod’s orientation (as determined using the iPod’s internal sensors) and differences in these measurements can be used to control various parameters of real-time effects processes. The mapping of such controls to real-time parameter updates is of course a major task given to the composer / performer / programmer of the system.

Figure 4. Left, the carbon fiber bow used with the Overtone Fiddle, and right, the electronics components before being attached to the bow (red CUI32 bottom, blue 802.11g radio module top, and green absolute orientation sensor middle).

3. MUSICAL PROGRAMMABILITY

Software used with the Overtone Fiddle can be written in a variety of applications, such as SuperCollider [4], PureData (using libpd [3] or RjDj [10]), or the MoMu the Mobile Music Toolkit [1], all of which can run in real-time on the iPod Touch.

In general, the author tends to use SuperCollider, but has experimented extensively with others as well. The instrument can thereby incorporate all of the flexibility of modern digital signal processing techniques, for example, altering the timbral structure of the sound being produced in response to player input.

In order to use the sensor data from the bow with any of these iPod Touch applications, the data must be formatted into either OSC as mentioned earlier, or another network format that the iPod application can receive via the WiFly module. For an example here, Figure 5 shows BASIC code that captures sensor values on the 16 analog input pins on the CUI32, and sends them as UDP to a custom “scene” in RjDj – really just a PureData patch, which is shown in Figure 6. In this case, the absolute orientation sensor was not used, as this was a simple test to verify connectivity. Nonetheless, there are many different types of analog sensors that can be used with the 16 analog input pins on the CUI32. Therefore, such a setup will almost surely be useful in the future.

To explain the BASIC code shown in Figure 5, it can be seen that on line 10 the 2^nd UART (serial port of the CUI32) is initialized. This UART is connected to the WiFly module via two wires: one for transmitting, and one for receiving. Lines 20 through 160 are declaring variables “a … p” that correspond to the 16 analog input pins on the CUI32. These are configured as

‘debounced’, which causes a simple 3-point running average filter to be executed on the incoming sensor values, in order to smooth out any glitches. Lines 170-210 create a connection

(30)

between the iPod Touch and the CUI32 (the iPod must already have joined the WiFly’s AdHoc network).

Figure 5. StickOS BASIC program that sends the CUI32’s analog sensor inputs to the [netreceive] object in the RjDj app on the iPod (running a corresponding RjDj “scene”, which is actually the PureData patch shown in Figure 6).

Finally, line 220 enables an internal timer in the CUI32 (functionally similar to the [metro] object in PureData), and configures it to cause events every 10 milliseconds. Every time one of these events happens, line 230 sends the actual sensor values to RjDj, which is running the PureData patch seen in Figure 6. The list of sensor values is always preceeded with a capital “A”, and appended with a semicolon. In PureData, the semicolon is needed by the [netreceive] object to signify the end of a packet, and the capital “A” is used as an identifier to signify the beginning of the packet. The top [match] object in Figure 6 always checks for the capital “A” (number 65 in ASCII-code) so that synchronization is maintained.

The same functionality can be achieved with other iPod applications with a few modifications to the code. For example, SuperCollider requires the use of OSC-format strings in order to receive network data, so the addition of proper OSC syntax (string identifiers and 4-byte boundaries) is added to the BASIC code in order to use it with SuperCollider running on the iPod Touch. For the sake of brevity, an example of such is not included here.

The CUI32 circuit board was designed by the author as an improved version of the CREATE USB Interface (CUI) [7], and is sold by SparkFun electronics and other online retailers.

4. NEW PERFORMANCE TECHNIQUES

Since any DSP algorithms in use can be controlled through gestural interaction using both the motion sensors in the iPod (accelerometers, gyroscopes, etc.) as well as the electronics on the bow, the system promotes new performance techniques for interactions above and beyond those supported by traditional acoustic instruments. For instance, in initial improvisational performances by the author, the timbre of the Overtone Fiddle changes is made to change dramatically when rapid movements are performed.

The multitouch screen surface on the iPod is also useful in certain musical contexts. While it clearly cannot be used while simultaneously bowing anything other than open strings, there can nonetheless be sections of a performance allowing the performer enough time to access the screen. For example, a

DSP algorithm can sustain a note beyond the time that the performer actually bowed it, thus allowing modifications to the timbre thereafter through interaction with the multitouch screen. Simple parameter changes can of course be executed in between notes as well, etc.

Figure 6. PureData patch used in RjDj to receive real-time analog sensor data from the CUI32. On the left side, sliders visually represent incoming sensor values, and on the right

side, a simple additive synthesizer used for testing that generates sounds in response to the sensor data.

4.1 New Sonic Possibilities

DSP algorithms are used to adjust the body vibrations of the acoustic part of the instrument, actively changing the harmonics heard in the musical tone quality. Consequently, the acoustic properties of the Overtone Fiddle are adjustable, rather than being permanently defined by the woodworking skills of a luthier. In other words, the internal actuator can cause the wooden body to exhibit new dynamic behaviors, and it is this methodology that distinguishes the Overtone Fiddle from prior instruments such as the Chameleon Guitar [14].

While the sound quality of a traditional acoustic instrument is fixed by its physical design, actuated musical instruments can malleably simulate even physically impossible acoustic properties, such as a violin with hundreds or thousands of sympathetic strings; this would be akin to extreme versions of instruments like the Norwegian Hardanger Fiddle, or the Viola d'Amore from the baroque period. For the performer and the audience, however, the perception is that the sound being produced by an actuated acoustic musical instrument such as the Overtone Fiddle is somehow physical – the resulting music is created through gestures on the instrument held close to the performer, who can use the technology to produce interesting timbres that might never have been heard before. It is an important consideration for the performer, that the Overtone Fiddle is not connected by cables to a computer, nor to any remote loudspeakers.

5. MUSICAL OBJECTIVES

The main objective of the development of the Overtone Fiddle is to pursue a long-term research project that is focused on the

(31)

development of new acoustic musical instruments – at first, in the bowed string family and then expanding to others. The addition of technological components to acoustic instruments is used in order to extend these existing instrument types with new expressive and performative possibilities. The project also aims to explore the potentials that these instruments have in both new compositions and new methods of performance.

It is the author’s hope that the development of such new instruments will help revive the evolution of more traditional musical instruments today, through a combination of some of today’s most advanced technologies with traditional instrument making and musical skill and practice. In this sense, traditional acoustic instruments are already seen as advanced technologies in their own right, because they have been refined over many years of development. The goal is to add new dimensions and expressive possibilities to the capabilities of traditional acoustic instruments, and to explore these in contemporary music and performance. The research project should not be construed as an attempt to improve the basic acoustic instrument designs themselves, but seen as an extension of said instruments’

expressive and performative range.

Because the acoustic properties of the Overtone Fiddle can be changed through mathematical processes in real time, it allows artists to make radical changes to their sound. This gives an opportunity to create music that explores new sound worlds, yet sill follows in the traditional musical training to a certain degree. Composers and performers can make use of the instrument’s programmability by means of sound synthesis, sound effects and generative algorithms, all of which can be configured to respond to input from the musician and allowing an almost infinite number of different instrument interaction methods.

6. CONCLUSION AND FUTURE WORK

The development of the Overtone Fiddle offers both technical and artistic challenges that the author enjoys embracing. It has been shown that the first prototype of the instrument is capable of many new musical interactions – future versions of the fiddle, as well as other bowed string instruments are currently in the works. While cables or even wireless audio transceivers could be used to enable the Overtone Fiddle to connect to a laptop for more powerful signal processing than is possible with an iPod Touch, keeping the instrument compact and self- contained is a high priority for this project. Nonetheless, initial experiments were done using a laptop, and OSC-sending remote-control apps such as TouchOSC or Fantastick.

Future prototypes of actuated bowed string instruments may incorporate more traditional violin bodies, instead of the

‘pochette’ type of design. They may also involve a new concept developed in this research project, that of a “bridgeless” violin that was tested in the process of building this first prototype. In this case, an extended fingerboard is used, with the wide end culminating in a raised nut where the bridge would normally be. The strings are then entirely supported by the fingerboard (never touching the body of the instrument). The instrument body is hung from a bracing system running underneath (that also supports the iPod and electronics), in order to separate the fingerboard vibrations from the actuated body. With this setup, a secondary instrument body is not used.

The author can be seen playing the prototype Overtone Fiddle in a video titled “An Evening of Actuated Instruments”

together with Edgar Berdahl on the Haptic Drum and Robert Hamilton on the Feedback Resonance Guitar in an improvisational setting. This video is located online at the Actuated Musical Instruments Guild website:

http://actuatedinstruments.com/.

7. ACKNOWLEDGEMENTS

I would very much like to thank Chris Chafe, Max Mathews, Edgar Berdahl, Rich Testardi, Don Rickert, and my wife Anne- Marie Hansen for all of their help with this project. The support they have given me has been a great help in many ways.

I also wish to acknowledge Lars Beer Nielsen at the Innovation Center Denmark, and Keith Devlin at Stanford University’s Human Sciences & Technologies Advanced Research Institute (H-STAR) for their financial support, providing the opportunity to spend 2 months at Stanford’s Center for Computer Research in Music and Acoustics (CCRMA) working on this research.

8. REFERENCES

[1] Bryan, N. J., Herrera, J., Oh, J., Wang, G. Momu:

A mobile music toolkit. In Proceedings of the.

International Conference on New Interfaces for. Musical Expression (NIME), Sydney, Australia, 2010.

[2] CH Robotics, http://www.chrobotics.com/ accessed January 29, 2011.

[3] LibPd, http://gitorious.org/pdlib/ accessed January 29, 2011.

[4] McCartney, J. Rethinking the Computer Music Language:

SuperCollider. Computer Music Journal, 26(4), 61-68.

2002

[5] Overholt, D. CUI32 microcontroller board,

http://code.google.com/p/cui32/ accessed January 29, 2011.

[6] Overholt, D. The Overtone Violin: a New Computer Music Instrument. Proceedings of the International Computer Music Conference, ICMC 2005, (Barcelona, Spain, 5-9 September 2005).

[7] Overholt, D. Musical interaction design with the CREATE USB Interface: Teaching HCI with CUIs instead of GUIs. Proc. of the 2006 International Computer Music Conference, New Orleans, 2006.

[8] REBO, http://www.uli-boesking.de/rebo/ accessed January 29, 2011.

[9] Rickert, D. http://www.adventurousmuse.com/ accessed January 29, 2011.

[10] RjDj, http://www.rjdj.me/ accessed January 29, 2011.

[11] Roving Networks (WiFly GSX module),

http://rovingnetworks.com/ accessed January 29, 2011.

[12] Testardi, R. (StickOS), http://cpustick.com/stickos.htm accessed January 29, 2011.

[13] Pochette (kit violin),

http://en.wikipedia.org/wiki/Kit_violin accessed January 29, 2011

[14] Zoran A. and P. Maes, Considering Virtual and Physical Aspects in Acoustic Guitar Design, Proceedings of New Instruments for Musical Expression (NIME) Conference, Genova, Italy, June 5-7, 2008.