Exploring Textile Controllers for Computer Music
Nanette Lindrupsen
Thesis submitted for the degree of Master in Robotics and Intelligent Systems
60 credits
Department of Informatics
Faculty of mathematics and natural sciences
UNIVERSITY OF OSLO
Exploring Textile Controllers for Computer Music
Nanette Lindrupsen
© 2021 Nanette Lindrupsen
Exploring Textile Controllers for Computer Music http://www.duo.uio.no/
Printed: Reprosentralen, University of Oslo
Abstract
The thesis aims to investigate the potential of textile sensors as controllers for computer music. The investigation has been divided into two parts.
The first part seeks to examine the possibilities and limitations of e- textiles through practical exploration. In this phase, there were made a variety of prototypes and sketches of textile controller units.
In the second part, a case study approach was used to investigate the musicality of textile controllers. Two computer musicians participated in the study, and they were handed a modular system made during the previous phase. The participants were asked to use the controller to produce a one-minute composition. Their feedback was collected during semi-structured interviews.
These studies suggest there are two traits that separate textile sensors from conventional control components, namely the values and the tangible properties. However, the context in which they are used and the musician’s mindset play a role if these qualities are seen as assets or liabilities.
Contents
I Introduction 1
1 Introduction 3
1.1 Research Question . . . 4
2 Related Works 5 2.1 Abstracted interfaces and minimalistic interactions . . . 5
2.1.1 Showcase the circuit: The Lilypad Embroidery . . . . 5
2.1.2 Make the circuit invicible: Project Jacquard . . . 6
2.1.3 In between: Wave Shaper . . . 7
2.2 Explicit textile controllers for music . . . 7
2.2.1 The Musical Jacket . . . 7
2.2.2 MiMu gloves . . . 8
2.2.3 FabricKeyboard . . . 8
2.3 Summary . . . 9
II Sensors 11 3 Preparatory Sensors 13 3.1 Cap Sense Matrix . . . 13
3.2 Binary Cap Sensors . . . 18
3.3 Analog Sensors . . . 21
3.3.1 Zipper pot . . . 21
3.3.2 Combo sensor . . . 23
4 Metaphors 27 4.1 Finding a Source for the Metaphors . . . 27
4.2 DJ controllers . . . 28
4.3 Rotary Knobs . . . 28
4.4 Linear components . . . 31
4.5 Buttons . . . 31
4.6 The capacitive break-out board . . . 33
4.6.1 The Biano sensor . . . 33
5 Sensors for Evaluation 39 5.1 The Modular System . . . 40
5.2 Sensors . . . 45
5.2.1 Bend sensors . . . 46
5.2.2 Stroke sensors . . . 47
5.2.3 Capacitive Sensor - Touch and proximity . . . 49
5.3 The ideas that were not materialized . . . 50
6 Case Study 55 6.1 Methods . . . 55
6.2 Play and Creativity . . . 58
6.3 Participants . . . 62
6.4 Results and Analysis . . . 63
III Conclusion 75 7 Discussion 77 7.1 Case study . . . 80
7.2 Key Findings . . . 82
8 Conclusion 85
List of Figures
3.1 capGrid.v02 . . . 14
3.2 capGrid.v03b . . . 15
3.3 capGrid.v04 . . . 16
3.4 capgrid.v05 . . . 17
3.5 A cloth with the BinaryCap sensor. . . 18
3.6 The Binary Cap’s embroidered button . . . 19
3.7 All of the capacitive squares were within on hand’s reach . . 19
3.8 The inner layers that can be part of press and bend sensors . 22 3.9 The Zipper Potentiometer . . . 23
3.10 Paths in the zipper potentiometer . . . 24
3.11 The Layers of the Combo Sensor . . . 25
4.1 The different types of rotary knobs . . . 28
4.2 A textile sensor metaphor of the rotation knobs . . . 29
4.3 The structure of the button platter . . . 30
4.4 The button platter prototype . . . 30
4.5 Linear components . . . 31
4.6 The metaphor for the modulation wheel . . . 32
4.7 Buttons . . . 32
4.9 The first layers in the Biano sensor . . . 35
4.10 The paths that determines the pitch are sewn as two parallel paths. Both sides are placed adjacent to the octave paths. In addition, the right branch will be combined with the sharp- path that transports the key a semitone up. . . 36
4.11 The paths for triggering semitone transpositions are sewed next to the right note-paths. . . 37
4.12 The Biano sensor . . . 37
5.1 The modular kit . . . 39
5.2 The motherboard . . . 40
5.3 Sensor snaps . . . 41
5.4 The break-out board with double wrapped wires that is attached with hand sewn stitches. . . 43
5.5 The connection of the break-out board and the silicone stranded core wire . . . 43
5.6 The press buttons on top of the wire’s core and conductive tape . . . 44
5.7 Textile wires . . . 45
5.9 The mindmap with the initial ideas . . . 46
5.10 Table Bend sensor . . . 47
5.11 Wrist Bend sensor . . . 48
5.12 Illustration of the conductors . . . 49
5.13 Fluffy Stroke sensor . . . 50
5.14 The threads on the Fluffy Stroke . . . 51
5.15 Fluffy Stroke sensor . . . 51
5.16 Pleated Stroke sensor . . . 52
5.17 CapSense sensor . . . 52
5.18 Capacitive Proximity Sensor . . . 53
6.1 A map over Gamma’s conceptual conncetions . . . 70
7.1 Techniques for the stranded wire . . . 78
7.2 Sewing machine foot used for the stranded wires . . . 79
List of Tables
Part I
Introduction
Chapter 1
Introduction
Most humans are surrounded with textiles during the majority of their lives. Textiles offer protection against the climate and other physical forces, and it is even used as a form for communication of who we are [10], [29], [1]. Although textiles have been indispensable for humans throughout history, its role has expanded through technologic advances. One of these inventions of particular interest in this thesis is the electronic-textile.
E-textile is the amalgamation of conductive materials and textiles. This fusion allows us to integrate electronics into textile objects while preserving the tactile qualities of fabric. Although conductive textile is not a new invention, it is not until recently that researchers’ attention have been aimed towards the possibilities that can be achieved by integrating electric properties into textiles [9], [22], [7].
Parallel to this, technologic advances have also led to an increase in the omnipresence of digitizations. Activities that before required physical efforts have been shifted into digital abstractions. One of these abstractions is music making. The rise of computer music has allowed new musical expressions to form and it has redefined the role of music instruments in many computer musicians’ practices. While the computer mouse and -keyboard are useful tools for creating electronic music, the decoupling of the musician and their instrument can also separate the connection between their input gestures and the sonic output. The academic literature on embodied cognition has shown that cognitive processing is not just product of the mind, but cognition is also significantly driven by sensorimotor functions from the body [40]. If we look at the digital abstraction of music making from this perspective, it seems that something will be lost when emotional musical expressions are conjured with pragmatic interfaces.
This issue brings us back to e-textiles. During the research for textiles’
role in humans life, I noticed a topic that had been neglected; namely textiles as tools. Textiles are more than the assembled materials in clothing.
For example, bags and pockets are containers that helps us collect and carry items, a towel transports liquid away from wet surfaces, oven mittens protect us from burning damages and airbag cushions require little space when inactive, but can quickly expand and maintain their integrity under
a high pressure of gas.
The textiles’ soft, yet strong qualities have made these objects more suitable than their potential rigid alternatives. If we look at these features in the context of e-textile, it is worth investigating if textile sensors can be viable controllers to make computer music. These features can potentially bring tangibility into technology and potentially make the digital representations less abstract. The aim of this thesis is to investigate these ideas. What follows is a more detailed description of research questions of this thesis and how I have approached the questions in my research.
1.1 Research Question
The objective for this thesis set out to assess the potential of textile sensors as controllers for computer music. This aim was approached by investigating the following questions.
1. How can the tactile qualities of e-textiles be used in a music controller and what are the materials’ advantages and disadvantages?
2. What can textile sensors contribute with as a controller for creative expressions?
3. What are the advantages and disadvantages of using textile con- trollers as computer music tools?
The first subquestion was adressed by exploring alternative designs for textile controller units. The process was structured into three phases.
The first phase was driven by an intuitive process in which exploration was prioritized over perfecting the prototypes. The process and results are described in chapter 3. The second part, which is explained in chapter 4, followed a more structured process, where the scope focused on sketching textile metaphors for DJ controllers. The observations from these two chapters were used to make a modular kit of textile sensors that could be assembled into a music controller. The details about the system is covered in chapter 5.
A case study was chosen to get insight into how the textile sensors could be used in practice. Two computer musicians were each delivered a modular kit and asked to record a one minute composition using the sensors. The methodology and results will be covered in chapter 6.
The findings from the studies in part II will be discussed in chapter 7.
We will first look at the findings seen from the creator’s and musician’s perspectives individually, and then examine how these findings together form the basis for my contribution to the field with respecth to the overarching research question – what is the potential of textile sensors as controllers for computer music?
Before proceeding to examine the outcome of this research, it is necessary to look at the relevant works. What follows are projects that have used e-textiles in the context of interactivity and sonic output.
Chapter 2
Related Works
This chapter covers works that have demonstrated inspirational achieve- ments in e-textile interfaces. We will first look at three projects that have approached soft circuit interfaces as passive components.
All of the projects in this chapter have used e-textiles as a tool to influence a sonic output, however the second section presents work that have designed interfaces that are primarily built to control music. As a consequence, these interfaces require a more intentional interaction and might be more difficult to master.
2.1 Abstracted interfaces and minimalistic interac- tions
This section covers projects that have used soft circuits that can sense gestures, without having to directly manipulate the material. Although the contexts in which these works have been created have not been to control music, they still react to human gestures through sonic feedback.
The projects in this section are abstract in the sense that they do not take the form of a specific object. However, as will become clear, the creators have all made efforts to enhance fabric by augmenting it with electronic properties.
Despite their conceptual similarities, each of these three projects have approached the visibility of the soft circuitry very differently.
2.1.1 Showcase the circuit: The Lilypad Embroidery
This project is an interactive embroidery that plays computer sounds when it is approached. Gestures are detected with a photo-sensitive sensor.
Blocking the sensor’s access to light will therefore alter the rate of output.
The piece unapologeticly displays the circuitry, by placing the Lilypad microcontroller in the center of the frame, use frame it with colorful embroi- dery that complements the colors and lines on the Lilypad. Additionally, electrical components, such as LED-lights, resistors and transistors resem- ble beads that are equally a part of the visual artwork as the non conductive ornaments.
Some of the conductive threads in the piece, are covered with tradi- tional thread. Since the artwork is hold firm with an embroidery hoop, there is a low risk of short circuits. However, the covering stitches demon- strates a commonly suggested method to insulate conductive thread.
The conductive threads on the market are currently uninsulated, and soft circuits often have a high risk for short circuits due to the flexibility of the fabric or conductive threads that are sewn too close. The risk of short circuits can be avoided by properly insulating the exposed wires. Not only would the insulation provide a more robust circuit, it would also improve the yarn’s conductance and decrease interference. However, covering the conductors with satin stitches might not be sufficient enough. Additionally, in a review of e-textile insulation techniques, [8] found that most of these techniques for are likely to influence the physical qualities of the textile.
The use of digital components in the Lilypad Embroidery is interesting because it combines traditions with technology. In other words, it combines the fast with the slow. I think this is a relevant comment to the current role of e-textiles in most cases. Embroidery done by hand can be a laboring and time consuming process. Nonetheless, the inclusion of circuitry suggests that there is a desire for some automation.
2.1.2 Make the circuit invicible: Project Jacquard
The Advanced Technology and Projects (ATAP) group at Google developed scale production methods for embedding conductors unnoticeable in garments [27].
The researchers developed a conductive yarn that addressed the shortcomings in standard conductive threads. Since these shortcomings are reflections of the standard production techniques, the following paragraphs will explain both.
Coating or plating non-conductive yarn with a conductive material, such as silver, stainless steel, copper, carbon or metal salt. This technique provides a very soft finish, but the coated layers can be inconsistent which means the conductivity will vary throughout the yarn. An- other disadvantage is that repeated washing and friction will wear down the coating. Over time the yarn’s conductance can weaken or be worn out.
Core spinning regards to twisting conductive filament around a non- conductive yarn. In this yarn, the thin metal filament is exposed to wear and tear. The thin filaments can break, which can cause the circuit to malfunction.
Spinning metal fibers. These plied yarns have a relatively low resistance and are quite robust. The yarns made out of staple fibers (short fibers) are soft and flexible, but are prone to shed metal fibers which increases the possibilities of a short circuit. Yarn plied with filaments (continuous fibers) are stiffer and harder to work with, but they do not fray as easily and can even be used as a bobbin thread without
putting the sewing machine mechanics at risk. Since the thread is 100% metal, there are no other color options than gray. Visible stitches in a soft-circuit can therefore end up with the same “electronic look”.
To preserve the typical visual and tactile qualities of textiles, they used a technique similar to the core spinning. As the core, they used copper filaments, The copper is then covered by wrapping non-conductive yarns (as silk, cotton, polyester, etc.) around it.
As might have become clear, I have found the Jacquard yarn to be the most promising feature of the project. In their paper ([27]), it is mentioned that one of the non conductive cover’s functions is to insulate the conductive core. If this insulation really is resistive enough, then the surface layer seems to solve many of the problems faced in commercially available conductive yarns.
Although their product is not a musical interface in itself, there are demonstrational videos of the Jacquard based Levi’s jacket that show the interface can be used to turn on the music streaming on a smart phone [15].
This decoupling between the interface and the actuator (the speakers), is one of the main characteristics that separate digital musical instruments (DMIs) from acoustic instruments.
2.1.3 In between: Wave Shaper
Pearla Pigao is a Norwegian textile artist and musician. In 2020 her work Wave Shaper was exhibited at Soft Gallery in Oslo. The work consists of two woven fabric cloths hangs from the ceiling and glass cylinders that contains water. Through what seems like capacitive sensing, music starts to play when people apprach the cloths. Pigao said that the work first starts to become alive when people are interacting with the structures [32].
The conductive filaments have been integrated into the fabric through weaving raw metal filaments with non-conductive yarns. Her work shows an impressive middle road of accepting the material as it is, without shouting that it is a circuit.
2.2 Explicit textile controllers for music
This section presents more direct approaches to control music with textiles.
The following works also require the user to actively engage with the systems.
In contrast to the latter section, these projects have based the interfaces on recognizable objects such as a denim jacket, a glove and a keyboard.
2.2.1 The Musical Jacket
In 1997, Maggie Orth and her team at MIT Media Lab presented the Musical Jacket [26]. The jacket consists of an embroidered keypad that senses touch by capacitive sensing. The wearer uses the keyboard to play sounds, record and loop sequences while getting audible feedback from the speakers in
the pocket. The circuit layout was designed in a CAD and then stitched with an embroidery machine. The conductive yarn is described as a mildly conductive thread consisting of polyester and stainless steel. To reduce the total resistance in the composite thread, the motive was stitched several times to create parallel resistance, thus improving the conductance.
Although this is an early example of a wearable music device, the jacket seems to be about the concept, rather than an actual music controller. From a performer’s perspective, the interface has an awkward placement. The keypad’s placement on the upper chest requires the wearer to maintain a static position with the active arm, which seems tiresome over time. Unless the wearer knows the keypad by visual- or muscle memory, they would have to bend their neck to see the interface which can strain the neck over time.
2.2.2 MiMu gloves
The hands as controllers have been a big interest in gestural musical instrument design [36], [30]. The development of MiMu is one of the most recent successful examples of this field. The gloves monitors finger flexion and hand orientation. The hand gestures are analyzed and recognised by an artificial neural network which then trigger preset musical parameters or modes.
As an example, a clasped hand can activate a continuously measure of the rotare displacement of the hand. The recognised gesture can also measure the rotary data only when the hand is clasped.
The creators wanted the MiMu glove to resemble the expressivity in traditional instruments. The expressivity in electronic controllers for music has been a widely discussed topic [35], [12] and the critics often point out that the mapping between gesture and output is too simple in digital interfaces. To deal with this issue, discrete and continuous control mechanisms were combined to allow a wider range of gestural control options.
In many computer music performances, the interface for these param- eter adjustments are usually not visible for the audience. The glove seems to be a successful initiative that visually invites the audience into detailed adjustments of the musical parameters.
2.2.3 FabricKeyboard
The progress of e-textiles has allowed a sub group of musical interfaces to take place, namely deformable musical interfaces [38]. One of these is the FabricKeyboard [39].
The FabricKeyboard is a soft and lightweight textile music controller.
The creators’ goal was to enhance the relationship between the physical interaction and the musical output by allowing performers to explore the tactile qualities of the instrument.
The keyboard contains sensors that detect touch, pressure, stretch, proximity and sliding. There are also two additional sensors that can
be attached to the keyboard; a ribbon slider and a fabric trackpad that respectively track the finger’s position along one and two axes.
The main device’s layout resembles one octave on a piano. Each key consists of six layers of fabric. The uppermost layers are capacitive and senses proximity and touch. To avoid interference with the pressure layers, the two sensors are separated by non-conductive layer. The pressure sensitive layers are two pieces of conductive fabric that are separated with a piezo-resistant fabric and a layer of polyester mesh that works as a physical spacer/buffer.
This work is different from the other mentioned since its about exploring the musical interaction rather than the performance. Similar to the MiMu gloves, they wanted the device to detect both discrete and continuous control options to enhance the expressivity.
2.3 Summary
How this chapter defines my thesis will be presented along with a summary of the projects.
This far, we have seen unique interpretations for fabric controllers that respond to gestures with sonic output. These systems have been divided into two groups. The first group detecs proximity or lightweight touches while the other group requires a more active interaction and seemed somewhat more robust.
What I find interesting with the Lilypad Embroidery and Wave Shaper are two artistic expressions that have augmented the viewer experience with adding auditory feedback to the visual aesthetics.
An additional comparison can be done with the Musical Jacket and the Levi’s Trucker Jacket. Both works have placed a capacitive interface on a denim jacket. And despite the latter was not intended as a musical tool, if the mapping only was changed to send its values to a synth app on a smart phone, it could suddenly be a musical interface. I find the ATAP group’s achievements in embedding conductors in fabric inspiring. However, it is worth pointing out that there are well-known ethical, environmental and political concerns with large scale production.
This thesis will approach the design of the sensors by attempting to maintain their textile tactility and aesthetic while focusing on small-scale production. Hopefully, this perspective can make the controller easy to customize and fix. Furthermore, I will not focus on wearables. As with other outfits, the non-verbal communication that lies within the style decisions of the smart garment brings a complexity that is beyond the scope of this thesis.
Continuing to the last two projects, the mix of discrete and continuous linked the MiMu Glove had similarities with the FabricKeyboard. Where the MiMu glove could be a bridge for the audience’s understanding of digital adjustments of the sonic expression, the FabricKeyboard offered an interface that was highly tactile and deformable.
The most relevant project in this chapter must be the FabricKeyboard
due to the direct interaction with the interface. However, the creators met usability issues with the system, and some of their decisions have made me questions if it was really necessary to include fabric in all parts of the device.
At any rate, I think it is important to take a step back and start to look at the atoms that constitutes a textile music controller. In the same way the weight of a piano key can alter a pianist’s performance, I assume the design of a textile sensor influences the interaction it encourages. Think about it as wine making. Before planting the whole crop, you would want to know what grape types to choose and whether or not they are compatible with your climate. The process I want to avoid is to pick out the seeds I liked the idea of, not consult with a wine-knowledgeable person and later expect others to drink the wine.
This thesis attempts to identify some of the strengths, weaknesses and possibilities in textiles as music controllers. The intention is not to claim that textile should be a replacement for controllers that already work just fine. By working with the material instead of against it, we will also explore the suggestions that lie within. Those suggestions can bring something new into the music. Digital music interfaces have gotten critized for decoupling the sound from the phsyical body and the mapping between them. I hope these suggestions can help connect the sonic output to the interaction that is happening. Hopefully, this work can contribute to the design of future textile controllers and help guide the minor decisions that together will result in a holistic, unique and functional tool for music making.
Part II
Sensors
Chapter 3
Preparatory Sensors
This chapter covers the first sensors that were made and the decisions that led to the next sensor. Each section will elaborate the details behind one sensor unit. We will look at the inspirations, the planned and actual function, followed by a discussion about improvements and possible prospects.
Since the ideas have been centered around the available materials and equipment, the overarching concept and the techniques applied on each sensor is what separates them.
• Inspiration and motivation
• Function
• Construction
• Tests and results
• Discussion
3.1 Cap Sense Matrix
The inspiration for the first group of sensors came from [28]. The capacitive touch grid senses the location of one or several fingers on the sensor surface. Several versions were made due to some issues explained later.
In the following text, the different versions will be referred as capGrid.v0n where thenis the order of which they were made.
Materials
• Embroidery fabric
• Conductive thread (stainless steel)
• Copper tape
• Conductive textile tape
Figure 3.1: capGrid.v02
The first two versions were embroidered by hand. The capGrid.v01 was embroidered with the stable fiber-based conductive thread, and capGrid.v02 with the plied conductive thread. See fig. 3.1 As the base, a cloth specialized for embroidery was used. The two trace layers had to be separated to avoid unintentional conductance. According to [28] this could be achieved by using impregnation spray on both sides of the fabric before sewing the second layer of traces. In addition, the stitches pierced the fabric only where there would not be any direct contact between the two layers.
Tests on grid.v01 and grid.v02
The capGrid.v01 was tested with a multimeter. Due to outlying conductive fibers on the thread, there was conductance between the layers at several points. These connections would have caused the system to output false positive signals. Consequentially, this cause the affected rows and columns to be unresponding to touch.
Instead of fixing the current sensor, it became evident that it would be more productive to make a new version with the plied filament-based thread instead. The new version (capGrid.v02) passed the multimeter test and had almost no conductance between the layers. The following step was then to test the sensor with a microcontroller and a middleware.
The tests were done with: Arduino Mega 2560, a breadboard, multi- plexer, prototype wires, and alligator clips.
The results were visualized by the use of a related Processing-library provided by the researchers in [28]. The activated signals were represented by an increase in brightness levels, the "dots" circumference and their respective locations. The visualization provided an intuitive approach to
interpret the data in real-time.
The tests showed a noisy output-image from Processing. Despite leav- ing the embroidery untouched, the image showed brief and inconsistent white points. Upon touch, some points on the embroidery led to oversat- urating the output images with white. Although conductance between the paths were not found during the first measures, later measures revealed that the layers were no longer sufficiently separated.
capGrid.v03 Materials:
• Muslin
• 2-ply conductive thread (stainless steel, plied, filament based)
• Non conductive thread
• Copper tape
Figure 3.2: capGrid.v03b
To deal with the weaknesses in capGrid.v02, I made a new version of the grid sensor. In this version, namely capGrid.v03, the capacitive surface was reduced to single stitch lines of conductive thread. To better separate the two conductive layers, a denser fabric was used as the base. The stitches were sewed with a sewing machine (Singer Quantum Stylist 9960) and were separated by approximately 1 cm. The conductive thread was used as a bobbin thread, while the upper thread was non-conductive.
Tests on capGrid.v03
To test it, I used the same setup as earlier. This time there were no points on the sensor that oversaturated the output. Other than that, there was little difference. I could see a difference in the noise frequency when I put and removed my hand on and off my laptop. To see if this was an effect that was particular to my laptop, I tried the same setup on a different computer.
This did not improve the level of noise.
To increase the capacitive surface, squares of double folded copper tape were placed at each cross section. See fig. 3.2. Expanding the capacitive surface provided a clearer output, although the image was still full of noise.
However, lowering the resistors to ground by from 100 kWto 90 kWseemed to make the sensors more responsive to touch.
Although the result at this point was not satisfactory, I wanted to look into how one these findings could be used to make a more stable textile version that could handle repeated friction.
Figure 3.3: capGrid.v04
To improve the capacitance further on these sensors, on path now contained four lines of conductive thread. The conductive tape squares were placed on top of the paths in capGrid.v04 and under the paths in capGrid.v05. The squares were then attached with narrow zigzag-stitches of non-conductive thread. Finally, I made an attempt to freshen up the typical "e-textile" look and add more space between the layers in the latter sensor. One layer of squares were covered by black silk organza and the hem was finished with bias tape See fig. 3.3 and fig. 3.4.
Despite none of these version performed better than the capGrid.v03, they do illustrate how the sewing technical finishing can be done in future attempts. Since I was not able to reproduce the slight success in
Figure 3.4: capgrid.v05
capGrid.v03, therefore improvement attempts were discontinued.
Discussion
If we compare hand stitching with a sewing machine, stitching by hand is more time-consuming, and there is a higher chance of inaccuracy. Where the sewing machine excells in sewing long lines fast, handsewing provides more flexibility in narrow and curved areas, and for situations where moving the needle in only two direction is not enough (e.g. attaching shank buttons and invisible seams).
However, in the context of e-textiles. The time consumption for handsewing do also mean that the textile will be exposed to more trauma.
The conductance between the layers in capGrid.v02 seems to be due to the structure of the fabric loosening over time. This bears resemblance of the reasons stay stitching is used. In dress making and tailoring - constructing a garment is usually done in a certain order. The areas that are saved for last will be exposed to movement and friction due to handling the fabric.
This can cause those areas, such as neck lines and arm holes to widen and become uneven. The stay stitches negates this by stabilizing the structure of the raw edges. I suspect handling the fabric over time might have not only loosened the fabric structure, but the heat and friction from the hands might have worn down parts of the insulating layers. For further attempt at embroiderying the grid, an e-textile equivalent for stay stitches should be identified.
The failure of the sensors might be due to the following reasons:
1. Human imprecision
2. Uneven thickness and outlying fibers on the thread.
3. Too coarse or thin base-fabric.
4. Worn down areas on the insulation film, caused by friction from handling the fabric while embroidering.
5. Not proper grounding
6. Too high susceptibility for noise due the lack of insulation.
Nonetheless, the algorithm that was used to read the matrix, inspired me to the next sensor which will be discussed in the following section.
3.2 Binary Cap Sensors
Figure 3.5: A cloth with the BinaryCap sensor.
The requirements for the next sensor were as follows:
• It should be controllable with one hand.
• Better accuracy: the sensor should only read touch and not proximity.
• The parameter range should be greater than the pins used: The combination of activations should either switch modes or send parameter values.
The Binary Cap sensor consists of five squares of embroidered conduc- tive thread - one for each finger. The touch buttons were made by embroi- dering a square with conductive thread. The technique is shown in fig. 3.7 and consisted of three steps.
1. Sewing vertical structure lines
2. Weaving the conductive thread on this structure
3. Carry the signal through a path of conductive tape. The tape runs through a buttonhole where the remaining length is attached to the backside of the fabric.
Figure 3.6: The Binary Cap’s embroidered button
To reduce the noise, the capacitive squares sensed touch but not proximity. Lastly, one of the issues with the capGrid-sensor was that they required too many pins than the wearable microcontroller could provide.
So, instead of mapping one pin to one parameter, the output results from the combinations of activated pins.
Figure 3.7: All of the capacitive squares were within on hand’s reach The Binary Cap-sensor was mapped to work as a naive approach to improvisation. The leftmost button sets the root note, and the adjacent
button sets the scale. The combinations of the remaining three buttons were based on the binary system. Thus the buttons provided eight different options (0-7). Changing the scale or root note could be done by holding any parameter buttons, and then confirm the value by pressing either the left or middle button depending on the mode you want to change.
Method
For evaluation the system was programmed to behave more like an arpeggiator than a pitch-triggering device. The microcontroller was connected with a smartphone (Huawei P10 VTR-L29) via BlueTooth. The output was sent as MIDI messages to a MIDI-synth app (FluidSynth MIDI) on the phone. The sensor was evaluated by listening to the scales it generated.
Results
All the scales and root notes were tested, and the system managed to output the expected values.
Discussion
Due to the one-to-one mapping, the system is more of a "pitch-in-scale- generator" than a digital musical instrument. Furthermore, even as a
"pitch-in-scale-generator", the tonal range is limited.
For further development, the parameter buttons could have triggered tones when neither A nor B was pressed (i.e., instead of triggering arpeggios, the parameter buttons could have been used to play a melody).
Removing the explicit message to trigger an action would have made the sensor exposed for accidental touches. Depending on the situation, that can be a weakness. In other cases, it might be welcome and even a part of the musical output. However, since the states cannot be adjusted during performance, any melodic movement between scales and root notes can not be fluently achieved.
Upon reflection, it could be possible to move the modes from A and B to A and AB. This would result in one button to select between eight root notes (A), a second to choose one of eight types of scales (AB), and a third mode (when A is not pressed) where the user can use four buttons to trigger 15 different notes within a scale (1-15).
Initially, I considered the alternatives to change modes and trigger pitches. For example, the pitch range could have been increased by removing the modes and use the ascending binary values to represent a chromatic scale. This would have required the user to be far more knowledgeable about music theory and the binary system, and it raises the question if a wider range of pitches actually would improve the system or just make it more difficult use. Another feature I considered was to use a small piezo motor as feedback when the user successfully changed any of the modes. Nonetheless, introducing hard materials would add bulk
to the fabric as there are no fabric-based piezo motors to my knowledge.
Furthermore, this would require a pin on the microcontroller. Due to the simplistic mapping, I would instead prioritize any remaining pins to sensors that could have introduced expressive parameters. When it is not possible to articulate tones, change the dynamics or alter the timbre, the current system’s expression level starts to resemble the expressiveness of a computer keyboard, which I believe would become boring to use rather quickly.
However, the possibility to choose between eight scales, eight root keys, and play 16 notes without moving the wrist, shows the potential of how the combination of a few "on and offs" can be used in a musical setting.
3.3 Analog Sensors
The next step was to look at how the pins could read a broader range of values instead of the binary on and offs. That was then I looked into textile analog sensors.
The requirements for the analog sensors were as follows: The sensors had to be sewable by hand or machine (e.g., no knitting or crocheting) The sensors had to require few analog pins (e.g., no matrices with many dependants) The sensors should be made out of the materials at hand and with the equipment available.
There seem to be two primary approaches to regulate the voltage in textile potentiometers. The first approach separates two conductive layers with a piezoresistive layer (e.g., velostat, eeontex). The second approach divides the voltage with a fixed number of resistors. The former provides a continuous increase in value. The latter produces fixed values where the difference depends on the resistors used.
The most common type of textile pressure sensors consists of a piezore- sistive material sandwiched between two conductive layers. During early experiments, one layer of velostat resulted in a rapid increase of the out- put value. In order to increase the pressure range, additional layers of the piezoresistive material were added. The layers can be seen in fig. 3.8. The two outer layers consist of non conductive fabric with conductive tape on them. The tape is arranged in a pattern so they will not overlap. The darker layer represents the layers were separated by velostat.
3.3.1 Zipper pot
The inspiration to make a zipper sensor came from the slide potentiome- ters. In the zipper potentiometer, the resistive material consisted of 10 kW resistors sewn in series—one end(terminal) connected to 3.3V, and the other connected to GND. The slider and a stitched line close to the zipper teeth represent the potentiometer’s wiper. See fig. 3.10. When the zipper head comes in contact with both sides, the circuit closes. The height of the head alters the variable resistance and thus the voltage that is read.
Figure 3.8: The inner layers that can be part of press and bend sensors
Weaknesses
I have a few thoughts about the zipper potentiometer. First of all, the physical size of the potentiometers limits the number of possible values.
Secondly, the conductive material at the side of the chain has to be near enough to connect with the zipper head and lie flat enough so that it will not jam the slider. Then, after repeated use, the friction can break down the conductive fibers and break the circuit.
Possibilities
The activity of zipping and unzipping could offer some unique tactile qualities which could make it suitable as a sonic controller. Suppose one would want to do more extensive research on musical zippers. In that case, they might find other aspects such as the pull friction (how much force is needed to slide the zipper head in either direction) and the sound and vibration from translating the slider and how these are different from zipping compared to unzipping (primary/secondary feedback). In addition, these variables can be influenced by changing the zipper length, teeth size, and materials of both the teeth and zipper head.
The following text describes the ideas I later got for the zipper sensor.
When the zipper head moves up and down, some vibration in the zipper and the adjacent fabric will occur. What would happen if we attached piezo microphones to the fabric to detect those vibrations? The translation of the zipper head emits a grainy sound when the zipper head moves slowly. However, when the teeth size decreases and the velocity of the pull increases, the pitch will go up. The concept draws some parallels
Figure 3.9: The Zipper Potentiometer
to the square wave in an oscillator. When the frequency is low, only the
"clicks" will be audible. Increasing the frequency will eventually make the clicks sound like a tone. With this in mind, could we use the pitches of one coarse-tooth zipper and one fine-tooth zipper to play a sort of duet? When closing and opening a zipper, the pull tab is tilted and subjected to pressure from the fingers. It could be interesting to read the forces on the pull tab by replacing it with a force-sensitive resistor. The pull tab could even be embedded with an accelerometer or a tilt sensor. Could we add stretch sensors on the adjacent fabric to detect pressure from the body when the zipper is fully closed?
What I found fascinating with zippers is that closing a zipper is seemingly a simple gesture, but the physical effects can be a source of multidimensional data. Furthermore, since the data reflects the physical restraints of the zipper and its adjoined fabric, it might be possible that the user could experience a more recognizable link between the tactile experience of manipulating the slider and the sonic output.
3.3.2 Combo sensor
The final experiment with the analog sensors was a combination of the resistor-based potentiometer and the piezoresistive potentiometer.
How it worked
Four actions determined the two output values; horizontal translation, vertical translation, pressure, and the pressure point size.
The sensor consisted of four parts. The backmost layer had a similar
Figure 3.10: Paths in the zipper potentiometer
arrangement as the zipper potentiometer. The resistive track consisted of seven 8 kWpotentiometers sewn in series with a combination of conductive thread and the 8 mm conductive tape. An was connected to analog-in.
Velostat separated the voltage source from Line 1. Additionally, a thick thread sewed 1 cm from the edges of the velostat provided additional physical space from the back layer. On top of the voltage source path, there were two additional layers of velostat. The velostat patches attached hand with blanket stitches. Blanket stitches gather the layers without applying too much pressure and are less likely to affect the baseline resistance in the piezoresistive layers. The third part was a muslin cloth with a path Am to the second analog-in. Finally, a window with black silk organza covered and framed the interface area.
Upon pressure, the conductivity of the velostat increases, thus allowing both analog-ins to pick up on the voltage level.
Horizontal placement affected An, pressure affected Am while increas- ing the pressure point by tilting the finger affected both. Pressing the upper half of the window had a more significant influence onAnand vice-versa.
I did not conduct any extensive tests on this sensor. However, to get an impression of how the one-to-many sensor could work, I used the inbuilt speaker and LED lights on the microcontroller. The signals from Ancontrolled the pitch, and the signals fromAmcontrolled the light level.
Results and Discussion
Although there were not collected data on how well the sensor performed, based on the pitch and light levels, it seemed to do what was expected. The
Figure 3.11: The Layers of the Combo Sensor
resistor-based path controlled the pitch. The input values were mapped to represent a pitch between 220 Hz and 440 Hz. In retrospect, the audible performance could have been easier to evaluate if the signals were mapped to a scale instead.
All the layers made the sensor relatively thick. In total, there were six layers. Velostat has a texture that resembles a mixture of printer paper and plastic. When layering pieces of velostat, the sensor starts to lose its textile qualities. Knitted fabric of semi-conductive yarn could be an alternative for the piezoresistive layers. However, it would likely add more thickness, and the structures of the fibers would likely introduce hysteresis.
Using one gesture to control several elements seems to add richness to the interaction. However, the sensor’s stiffness makes it less than ideal for a wearable device. Placing the sensor on a hard surface will also provide a better consistency on the ratio between the force applied and the force detected. Luckily, textiles are more than garments. For example, variations of the sensor could be applicable to a rug, table cloth, or wall installation.
Chapter 4
Metaphors
Until now, the sensor’s design decisions were lead by using the data to trigger and generate sound. After all, there will not be any music without sound (unless you ask John Cage [6]). The gridCap sensor looked promising. However, the sensor occupied resources that could have been used to introduce more degrees of freedom. If the sensor had been used alone, the system would be too simplistic. Furthermore, the analog sensors required fewer pins and provided a wider range of values. Yet, their signals fluctuated, and controlling them to a specific value introduced challenges.
Although these unmanegeable signals can be used intentionally, their utilization area as melody-generating sensors seemed too narrow.
The idea process started to decelerate as a consequence of this dilemma.
I felt I had to go back a few steps and change the criteria for the textile sensors’ role in creating computer music. I started to look at two things – established musical interfaces and how gestures are used to embody musical elements. The investigation finally brought my awareness to how the sensors could serve as metaphors for common music-related interfaces.
Each of the following four sections is divided into two parts. The first part begins by analyzing the components found in the interfaces of a standard DJ-setup. The second part presents a possible metaphor for the respective component.
The last sections covers two miscellaneous sensors. The first is a capacitive proximity sensor, and the second is an ad hoc metaphor for the bass guitar. Their commonality – the Trill Craft break-out board – will be discussed there as well.
4.1 Finding a Source for the Metaphors
In accordance with many of the proposed models of the creative stages (and unknowingly of which), the process started with thinking divergently.
First, I noted the possible interfaces that could be represented into a mindmap. The notes were then structured into groups and subgroups and then sorted into a hierarchical tree.
The metaphor’s inspiration source had to be an interface that I knew well. Having practical experience with the source would prevent
assumptions from replacing the knowledge gaps. If the metaphors were based on assumptions, the sensors would instead be a manifestation of the audience’s perspective rather than the performer’s.
Therefore, the metaphor’s inspiration source became a interface I knew well, namely the DJ-controllers. The controllers included a turntable, a digital music player, and a DJ-mixer.
4.2 DJ controllers
The specific models I looked at were: Technics SL1210MK2 (turntable), Pioneer CDJ 2000 (digital music player) and Pioneer DJM 900 (mixer). We will look at how I have categorized the components in the interfaces in the followin sections. Each category consists of a short analysis of the different characteristics within each group and the resulting sketches for the metaphors.
4.3 Rotary Knobs
Figure 4.1: The different types of rotary knobs
There were observed two types of variations that separated the rotary knobs. Although it might be hard to identify visually, the knobs in fig. 4.1 illustrates three knobs with different combinations of these traits found on the interfaces.
The first trait is the rotation angle, which could be fixed or unlimited.
The first group, which is the majority of the knobs, rotate within a fixed range (e.g., The knobs for EQ, effects, and cue level). The other group has an unlimited rotation angle. There were only three knobs with this trait in all of the interfaces combined. They control the following: adjust the beat effects’ time parameter (mixer), temporarily manipulate the speed of the track, and move the cursor on the menu display (CDJ). The last knob can also be pressed like a button in order to select actions on the menu. This might suggest that the continuous knobs control parameters that vary in size (e.g., the number of options in the display menu or the length of the track.) Although the effects’ time parameter knob control parameters that have a maximum size, changing the effect (with another knob) can set the
parameter back to zero, which complements the unlimited range of rotation of the time parameter knob.
Moving to the next trait: the resolution. The resolution could be discrete (i.e., stepwise) or continuous (i.e., smooth). In the first subgroup, the rotation sequence is divided into into discrete steps. The fixed positions for each increment can be felt as they "click in place." These are often used for select actions (e.g., channel, track, effects). The other group rotates continuously, which means they do not have a fixed stopping point other than the max and min values. The continuous knobs seem to control parameters that can require gradual changes and detailed adjustments, such as EQ, filters, and gain.
Figure 4.2: A textile sensor metaphor of the rotation knobs
The metaphor for the rotation knob can be seen in fig. 4.2. The sensor has kept the unlimited rotation angle and discrete values. It consists of seven squares of conductive fabric and an arm with a conductive underlayer. When I was planning to make this sensor, I intended to put a coin inside of the arm, in order to weight the tip down and sequre the contact point without locking it permanently. As discussed in the first capacitive sensors. Then there was conductance between the channels, their values peaked. For this sensor, the intention was to use that error as a benefit. Furthermore, this sensor is the only sensor in this thesis that holds it values after a change.
Additionally, the platter on the CDJs and the turntables have charac- teristics that puts them under the rotary knob section. The platter is the
part on a vinyl player that spins and supports the record. As mentioned in the section about the linear components, The BPM slider adjusts the speed platter’s rotation. However, nudging the platter can briefly increase or de- crease the speed. This action helps fine-tune the synchronization of the different tracks.
In this metaphor represents the gesture rather than the control unit.
The gestures that adjust the BPM with the platter are usually short and jerky. This idea lead me to sew a button to a stabilized fabric and attach a conductive tape strip under it. Then, I explored how far the button could rotate before snapping back to its original positin. These limits were used to estimate the placement of two additional capacitive tape strips on the fabric. If these two pieces of tape were connected to a capacitive channel, they could detect nudge that rotated the button both directions. fig. 4.3
Figure 4.3: The structure of the button platter
Figure 4.4: The button platter prototype
(a) Continuous slider (b) Switch Figure 4.5: Linear components
4.4 Linear components
The sliders can be split into two groups. First are the continuous sliders (see fig. 4.5a) In all of the three interfaces combined, all of the continuous sliders either faded the volume or adjusted the BPM.
The other group was the switches which usually had a range of two or three options (see fig. 4.5b). The majority of the switches were dedicated to functions that let the DJ customize the mixer to their preferences (e.g., assign channels to the crossfader and assign curve to the faders). If these parameters are adjusted, this usually happens before or early in the set, which likely explains why they are flat and considerably smaller than the other components.
Similar to the rotary buttons it seems that the continuous control units are commonly mapped to parameters that require precise control, while the discrete units affects different modes.
The zipper potentiometer could be a suitable metaphor for how these control units. Additionally, the linear control gestures also includes stroke sensors, such as the stroke sensors were made for the case study.
During the examination of these control units, I also experimented with the modulation wheel that can be found in synths. Although they are not used in any of the DJ controllers that have been mentioned, they are linear controllers for music control.
The process for this metaphors started by exploring the shape of the modulation wheel and how this can be transfered using textile materials.
The shape was supported by a pocket for a plastic boning that is usally used for garments like the corset. And the top surface was sewn with two layers so their space could be used as a pocket to hold the structuring element in place. However, when I tried to interact with the patch, the structure tended to drift to the sides. Additionally, upon press, it gave in by rotating the boning, instead of going in the same direction as the finger. This is illustrated in fig. 4.6.
4.5 Buttons
The buttons consisted of three attributes: size, hardness, and visual feedback. All buttons bounce back to their original position after a press.
Figure 4.6: The metaphor for the modulation wheel
Figure 4.7: Buttons
The toggle buttons use visual feedback in the form of light and color to separate its states.
A second discovery that stands out is that the most frequently used buttons are the largest in size (such as cue and play/pause).
With three exceptions, all of the buttons are elevated. The exceptions control USB and CD ejection and cue point deletion. Accidentally pressing these first two would disconnect the music source.
The capacitive surfaces on the first sensors in chapter 3 are some examples of textile metaphors of buttons. However, these are not elevated.
Elevating fabric can be done by adding darts. In dressmaking, darts are sewn triangular tucks that transform flat patterns into a three dimensional garment. There are examples of this technique in [41]. If darts were used to get an elevated button, the height of the elevation could be increased by putting the apex further away from the edge.
If the buttons react when the top layer comes in contact with the bottom layer, then there should be a material that separates the layers.
Neoprene is a commonly used filler in e-textile buttons. Since neoprene is not conductive, there has to be cut holes to allow contact between the layers upon pressure. Alternatives to neoprene might have been an anti- slip mat, cotton fibre fill, a matrix of multi-layered wondertape and maybe even textured embroidery and faux fur.
—————————————————–
4.6 The capacitive break-out board
In the midst of the work with metaphors, I was informed of an e-textile break-out board: the Trill Craft from Bela. The break-out board can extend the number of input pins with additional 30 capacitive channels. The implementation of the board opened up possibilities that had previously not been achievable. In the following sections, I will present one sensor that could not have been made with the break-out board.
Preparatory Work
The break-out board was hand-sewn attached to the base by conductive thread. The end of each path were stitched multiple times on a square of the conductive tape. Then conductive snap buttons were placed on top of these points. The snap buttons facilitated for quick connections with different sensors. Although this approach require more materials that cannot be reused, it allowed more flexibility when working with different versions of the prototypes.
The first nine pads of the Trill Craft was tested without sensors in order to see the behaviour of the pins. To do this, the Arduino compatible library for Trill was used. After repeated measures, the channels closest to the I2C pads showed significantly higher values.
The Trill library allows the break-out board to be read in a differential mode. This mode stores the baseline values upon start and subtracts these values from the raw values. The output values will therefore be set to zero when no pins are activated. Despite this calibration, the possible max values would still be reflect this dissimilarity.
The results made me question how useful the pins could be for a multi- channeled sensor for capacitive sensing.
In order to combat this difference, I made an attempt at insulating the threads in this area. Insulating the stitches with satin stitches alone gave showed little improvement. Interestingly, there were cases where experiments with electrical tape lowered the baseline values. The seventh attempt showed that the average values of six channels has a average difference of -50.98 (stdev approx ⇡ 28.61), while the remaining four channels has an average increase of⇡27.9 (stdev⇡16.75).
Although the use of electrical tape demonstrates that insulation can improve the baseline values, the solution was temporary since it was not sewable.
In the work with the following sensor, I worked around this issue by using the channels for capacitive touch sensing (on and off). As with the capacitive sensing, the issue was avoided by connecting the sensor to a pin with lower baseline values.
4.6.1 The Biano sensor
The idea for the Biano came to mind when I was learning the major scales on the bass. While I tried to memorize the fret and string patterns, I
(a) The Biano’s path arrangement (b) The Biano sensor with cover was annoyed by how unintuitive the interface was. It is, however, worth mentioning that my mental image of tone intervals and chords has been influenced by the arrangement of the keys on a piano. I realized that this annoyance is likely to go both ways and that the piano interface was not perfect. In some sense, we might argue that the bass interface displays the tones in a more impartial manner. Transposing a scale can be done by moving the grip along the neck of the bass and follow the same intervals for the original melody.
In contrast, the piano interface can seem more disorienting due to its key pattern. Although the visual arrangement of the octaves make them easy to recognize, I would claim the interface can complicate transpositions and encourage certain players to remember key signatures by visual memory instead of as tonal relations.
What follows is a sensor inspired by the frets and strings of a bass guitar and the octaves of a piano, namely: The Biano (section 4.6.1).
The interface consists of three windows where each represent a string on the bass guitar. The pitch increases chromatically during horizontal control gestures from left to right. The piano’s influence came into play by having an octave’s equivalent of semitones in each window and by separating the possible tones in the windows with an octave. The latter was a firmware decision and could easily have been changed to mirror the adjacent strings in a bass (where there usually is a five semitones difference between the strings).
However, separating the windows with an octave gave the widest range of chromatic tones for this particular design. The range is 36 pitches (i.e., three octaves with the chromatic scale) and the output is determined by the combination of the activated pins. In the firmware, each note is defined by the channels in three categories. These groups will be presented in the order they are placed on the fabric.
The Layers
In the first group, there are three channels that are metaphors for the bass strings. These are sewn horizontally in a square-wave shape. To avoid conductance with the other groups, the horizontal lines are covered with a non conductive lining. See fig. 4.9.
(a) The base layer with paths sewn in the shape of a square-wave. Touch- ing one of the paths tells the system which octave to play.
(b) The horizontal lines are then in- sulated with heat adhesive lining
Figure 4.9: The first layers in the Biano sensor
Each channel in the second group represents a tone and are split into two vertical lines. The interval between each tone is two semitones. See fig. 4.10 The stitch length were manually controlled to avoid contanct in the intersections with the octave paths.
The third group represents the sharp (i.e. the "#" in music notation).
This path is connected to one channel, but has branches that run parallel with each second tone-path. See fig. 4.11.
To output sound, the system needed one activated channel from the group of octaves and one from the group of tones. Then the #-channel was checked to see if the tone needed to be transposed a semitone up.
Discussion
To test the sensor, the activated pins triggered their names to be outputted on the Arduino serial monitor. The sensor seemed to provide the expected values with a few exeptions. There was some inconsistent activation of the rightmost side of the sensor. This may be to the outer layer’s thickness around the window and not enough capacitance due to the thin thread. The window was framed by the outer layer, interfacing and the seam allowance of both. The thickness from these four layers were enough to inhibit the finger reaching the twelfth notes.
While interacting with the sensor, the horizontal movements felt unrestricted, but the height of the cover-layer forced the finger to be lifted between the vertical direction. Although this doesn’t have to be seen as a problem, the intention of the cover layer was to provide a finished look as
Figure 4.10: The paths that determines the pitch are sewn as two parallel paths. Both sides are placed adjacent to the octave paths. In addition, the right branch will be combined with the sharp-path that transports the key a semitone up.
well as protecting the circuit, while being fairly unnoticeable.
Figure 4.11: The paths for triggering semitone transpositions are sewed next to the right note-paths.
Figure 4.12: The Biano sensor
Chapter 5
Sensors for Evaluation
Figure 5.1: The modular kit
This chapter covers the the ideas behind and the construction process of the system that was tested by practicing musicians. The sensors were abstracted to swatches, so there could be more focus on the what the sensors did and what they could be instead of presenting a concluded product. This was done by making the system modular, which could
allow the participants to attach, detach and rearrange the sensors to some degree. The following section will cover the motherboard and the process of making it. This part also includes the accompanying accessories and how the signals were managed in the code. Then we will look at the sensors and the design process that lead to them. The complete system is shown in fig. 5.1
5.1 The Modular System
Figure 5.2: The motherboard
Two kits were constructed and each kit consisted of the following:
• a main cloth (the motherboard/mothercloth)
• sensors
• textile wires in different lengths
• textile bridges that could be used to extend the wires
• sensor snaps Motherboard
The motherboard contained two sewable hardware components, namely a microcontroller (nRF52840) and a breakout board (Trill Craft).
The break-out board communication pins were connected to the respective pins on the micro-controller, namely the SDA-, SCL-, 3.3V- and GND pins. The remaining and unused pins on the Trill Craft – Event and Reset pins – were connected to ground on the microcontroller.
Layout
The set consisted of separate modules that could be easily attached and detached with the snap fasteneres and texile wires. This idea was inspired by Cas Holman and her work with Rigamajig, which is a building kit for hands-on free play and learning 1. This "build-for-play"-approach could allow the users to imagine how they want the controller to be structured, assemble it and then use it.
I did not know whether the participant preferred to focus on one sensor, or experiment with several simultaneously. Therefore, instead of imposing an already fixed layout, making the set modular could allow them to customize the layout to their own preferences and mix and match the sensors as they wished.
Before deciding the final shape, I sketched several alternatives for the modules. The overarching concept was to use similar building blocks that fit easily togheter without adding too much bulk under the sensors.
During the skething process, pentagonic shapes and puzzle brick-looking units were considered. Despite these explorations, the final shape ended up as squares and rectangles, as these were more practical to make and ornamented modules was not a goal for the study that followed.
The sensor-modules were 20 cm long and 10 cm high. There were white non conductive sockets on each corner. These allowed the sensors to be physically attached with the sensor snaps. The sensor snaps were square pieces of fabric that contained four plastic press buttons (see fig. 5.3).
Figure 5.3: Sensor snaps
Code
When the microcontroller was powered it stayed idle until it was connected to a device via BlueTooth. A successful connection initiated the calibration
1https://casholman.com/rigamajig/
sequence.
The CapSense was calibrated first, and then the analog sensors. After the calibration, the values were read and immediately mapped to the range 0-127. A change in any of the values, would send a new control message with the respective converted value.
If the participants found the border values (minimum and maximum) to drift, they could press on one of two buttons on the microcontroller to recalibrate the CapSense or the analog sensors. The microcontroller also had a switch that was set to toggle the filters on and off.
Calibration The analog calibration process started with a tone sequence.
When the participants heard this sequence, they were told to use the sensor as firmly as they would use it to get to the maximum value. During an interval of 211ms (2048 ms), the values were compared and the highest value was saved as a global variable. After this, the tone sequence played in the opposite direction, which was a cue for them to let go of the sensor and that the firmware would start to look for the lowest value.
In contrast to the latter approach, the minimum value was calculated by finding the average of the measurements during 211ms. Furthermore, when trauma has been applied to fabric, it takes time before the inner arrangements of the fibres comes back to its baseline structure. Measuring the values in the resting state after the finding the maximum, achieved to reflect that.
The third step was to get the range by subtracting the minimum from the maximum value. These three values were then stored as global variables. This process was repeated four times, one for each analog channel. Finally, after the calibration process, the values were mapped to the MIDI-range.
To counteract this problem, there was added a calibration algorithm.
This would also make it easier to use the sensors in the modular system and allow the participants to connect the sensors to the analog-in channel they preferred.
During these tests, it was difficult to decide the strength of the filter as I did not know if the participants would prefere reactivity over noise reduction. The solution was to let the participants turn on and off the filters with a switch on the microcontroller.
Wires
During the earlier experimentations with the break out board, I had noticed interference in the channels closest to the I2C pins. To reduce this, the silicone insulated wire was used on all connections from the rigid components’ pins.
Due to the close proximity between the capacitive channels, it was not possible to fasten the wires using a sewing machine, therefore the wires were stitched onto the fabric by hand (fig. 5.4).
The wire was wrapped two times through each channel. To avoid accidental contact between the channels, the wire’s insulation had to be