Conclusion: Ontic engineering of musical objects, processes and worlds

Chapter 2 – Processes and Objects – Musical intelligence and representations

6. Conclusion: Ontic engineering of musical objects, processes and worlds

In ch2 we naturalized the sources of our inborn musicality [2.1], restrained the powers of sector-building proficiencies and recommended “full-cycling” musicianship [2.2]. Finally, we looked at representational issues in both machines [2.3] and humans [2.4, 2.5]. It is now time to bundle the threads from ch1 about algorithms with the objects from representations. Algorithm-like schemes of composers will become full-blown computational algorithms in a compositional agent that uses percepts from environments to act on its environment by responding to and emitting musically meaningful structures, structures we ultimately may call compositions (AIMA, general agent model,

fig.2.1, 32; below).

Such an agent will have both goal-based and utility-based to facilitate an optimal blend between autonomous (“happiness”) and heteronomous goals of artistic scope.

To achieve these ends we must engineer an ontology of objects and processes for the virtual worlds compositional agents will have to reside in. Objects may be MIDI-messages or affective topological atoms; processes may be the generative principles, reverse engineered from the generative theory of the tonal music by Lerdahl/Jackendoff, or any inductive or learning algorithms meant to be

executed on one of the in 2.3 mentioned machine architectures. Environments may be open or closed.

The “psychological model” for our agent should be based on constructivism for its innovative or creative power (constructing new world perspectives), on cognitivism for its variational reason (rational and planning potentials), and finally behaviorism for its reproductive potentials (learning and pattern based logic). These three models are reflected in the three “master modes” of musical creativity and learning (learning cycle!, 2.2), namely contrast, variation and repetition.²⁶³ This trinity of expressional laws will be the driving motor for algorithmical logic in Machine Composition.

Otherwise, there should not be 'a priori' frameworks to delimit, inadvertantly or conventionally, the constructive approaches of Machine compositional agents. It will be their success and not there strategic choices under the building processes that decide their survival and hence truthfulness to their world model. This is in alignment with Dennett's generalized concept of design, where:

...my examples in this chapter have wandered back and forth between the domain of organisms or biological design, on the other hand, and the domain of human artifacts – books, problems solved engineering triumphs on the other ... There is only one Design Space, and everything actual in it is united with everything else.²⁶⁴

We may add 'musical compositions' to the list of human artifacts, that are proposed in the General Design Space, in analogy to the search spaces of 1.3, and that alouds “authorized configurations” to emerge as solutions after the necessary number of trials have perished in this “space” of designs.

Such systems will also have to modularize into hierarchic and non-hierarchic societies of composers (Society of Minds, Marvin Minsky), e.g. modeling the evolutionary strategies of Peretz modular model from 1.1. But equally relevant are modules of reflective functionalities describing the three master modes of expressing, mentioned several times already: contrast, variation and repetition.

An important bundle of problems related to design, will be the various challenges in mapping sound features to higher features, and sound features to control parameters.

Bio-inspired systems²⁶⁵ with non-neglectable unpredictableness²⁶⁶, are the horizon of Machine Composition. In addition, there will be rational assistents to composers that will use such tools, that are built on tools for building them in the first place. We will have to build on optimism too, in order to to aloud for some degree of the Sourcerer's apprenticeship in our new tools of

compositional creation. In ch7, we will even look at what was called Frankensteinian models of Composition. We might with time see the growth of affective and other higher-level properties inside MC systems as well.

As to the question of which epistemological choices one should proceed with, we may look for

naturalized epistemologies, i.e. expecting answers from theoretical psychology²⁶⁷ or in the present case from evolutionary musicology. For constructive purposes, we might content ourselves with a procedural epistemology, a non-commiting and pragmatic stance for the ontic design and

engineering. Such an attitude asks how-questions instead of why-questions during the development of musical objects, algorithmic processes and environmental “worlds”. Abelson and Sussman:

Computer revolution is a revolution in the way we think and the way we express what we think. The essence of this change is the emergence ... of the structure of knowledge from an imperative point of view, as opposed to the more declarative point of view taken by classical mathematical subjects.²⁶⁸

The first two chapters attempted to provide

• clarifying concepts and a procedural sharpening of our tool boxes [ch1] and

• a view of musical composition in integrative perspectives (both in the evolutionary time scales as well as learning time scales, ch2) and

• a more balanced and unified approach to human activities as a whole (i.e. viewing creative activities as normal and naturalized) incorporating both instrumental and expressive/artistic cultures as recombining and

meaningful parts of life.

Hopefully, this may contribute to the liberation of the powers of “engineering imagination” and new man-machine contexts that satisfy our inherited senses and understandings of musical composition, as well as they may build new levels of culturally significant Machine Composition. In ch9 we will revisit and somehow revision the intuitions inspired by these thoughts. The next six chapters will overview some of the most influential examples and applications of Machine Composition in found in academic research today.

Ex nihilo deus → ex nihilo genius → ex nihilo machina ex dubitatio nihil crescit → ex nihilo nihil fit!

Chapter 3 Objects and Messages : Machine composition with MAX

Patches with Max/MSP, Pd and jMax

1. Introduction

2. Early computer music 3. Basics of MAX 4. Types of objects 5. Types of messages

6. Programming methods and approaches 7. Composer objects and examples

8. Max/MSP, Pd, and jMax: dialects of MAX 9. Conclusion: MAX and machine composition

The computer should ideally feel in the musician's hands like a musical instrument, needing only to be tuned up and then played.

(Miller Puckette, 2002²⁶⁹)

3.1. Introduction

In ch1 we looked at common ground of algorithmic thinking in computer programming and music composition. In ch2 we discussed problems related to choosing a representational language for machine composition. In the following I concentrate on MAX as a programming tool for MC. MAX uses MIDI as its primary representation. It was developed at IRCAM (Paris) around 1986 and was in 1991 commercialized by OpCode (since 1999 Cycling74). But first we make an excursion into the historical background of MAX: early 'computer music'.

3.2 Early computer music

Some composers showed early an interest in electronic technology and used tape recorders to manipulate recorded material. Computer composition is generally assumed to have started around 1960 with Mathew's general-purpose music programs at Bell Laboratories. Computers worked extremely slow from our perspective, and it therefore required both time and patience to get results.

Nevertheless, some composers enjoyed the precise control over many aspects of the sound result.

Still, by substituting human performers on a stage with tape players one impoverished the experiential content of concerts. Audiences often got “disconnected” and failed to see a human dimension. A tape as performer may satisfy the composers intentions more than the expectations of an audience. Live electronic systems seem to close this gap. Typical set-up consisted of modules that delayed tapes or processed sound from acoustic instruments adding thereby effects, such as reverberation, distortion etc. Many early experiments were done in a spirit of control and determinacy. John Cage provoked aesthetic discourse about indeterminacy and improvisation contributing to the growing and ongoing interest in interactivity in musical composition.

“Spontaneous generation”

Romanticism Computer music

“Com plete determ ination”

Aleatorics (eg.Cage))

“Anti-determ ination”

Machine Composition

Interactive system s

Voltage-controlled synthesizers soon became the standard in computer music. Similar to MIDI becoming a de-facto standard for control, any synthesizer module now responded in the same basic way to varying voltages in changing the parameters of emitted sounds. (see also Subotnick²⁷⁰). In the 1970's CM mostly meant sound synthesis and sound processing on mainframe computers. This changed when two important developments fused around 1984: digital technology and MIDI language (International CM conference, 1984). Digital technology made computers and

synthesizers (e.g. Yamaha's DX7) affordable for people outside research labs. The MIDI-protocol (Musical Instrument Digital Interface) became the universally employed method for sending and receiving musical information digitally. When music hardware became digital as well, these technologies furthered flexibility and modularity of systems. Timing problems for parallel streams of notes were now possible to control. Once instrument manufactures had agreed on MIDI,

practitioners and researchers established common software grounds for CM systems. Research at IRCAM (Paris), STEIM (Amsterdam), MIT and Carnegie Mellon was necessary before todays consumer music programs such as sequencers, sound editors, notational programs and interactive software could be developed and standardized (M and Jam Factory, 1986 , ch4).

In the last 15 years many CM-friendly programming environments or systems have been developed.

We shall take a closer look at Cypher, EMI, CommonMusic and others in ch4,5,6,7. For now we look at the perhaps most used platform for MC: MAX.

3.3 Basics of MAX

Miller Puckette created MAX originally as native code control system for IRCAMS 4X

synthesizers. Later it was generalized and adapted to Macintosh by David Zicarelli. Puckette made a version for IRCAMs new Signal Processing Workstation (ISPW). In this MAX-version for NeXT computers audio signal production and processing was now possible as well. The audio-part of the software (FTS) evolved later into the 'creative commons' version of MAX, called "Pure Data" or PD.

PD contains numerous extensions contributed by the user community of MAX, among them interfaces for video programming, 3D-graphics (GEM) and light systems for multimedia applications.

MAX is an object-oriented Fourth-generation language (4GL). Building MAX-patches means building collections of interacting objects (“objects are algorithms that precipitate action”²⁷¹). Each object has a function and depends on its neighboring objects through communication of messages.

Messages are information passing through the objects, received only, stored, answered or passed on unchanged or processed. Messages can be instructions for start or control of processes. MAX is a specific purpose language and does therefore not offer the full flexibility of other fully object-oriented-languages. MAX makes modular programming very easy. Small subroutines (objects) build up bigger subroutines (more objects) and so on. MAX programming is therefore intuitive, bug resistant, reusable, modifiable and as a consequence rather easy to learn.

MAX is an algorithmic and graphical programming environment that is widely used in machine composition circles today. It incorporates a MIDI-interface to facilitate programming with MIDI -sounds. MAX both listens to streams of MIDI-messages (input) and plays MIDI-messages (output) to a synthesizer (software or hardware) on the same computer or an external sound module.

What can we do with MAX? We can make a little game with MAX. MAX listens to small melodies and answers them by playing them back transposed (like an “echo spirituoso”). This is not exactly

x

^listens ^MAX^f ^plays

y

impressing MC, but it illustrates the basic parts of such a system. It involves both transformation and decision-making. How do we solve such a task in MAX? We will fill in the unknowns throughout this chapter.

MAX is an object-oriented system i.e. data are represented as objects that connect in a network (called patch in MAX-lingo referring to the physical wiring and rewiring in analog synthesizer systems). We can group MAX-objects into functional categories. Some objects parse and process input-streams of MIDI-messages (listening objects); others generate or compose sound and send output-streams of MIDI-messages. Other MAX-objects work with notational or graphical

representations (score objects) while performance directed objects increase interactive and interface functionality. Recent versions of MAX add multi-media objects for use with pictures, videos and lights. These extensions transform MAX increasingly into a general-purpose control programming language. But its history and basic approach (with MIDI as fundamental message system) classifies as music composition tool.

In its first versions MAX was limited to handling MIDI-messages. Today, MAX-programmers have tools for working with raw data of sound, permitting manipulation of physical sound structures (at a lower level than MIDI). Using these tools, machine composition can include sound synthesis,

mixing and signal processing. In this chapter we limit ourselves to programming of MIDI-messages;

MAX typically operates in an interactive or interpreter-type programming style. The resulting MAX code is compilable, but the graphical interface with objects/messages and connecting lines invite to continuously test the audible results of patches (in RUN-mode).

Interactivity is a property we typically find in performance and improvisation (even solo impro-vising is a kind of answering ones own questions). But, as we saw in ch2, composition feeds on learned material. More importantly, most serious and interesting works seem to evolve from doubts and reflections similar to generate-and-test principles in creative computer programming [ch1]. In Marvin Minsky's “Society of mind” the complexities of mind are broken down into smaller functions. Using MAX we may break up musicality into constituent parts or functions (objects).

Objects output messages in response to receiving messages. What happens inside objects can be hidden (encapsulated) on the screen, so users can concentrate on what goes on without distraction by how it is implemented. MAXobjects process data i.e. manipulate, generate and store data.

Objects are algorithms that precipitate actions and they listen (observe) and play (act) according to messages.

Graphically, objects and messages (i.e. patch) are displayed in a window by boxes (objects) and lines (connections), messages are not always visible, but can for various reasons be displayed by indicator boxes. Messages originate either spontaneously from inside objects (random), or come from a user (keyboard/mouse or MIDI). Messages then flow downwards in the patch (along lines) depending on the definition of the objects and their connections. Sub patches can be hidden in a sub-window. Programming or editing of patches is done in «Edit mode». Execution of code/objects occurs in locked «Run mode». Easy toggling between modes, makes MAX a highly interpretive

y

x

^listens ^MAX^f ^plays

box1

line1 outlet3

environment. Finished projects can be compiled for efficiency. Programs can be read from graphical representation or textual print of patch definitions.

A first patch: zero-patch

Knowing the different types of objects and messages is important for understanding MAX-patches.

Objects have different numbers of input and output, called inlets and outlets. Some inlets and outlets must be connected, others are optional. Objects without any connections are inactive and hence without meaning. We may call a patch zero-patch when it is the simplest possible or atomic program with information entering and leaving unprocessed (e.g. input-object midiin and output-object midiout). It corresponds to an identity function in mathematics: f(x)=x. Between those two objects there must be at least one line or connection, sending everything from the input-port of the first object to the output-port of the second object.

Using notein and noteout instead we can process pitch and volume separately since the objects split note information into its constituent aspects. The left outlet of notein sends the MIDI-number for its pitch of the notes passing through. The middle and right outlets sends MIDI-numbers for volume and duration. Since the inlets of noteout-objects reflect this layout, we can make a slightly more advanced zero-patch or neutral program, that does not change anything, by connecting the three outlets of notein to corresponding inlets of noteout. As a result, information about notes are now split into three aspects and sent further through its three connections. For each note, noteout now receives three messages, not one as in midiout.

The objects used so far receive and send/transmit messages. Sometimes we need objects for displaying messages and stopping messages. We think of three “empty” 'indicator boxes' between notein and noteout lines. We now have a window into the programs message flow. Any time a note plays (travels through), indicator boxes display from left to right the value for pitch,volume and duration. Values are only visible/stored until the next incoming note initializes these values. Our patch is still neutral, doing nothing other than displaying input, so here is what we got so far: boxes, lines and visible messages in a working and transparent patch.

Processing patch (“note-amplifier”)

We were able to confirm that patches change nothing; zero-patches receive, display and transmit everything entering it at the top. We expand this neutral patch into something that does a little bit more. If we connect empty boxes under the display-boxes and fill '+1' inside them, the patch increases pitch, volume and duration with one step corresponding to one integer in MIDI-values.

That means in practice incoming notes will be played one semi-tone higher, one MIDI-value louder and longer than before. We can again add three indicator-boxes to demonstrate it. Any other

arithmetic operator and value can be used to process input, i.e.change values passing through the patch. We can compare the contents of indicator boxes before and after the processing objects (“+ 1”) to see that all incoming note values increase by one. We might call this patch a “note-amplifier” patch, since each received note is transmitted a little higher, louder and longer. We now comment the patch: comment boxes are objects that show literal and static text explaining the intention of the patch creator.

notein

noteout midiin

midiout

It is easy to see how more complex arithmetic processing will lead to programs of increasing

complexity. Our echo game example from above should involve processing objects that control time delay and decide on the distance of transposition. Another example is a patch that increases

loudness negative proportional to pitch values, i.e. low notes (midi values for pitch) means high volume (midi values for volume). Or any other logical relations between loudness and duration of notes, or any other variables of input. A zero-patch just accepts input and passes on. A processing patch analyzes input or listens and changes/manipulates output according to the intentions in the patch. Generally we may divide patches into negative and positive patches, i.e. patches that filter/scale or subtract information and patches that construct or add information.

Positive patches use incoming information as triggering data, building blocks or variables used for more complex output. Negative patches use incoming information as raw data for more ordered structures. Patches may use non-human data as input for meaningful constructs. This is typical for

“computer music” . MC employs both types of patches, so did and does classical “computer music”.

3.4 Types of objects

MAX objects are graphically represented as boxes in MAX windows displaying patches. In addition to predefined or standard built-in classes of objects, indicator objects and comments, there are a number of interface objects. They give the user direct control during run-time through mouse or keyboard. Examples are toggle-switches, sliders and bangs (emitting «do-it»-messages)

Predefined objects are the heart of MAX. In addition to arithmetic objects, we find objects for storing or recording consecutive data (seq), objects that generate random numbers (random), objects that generate regular streams (metro), objects that delay messages with a variable or fixed time (delay and pipe), filtering objects such as stripnote, unpack and constructing objects such as makenote, pack as well many others. Input/output-objects for other types of MIDI-information deal

In document Machine Composition - Between Lisp and Max : Between AI and Music. Lisp, Max, maxlisp and other recombinations (sider 54-0)