Six disciplines for designing sound, motion and space.
What do we define, what do we change, and how do we evaluate it by listening? We explore six disciplines through their design objects, variables, musical uses, creative examples and evaluation methods.
The creative examples are designs for further experiments. Each discipline ends with a description of its current implementation and research questions.
01 / SPACE
Space
We treat space as a set of variables for composing relationships, extending its role beyond placing sounds. Sources, groups, regions, listening positions and output destinations are described separately, so we can examine which changes affect musical structure.
Describe positions and relationships separately
Three axes describe position, with separate definitions for a source trajectory, a group centre and a region centre. Moving a source and moving a surrounding region are different operations, allowing the same trajectory to be heard under different conditions.
Rules can be evaluated in an object’s local coordinates or after group and spatial transformations. This allows a comparison between preserving a timbral arrangement as a group rotates and moving that group through a fixed timbre field.
Place sonic rules in regions
Regions can be represented by spheres, boxes, planes or grids recording distance to a boundary. Their centres, orientations, sizes and transition widths are design variables. An effect can increase as a source enters a region, while entry and exit are recorded as events.
Overlapping regions have explicit combination rules: taking the stronger influence, adding influences, selecting by priority or blending. Because changing the overlap rule changes the result, it is saved as part of the composition.
Connect sources to outputs
A sound source is not identified with a single speaker. Its signal is distributed using weights based on distances to output points. Source IDs and composition rules remain distinct from the output layout.
Musical uses include closely grouped voices, separated responses and sounds handed between regions. Numerical distance need not equal perceived distance. Stereo audition and evaluation across multiple outputs are treated separately.
What to compare
Keep trajectories and note events fixed, then change either a region’s position or the output layout. Examine voice separation, continuity at boundaries and reproducibility on returning to the same location. Listening position and level are recorded as comparison conditions.
CREATIVE EXAMPLE
moving a source and a region
Move a steady sound along an orbit while keeping a timbre field fixed. Record where the source passes and how its timbre changes.
Next, hold the source still and move the field. Listen separately for source movement and timbral change.
Combine both motions and vary their speeds. Save the conditions and include comparisons made without the visual display.
Current implementation
The production environment implements coordinate and group transformations, moving regions, distance-based influence, entry and exit records, and distribution to output points. It is not a complete model of sound fields or hearing.
Research questions
Further research will examine how region arrangements affect grouping and shifts of attention. Comparisons across listening positions will help distinguish layout-dependent results from compositional rules that transfer to other arrangements.
We design a source as a unit with pitch, timbre, onset, modulation and position. By changing how a consistent synthesis method connects to space, we aim to build instruments whose sonic changes can be traced to their causes.
The basic waveforms are sine, triangle, sawtooth and square. Their proportions are mixed alongside noise level, filter cutoff and resonance. Waveform mixture and pitch are separate, so timbre can change while a melody stays the same.
A sound’s level over time is described by attack, decay, sustain and release. A brief, point-like sound and a sustained sound offer different cues along the same trajectory. This difference is another variable to compare.
Arrange timbre in space
A timbre field contains anchors with waveform mixtures and filter values. Nearby anchors are interpolated to calculate the timbre at a source’s position, including the transitions encountered while moving between settings.
A faster source passes through the same timbral region in less time. Speed affects timbral change through position in this case. There is no automatic rule converting velocity into pitch or adding a Doppler effect.
Combine modulation and motion
Two periodic modulators can vary pitch, filter cutoff, level and other parameters. Matching their periods to a trajectory, or setting them slightly apart, offers ways to make the points of alignment change across repetitions.
Synthesized sounds carry IDs, keeping originating sources distinct from delayed or granular descendants. Changes made in synthesis can be examined separately from changes made by spatial distribution or effects, adding complexity in stages.
Evaluate timbral change
Changing waveform, level and filter together makes causes difficult to separate. Vary them individually and inspect signal level, abrupt discontinuities and voice limits. Listening comparisons need conditions that do not confuse level differences with timbral preference.
CREATIVE EXAMPLE
read one trajectory through different timbres
Fix pitch, note timing and trajectory. Use a constant timbre without a field as the reference condition.
Vary only the waveform mixture with position. Then fix the mixture and vary only the filter.
Save a combined condition as well, and compare which parts of the trajectory can be identified through timbre.
Current implementation
The production environment supports up to 16 voices with four-waveform mixing, FM, additive and decaying 16-partial banks, ring modulation and AM. The decaying bank uses sine oscillators that fade after note-on; it is not a physical resonator model. Noise, amplitude envelopes, filtering, two periodic modulators and position-based timbre interpolation are also available. Pitch, level and synthesis-parameter changes are smoothed in the audio processor.
Research questions
Further research will examine whether listeners can learn a position–timbre relationship and use it with a different trajectory. Timbral range, movement speed and repetition count will be varied separately to explore useful ranges for an instrument.
Motion is treated as material for composing the timing and relationships of voices, extending beyond visual decoration. Individual trajectories, group transformations, spatial warps and time mappings remain separate, so their effects on listening can be traced.
Fixed positions, straight paths, pass-bys, orbits, pendulums and spirals are described through starting positions, radius, speed, phase and duration. Two sounds following the same shape can meet at different places and intervals when their start times or phases differ.
A motion name does not by itself specify a sonic change: we check whether it affects position, gain or both. Repeating a sound at a fixed location is distinct from moving it around an orbit. Movement and rhythm can be designed independently.
Layer objects, groups and space
A group can rotate, drift or scale while retaining the trajectories of its sources. Twisting, bending or mirroring can then transform space. Separating individual voices from group motion allows local repetitions to coexist with larger changes.
Transformation order is part of the result. Rotating before translating produces a different path from translating before rotating. The order is saved; comparisons involving swarms or interactions also fix initial states and random seeds.
Motion time and audio time
Time used to read a trajectory can be scaled, looped, held or reversed. Reading a path backwards is distinct from playing captured audio backwards. The time domain affected by an operation is made explicit.
Reference time advances from the audio processor, while display refresh is used for presentation. Swarms and other stateful models also use fixed calculation steps. Visual smoothness is checked separately from the timing of sound and events.
Can a trajectory be heard?
Using the same sound material, compare a fixed source, a single orbit and crossing voices. Change speed, phase offset and note spacing individually to examine voice tracking and grouping. Comparisons without visuals help distinguish seen motion from heard motion.
CREATIVE EXAMPLE
form voices from two orbits
Use two sounds with the same orbital radius and speed, offset in phase. Keep pitch and level fixed while listening to their relationship.
Reverse one direction and compare meeting points and passage intervals. Then rotate both sounds as a single group.
Save the motion settings and compare a condition with a timbre field. Record trajectory differences separately from timbral differences.
Current implementation
The production environment implements 24 basic motions, group and spatial transformations, time mappings and boundary handling. Backward seeking and repeatability under the same conditions have been tested. Swarms and material-like gestures are treated as numerical models for composition.
Research questions
Further research will compare how trajectory shape, speed and crossings affect voice separation and perceived tension. We will examine which cues survive changes in playback environment or listening position, relating the motion vocabulary to perceptual evaluation.
Composition combines pitch and duration with place, movement and relations between voices. Alongside when and where a sound occurs, we describe how one event changes the conditions for the next, asking when those changes become an audible musical structure.
From notes to related events
A voice has a musical figure, position, trajectory, start time and lifetime. Handoffs and branches retain a parent relationship; interactions can carry a coupling strength and a delay. When a figure moves to another location, its relation to the preceding voice remains traceable.
Stored source identities and states let us return to a point in a composition and compare conditions. When a study uses randomness, a fixed seed helps separate the effect of a changed rule from variation caused by a different random sequence.
Adjusting relations between voices
Variables include onset intervals, phase offsets, response latency, branching, rests and coupling strength. Two voices may travel in opposite directions while retaining simultaneous onsets. Varying spatial separation and temporal alignment independently lets us investigate how voices form a group.
Making history affect the next sound
A passage can alter a later route; an unused connection can weaken; an absent cue can change the attention allocated to a later event. Branching from a shared pulse in Room A, persistent paths in Room B and changing responses in Room C are themes of composition studies that explore these relations.
Developing a form from one condition
A form can establish a shared figure, change one relationship, then allow time for its consequences to persist. Comparisons keep the sound material and input sequence fixed while changing one condition, such as whether history is retained. Planned observations include which voice listeners can follow, whether they can describe the changed rule and how level differences affect their judgments.
CREATIVE EXAMPLE
a handoff between four voices
Present a newly authored three-note figure at four invented positions in sequence. Keep pitch and onset spacing fixed, allowing time to become familiar with the handoff order.
Delay one connection. From the same starting state, create one condition in which the delay propagates to later voices and another in which each voice retains its original schedule.
Repeat both conditions with a short pause, reversing their presentation order. Examine whether listeners can identify where the delay began and describe the coherence of the whole.
Current implementation
The production environment implements state save and recall, ordered transformations, deterministic history models and studies with comparison conditions. History and prediction are computed with small synthetic state models. The four-voice example above proposes a composition method.
Research questions
Future work will examine the density and duration at which relational changes remain audible, and how much a voice can change while retaining its identity. Computational reproducibility and the perception of musical relationships require separate evaluation.
Effects can transform a sound or produce another sound at a new location. We design where delayed, sustained or spectrally separated signals appear, how long they last and how they relate to their source.
The same delay can repeat at its source position or pass to another location. Delay time, decay, repetition and destination become separate variables. Keeping the relationship to the parent source lets us trace which sound produced a particular response.
More particles on a display do not establish that more sound is being produced. Signals sent to independent positions carry actual PCM audio. Measurements distinguish the original source, the processed signal and the final output.
Changing an effect through spatial crossings
A sphere, box or other region can define an effect amount and a gradual boundary. Approaching a region can increase reverberation; a crossing can trigger a capture. Overlapping regions use explicit combination rules, such as addition or priority, and transitions are checked for abrupt changes.
Turning time and frequency into additional voices
Grain duration, density, playback rate and capture interval provide ways to explore a transition from discrete events to sustained sound. Sending low, middle and high bands to different positions, or separating attacks from sustained content, changes how much of the original contour remains intact. These splits are checked against the actual signal.
Comparing level and event density
Changing the source-to-effect balance, decay time and spatial spread together makes the cause of a difference hard to identify. Comparisons vary one condition at a time and inspect output peak and level. Planned listening questions ask whether an effect is heard as the source’s tail or as another voice, including a condition without visual guidance.
CREATIVE EXAMPLE
one strike, three responses
Present a new short synthesized sound without effects. Set its level and onset spacing, leaving enough time for the later responses to remain distinct from the next event.
Use that sound to create a repeat at the source, a repeat elsewhere and a dispersed reverberant tail. Present each separately to distinguish the roles of delay and spatial spread.
Audition the source and processed signals separately, then together. Compare their measurements with listening judgments to investigate when one sound begins to be perceived as several voices.
Current implementation
Implemented mechanisms include PCM delay, granular processing, time-domain capture and repetition, band splitting, delay-network reverberation and decorrelation. FFT-based spectral freezing and convolution with measured impulse responses are not implemented. The three-condition example above proposes a production method.
Research questions
Future studies will examine when delay and spatial separation produce an independent voice, and how much decorrelation permits a direction to remain traceable. Signal-processing accuracy, perceived spread and the reflections of a physical venue require separate evaluation.
Experience design considers when a person arrives, what they notice and how long they stay, alongside where sounds occur. The relation between sound and image, time spent reading and the timing of interactions all affect whether a change in relationships can be followed.
Entering, noticing and listening again
An experience can establish a figure that is easy to follow, introduce one change, then repeat it for comparison. Chapter duration, event density and pauses determine how much time is available to notice a relationship. Repeated cues may also help a listener who joins partway through.
Silence can preserve an impression of the preceding sound or signal a boundary. A tail that continues after a source stops has a different role from a pause between sections. The design also considers whether a cue is needed when sound resumes.
Showing the shared cause of sound and image
Sound and image share source identity, time, position and state, allowing a change in sound to be traced to an event or movement. Scheduled activity, actual signal energy and final output level are distinct quantities. Visual brightness needs a consistent meaning so that a graphic does not imply sound that is absent.
Treating interaction and listening position as conditions
Playback and stop controls should be clear, with an opportunity to adjust level before listening. Comparisons limit the controls that change and provide a way to restore the same state. Planned venue studies distinguish central and peripheral positions, moving and stationary listening, and differences in visibility caused by orientation or other visitors.
Asking specifically what was perceived
Alongside an overall impression, planned questions ask which voice a listener followed, what changed and which event they expected next. Conditions can vary visual guidance, presentation order and whether explanatory text has been read. Familiarity, preference and listening environment should be recorded, with conclusions kept within the limits of the participant group.
CREATIVE EXAMPLE
comparing a cue and its response
Author a new cue and two responses, presenting them as audio only. Begin with a fixed response order and allow enough time to identify which response arrived first.
Add an explanatory view generated from the same events. Preserve the audio and visual conditions, then compare whether the view changes judgments of order or the focus of attention.
After a short pause, change only the response delay and present it again. Restore the original condition for repeated comparisons, exploring how to record the noticed change and confidence in that judgment.
Current implementation
The production environment provides transport and seeking, chapters and pauses, saved state, actual PCM measurements and AV studies driven by a shared record. The public page presents recordings of the local application with stereo auditions of an illustrative space. The listening example above proposes an evaluation procedure; it does not report experimental results.
Research questions
Listening evaluation remains future research. Reproducible data, device behavior, venue acoustics and audiovisual synchronization, and human understanding will be assessed separately, informing the different temporal and relational themes explored in Room A/B/C.
This proposed 60-second piece is an exercise in working from a shared description. Introduce one recognizable change before combining several, and retain a way to return to the comparison.
Establish a reference
Repeat one short sound at a fixed position. Set timbre, level and onset interval to establish a listening reference.
Change position alone
Move the same sound in an orbit with a fixed period. Examine how motion changes the listening experience with the source, listener and output model explicitly defined.
Add relationships
Add a second voice moving in the opposite direction and a delayed sound. Establish the trajectory relationship, then adjust timbral contrast, delay and decay.
Leave space for reflection
Stop new onsets and listen as delayed sounds disappear. Return to silence, then compare the fixed, orbiting and layered conditions in notes and recordings.
Keep a reproducible record
Save the scene, random seed, source IDs, output order, gains, timing and listening position. Compare conditions with controlled presentation order and levels, rather than treating one impression as a conclusion. For Room A/B/C, preserve each room’s mechanisms and unresolved requirements while building distinct works on the shared foundation.