Week 12 – Final Project Proposal

Late Hour Steer

Concept

I adore music, and I truly think that the art of making and producing music is something magical. I always listen to different kinds of music, and when I listen to jazz specifically, I think about how cool it would be if I could make music like that.
I don’t know how to play instruments, but sometimes the desire to make, adjust, produce something sounding so beautiful strikes me.
Looking at different animations, cinematics, and paintings makes me feel exactly the same.
This is how my idea emerged: I want to make an Arduino hardware controller pad paired with a generative audio-visual system in p5, where physical input through knobs and buttons shape a particle world that produces atmospheric jazz sound in real time.

Some of the core traits I want to maintain in this work are:

• • The user isn’t playing note by note, and they neither record anything. What they do is steer the autonomous system that can exist and work on its own by always producing sound and image
  • Rather than “performing” type of interaction, this is closer to “conducting”: the user shapes conditions, and the system acts within them
  • The overall vibe and aesthetic I want to maintain is moody, slow, noir-jazz, dark-tech style. Music references to Bohren & der Club of Gore or something close, and visual references to Y2K Sony industrial design, and physical instruments like the OP-1
  • The core of the idea is that both generative parts (audio and visuals) are controlled by physical tactile control, neither of them is most important. One of my main goals is to build system where both parts are equally important and depend on each other but can be autonomous at the same time

Interaction

The way I see the interaction is the following:
As user approaches the small panel, laptop screen can still show a dark, moving visual world that produces sound and visuals. The audio playing should be some slow, almost ambient, jazz that stays neutral if nothing is happening in the hardware. As the user turns the knob or presses a button, both the visuals and the sound will shift accordingly: is the “tension” knob raised, the sound becomes more dissonant and the the visuals become tighter.

Some technical parts of the interaction are the following:

• • Arduino reads knobs and buttons states somewhere around ~30 times per second
  • p5 receives this data via p5.serialport library (most probably), updates a shared state object, and then runs both audio and visual engine based on this state
  • Audio will play via laptop speakers, and visuals will render on screen. Some LEDs integration right in the panel to show the state of the knobs/buttons can be implemented, but I’m not fully sure about it yet

In terms of design, I want to maintain the following:

• • Macro controls and not direct micro controls: each knob and button affects multiple parts of the system and not just one minor thing
  • Both audio and visuals will depend on one state, so the system will have one “brain” every other “organ” will refer to in order to act
  • The system is autonomous, so even with no input, it will run on its own, idle behavior will be implemented as part of the design

Technical Implementation

Arduino:

• • Microcontroller: Arduino UNO
  • Components on panel:
    • 3-4 potentiometers: continuous parameters (density, tension, register/tone, etc.)
    • 4-6 buttons: discrete actions (mode change, reset, accent triggers, acceleration)
    • **1-2 indicator LEDs: mode/state feedback on the panel (up to consider)
  • Components hidden under the panel:
    • Breadboard with all wiring
    • Wires running from panel components to breadboard
  • The panel: cardboard (or wood if possible) as the surface, and breadboard mounted underneath
  • The design aesthetic: clear tech, dev-kit with labeled controls, clean layout, visible&intentional construction

p5.js:

• • Architecture: a single shared state object holds all current parameters: both the audio engine and the visual engine read from it
  • Serial communication: p5.serialport library (?) receives strings from Arduino, parses them, smooths the values, updates state
  • Generative engine: particle system
    • Particles drift in 2D space
    • Particles have properties (size, speed, color) influenced by state (impacted via physical input)
    • Collisions, threshold crossings, or other particle events trigger audio events
    • Behavior parameters (count, attraction/repulsion, speed) controlled by state
  • Audio engine: using p5.sound
    • Multiple voices: a sustained drone layer, a triggered note layer, possibly a percussive layer
    • All notes constrained to a single scale (Dorian or natural minor) to allow randomness only within this constraint
    • Regulated timing: events trigger on a slow tempo grid (~60-80 BPM) so rhythm feels intentional
    • Sound design: detuned sine/triangle oscillators, lowpass filter, reverb to maintain the dark jazz aesthetic
    • I need to research more on the sound design and p5 sound library

References

• • Sound: Brian Eno (Music for Airports), Tim Hecker, William Basinski, Bohren & der Club of Gore
  • Visual: Lacuna’s interface aesthetic, Signalis, Y2K-era Sony industrial design, OP-1
    

    

    

• • Hardware: Teenage Engineering Pocket Operators, Monome Norns
    

    

Risks

There are quite a lot of risks I need to mitigate in order for this project to not sound like random sounds but a cohesive melody with the intendent mood

• • Serial latency or audio glitches
    • Prototype the serial connection and audio engine first and then move to the design
  • Generative system feels random rather than intentional
    • Scale constraints, quantized rhythm, careful voice design — needs additional research on music design, p5 sound and overall music rules I might not be familiar with + a lot of tuning and testing needed
  • Knob mappings feel unintuitive
    • Spend some time on parameter tuning and intentional design of the panel: first sketch it, prototype the best and ask family/friends for feedback to choose the final version
  • Physical build looks rough
    • Design the panel layout on paper first, source proper knob caps and clean labels, hide the breadboard; Maybe consider painting some materials, asking dad for some wood-cutting help or consider using ready for implementation materials (old plastic boxes as a panel) — needs some brainstorm

Overall Review

I believe this project can become a really cool one if I spend enough time on music tuning and design prototyping, as well as on doing some proper research both on the aesthetic and how to maintain it with the available materials for now.
I will start off with some simple pseudo-code algorithms to structure the code I will need to write, and sketching the visual generative system and the physical panel. Then, I’ll move onto the coding, and finally, when everything will work properly (hopefully) I will build the design of the panel itself and try to organize everything so it looks good.