For my final project, I am creating a 3D physical music interface that uses Arduino and p5.js to produce both sound and visual feedback in real time. The interface will be built from cardboard and tinfoil touchpads connected to an Arduino. Each physical input will trigger 3D animations on screen and simultaneously play musical notes or sound effects through p5.js’s sound library.
The goal is to design an experience where physical gestures directly generate both visual creativity and musical expression, offering a playful and intuitive interaction. Touching different areas results in unique audiovisual combinations, allowing users to “perform” across both sound and visual mediums.
Inspired by projects like Browser Synth, Interactive Soundboard with p5.js, and Visual Music Keyboards (examples attached), my final project will explore how homemade interfaces can connect physical creativity to browser-based multimedia environments.
Implementation:
-
Arduino: Detecting touch interactions
-
p5.js:
-
Processing serial input from Arduino
-
Triggering real-time 3D graphics (WEBGL)
-
Playing sound using the p5.sound library
-
Vision:
For this project, my vision is to create an intuitive and playful instrument that connects physical touch to both sound and visual expression. I want the final product to feel immediate and responsive so that when a user touches different parts of the physical interface (buttons/controllers), they should instantly hear a corresponding musical note and see a visual reaction on the screen. My goal is to ensure that the interaction feels clear and easy to discover, even for someone using it for the first time, while also offering enough variety for users to improvise and experiment with the music instrument. Consistency and reliability are also important: every touch should reliably trigger a specific, predictable response without delay.
