Category: S2025 – Mang

Week 11 – Final Project Idea

For my final project, I am creating a 3D physical music interface that uses Arduino and p5.js to produce both sound and visual feedback in real time. The interface will be built from cardboard and tinfoil touchpads connected to an Arduino. Each physical input will trigger 3D animations on screen and simultaneously play musical notes or sound effects through p5.js’s sound library.

The goal is to design an experience where physical gestures directly generate both visual creativity and musical expression, offering a playful and intuitive interaction. Touching different areas results in unique audiovisual combinations, allowing users to “perform” across both sound and visual mediums.

Inspired by projects like Browser Synth, Interactive Soundboard with p5.js, and Visual Music Keyboards (examples attached), my final project will explore how homemade interfaces can connect physical creativity to browser-based multimedia environments.

Implementation:

  • Arduino: Detecting touch interactions

  • p5.js:

    • Processing serial input from Arduino

    • Triggering real-time 3D graphics (WEBGL)

    • Playing sound using the p5.sound library

Vision:

For this project, my vision is to create an intuitive and playful instrument that connects physical touch to both sound and visual expression. I want the final product to feel immediate and responsive so that when a user touches different parts of the physical interface (buttons/controllers), they should instantly hear a corresponding musical note and see a visual reaction on the screen. My goal is to ensure that the interaction feels clear and easy to discover, even for someone using it for the first time, while also offering enough variety for users to improvise and experiment with the music instrument. Consistency and reliability are also important: every touch should reliably trigger a specific, predictable response without delay.

Introducing p5.js - Building a Generative AI Music Visualizer with JavaScript and React Codédex | Build an Interactive Soundboard with p5.js
Build a Web Audio Synthesizer in the Browser with p5js

Finalized Concept-week12

For my final project, I’m creating an interactive Mood Tree Visualizer—a small Christmas tree decorated with RGB LEDs that change colors based on sound  input. The physical tree will be paired with lava lamp-inspired digital visuals in P5.js, creating a cohesive, immersive experience that blends the physical with the virtual. The LEDs on the tree and the blobs on the screen will respond dynamically to sound levels (like clapping, talking, or music) and mood selections .

This playful project is about visualizing emotions in a cozy, festive, and engaging way, letting users express energy levels through both the physical glow of the tree and the flowing digital visuals.

inspiration:

Arduino : Inputs & Outputs:

sound sensor:Captures ambient sound intensity. Louder sounds trigger more active LED patterns and more dynamic P5.js animations.

  • Calm: Blues/Purples

  • Medium: Greens

  • Excited: Reds/Orange

    P5.js Program Design: Inputs & Outputs

    • Inputs from Arduino:

      • Sound Levels:
        Modulate the size, speed, and behavior of the lava blobs.

      • Mood Selection:
        Switch between color palettes and animation styles that match the LED colors.

or instead I will create My final project is a Mood House Visualizer a small interactive installation where a miniature house lights up in response to sound. Using an Arduino with a microphone sensor, I will control five RGB LEDs embedded because Arduino doesn’t have enough current for more than 5 led lights or I might just add the led lights for each colors and I will make a mini garden it will act the same way the tree should act the difference is going to be in the appearance of the project

Week 11 Project Exercises

 

 

 

Exercise 1.1

let port;
let connectBtn;
let baudrate = 9600;
let lastMessage = "";

function setup() {
  createCanvas(400, 400);
  background(220);

  port = createSerial();

  // in setup, we can open ports we have used previously
  // without user interaction

  let usedPorts = usedSerialPorts();
  if (usedPorts.length > 0) {
    port.open(usedPorts[0], baudrate);
  }

  // any other ports can be opened via a dialog after
  // user interaction (see connectBtnClick below)

  connectBtn = createButton("Connect to Arduino");
  connectBtn.position(80, 200);
  connectBtn.mousePressed(connectBtnClick);

}

function draw() {
  background("255");
  

  
  // Read from the serial port. This non-blocking function
  // returns the complete line until the character or ""
  let str = port.readUntil("\n");
  if (str.length > 0) {
    // Complete line received
    // console.log(str);
    lastMessage = str;
    
  
  }
  
      
  
  // Display the most recent message
  text("Last message: " + lastMessage, 10, height - 20);

  // change button label based on connection status
  if (!port.opened()) {
    connectBtn.html("Connect to Arduino");
  } else {
    connectBtn.html("Disconnect");
  }
  
  
  ellipse(lastMessage,10,20,20)
}

function connectBtnClick() {
  if (!port.opened()) {
    port.open("Arduino", baudrate);
  } else {
    port.close();
  }
}

 

Exercise 1.2

let port;
let connectBtn;
let baudrate = 9600;
function setup() {
  createCanvas(255, 285);
  port = createSerial();
  let usedPorts = usedSerialPorts();
  if (usedPorts.length > 0) {
  port.open(usedPorts[0], baudrate);
 } else {
  connectBtn = createButton("Connect to Serial");
  connectBtn.mousePressed(() => port.open(baudrate));
 }
}
function draw() {
  background(225);
  circle(140,mouseY,25,25):
  let sendtoArduino = String(mouseY) + "\n"
  port.write(sendtoArduino);
}

Exercise 1.3

 

 

int led = 5;
void setup() {
  Serial.begin(9600);
}
 
void loop() {
  if (Serial.available()) {
    int ballState = Serial.parseInt(); 
    if (ballState == 1) {
      digitalWrite(led, HIGH); // ball on ground
    } else {
      digitalWrite(led, LOW); // ball in air or default
     }
  }
  // read the input pin:
  int potentiometer = analogRead(A1);                  
  int mappedPotValue = map(potentiometer, 0, 1023, 0, 900); 
  // print the value to the serial port.
  Serial.println(mappedPotValue);                                            
  
  delay(100);
}

 

 

 

 

Week 11: Design Meets Disability Reading Response

One thing I really liked about Design Meets Disability by Graham Pullin is the way it focuses on reframing disability not as a limitation, but as a creative lens through which to innovate design. Rather than treating these assistive technologies like hearing aids or prosthetic limbs as mere medical interventions meant to blend in or be hidden, Pullin instead emphasizes their potential to become expressions of identity, taste, and personal style. The idea that a prosthetic limb could be as much a fashion statement as a handbag or a pair of shoes completely challenges traditional views of medical devices. This shift from invisibility to visibility, where these aids can be beautiful, bold, and integrated into one’s sense of self, was fascinating for me because there’s often a sense of shame associated with these assistive technologies instead that is now being converted into a sense of pride. It reveals how design can be an empowering force when it treats users as people with preferences and personalities, not just as patients.

Overall, I think Pullin’s work makes a very compelling case for bringing designers and disabled individuals into a closer collaboration, arguing that inclusive design should not be about achieving an average but about expanding the range of what’s possible. By introducing examples like sculptural hearing aids or prosthetics designed with aesthetic flair, Pullin invites us to think beyond function and consider meaning, identity, and even fun. It is a very unique take that kind of steps away from more utilitarian design approaches, and it challenged my own assumptions about what disability-related products are supposed to look like. Thus, Design Meets Disability doesn’t just advocate for better products, it advocates for a broader, more human-centered vision of design itself.

Week 11: Design meets Disability

In Design Meets Disability, Graham Pullin talks about how design and disability don’t have to be separate things. He makes the point that products for people with disabilities are often made just to be functional, not stylish or personal. But he believes design should consider both usefulness and aesthetics. For example, he compares hearing aids to glasses—how glasses have become a fashion item, while hearing aids are still usually hidden. This really made me think about how much design influences how people feel about themselves and their devices.

What stood out to me is how Pullin encourages designers to think differently and challenge old assumptions. Instead of just trying to make something look “normal,” why not celebrate individuality and make things that people are proud to use? It made me realize how much power design has—not just in how things work, but in how they make people feel. Overall, the reading made me think more deeply about what inclusive design really means and how it can actually improve lives in more ways than just solving a problem.

Reading response

Design Meets Disability is a new take on how design can play a much bigger role in the world of assistive technology. Instead of treating medical devices as something to hide, it encourages us to see them as an opportunity for creativity and personal expression. One of Graham’s coolest examples is the leg splint designed by Charles and Ray Eames during World War II—originally meant to help injured soldiers, but it later inspired their famous furniture. It’s a great reminder that functional design can also be beautiful and impactful beyond its original use.

Pullin makes a strong case for reimagining everyday medical devices, like hearing aids and prosthetic limbs, as something more than just tools. Think about glasses: they used to be seen purely as medical necessities, but now they’re a full-on fashion accessory. So why not give the same design attention to other devices? By making them customizable and stylish, users could feel more confident wearing them, and society might begin to see disability in a new, more positive light.

What’s especially great is Pullin’s emphasis on involving people with disabilities in the design process. He believes that assistive tech should reflect the user’s identity, not just meet a need. That mindset leads to more thoughtful, inclusive designs, and it helps everyone, not just those with disabilities. The reading is an inspiring call to designers to be more empathetic, open-minded, and creative in how they think about ability, aesthetics, and innovation.

Week 11 – Reading Response

This week’s reading was very eye-opening, and addressed a number of design aspects that I had never really considered. The title Design Meets Disability left me with a preconception of what that entailed, which was quickly proven wrong. I’ve worn glasses since I was nine years old, so seeing one of the first sections focus on them came as a surprise to me. I am obviously aware of things like contacts and non-prescription lenses, but it was fascinating to look at the shift from medical spectacles to ‘eyeware’ and see how the perception completely changed. The hearing aid section served as a contrast to that change, where the emphasis is on hiding them from view. There have been a number of times where it took me years of knowing someone to ever notice that they wear hearing aids, which speaks to how far the technology has come. Another tidbit was the mention of using pink plastic to try and hide wearable devices like those, which stood out to me at the very beginning of the text and was addressed again here.

I found it harder to identify with the later portions on prosthetic limbs, mostly due to lack of exposure, but I also feel like it is harder to think about in general. Glasses and hearing aids just focus the ambient light/sound to fit your needs, but trying to design a fully-functional arm seems much more complex. The aesthetic side of things is also a bit strange. Instead of being worn on your head and framing your face, they are harder to work with in that they are simply much larger, but can also be obscured by clothing much more easily. The battle between functionality and aesthetic also becomes much more important, with some prosthetics aiming to look as realistic as possible versus using inorganic structures like hooks to maximize their usefulness. From there you can even ask whether we should allow prosthetics to be constrained by the rest of the body, or to instead push into the realm of transhumanism.

Week 11 Reading

In the article, one of the most interesting points made was the how solving these design problems for those with disabilities can benefit everyone by creating better products overall, not just tailored to those with a disability. This really goes against what ideas we would intuitively have about design, creativity and innovation; namely that more is more. But this article argues that working with constraints, in this case, the unique needs of a person with a disablilty, actually leads to more creative outcomes that can branch out into better designs for general use. Namely, the article says working with disabilities in mind pushes designs to be more simple to increase ease of use, to be more discreet and smaller, and generally more usable. This pushes products, in general to also be better and more easily used by those without disabilities, which is the same point of design. Working for disabilities actually pushes design in the same direction it should be going in anyway in the majority of cases.

Week 11 – Reading Response – Design meets Disability

This reading explores how design and disability are intricately related, and how a constraint in designing from disability can lead to greater innovative efforts and contributions. The constraints due to disability basically serve as a catalyst for innovation. The author brings up several examples in history that illustrate the evolution of design with disability. Designing for disability shouldn’t solely focus on the functionality but it should also consider the aesthetics of the product, challenging the stigma around people with disabilities. Every single design, from furniture to eyewear to prosthetics to hearing aids, serve a purpose. That purpose is not only defined by its functionality, but also its aesthetic form. I was very interested in the aesthetics of Aimee Mullins’ prosthetics, and how it combines function, form and aesthetics. I do believe that inclusive designing helps to create a sense of identity and belonging for differently abled people. As a person who wears glasses, I think it is definitely important to consider the design of such products; it truly does give a sense of identity and expression. It is also important to create diverse, inclusive and collaborative work environments that promote the designing for the differently abled. I loved this quote from Steve Jobs mentioned here, “Most people make the mistake of thinking design is what it looks like. That’s not what we think design is. It’s not just what it looks like and feels like. Design is how it works.” This just highlights how all spheres of design must come together for it to be innovative and inclusive; both in functionality and aesthetics. 



Final Project Proposal

Plant Whisperer

I want to make an interactive system that allows a plant to “communicate” its environment and feelings through a digital avatar. Using an Arduino, the project reads light levels and touch interactions, and sends this data to a p5.js sketch. The p5.js interface visualizes the plant’s mood in real time using simple animations, color changes, and sound.

For example, if the plant is in low light, the avatar may appear tired or sleepy, and the RGB LED on the Arduino will turn a soft blue. If the user touches the plant (using a button or capacitive sensor), it responds with a cheerful animation and sound, and the LED may flash in a brighter color.

It’s intended to be playful, meditative, and emotionally engaging—bridging physical and digital experiences in a way that encourages users to care for their environment.