Category: S2025 – Mang

Final Project Preliminary Concept : Drawing With Motion

For my final project, I will design an interactive hand-controlled drawing device that allows users to sketch on a digital canvas by waving their hand in the air. I will use a distance sensor to track the distance of a hand from the sensor. The distance information will be sent to p5.js, which will use iit to manage various features of a virtual brush, like how it draws and how thick the lines will be. Thus, the graphics in p5.js will change instantly according to the user’s hand movements. For instance, as the hand moves closer to the sensor, the brush could create bolder, thicker lines. However, bringing the hand farther away might result in thinner lines. The system allows individuals to “paint” on the screen without using a mouse or stylus at all.

The p5.js sketch on the screen will show a white canvas featuring a circular brush that moves in sync with the user’s hand motions. When the user holds their hand over the sensor, the brush will emerge and begin drawing as they move their hand side to side. To clarify when to begin, the canvas will show a “Place your hand to start” message. As soon as a hand is identified near the sensor, the message will vanish and drawing will start automatically. In addition, it might be possible to add a color wheel for the user to click the desired color when “painting” on p5.

Once the user finishes their drawing, they can easily press a button next to the distance sensor. After pressing, the canvas will pause and show a cheerful message: “Awesome! “Thanks for your artwork!” In a few seconds, the sketch will return to the starting screen, prepared for the next attempt to draw.

I will create an interactive mood light visualizer that responds to both sound and touch. I’ll use an Arduino with a microphone sensor to detect sound levels like claps or voice, and a touch sensor or button to switch between different mood settings. The sound data will be sent from Arduino to P5.js, where I’ll visualize it with animated shapes and changing colors on the screen.

The visuals in P5 will shift depending on how loud the sound is, creating a more active or calm display. For example, soft sounds might create slow-moving blue waves, while loud sounds can trigger bright, fast animations. With the touch sensor, users can choose between different color moods like calm (blues), excited (reds), or dreamy (purples). I’ll also add LED lights to the Arduino that change color to match the selected mood.

This system will allow people to express their emotions or interact through sound and touch in a playful, visual way. It’s a simple setup, but it creates an engaging experience by combining physical input with real-time digital response.

Reading Design Meets Disability really changed the way I think about design. Before this, I used to see design as something focused on looks or style, not really about who it’s for. But this reading made me realize that design is also about who gets included or left out. I liked the part where it said that designing for disability actually helps everyone. It reminded me of curb cuts on sidewalks. They were made for wheelchair users but also help people with strollers or suitcases. It made me think that maybe good design should always start with accessibility, not add it later.

Another thing that stood out to me was how design shows what we think is “normal.” I never thought about how something like a prosthetic leg can carry stigma, while glasses don’t, even though both are assistive. The difference is that glasses have been accepted and even turned into fashion, while prosthetics haven’t been treated the same way. It made me wonder if more expressive or creative prosthetic designs would change how people see them. This reading made me realize that design isn’t just about making things work,it can also change how we feel about our bodies and each other.

For my final project, I want to create a game called “Go Ichi-Go!”, something like the Chrome Dinosaur Game, but a bit different. So the game would have a character called Ichigo (strawberry), who runs and has to jump over obstacles like jumping puddles of whipped cream, and towers of chocolate, and slide under floating slices of cake. After each jump/dive, there would be randomised text giving cute strawberry themed puns. There would also be sound effects for each jump or dive. There would be two buttons to jump or dive, using Arduino, and a button to start/restart the game. The player wins after successfully overcoming ten obstacles. This would display a congratulations image and a restart option. If they fail to do so, then it would just go yo a Game Over state with a restart button. If I do have the time/skill set for it, I want to add a treat component, where when they win, a sweet will be ejected out, probably using DC motors, but I’m not sure about this yet (really really want to try this). I want to make this game in a very kawaii retro vibe, but also focus more on the interaction design on the game itself.

Group members : Liya Rafeeq, Kashish Satija

Exercise 11.1 :

1. Make something that uses only one sensor  on Arduino and makes the ellipse in p5 move on the horizontal axis, in the middle of the screen, and nothing on arduino is controlled by p5 – for this we used a potentiometer. We mapped the values of the potentiometer to change the X coordinate of the ellipse, making it move along the horizontal axis.

P5.JS CODE :

let port;
let connectBtn;
let baudrate = 9600;

function setup() {
  createCanvas(400, 400);
  background(220);

  port = createSerial();

//serial connection
  let usedPorts = usedSerialPorts();
  if (usedPorts.length > 0) {
    port.open(usedPorts[0], baudrate);
  }
  let connectBtn = createButton("Connect to Serial");
  connectBtn.mousePressed(() => port.open(baudrate));
}

function draw() {
  
  
  //read from the serial port, complete string till the ""
  let str = port.readUntil("\n");
  if (str.length > 0) {
    background("white");
    ellipse(int(str),200,40,40); 
  }

}

ARDUINO CODE:

void setup() {
  Serial.begin(9600); // initialize serial communications
}
 
void loop() {
  // read the input pin:
  int potentiometer = analogRead(A1);                  
  // remap the pot value to 0-400:
  int mappedPotValue = map(potentiometer, 0, 1023, 0, 400); 
  // print the value to the serial port.
  Serial.println(mappedPotValue);
  // slight delay to stabilize the ADC:
  delay(1);                                            
  
  // Delay so we only send 10 times per second and don't
  // flood the serial connection leading to missed characters on the receiving side
  delay(100);
}

Exercise 11.2 :

2. Make something that controls the LED brightness from p5. For this, we made a circle that moves along the Y axis. According to the Y coordinates, the LED turns brighter or lower.

P5.JS. CODE:

let port;
let connectBtn;
let baudrate = 9600;

function setup() {
  createCanvas(255, 285);
  port = createSerial();

  // in setup, we can open ports we have used previously
  // without user interaction

  let usedPorts = usedSerialPorts();
  if (usedPorts.length > 0) {
  port.open(usedPorts[0], baudrate);
} else {
  connectBtn = createButton("Connect to Serial");
  connectBtn.mousePressed(() => port.open(baudrate));
}
}

function draw() {
  background(220);
  circle(128,mouseY,30,30)
  let sendtoArduino = String(mouseY) + "\n"
  port.write(sendtoArduino);
}

ARDUINO CODE:

int led = 5;

void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);
  pinMode(led, OUTPUT);
}

void loop() {
  // put your main code here, to run repeatedly:
  while (Serial.available()) 
    {
    digitalWrite(LED_BUILTIN, HIGH); // led on while receiving data
    int brightness = Serial.parseInt(); //get slider value from p5
     if (Serial.read() == '\n') {
       analogWrite(led, brightness);
    }
  }
}

Exercise 11.3:

let baudrate = 9600;
let velocity;
let gravity;
let position;
let acceleration;
let wind;
let drag = 0.99;
let mass = 50;
let str="";
let val;
let heightOfBall = 0;

function setup() {
  createCanvas(640, 360);
  noFill();
  position = createVector(width/2, 0);
  velocity = createVector(0,0);
  acceleration = createVector(0,0);
  gravity = createVector(0, 0.5*mass);
  wind = createVector(0,0);
  port = createSerial();

  // in setup, we can open ports we have used previously
  // without user interaction

  let usedPorts = usedSerialPorts();
  if (usedPorts.length > 0) {
  port.open(usedPorts[0], baudrate);
} else {
  connectBtn = createButton("Connect to Serial");
  connectBtn.mousePressed(() => port.open(baudrate));
}
}

function draw() {
  background(255);
  applyForce(wind);
  applyForce(gravity);
  velocity.add(acceleration);
  velocity.mult(drag);
  position.add(velocity);
  acceleration.mult(0);
  ellipse(position.x,position.y,mass,mass);
  if (position.y > height-mass/2) {
      velocity.y *= -0.9;  // A little dampening when hitting the bottom
      position.y = height-mass/2;
      heightOfBall = 1;
    } else {
      heightOfBall = 0;
    }
  str = port.readUntil("\n");
  val=int(str);
  if (!isNaN(val)) {
  breeze(val);
  }
}
function applyForce(force){
  // Newton's 2nd law: F = M * A
  // or A = F / M
  let f = p5.Vector.div(force, mass);
  acceleration.add(f);
}

function breeze(val){
  if (val<400){
    wind.x=-1;
  }
  else if (val>500 && val<900){
    wind.x=1;
  } else {
    wind.x=0
  }
  let sendToArduino = String(heightOfBall)  + "\n";
  port.write(sendToArduino);
}
function keyPressed(){
  if (key==' '){
    mass=random(15,80);
    position.y=-mass;
    velocity.mult(0);
  }
}

int led = 5;

void setup() {
// put your setup code here, to run once:
Serial.begin(9600);
pinMode(led, OUTPUT);
}

void loop() {
// put your main code here, to run repeatedly:
while (Serial.available())
{
digitalWrite(LED_BUILTIN, HIGH); // led on while receiving data
int brightness = Serial.parseInt(); //get slider value from p5
if (Serial.read() == '\n') {
analogWrite(led, brightness);
}
}
}

Reading Design Meets Disability made me rethink how I’ve traditionally understood assistive devices—as purely functional tools. Pullin challenges that limited view by showing how design and disability can intersect in creative, expressive, and even fashionable ways. What stood out most to me was the idea that disability devices, like hearing aids or prosthetics, shouldn’t have to be hidden or neutral—they can be bold, beautiful, and part of someone’s personal identity. The example of Aimee Mullins using prosthetic legs designed by Alexander McQueen was especially powerful. It showed how design can shift perceptions of disability from something to be fixed or minimized to something that can be celebrated and uniquely expressed.

This reading made me reflect on how design influences the way we feel about ourselves and how others see us. It made me realize how much design has to do with dignity, pride, and empowerment—not just function. I found myself thinking about how many products I use daily that are designed to be sleek or stylish, and how unfair it is that many people with disabilities are given tools that feel like medical equipment instead. Pullin’s emphasis on co-design really resonated with me; involving disabled people directly in the design process isn’t just practical, it’s respectful. This reading left me inspired to think more inclusively about design and more critically about who gets to have choice, beauty, and individuality in the products they use.

1: Ellipse

/*
 * Week 11 Production (1)
 *
 * Inputs:
 *   - A1 - 10k potentiometer connected to 5V and GND
 *
 */

int interval = 100;
int lastMessageTime = 0;

int potPin = A1;

void setup() {
  Serial.begin(9600); // initialize serial communications
}
 
void loop() {
  // read the input pin:
  int potentiometer = analogRead(potPin);                  
  // remap the pot value to 0-255:
  int mappedPotValue = map(potentiometer, 0, 1023, 0, 255); 
  // print the value to the serial port.
  Serial.println(mappedPotValue);
  // slight delay to stabilize the ADC:
  delay(1);                                            
  
  delay(100);
}

let port;
let connectBtn;
let baudrate = 9600;
let lastMessage = "";
let currX;

function setup() {
  createCanvas(400, 400);
  background(220);

  port = createSerial();

  let usedPorts = usedSerialPorts();
  if (usedPorts.length > 0) {
    port.open(usedPorts[0], baudrate);
  }

  connectBtn = createButton("Connect to Arduino");
  connectBtn.position(80, height-60);
  connectBtn.mousePressed(connectBtnClick);

  currX = width/2;
}

function draw() {
  background("white");
  fill('grey');
  circle(currX, height/2, 100);
  
  let str = port.readUntil("\n");
  if (str.length > 0) {
    // console.log(str);
    lastMessage = str;
  }
  
  // Display the most recent message
  text("Last message: " + lastMessage, 10, height - 20);

  // change button label based on connection status
  if (!port.opened()) {
    connectBtn.html("Connect to Arduino");
  } else {
    connectBtn.html("Disconnect");
  }
  
  // // Move shape based on received value
  if (!lastMessage) {lastMessage = "127"}
  currX = map(int(lastMessage), 0, 255, 0, width);
  currX = floor(currX);
  // console.log(currX);
}

function connectBtnClick() {
  if (!port.opened()) {
    port.open("Arduino", baudrate);
  } else {
    port.close();
  }
}

2: LED Brightness

/* 
 * Week 11 Production (2)
 * 
 * Outputs:
 * - 5 - LED
 * 
*/

int ledPin = 5;

void setup() {
  Serial.begin(9600);

  pinMode(LED_BUILTIN, OUTPUT);
  pinMode(ledPin, OUTPUT);

  // Blink them so we can check the wiring
  digitalWrite(ledPin, HIGH);
  delay(200);
  digitalWrite(ledPin, LOW);

  // start the handshake
  while (Serial.available() <= 0) {
    digitalWrite(LED_BUILTIN, HIGH); // on/blink while waiting for serial data
    Serial.println("0,0"); // send a starting message
    delay(300);            // wait 1/3 second
    digitalWrite(LED_BUILTIN, LOW);
    delay(50);
  }
}

void loop() {
  // wait for data from p5 before doing something
  while (Serial.available()) {
    digitalWrite(LED_BUILTIN, HIGH); // led on while receiving data

    int brightness = Serial.parseInt();
    if (Serial.read() == '\n') {
      analogWrite(ledPin, brightness);
    }
  }
  digitalWrite(LED_BUILTIN, LOW);
}

let port;
let baudrate = 9600;

// Show button to connect / disconnect
let showConnectButton = false;

function setup() {
  createCanvas(640, 480);
  textSize(20);

  // Create the serial port
  port = createSerial();

  // If the user previously connected, reopen the same port  
  let usedPorts = usedSerialPorts();
  if (usedPorts.length > 0) {
    port.open(usedPorts[0], baudrate);
  }

  // any other ports can be opened via a dialog
  if (showConnectButton) {
    connectBtn = createButton('Connect to Arduino');
    connectBtn.position(80, 350);
    connectBtn.mousePressed(setupSerial);
  }
  
}

// Show serial port connection dialog in response to action
function setupSerial() {
  if (!port.opened()) {
    port.open('Arduino', baudrate);
  } else {
    port.close();
  }
}

function draw() {
  background('white');
  fill('black');
  
  if (showConnectButton) {
    // changes button label based on connection status
    if (!port.opened()) {
      connectBtn.html('Connect to Arduino');
    } else {
      connectBtn.html('Disconnect');
    }
  }

  if (!port.opened()) {
    text("Disconnected - press space to connect", 20, 30);
  } else {
    text("Connected - press space to disconnect", 20, 30);
    
    // // Transmit brightness based on mouse position
    mappedX = floor(map(mouseX, 0, width, 0, 255));
    console.log(mappedX);
    let sendToArduino = mappedX + "\n";
    port.write(sendToArduino);
  }
}

function keyPressed() {
  if (key == " ") {
    setupSerial();
  }
}

3: Wind Gravity

/* 
 * Week 11 Production (3)
 * 
 * Inputs:
 *   - A1 - 10k potentiometer connected to 5V and GND
 * 
 * Outputs:
 * - 5 - LED
 * 
*/

int potPin = A1;
int ledPin = 5;

int interval = 100;
int lastMessageTime = 0;

void setup() {
  Serial.begin(9600);

  pinMode(LED_BUILTIN, OUTPUT);
  pinMode(potPin, INPUT);
  pinMode(ledPin, OUTPUT);

  // Blink them so we can check the wiring
  digitalWrite(ledPin, HIGH);
  delay(200);
  digitalWrite(ledPin, LOW);

  // start the handshake
  while (Serial.available() <= 0) {
    digitalWrite(LED_BUILTIN, HIGH); // on/blink while waiting for serial data
    Serial.println("127"); // send a starting message
    delay(300);            // wait 1/3 second
    digitalWrite(LED_BUILTIN, LOW);
    delay(50);
  }
}

void loop() {
  // wait for data from p5 before doing something
  while (Serial.available()) {
    digitalWrite(LED_BUILTIN, HIGH); // led on while receiving data

    // blink LED based on p5 data
    int status = Serial.parseInt();
    if (Serial.read() == '\n') {
      digitalWrite(ledPin, status);
      delay(10);
      digitalWrite(ledPin, LOW);
    }

    if (lastMessageTime > interval) {
      lastMessageTime = 0;
      // send mapped potentiometer reading to p5
      int potentiometer = analogRead(potPin);                  
      int mappedPotValue = map(potentiometer, 0, 1023, 0, 255); 
      Serial.println(mappedPotValue);
      // slight delay to stabilize the ADC:
      delay(1);
    }
    else {
      lastMessageTime++;
    }

  }
  digitalWrite(LED_BUILTIN, LOW);
}

let velocity;
let gravity;
let position;
let acceleration;
let wind;
let drag = 0.99;
let mass = 50;

// // Arduino
let port;
let connectBtn;
let baudrate = 9600;
let lastMessage = "";
let showConnectButton = false;

function setup() {
  createCanvas(620, 400);
  noFill();
  position = createVector(width/2, 0);
  velocity = createVector(0,0);
  acceleration = createVector(0,0);
  gravity = createVector(0, 0.5*mass);
  wind = createVector(0,0);
  
  // // Arduino
  port = createSerial();
  let usedPorts = usedSerialPorts();
  if (usedPorts.length > 0) {
    port.open(usedPorts[0], baudrate);
  }
  if (showConnectButton) {
    connectBtn = createButton('Connect to Arduino');
    connectBtn.position(80, 300);
    connectBtn.mousePressed(setupSerial);
  }
}

function draw() {
  background(255);
  applyForce(wind);
  applyForce(gravity);
  velocity.add(acceleration);
  velocity.mult(drag);
  position.add(velocity);
  acceleration.mult(0);
  ellipse(position.x,position.y,mass,mass);
  if (position.y > height-mass/2) {
      velocity.y *= -0.9;  // A little dampening when hitting the bottom
      position.y = height-mass/2;
      flashLight();
    }
  
  // // Arduino
  if (showConnectButton) {
    if (!port.opened()) {
      connectBtn.html('Connect to Arduino');
    } else {
      connectBtn.html('Disconnect');
    }
  }
  fill('black');
  if (!port.opened()) {
    text("Disconnected", 20, 30);
  } else {
    text("Connected", 20, 30);
  }
  let str = port.readUntil("\n");
  if (str.length > 0) {
    // console.log(str);
    lastMessage = str;
  }
  
  // Display the most recent message
  text("Last message: " + lastMessage, 10, height - 20);
  
  // // Convert received value to wind.x value
  mappedPot = map(int(lastMessage), 0, 255, -1, 1);
  wind.x = mappedPot;
  let windSpeed = "Wind speed: " + wind.x
  text(windSpeed.substring(0,20), 10, height - 5);
  
  fill('white');
}

function applyForce(force){
  // Newton's 2nd law: F = M * A
  // or A = F / M
  let f = p5.Vector.div(force, mass);
  acceleration.add(f);
}

function keyPressed(){
  if (keyCode==LEFT_ARROW){
    wind.x=-1;
  }
  if (keyCode==RIGHT_ARROW){
    wind.x=1;
  }
  if (key==' '){
    mass=random(15,80);
    position.y=-mass;
    velocity.mult(0);
    port.write("0\n"); // reset light
  }
}

// // Arduino
function setupSerial() {
  if (!port.opened()) {
    port.open('Arduino', baudrate);
  } else {
    port.close();
  }
}

function flashLight() {
  if (port.opened()) {
    port.write("1\n");
    // port.write("0\n");
  }
}

Video (Google Drive)

The text “A Brief Rant on the Future of Interaction Design” takes a critical look at modern digital interfaces and points out how they often force users to adapt to strict, outdated design rules instead of the other way around. The author argues that interfaces should be more in tune with natural human thought processes, which could lead to more flexible and easier-to-use systems. For example, the text challenges the reliance on traditional metaphors in design that can limit how people interact with technology, suggesting that a rethinking of these strategies would better serve everyone.

In the responses, various designers and thinkers share their own views on what works and what doesn’t in today’s interaction design. Many contributors agree that sticking to rigid structures can suppress innovation and user engagement, while others offer practical examples from their work. One common point is that when interfaces are redesigned to be more intuitive, it often results in smoother and more productive user experiences, showing that a change in approach can have positive real-world benefits.

Overall, both readings encourage a move toward interaction design that feels more natural and accommodating to users. The discussion emphasizes the importance of creating technology that adapts to how people actually think and work, rather than forcing users to learn and conform to outdated digital patterns. This friendly call for change makes it clear that smarter design is not just a theoretical goal; it can lead to improvements in everyday technology that benefit us all.

A Brief Rant on the Future of Interaction Design

The article, “A Brief Rant on the Future of Interaction Design” Criticizes the “Pictures Under Glass” approach in interaction design, where the author emphasizes the importance of tactile and hands-on engagement, which is frequently disregarded in preference to visual interfaces. The author asserts that touch is essential to our interactions with the world, highlighting the extensive tactile sensations we experience from common items such as books or glasses of water, and underlining the variety of hand motions required for activities like opening a jar. He analyzes the drawbacks of touchscreens, which primarily provide a flat, sliding motion that fails to reflect the depth of physical interaction we typically experience. Although he recognizes the visual energy of digital interfaces, Victor proposes that present technologies, such as the iPad, might not have lasting potential unless they develop to utilize more of our physical abilities. This made me think about how often we overlook tactile feedback in our daily activities and how digital interfaces, although visually engaging, frequently fail to deliver that deep physical interaction.