Author: Siyona

Make a Wish ✨

Concept

My final project is a Disney’s Tangled themed reaction-time game. A major focus in Tangled is the floating lanterns that appear every year on Rapunzel’s birthday. These same lanterns are also popularly known as “wish” lanterns and I wanted to mix these two ideas to create my game. The overall flow of the game is that the user catches enough lanterns to earn a wish, and can then type and send out their wish into the universe with the other lanterns.

Video Documentation

Interaction Design

Once the user starts the game, the lanterns in front of them start to light up and turn off rapidly, and lively music, specifically the Kingdom Dance music from the film, plays in the background. The user needs to press a lit up lantern’s corresponding button to “catch” it. Every time they catch a lantern, their score (displayed on the p5js screen) goes up. Once they’ve reached a score of 20, the lanterns turn off and I See the Light begins to play in the background. They can input their wish on the screen and press ‘Enter’ to send it up with the floating wish lanterns. The physical lanterns blink pleasantly as the user’s wish floats upwards. Once out of sight, the light and music stops and the experience finishes. The user then gets the option to restart if they wish.

Arduino

Code:

// 4 lanterns (1 pin per lantern, 4 LEDs in parallel) + 4 buttons

// LED pins - one pin controls all 4 LEDs per lantern
const int lantern1 = 2;
const int lantern2 = 5;
const int lantern3 = 8;
const int lantern4 = 11;

// Button pins (using internal pullup resistors)
const int button1 = A0;
const int button2 = A2;
const int button3 = A3;
const int button4 = A4;

// track lit up lanterns
bool lanternActive[4] = {false, false, false, false};

// Button state tracking for debouncing
bool lastButtonState[4] = {HIGH, HIGH, HIGH, HIGH};   // last stable reading
unsigned long lastDebounceTime[4] = {0, 0, 0, 0};
const unsigned long debounceDelay = 50;   // 50ms debounce duration

void setup() {
  Serial.begin(9600);
  
  // Initialize LED pins
  pinMode(lantern1, OUTPUT);
  pinMode(lantern2, OUTPUT);
  pinMode(lantern3, OUTPUT);
  pinMode(lantern4, OUTPUT);
  
  // Initialize button pins with internal pullup
  pinMode(button1, INPUT_PULLUP);
  pinMode(button2, INPUT_PULLUP);
  pinMode(button3, INPUT_PULLUP);
  pinMode(button4, INPUT_PULLUP);
  
  // Turn off all LEDs initially
  turnOffAllLanterns();
  
  // Wait for serial to stabilize
  delay(1000);
  
  // debugging
  Serial.println("READY");
  Serial.println("Button test: Press each button");
  Serial.flush();
}

void loop() {
  // Check for commands from p5.js
  if(Serial.available() > 0) {
    String command = Serial.readStringUntil('\n');
    command.trim();
    
    if(command.startsWith("ON:")) {
      int lanternNum = command.substring(3).toInt();
      turnOnLantern(lanternNum);
    }
    else if(command.startsWith("OFF:")) {
      int lanternNum = command.substring(4).toInt();
      turnOffLantern(lanternNum);
    }
    else if(command == "ALLON") {
      turnOnAllLanterns();
    }
    else if(command == "ALLOFF") {
      turnOffAllLanterns();
    }
  }
  
  // Check buttons with debouncing
  checkButton(0, button1);
  checkButton(1, button2);
  checkButton(2, button3);
  checkButton(3, button4);
}

// Button check function
void checkButton(int buttonIndex, int buttonPin) {
  bool reading = digitalRead(buttonPin);
  
  // check if reading is bouncing, reset debounce timer
  if(reading != lastButtonState[buttonIndex]) {
    lastDebounceTime[buttonIndex] = millis();
    
    // If state is now LOW (pressed), send immediately
    if(reading == LOW) {
      Serial.print("BTN:");
      Serial.println(buttonIndex + 1);
      Serial.flush(); // data is sent immediately
    }
  }
  
  lastButtonState[buttonIndex] = reading;
}

// LED Control Functions

void turnOnLantern(int lanternNum) {
  lanternActive[lanternNum - 1] = true;
  
  switch(lanternNum) {
    case 1:
      digitalWrite(lantern1, HIGH);
      break;
    case 2:
      digitalWrite(lantern2, HIGH);
      break;
    case 3:
      digitalWrite(lantern3, HIGH);
      break;
    case 4:
      digitalWrite(lantern4, HIGH);
      break;
  }
}

void turnOffLantern(int lanternNum) {
  lanternActive[lanternNum - 1] = false;
  
  switch(lanternNum) {
    case 1:
      digitalWrite(lantern1, LOW);
      break;
    case 2:
      digitalWrite(lantern2, LOW);
      break;
    case 3:
      digitalWrite(lantern3, LOW);
      break;
    case 4:
      digitalWrite(lantern4, LOW);
      break;
  }
}

void turnOnAllLanterns() {
  for(int i = 0; i < 4; i++) {
    lanternActive[i] = true;
  }
  
  digitalWrite(lantern1, HIGH);
  digitalWrite(lantern2, HIGH);
  digitalWrite(lantern3, HIGH);
  digitalWrite(lantern4, HIGH);
}

void turnOffAllLanterns() {
  for(int i = 0; i < 4; i++) {
    lanternActive[i] = false;
  }
  
  digitalWrite(lantern1, LOW);
  digitalWrite(lantern2, LOW);
  digitalWrite(lantern3, LOW);
  digitalWrite(lantern4, LOW);
}

Description:

This Arduino sketch controls four LED-based lanterns and four physical push buttons while communicating with the p5.js application through the Serial connection. Each lantern consists of four LEDs connected in parallel and is controlled by a single digital output pin (2, 5, 8, and 11), while the buttons are connected to analog pins (A0, A2, A4, and A3) and use internal pull-up resistors, meaning they read HIGH by default and LOW when pressed.

The program tracks the active state of each lantern using an array and also monitors button activity with additional arrays that store the last button state and the last debounce time, applying a 50-millisecond debounce delay to prevent false or repeated presses caused by mechanical noise.

Inside the main loop, the Arduino continuously listens for serial commands sent from p5.js, such as instructions to turn individual lanterns on or off using formatted messages like “ON:1” or “OFF:3”, as well as commands that control all lanterns at once using “ALLON” and “ALLOFF”. At the same time, the loop constantly checks each physical button using a dedicated debouncing function that detects state changes, filters out signal noise, and immediately sends a message like “BTN:1” through the Serial port when a valid press occurs so that the visual system can respond instantly.

Schematic

P5.js

(you can enter fullscreen mode by double-clicking on the canvas in p5.js)

Description:

The p5.js sketch controls the full digital side of the interactive experience by handling the visuals, sound, gameplay logic, and communication with the Arduino. It manages multiple game states including the start screen, instructions, active gameplay, and the final wish sequence. The program sends commands to the Arduino to activate and deactivate physical lantern LEDs while simultaneously listening for incoming button press data through the Web Serial API. Animated lanterns are continuously spawned and float upward across the screen, a live score is tracked, and background music changes based on the game state. Once the user successfully collects enough lanterns, a text input appears to capture and animate their written wish before the experience resets.

Arduino + p5.js communication

The communication between the Arduino and p5.js is handled through serial data exchange. The Arduino continuously listens for text-based commands sent from p5.js, such as “ON:1”, “OFF:3”, ALLON, and ALLOFF, which control the physical LED lanterns. At the same time, the Arduino sends messages like “BTN:1” whenever a physical button is pressed. These messages are read and interpreted by p5.js to update the game state, score, and visuals. This two-way communication allows the physical hardware and the digital game to stay perfectly synchronized.

What I’m most proud of

I think what I’m most proud of is the physical components and setup of this game, as the software side of it is not too complex. I got to use a lot of new tools such as 3-D printing for the lanterns and laser cutting for the panel box for the buttons which I may not have used otherwise. Moreover, I got through several issues, such as the not visible yellow light from LEDs (which was the initial plan) by switching to blue, and the mess of wires going into the Arduino (that was an eyesore) by adding the cardboard base under the lanterns to hide the wires under. Just yesterday, I found that one of the buttons in my panel box was no longer working and I had to remove the solder from it to free the wires and replace the system with a new button. I’m also still very proud of the castle that’s part of the visuals in p5.js as I made every single individual shape of that silhouette through trial and error with the shape coordinates.

How this was made

Media sources:
The starting page’s background image came from this website: wallup.net
The font Homemade Apple cam from fonts.google.com
The background music was from the Youtube videos linked previously in the post

The game concept, code, and writeup were done by me, with AI being used for debugging the p5.js code. For example, it helped me debug the connectSerial() and readSerial() functions when they weren’t working as I still was not fully clear on how they connected and functioned, and it’s also how I found out about the windowResized() function.

// Keeps the layout responsive when the browser window is resized
function windowResized() {
  resizeCanvas(windowWidth, windowHeight);
  // Reposition input box
  if (wishInput) {
    wishInput.position(windowWidth/2 - 180, windowHeight - 100);
    // Keeps the wish input centered at the bottom of the screen
  }
  if (restartButton && experienceFinished) {
    restartButton.position(windowWidth / 2 - 110, windowHeight / 2 - 25);
    // Keeps the restart button centered on the screen after resizing
  }
}

My grammar and sentence phrasing in general in this writeup was occasionally polished by Grammarly to help make it more clear and concise.

Future Improvement

I think there are many things I would like to improve in the future:
1) Make a full platform (similar to the one for the buttons) for the lanterns to sit on under which I can hide the Arduino and the full wire.
2) Create a mechanism that allows the player to adjust the game speed, maybe through a potentiometer.
3) Use a brighter source of light such a neo pixels instead of LEDs so that lanterns shine brighter.
4) On the software side the user can still click anywhere on the screen and that creates their own lantern that begins floating upwards, but currently there is no corresponding reaction in the physical setup. It would be interesting to set up such a reaction for every new lantern that is spawned by clicking.

User Testing Video Documentation

Based on my user testing, it seems like people mostly are able to figure out my project. There weren’t any points of confusion in terms of what needs to be done, and it was clear that every time they pressed a button for a lit up lantern, the score went up.

However, one point of concern by one user was that after connecting to the Arduino, the game started so quickly that they did not realize that it had even started. Maybe I could add a kind of countdown timer after the Arduino is connected so that the user can prepare for the game to begin.

I didn’t have to explain anything to the users during any of the tests, and the simple 3 pointer instructions at the beginning were enough for the users to understand what they had to do. The project is obviously still in progress, and there are parts missing including the panel box for the buttons, the speaker attached to the laptop for the music, and maybe even a shade over the lanterns so the light is more visible. Overall, I believe the interaction design was clear and the communication between the p5 interface and Arduino was smooth.

My final project will be a short game/experience inspired by the wish lanterns in Disney’s Tangled and will use the artwork I did all the way back in week 3:

I want users to be able to send out their own wishes. Before they can enter their wish, they must complete a short challenge. The challenge is a physical reaction-time game: four lanterns (each with an LED inside) are paired with four corresponding push buttons. The lanterns light up randomly at a fast pace, and the user must quickly press the button for the lantern that lights up, essentially “whacking” it. Their successful hits are tracked and displayed in p5, with a similar sketch as the one above as the background.
Once the user reaches 10 successful hits, a text box appears in the p5 interface (as already present in the sketch above) where they can type their wish. That wish is then displayed on an Arduino screen, all the lanterns light up together, and the Tangled music plays to complete the experience. After this, the p5 screen gives the user an option to restart the experience all over again.

The Arduino handles all the feedback related to the physical input:
it flashes the LEDs in a random, fast paced manner and detects corresponding button presses by the user. Every time a lit up LED’s button is pressed on time, it sends a signal to p5 to increase the score of the user by 1.

On the p5 side, the sketch handles the game logic, tracks the score, maintains the aesthetics, provides the textbox to enter the wish, and sends the wish back to the Arduino. It also gives instructions at the start of the experience to the user and gives them an option to restart at the end.

To get started, I tried to implement the reaction-time game logic with the score-tracking on a small scale using just my own Arduino, breadboard and 4 LEDs. It mostly works, but the LEDs light up too slowly (ignore the interface also):

Group Member: Aditi

Task 1: make something that uses only one sensor on Arduino and makes the ellipse in p5 move on the horizontal axis, in the middle of the screen, and nothing on arduino is controlled by p5.

We used a potentiometer as the analog sensor on Arduino to move the ellipse on the horizontal axis.

p5js code:

// Serial port object
let port;

// Latest sensor value
let sensorVal = 0;

function setup() {
  createCanvas(400, 300);
  background(220);

  // Create the serial connection object
  port = createSerial();

  // If a port was used before, auto-reconnect
  let used = usedSerialPorts();
  if (used.length > 0) {
    port.open(used[0], 9600);
  }
}

function draw() {
  background(220);

  // Read one line of text until newline "\n"
  let str = port.readUntil("\n");

  // Make sure we actually received something
  if (str.length > 0) {

    // Convert the string into an integer
    let val = int(str.trim());

    // Map sensor value (0–1023) to screen X position (0–400)
    let x = map(val, 0, 1023, 0, width);

    // Draw a circle at mapped position
    ellipse(x, height / 2, 40, 40);

  } else {

    // If empty data is received, print it for debugging
    console.log("Empty:", str);
  }
}

Arduino code:

const int sensor=A2;
int sensorValue;
void setup() {
  // put your setup code here, to run once:
  pinMode(sensor,INPUT);
  Serial.begin(9600);
  Serial.println(0); // send a starting message
}

void loop() {
  // put your main code here, to run repeatedly:
  sensorValue=analogRead(sensor);
  delay(60);
  Serial.println(sensorValue);  
}

Task 2: make something that controls the LED brightness from p5.

We decided to control the LED’s brightness through four buttons on p5js.

p5js code:

let port;
let brightnessToSend = 0;
const baudrate = 9600;

function setup() {
  createCanvas(400, 400);

  // Create the serial port
  port = createSerial();

  // connect automatically if used before
  let usedPorts = usedSerialPorts();
  if (usedPorts.length > 0) {
    port.open(usedPorts[0], baudrate);
  } else {
    connectBtn = createButton("Connect to Serial");
    connectBtn.position(10, 10);
    connectBtn.mousePressed(() => port.open(baudrate));
  }
}

function draw() {
  background(30);

  drawStarButton(80, 150, 40, 20, 5, 64);
  drawStarButton(160, 150, 40, 20, 5, 128);
  drawStarButton(240, 150, 40, 20, 5, 192);
  drawStarButton(320, 150, 40, 20, 5, 255);

  fill(255);
  textAlign(CENTER);
  text("Current brightness: " + brightnessToSend, width / 2, 280);
}

function mousePressed() {
  if (!port.opened()) return; // p5.webserial function to check if open

  let stars = [
    { x: 80,  brightness: 64  },
    { x: 160, brightness: 128 },
    { x: 240, brightness: 192 },
    { x: 320, brightness: 255 }
  ];

  for (let s of stars) {
    if (dist(mouseX, mouseY, s.x, 150) < 30) {
      brightnessToSend = s.brightness;

      // Send brightness (0–255)
      port.write(brightnessToSend);
    }
  }
}

function drawStarButton(x, y, radius1, radius2, points, brightness) {
  let angle = TWO_PI / points;
  let halfAngle = angle / 2;

  if (brightnessToSend === brightness) fill(255, 200, 0);
  else fill(255);

  push();
  translate(x, y);
  beginShape();
  for (let a = 0; a < TWO_PI; a += angle) {
    vertex(cos(a) * radius1, sin(a) * radius1);
    vertex(cos(a + halfAngle) * radius2, sin(a + halfAngle) * radius2);
  }
  
  endShape(CLOSE);
  pop();
}

Arduino code:

int ledPin = 9;

void setup() {
  Serial.begin(9600);
  pinMode(ledPin, OUTPUT);
}

void loop() {
  if (Serial.available() > 0) {
    int brightness = Serial.read();  // 0–255
    analogWrite(ledPin, brightness); // PWM LED brightness
  }
}

Video:

Task 3: take the gravity wind example and make it so every time the ball bounces one led lights up and then turns off, and you can control the wind from one analog sensor.

p5js code:

let velocity;        // velocity vector of the ball
let gravity;         // gravity force vector
let position;        // position of the ball
let acceleration;    // accumulated acceleration each frame
let wind;            // wind force vector
let drag = 0.99;     // drag factor to slow down velocity a bit
let mass = 50;       // "mass" of the ball, also used as its size
let port;            // serial port object for Arduino ↔ p5 communication
let button;          // button to trigger serial connection popup
let open=false;      // flag to track if we've tried to open the port

function setup() {
  createCanvas(640, 360);
  noFill();
  position = createVector(width/2, 0);   // start ball at top center
  velocity = createVector(0,0);          // start with no movement
  acceleration = createVector(0,0);      // acceleration accumulates forces
  gravity = createVector(0, 0.5*mass);   // gravity pulls ball downward
  wind = createVector(0,0);              // wind starts with no force
  port = createSerial(); //port is an object that will be used to send message and receeieve message from arudnio
 
  //createbutton creates button in p5js with quote
  button=createButton("Connect to Arduino");
  button.position(width/2,height/2); //where you want your button to be
  button.mousePressed(openArduino); 
}

function openArduino(){
  if(!port.opened()){ //to check if the port is opened or not we use function opened
    port.open(9600) // we open the port for communication with 9600 frequency
   
    button.remove() // once arudino is opended we removed the button to connect to ardunio
   
    open=true; //just a variable to check if we opended the port or not
  } 
}

function draw() {
  if(port.opened()){             // run main logic only if serial port is open
  background(255);               // clear canvas every frame
  applyForce(wind);              // apply wind force first
  applyForce(gravity);           // then apply gravity force
  velocity.add(acceleration);    // a → v
  velocity.mult(drag);           // apply drag to slowly reduce velocity
  position.add(velocity);        // v → position
  acceleration.mult(0);          // reset acceleration so next frame starts fresh
  ellipse(position.x,position.y,mass,mass);  // draw the ball
 
  }

  // collision + LED control are outside the port.opened() block,
  // so they always run based on current position
  if (position.y > height-mass/2) {     
      velocity.y *= -0.9;         // A little dampening when hitting the bottom (bounce)
      position.y = height-mass/2; // keep ball from sinking below the "floor"
      port.write("1\n")           // send "1" to Arduino → turn LED ON
    }
    else {
      port.write("0\n")    // otherwise send "0" → turn LED OFF
    }
    potvalue=port.readUntil("\n");  // read a line from Arduino (pot value as text)
    // console.log(potvalue);
 
    if(potvalue>514){      // simple threshold: above midpoint = wind to the right
      wind.x=1
    }
    else{                  // below midpoint = wind to the left
      wind.x=-1
    } 
}

function applyForce(force){
  // Newton's 2nd law: F = M * A
  // or A = F / M
  let f = p5.Vector.div(force, mass);  // scale force by mass → acceleration
  acceleration.add(f);                 // accumulate acceleration for this frame
}

function keyPressed(){
  if (keyCode==LEFT_ARROW){
    wind.x=-1;              // keyboard override: push ball left
  }
  if (keyCode==RIGHT_ARROW){
    wind.x=1;               // keyboard override: push ball right
  }
  if (key==' '){
    mass=random(15,80);     // randomize mass (and drawn size)
    position.y=-mass;       // reset ball above the top so it falls back down
    velocity.mult(0);       // clear velocity so it restarts clean
  }
}

Arduino code:

int greenLight=7;
int sensor=A0;
void setup() {
  // put your setup code here, to run once:
  pinMode(greenLight,OUTPUT);
  pinMode(sensor,INPUT);
  Serial.begin(9600);
  digitalWrite(greenLight,0);
}

void loop() {
  // put our main code here, to run repeatedly:
  // Serial.print("0");
 
  int value= analogRead(sensor);
  Serial.print(value);
  Serial.print("\n");

  while(Serial.available()>0){ //avaiable checks if there is any message in inbox of ardino
    String a=Serial.readStringUntil('\n');
    a.trim();
    if(a=="1"){
       digitalWrite(greenLight,1);
    }
    else{
      digitalWrite(greenLight,0);
    }   
  }
}

Video:

A (Disney) Tangled Experience

Concept Overview

For my final project, I wanted to extend the assignment that I did all the way back in Week 3, Make a Wish. It was based on one of my favorite Disney films, Tangled, and I thought it would be interesting to extend it into a physically interactive system. Here is the sketch from Week 3:

For this project, I want users to “send out a wish” using Tangled-style wish lanterns. Before they can enter their wish, they must complete a short challenge. The challenge is a physical reaction-time game: four lanterns (each with an LED inside) are paired with four corresponding push buttons. The lanterns light up randomly at a fast pace, and the user must quickly press the button for the lantern that lights up, essentially “whacking” it. Their successful hits are tracked and displayed in p5, with a similar sketch as the one above as the background.

Once the user reaches 5 successful hits, a text box appears in the p5 interface where they can type their wish. That wish is then displayed on an Arduino screen, all the lanterns light up together, and the Tangled music plays to complete the experience. After this, the p5 screen gives the user an option to restart the experience all over again.

Projected Required Materials

• Arduino and consumables that come with it
• 4 large push buttons
• 4 3-D printed lanterns big enough to fit small breadboard with yellow LEDs
• Arduino screen and speakers
• Cardboard and Paint

Design Meets Disability

I do not associate medical or disability-assistive products with design. To me, it only mattered how functional or practical they were, and not their aesthetic value. What I appreciated about the author’s perspective was how they treated people with disabilities as a distinct user group whose preferences and needs are often overlooked by designers and engineers.

Further into the reading, I started to feel bad that people with disabilities are often forced to choose between a device that functions well and one that’s subtle or aesthetically pleasing. These goals seem to conflict with current design approaches. Even within the category of disability, there’s a wide range of experiences that should shape how products are created.

I really liked the author’s example of eyeglasses. Glasses are no longer seen as a disability aid, but are now a fashion statement. Although, to be honest, personally glasses have always just been a medical necessity for me. But also I refuse to get laser because I now think I look better with glasses anyways. I could think of some other examples back from high-school as well: hearing aids and braces for teeth.

I strongly believe that the notion that assistive devices must remain discreet reflects a broader limitation or bias in design thinking. It is a kind of hesitation in creating bold, confident products that users would actually be proud to display. However, I do think that with every passing year, adaptive fashion is becoming increasingly popular, and this will help begin a new era of accessibility.

Concept

For this week’s assignment, we had to make a musical instrument involving at least one digital sensor and one analog sensor. Aditi and I decided to create a simple piano-like instrument with lights whose pitch level can be controlled by a potentiometer. There are 4 buttons (switches) that act as the piano “keys” and play different sounds, while the potentiometer has been mapped to three different levels so that the keys produce a high-pitched, middle-pitched, and low-pitched sound.

Materials

• Analog sensor: 10K Trimpot
• Digital Switch: Red, Blue, Yellow, and Green tactile buttons
• Output: Piezo Buzzer to produce the sound and LEDs for light output

Schematic

Video Documentation

The Code

const int button_yellow=8;
const int yellow_light=7;
const int button_blue=9;
const int blue_light=6;
const int green_light=5;
const int button_green=10;
const int red_light=4;
const int button_red=11;
#define BUZZER 12
int potValue=0;
const int potPin=A0;
int melody[]={262,294,330,349,392,440,492,523}; // notes from C4-C5
int melody2[] = {523, 587, 659, 698, 784, 880, 988, 1047}; // notes from C5–C6
int melody3[] = {1047, 1175, 1319, 1397, 1568, 1760, 1976, 2093}; // C6–C7
int nodeDurations=4;
int increase=0;
int potValue_p;

void setup() {
  // put your setup code here, to run once:
  pinMode(button_yellow, INPUT);
  pinMode(yellow_light,OUTPUT);
  pinMode(button_blue, INPUT);
  pinMode(blue_light, OUTPUT);
  pinMode(button_green, INPUT);
  pinMode(green_light,OUTPUT);
  pinMode(button_red, INPUT);
  pinMode(red_light, OUTPUT);

  pinMode(BUZZER, OUTPUT);
  Serial.begin(9600);
}

void loop() {
  potValue = analogRead(potPin);
  
  if (digitalRead(8) == HIGH) {
    if (potValue <= 300) {
      tone(BUZZER, melody[1]);
    } else if (potValue >= 300 && potValue <= 550) {
      tone(BUZZER, melody2[1]);
    } else if (potValue >= 550 && potValue <= 1023) {
      tone(BUZZER, melody3[1]);
    }
    digitalWrite(yellow_light, HIGH);

  } else if (digitalRead(9) == HIGH) {
    if (potValue <= 300) {
      tone(BUZZER, melody[2]);
    } else if (potValue >= 300 && potValue <= 550) {
      tone(BUZZER, melody2[2]);
    } else if (potValue >= 550 && potValue <= 1023) {
      tone(BUZZER, melody3[2]);
    }
    digitalWrite(blue_light, HIGH);

  } else if (digitalRead(10) == HIGH) {
    Serial.print("green");
    if (potValue <= 300) {
      tone(BUZZER, melody[3]);
    } else if (potValue >= 300 && potValue <= 550) {
      tone(BUZZER, melody2[3]);
    } else if (potValue >= 550 && potValue <= 1023) {
      tone(BUZZER, melody3[3]);
    }
    digitalWrite(green_light, HIGH);

  } else if (digitalRead(11) == HIGH) {
    if (potValue <= 300) {
      tone(BUZZER, melody[4]);
    } else if (potValue >= 300 && potValue <= 550) {
      tone(BUZZER, melody2[4]);
    } else if (potValue >= 550 && potValue <= 1023) {
      tone(BUZZER, melody3[4]);
    }
    digitalWrite(red_light, HIGH);

  } else if(digitalRead(8)==LOW && digitalRead(9)==LOW && digitalRead(10)==LOW && digitalRead(11)==LOW) {  
    Serial.println("Pin is LOW");
    noTone(BUZZER);
    digitalWrite(yellow_light, LOW);
    digitalWrite(blue_light, LOW);
    digitalWrite(green_light, LOW);
    digitalWrite(red_light, LOW);
  }
}

Reflection & Future Improvements

We had fun making this project, however it was also challenging to make all of the connections. Initially, we planned to have at least 6-7 switches, but the setup became so crowded with just 4. Aditi drew the schematic diagram on paper before we began to build the connections physically, and this made the process much more manageable. We definitely could not have done it without having the diagram in front of us first. Sometimes, the setup for a particular switch would not work and we would face trouble in figuring out whether the issue was in the code or the wiring. In future versions, we would love to have more keys, as well as a tiny screen that displays the letter for the current note that is being played. We would also want to research on more “melodic” or pleasant sounding notes to add to the code.

A Brief Rant on the Future of Interaction Design and A follow-up article

I have never really thought much about it, but if someone were to put me on the spot and ask me which is more important, the visual or the tactile senses, I would probably choose the visual. This rant by Bret Victor has successfully changed my mind, though. I think what really convinced me was the example he gave on tying shoelaces. Indeed, I could easily tie my laces with my eyes closed, but if I numbed my hands or fingertips, I wouldn’t be able to do it anymore. This enlightenment has made me realize that I’ve never even considered the tactile aspect of any of my works, even though the first part of the course didn’t have room for it. I’m excited to take this into account for my upcoming projects since they involve physical human interaction.

I really liked the way Victor described today’s technology of iPads and phones as Pictures Under Glass. When phrased in this way, it actually makes the technology seem so boring and unintuitive. Like what do you mean our “revolutionary” technology doesn’t use the human tactile sense at all, probably one of our greatest assets? It actually gets more absurd the more I think about it.

When I moved on to the follow-up article, I found some of the comments and his response to them quite funny. But amongst all of this, the line that really stood out to me was, “Channeling all interaction through a single finger is like restricting all literature to Dr Seuss’s vocabulary.” I believe this line perfectly sums up the point Bret was trying to bring across about iPad technology only using your fingers. Where real literature’s scope goes to Shakespeare and beyond, leaving a human being with words only from a Dr. Seuss book is the dumbest thing, anyone can see that. So why can’t people see that limiting one’s fingertips- the body part with the highest density of nerve endings- to a flat, smooth screen, is dumb too? Overall, this has been one of my favorite readings so far, very eye-opening.

Concept

I got the idea for this assignment after watching a campus cat swiftly pounce upon a tiny lizard on the wall in front of the arts center. It got me thinking about reflexes and reaction times and how I can create a mini-game out of this using Arduino. The game basically works like this:
There are two LEDs, one red and one green, one button, and one potentiometer. When the green LED turns on, the player must press the button as fast as they can. If they press it in time, the green LED flashes thrice, indicating a win, otherwise the red LED flashes, indicating the loss. If the green LED flashes, they can see their reaction time in milliseconds on the screen as well. The potentiometer controls the difficulty of the game in two ways. Firstly, it controls the random waiting period (i.e. how long the Arduino waits before turning on the green LED). A decrease in the delay between instances would need the player to be on higher alert. Secondly, it controls the reaction time threshold (i.e. how fast the player must press the button to win). At the easiest setting, the player has 400ms to react, and at the hardest, only 100ms.

Image & Video Documentation

The code

// potentiometer (A0), button (2), red LED (9), green LED (8)

const int potPin = A0;
const int buttonPin = 2;
const int greenLED = 8;
const int redLED = 9;

void setup() {
  pinMode(greenLED, OUTPUT);
  pinMode(redLED, OUTPUT);
  // use a resistor in the microcontroller instead of on the breadboard
  // causes inverted logic: not pressed - HIGH, pressed - LOW
  pinMode(buttonPin, INPUT_PULLUP); // Internal pull-up resistor
  Serial.begin(9600);
  randomSeed(analogRead(A5)); // Add some randomness
}

void loop() {
  Serial.println("Get ready");
  digitalWrite(redLED, LOW);
  digitalWrite(greenLED, LOW);
  delay(2000); // brief pause before start

  // Read difficulty from potentiometer
  int potValue = analogRead(potPin);
  int waitTime = map(potValue, 0, 1023, 1000, 3000); // 1s to 3s random wait range
  // potValue = 0 then threshold = 400 ms; 
  // potValue = 1023 then threshold = 100 ms (hard)
  int threshold = map(potValue, 0, 1023, 400, 100);  

  // Random gap time before LED turns on
  int randomDelay = random(1000, waitTime);
  delay(randomDelay);

  // Start the test
  digitalWrite(greenLED, HIGH);
  unsigned long startTime = millis();

  // Wait for button to be pressed or timeout
  bool pressed = false;
  unsigned long reactionTime = 0;

  while (millis() - startTime < 2000) {       // 2 seconds max to press button
    if (digitalRead(buttonPin) == LOW) {
      pressed = true;   // player successfully pressed the button
      reactionTime = millis() - startTime;    // calculate the player's reaction time
      break;
    }
  }

  digitalWrite(greenLED, LOW);    // turn off green LED

  if (pressed && reactionTime < threshold) {
    // Player wins
    // Display reaction time
    Serial.print("Your reaction time: ");
    Serial.println(reactionTime);

    // Flash green LED thrice to indicate win
    for (int i = 0; i < 3; i++) {
      digitalWrite(greenLED, HIGH);
      delay(150);
      digitalWrite(greenLED, LOW);
      delay(150);
    }

  } else {
    // Player loses
    Serial.println("Too slow!");
    // Flash red LED thrice to indicate loss
    for (int i = 0; i < 3; i++) {
      digitalWrite(redLED, HIGH);
      delay(200);
      digitalWrite(redLED, LOW);
      delay(200);
    }
  }

  delay(1000); // small 1s pause between game rounds
}

Reflection & Future Improvements

I had a lot of fun making this project. I think the physical setup was not too complicated, and still gave an effective output through the implemented code. In future versions, I think it would be nice to have a buzzer that gives different sounds on win/loss conditions, and maybe also implement a score counter that keeps track of how many times the player won throughout multiple rounds. I believe it would also be effective to add another LED (probably yellow) that blinks during the “get ready” phase of each round.

Physical Computing’s Greatest Hits (and misses)

Sometimes I feel like the authors of our readings can read my mind. Usually, my first thought when thinking of a project idea is that I don’t want to do something that’s already done, but the author starts by saying not to think that way. I guess he is right, and that no work is ever truly ‘original’, but always takes inspiration from somewhere or something.

One of the applications of physical computing that really struck me was the Dance Dance Revolution. I realized that I never really thought much about the mechanics of it, but it’s crazy to think that just having a few switches on the floor managed to create this dynamic that took over the world. I also don’t think that I’ve ever encountered the theremin instrument, but it seems very interesting that even though it’s a physical object affected by your actions, you can’t actually think about the actions, but must actually think about the music.

The author’s section on Dolls and Pets also got me thinking about how that concept has evolved compared to the year 2008. The image in the reading shows normal dolls physically connected to a device, but nowadays we have toys with the screens built in, so that the doll/pet seems even more life-like. An example of this is the Eilik Interactive Robot Pet, which went viral on social media a while ago (I actually really want one, but they’re so expensive).

Making Interactive Art: Set the Stage, Then Shut Up and Listen

I agree with everything the author says throughout this reading, however, I do think that it’s easier said than done. He mentions, “Some will be emotionally moved, some will not get it, others won’t care.” This dynamic is why I believe that artists tend to offer interpretation notes about their artwork to their audience. I think it’s quite painful to see people not understand or not care about an interactive work you poured hours of your time into. Therefore, to save themselves from this struggle, artists tend to offer the script beforehand. On a similar note, this is also the approach I took with the first half of this class. Every time I showed someone my work, I would be quick to explain the concept behind it before they even began to explore, making sure that they didn’t do anything “wrong”. But after reading the insightful comparison done by the author between planning interactive artworks and a director working with actors, I think I’ve really understood that an interactive artwork is only completed by the audience’s curiosity. I’m excited to apply this new approach to my own interactive artworks, where the audience is nudged to discover the statement of the artwork by themselves.