Final Project Proposal

Inspiration

I’ve always tried making most of my projects personal/unique to me in some way,  and really I wanted to do the same for the final project! When I tried thinking about things I love, the first thing that came to mind was books and reading in general. Reading has always been one of my favorite activities and for this project, I tried to think of creating something that could make the experience of reading even more enjoyable and interactive.

I then remembered how a few years ago, I had stumbled upon a video of an automatic page turning mechanism and while it was cool, I couldn’t help but think: What if this was more interactive, more human-like? That’s when the concept of a reading buddy robot began to take shape in my mind. I envisioned a small, friendly robot sitting across from me, holding my book as I read. Not just flipping pages mechanically, but responding to my actions—like looking at the left or right page when I’m reading—and maybe even bringing some humor or companionship to the reading experience. The idea of combining robotics with interactivity felt like a perfect challenge that merges both technical and creative elements.

Concept

The reading buddy robot will be designed to face me while holding a book flat out on a  table, almost like a tiny librarian or companion. Its main features will include:

  1. Page Flipping: With a simple gesture or button press, the robot will flip the book’s pages one at a time. I want this mechanism to be smooth and reliable, even for thinner pages.
  2. Interactive Movements: The robot will “look” at the left page when I’m reading on the left side and the right page when I’m reading on the right. This can be achieved using a combination of servo motors and sensors or even a webcam paired with eye-tracking software. It could also give some sort of indication to about my distance from the book, so that I don’t strain my eyes too much.
  3. Personality Through Interaction: The robot won’t just be functional; it will have a personality! For example, if I take too long on one page, it could nod or tilt its “head” as if it’s waiting for me to move on. I’m thinking of adding some sound effects or p5.js animations to make it feel like the robot is alive.

To add a modern twist, I want to connect the robot to a digital interface using p5.js, allowing it to display fun visuals or stats about my reading progress. Maybe it could track how long I’ve been reading and cheer me on, or even throw a playful tantrum if I ignore it for too long!

Parts I’m Uncertain About

While I’m excited about this project, there are a few areas that I’m still figuring out:

  1. The Page Flipping Mechanism:
    I’ve seen examples of flat arm page-turners, but adapting this mechanism to hold the book upright and flip pages smoothly is going to be tricky.  Thin book pages might make this more challenging, so I’ll need to experiment with different designs and materials.
  2. Making It Human-Like:
    The goal is to make the robot feel interactive and expressive, but that’s easier said than done. I can imagine using servos to give it a moving “head,” but what if it feels too mechanical? Adding personality through sounds, lights, or p5.js animations could help, but I’m still brainstorming the best way to tie all of this together.

Final Thoughts

This project is still in its early stages, but I’m really excited about where it’s heading. The idea of combining robotics, interactivity, and reading feels like a perfect way to push myself creatively and technically. I know there will be challenge but I look forward to working on it and seeing how it turns out!

Assignment 8 – In-Class Exercises (by Jheel and Linda)

EXERCISE 01: ARDUINO TO P5 COMMUNICATION 

Make something that uses only one sensor on arduino and makes the ellipse in p5 move on the horizontal axis, in the middle of the screen, and nothing on arduino is controlled by p5.

In our solution, the we used a photoresistor to detect the surroundling light level. The ball in the screen then moves to the left when it’s dark and to the right when it’s bright.

Please find the demonstration video here:

Exercise 1

Please find the p5.js sketch here 

let port, reader, writer;
let serialActive = false;

async function getPort(baud = 9600) {
  let port = await navigator.serial.requestPort();
  await port.open({ baudRate: baud });

  // create read & write streams
  textDecoder = new TextDecoderStream();
  textEncoder = new TextEncoderStream();
  readableStreamClosed = port.readable.pipeTo(textDecoder.writable);
  writableStreamClosed = textEncoder.readable.pipeTo(port.writable);

  reader = textDecoder.readable
    .pipeThrough(new TransformStream(new LineBreakTransformer()))
    .getReader();
  writer = textEncoder.writable.getWriter();

  return { port, reader, writer };
}

class LineBreakTransformer {
  constructor() {
    this.chunks = "";
  }

  transform(chunk, controller) {
    this.chunks += chunk;
    const lines = this.chunks.split("\r\n");
    this.chunks = lines.pop();
    lines.forEach((line) => controller.enqueue(line));
  }

  flush(controller) {
    controller.enqueue(this.chunks);
  }
}

async function setUpSerial() {
  noLoop();
  ({ port, reader, writer } = await getPort());
  serialActive = true;
  runSerial();
  loop();
}

async function runSerial() {
  try {
    while (true) {
      const { value, done } = await reader.read();
      if (done) {
        reader.releaseLock();
        break;
      }
      readSerial(value);
    }
  } catch (e) {
    console.error(e);
  }
}

let rVal = 0; // Value from photoresistor

function setup() {
  createCanvas(640, 480);
  textSize(18);
}

function draw() {
  background(245, 245, 200);
  fill(0);

  if (!serialActive) {
    text("Press Space Bar to select Serial Port", 20, 30);
  } else {
    text("Connected", 20, 30);
    text('Photoresistor Value = ' + str(rVal), 20, 50);

    // Map the sensor value to the x-position of the ellipse
    let xPos = map(rVal, 0, 1023, 0, width);
    fill(100, 123, 158);
    ellipse(xPos, height / 2, 50, 50);
  }
}

function keyPressed() {
  if (key == " ") {
    setUpSerial();
  }
}

function readSerial(data) {
  if (data != null) {
    let fromArduino = trim(data); // Remove any whitespace
    rVal = int(fromArduino); // Convert the string to an integer
  }
}

Please find the Arduino code here:

void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);
}


void loop() {
  // put your main code here, to run repeatedly:
  int sensorValue = analogRead (A2);
  Serial.println (sensorValue);
  Serial.write(sensorValue);
  delay (1);
}

EXERCISE 02: P5 TO ARDUINO COMMUNICATION 

Make something that controls the LED brightness from p5.

In our solution, we used an LED whose brightness changes according to the slider value.

  • Please find the demonstration video here

Exercise 2

  • Please find the p5.js sketch here 
let brightness = 0; 
let slider;

function setup() {
  createCanvas(400, 200);
  //make the slider
  slider = createSlider(0, 255, 127); 
  slider.position(10, 10);
  slider.style('width', '300px');

  let serialButton = createButton("Connect to Arduino");
  serialButton.position(10, 50);
  serialButton.mousePressed(setUpSerial);
}

//troubleshoot
function readSerial(data) {
  console.log("Received data:", data); // Log the received data to the console
}

function draw() {
  background(220);
  brightness = slider.value(); 
  fill(0);
  textSize(16);
  text(LED Brightness: ${brightness}, 10, 100);

  // Send brightness value to Arduino via Serial
  if (serialActive) {
    writeSerial(brightness + "\n"); // Append a newline character
  }
}
  • Please find the Arduino code here:
int ledPin = 9; // PWM pin connected to LED

void setup() {
  Serial.begin(9600);
  pinMode(ledPin, OUTPUT);
}

void loop() {
  int bright;

  if (Serial.available() > 0) {
    bright = Serial.parseInt();
    analogWrite(ledPin, bright);
  }
}

EXERCISE 03: BI-DIRECTIONAL COMMUNICATION

Every time the ball bounces one led lights up and then turns off, and you can control the wind from one analog sensor.

  • Please find the p5.js sketch here 
let port, reader, writer;
let serialActive = false;

async function getPort(baud = 9600) {
  let port = await navigator.serial.requestPort();
  await port.open({ baudRate: baud });

  // create read & write streams
  textDecoder = new TextDecoderStream();
  textEncoder = new TextEncoderStream();
  readableStreamClosed = port.readable.pipeTo(textDecoder.writable);
  writableStreamClosed = textEncoder.readable.pipeTo(port.writable);

  reader = textDecoder.readable
    .pipeThrough(new TransformStream(new LineBreakTransformer()))
    .getReader();
  writer = textEncoder.writable.getWriter();

  return { port, reader, writer };
}

class LineBreakTransformer {
  constructor() {
    this.chunks = "";
  }

  transform(chunk, controller) {
    this.chunks += chunk;
    const lines = this.chunks.split("\r\n");
    this.chunks = lines.pop();
    lines.forEach((line) => controller.enqueue(line));
  }

  flush(controller) {
    controller.enqueue(this.chunks);
  }
}

async function setupSerial() {
  noLoop();
  ({ port, reader, writer } = await getPort());
  serialActive = true;
  runSerial();
  loop();
}

async function runSerial() {
  try {
    while (true) {
      const { value, done } = await reader.read();
      if (done) {
        reader.releaseLock();
        break;
      }
      readSerial(value);
    }
  } catch (e) {
    console.error(e);
  }
}


////////////////////////////////////////////////////////////

let velocity;
let gravity;
let position;
let acceleration;
let wind;
let drag = 0.99;
let mass = 50;
let windSpeed;


function setup() {
  createCanvas(640, 360);
  noFill();
  position = createVector(width / 2, 0);
  velocity = createVector(0, 0);
  acceleration = createVector(0, 0);
  gravity = createVector(0, 0.5 * mass);
  wind = createVector(0, 0);
  textSize(16);
}

function draw() {
  background(0);
  
  if (serialActive){
    wind.x = map(windSpeed, 0, 1023, -1, 1);
  }
  applyForce(wind);
  applyForce(gravity);
  velocity.add(acceleration);
  velocity.mult(drag);
  position.add(velocity); 
  acceleration.mult(0);
  ellipse(position.x, position.y, mass, mass);
  console.log("Wind Speed: " + windSpeed); // For debugging

  if (position.y > height - mass / 2) {
    velocity.y *= -0.9; // A little dampening when hitting the bottom
    position.y = height - mass / 2;

    // Send signal to Arduino on bounce
    if (serialActive) {
      sendBounceSignal();
    }
  }

  if (!serialActive) {
    fill(255);
    text("Press SPACE to connect to Serial Port", 20, 30);
  } else {
    fill(0, 255, 0);
    text("Connected to Serial Port", 20, 30);
  }
}

function applyForce(force) {
  // Newton's 2nd law: F = M * A or A = F / M
  let f = p5.Vector.div(force, mass);
  acceleration.add(f);
}

function keyPressed() {
  if (keyCode == LEFT_ARROW) {
    wind.x = -1;
  }
  if (keyCode == RIGHT_ARROW) {
    wind.x = 1;
  }
  if (key == " ") {
    if (!serialActive) {
      setupSerial();
    } else {
      mass = 50;
      position.y = -mass;
      velocity.mult(0);
    }
  } 
}

async function setupSerial() {
  try {
    noLoop();
    // ({ port, reader, writer } = await getPort());
    
    port = await navigator.serial.requestPort();
    await port.open({ baudRate: 9600 });
    
    
    // Create a TextDecoderStream to decode incoming bytes to text
    const textDecoder = new TextDecoderStream();
    const readableStreamClosed = port.readable.pipeTo(textDecoder.writable);

    // Create the reader to read from the decoded text stream
    reader = textDecoder.readable
    .pipeThrough(new TransformStream(new LineBreakTransformer())) // Optional: split data by lines
    .getReader();
    
    writer = port.writable.getWriter();
    serialActive = true;
    // Start reading data after successfully opening the port
    runSerial();
    loop();
  } catch (err) {
    console.error("Serial connection failed:", err);
    serialActive = false;
  }
}

function readSerial(data) {
  if (data != null) {
    let fromArduino = trim(data); // Remove any whitespace
    console.log(data);
    if (fromArduino !== "") {
      fromArduino = parseInt(fromArduino, 10);
      windSpeed = int(fromArduino); // Convert the string to an integer
      
    }
  }
}


async function sendBounceSignal() {
  try {
    if (writer) {
      await writer.write(new TextEncoder().encode("bounce\n"));
    }
  } catch (err) {
    console.error("Failed to send bounce signal:", err);
  }
}

  • Please find the Arduino code here:
int ledPin = 13; // LED pin#
int sensorPin = A0;
int windValue;

void setup() {
  pinMode(ledPin, OUTPUT);
  Serial.begin(9600);
  
}

void loop() {
  windValue = analogRead(sensorPin); // Read the sensor value
  Serial.println(windValue); // Send the value to the serial port
  // Serial.write(windValue);
  

  if (Serial.available() > 0) {
    String data = Serial.readStringUntil('\n'); // Read data until newline
    if (data == "bounce") {
      digitalWrite(ledPin, HIGH);
      delay(100); // Keep the LED on for 100ms
      digitalWrite(ledPin, LOW);
    }
  }
}

 

Week 11 – Reading Reflection

While reading Design Meets Disability, I found myself thinking about the tension between discretion and visibility in design for disability. The idea that glasses have successfully transitioned from being a purely functional medical device to a fashionable accessory made me wonder whether this will also happen for hearing aids or prosthetics. Are we, as a society, more comfortable with visible “disabilities” like poor eyesight because they’ve been normalized for centuries? If so, what would it take for other assistive devices to undergo a similar transformation? I thought about a friend’s reluctance to wear a brace for a sprained wrist in high school—she felt like it felt like it called too much attention to itself, even though it was helping her heal. Why does function often have to feel like a trade-off with self-expression?

The discussion of the Eames leg splint also sparked questions about the role of aesthetics in design born out of necessity. I found it very interesting how such a utilitarian object could later inspire iconic furniture. Does this mean that every functional object has hidden potential for beauty, or does it require someone with a particular eye to uncover it? It made me think about the way my grandfather modified his walking cane to reflect his personality—adding small carvings and even painting parts of it to make it less clinical. Could design for disability embrace more of this individuality from the start? The reading left me questioning not just how we design for diverse bodies, but how we might better celebrate the creativity that comes from these constraints.

Assignment 7 – Music Box! (by Jheel and Linda)

For this assignment, our task was to create a musical instrument using Arduino (involving at least one digital and one analog sensor).

Inspiration

While initially brainstorming ideas for this project I thought a lot about conventional musical instruments (like the piano and guitar) and about recreating them in some way. But in the end, I decided that I wanted to do something more personal and unique. I suddenly remembered how I’ve always loved music boxes (since I was a kid) and found them really cute. If you don’t know what a music box is, it’s basically a small, nicely decorated box that plays music when you open it. Many of them also involve twisting a handle multiple times in order to get the music to play. Also, the resulting music is usually just a soft and soothing instrumental song.

So, for this project, we decided to make a simple version of a music box!

Concept

 The basic concept of this project is that the music box will play music when exposed to light and pause when it’s dark. To achieve this, we used a photoresistor as the analog sensor to detect light and a button as the digital sensor to be able to switch between songs. For simplicity, we used three tracks that keep rotating among each other every time the button is pressed.

Implementation

Components and Setup

To bring this vision to life, we used:

  • Adafruit Arduino Music Maker Shield: This allows us to play music files from an SD card, controlled via the Arduino. By downloading the corresponding library and integrating the shield on the Arduino, we ensured smooth playback of direct mp3 files.
  • Speaker Attachment: We initially thought we could use a buzzer, but we realized it produced a harsh, mechanical sound that didn’t match our desired soothing effect. We then switched to a consumable speaker from the IM lab, which provided a richer and more pleasant sound. We connected the speaker to the Music Maker Shield for optimal sound quality.
  • Circuit with Photoresistor: We programmed the photoresistor to detect light levels, pausing music playback when it’s dark and resuming when light is detected.
  • Push Button: We added a push button to switch between multiple music files, allowing users to vary their listening experience.
  • Prototyping Shield and Battery: To make the setup compact enough to fit inside the box, we transitioned from a breadboard to a prototyping shield. Adding a battery also made the setup portable, enhancing usability.
Crafting the Wooden Box

Since traditional music boxes are often crafted from wood, we decided to make an elegant wooden enclosure for the circuit. We sourced templates for the box online and used a laser cutter (with the help of lab assistants of course) to neatly cut the different pieces of the box, which could then be glued together to create a polished exterior.

 After a few rounds of trial and error with different templates, we arrived at a design that worked well for housing the components. To personalize it further, we designed a music-themed engraving for the top using Adobe Illustrator and laser-cut the design, adding a refined aesthetic touch.

Code

Github URL

Here’s some of the code that’s responsible for switching between songs:

// Check if button was pressed
  int buttonState = digitalRead(buttonPin);
  Serial.println(buttonState); // Prints the button state for debugging
  Serial.println(songs[songIndex]);

  if (buttonState == 1 && lastButtonState == 0) {  // Button press detected
    // musicPlayer.pausePlaying(true);
    musicPlayer.stopPlaying();
    delay(50);  // Debounce delay
    songIndex = (songIndex + 1) % 3;  // Move to the next song in the array
    musicPlayer.startPlayingFile(songs[songIndex]);  // Play the next song
    // Serial.println(songs[songIndex]);
    
  }
  lastButtonState = buttonState;

  // File is playing in the background
  if (musicPlayer.stopped()) {
    musicPlayer.startPlayingFile(songs[songIndex]);
  }

 

Challenges

One major challenge was achieving the desired sound quality. As tested in class, the initial buzzer didn’t deliver the calm, melodic experience we wanted, so switching to a consumable speaker improved the overall audio quality. We occasionally encountered malfunctions, likely due to wiring or compatibility issues with the speaker, and are continuing to troubleshoot these.

Final Assembly

After laser-cutting the box and engraving the design, we assembled all the components inside. The transition from breadboard to prototyping shield and the compact battery setup allowed everything to fit neatly within the box.

Reflections and Further Improvements

In the end, we were pretty happy with the modernized feel of the music box that combines traditional charm with interactive technology. By responding to light, the music box offers a unique, customizable listening experience that blends classic design with new-age tech.

Further improvements could include adding more sound files and testing different speakers to improve sound quality. We could even attempt to make it like a traditional music box with a rotating handle on the side and maybe even some visual rotating elements.

Week 10 – Reading Reflection

I found Bret Victor’s A Brief Rant on the Future of Interaction Design really interesting because of how it critiques the touchscreen-focused “pictures under glass” model that limits our physical interaction with technology. This definitely changed my perspective on the direction technology is taking. Earlier, I thought that touchscreens are revolutionary and easy-to-use but I am now thinking about why we are willing to settle for such a limited interaction, devoid of the tactile feedback that has been essential to human experience for thousands of years. Shouldn’t we be designing technology that truly connects with the way we physically engage with the world?

I was also interested in Victor’s thoughts about the stagnation of interaction design. Instead of visionary advancements, we get small incremental changes that feel innovative but don’t really use the full potential of human capabilities. For example, I often find myself reading on my Kindle or jotting down notes on my iPad for the sake of convenience, even though I’m sacrificing the tactile feel of a physical book or the natural flow of writing on paper. This makes me further question: Are we sacrificing sensory richness for convenience? What would it take for the tech industry to prioritize deeper, more meaningful interactions over merely efficient or visually impressive ones? His argument has led me to reevaluate my own ideas about technology and wonder whether our devices could one day feel like a natural extension of our bodies, rather than just tools for quick tasks.

Assignment 6 – Fluttering Pulse

For this pr0ject, the task was to get information from at least one analog sensor and at least one digital sensor (switch), and use this information to control at least two LEDs, one in a digital fashion and the other in an analog fashion, in some creative way.

Inspiration

My project was inspired by a few of my favorite things. Firstly, I’ve always loved butterflies and origami, and I often find myself folding little origami butterflies from random bits of paper I find lying around. Adding on to that, my all time favorite chocolate is Ferrero Rocher, and sometimes, after enjoying one, I fold the shiny gold wrapper into a butterfly. They look beautiful, with the gold and shimmer giving them an extra touch.

I wanted this project to feel personal, so I thought, why not bring these gold butterflies to life? It’s a fun “best out of waste” idea that lets me reuse something in a creative way. I was also inspired by the 2023 Abu Dhabi art installation Pulse Island by Rafael Lozano-Hemmer. It basically consists of an array of over 4,000 Edison lightbulbs that illuminate the plant life of Lulu Island. Each lightbulb glimmers to the rhythm of the heartbeat of a different participant. The way this piece interacts with people’s heartbeats was fascinating and sparked the idea of making something that reacts to simple sensor inputs.

Concept

For this project, I created three butterflies from Ferrero Rocher wrappers. I placed a yellow LED on each butterfly, intending to bring them to life in a way that responds to a person’s heartbeat. Using a pulse sensor, the setup detects the person’s heart rate, sending analog input to the LEDs. This means the yellow lights on each butterfly will pulse in sync with the person’s heartbeat, with the intensity of the lights varying based on the heart rate’s intensity (BPM value). Also, a blue LED is used to indicate when the setup is active, turning on and off with a button that acts as a digital switch. When the button is pressed, the blue LED lights up, showing that the butterflies are ready to respond to the heartbeat input.

The overall blinking effect of the LEDs  gives each butterfly a sense of “life,” as if they are fluttering gently in time with the person’s pulse.

Here’s what the butterflies and pulse sensor looked like:

 

 

 

 

 

Implementation

For this project, I used an Arduino UNO board, a pulse sensor, three yellow LEDs (connected in parallel), one blue LED, one push button, two 330-ohm resistors, and one 10K-ohm resistor. The setup is designed to illuminate the yellow LEDs in sync with a person’s heartbeat and control the blue LED as an indicator of the system’s state.

The push button acts as a digital input, turning the setup on or off. When the button is pressed, the blue LED is activated (handled through digitalWrite) to indicate that the system is running. Meanwhile, the pulse sensor detects the user’s heart rate, sending the analog data to control the brightness of the yellow LEDs. The brightness/intensity is mapped to the person’s beats per minute (BPM). Also, the yellow LEDs pulse in sync with the heart rate (they glow every time a heartbeat is detected), creating a visual heartbeat effect. The code initializes and configures the pulse sensor to read the heartbeat, and then it uses a map function to translate the BPM into a suitable brightness level for the LEDs.  The analogWrite function is used to control the intensity of the yellow LEDs based on the BPM.

For the pulse sensor, I used the PulseSensorPlayground library, which simplifies the integration of heart rate monitoring by providing functions to read and interpret heartbeat data easily.

I also created a custom delay function to avoid the blocking nature of the regular delay() function. Normally, delay() pauses all processes in the Arduino, which would prevent the button presses from being detected during the delay period. My custom delay function allows for non-blocking delays, so the button state can still be read while waiting.

GitHub URL

Here’s some of the code (which controls the mapping of pulse rate to LED brightness and makes the LEDs beat in sync with the person’s heartbeat) that I am proud of:

// Checks if the setup is active
  if (flag == true){ 
    digitalWrite(digitalLED, HIGH); // Turns on the blue LED to indicate active state
    if (pulseSensor.sawStartOfBeat()) {            // Constantly test to see if "a beat happened".
      int myBPM = pulseSensor.getBeatsPerMinute();  // Calls function on our pulseSensor object that returns BPM as an "int".
                                                    // "myBPM" hold this BPM value now. 
      Serial.println("♥  A HeartBeat Happened ! "); // If test is "true", print a message "a heartbeat happened".
      Serial.print("BPM: ");                        // Print phrase "BPM: " 
      Serial.println(myBPM);                        // Print the value inside of myBPM. 
      int val = map(myBPM, 0, 180,0, 255);          // Maps the BPM value to an LED brightness level
      Serial.print("val:");
      Serial.println(val);                          // Prints the mapped brightness value
      analogWrite(LED, val);                        // Sets the brightness of the yellow LED based on the BPM
      myDelay(100);                                 // Waits for 100 milliseconds
      analogWrite(LED, 0);                          // Turns off the yellow LED to create a pulsing effect
    }
  }

Schematic Circuit Diagram

Reflections and Further Improvements

I’m really happy with how this project turned out and how the concept came to life. It was really rewarding to incorporate elements that are personally meaningful to me, like the origami butterflies, and also to successfully learn how to use a new type of sensor.

In the future, I’d love to add sound to enhance the interactive aspect, maybe syncing gentle tones with the heartbeat to make the experience more immersive. I’d also like to expand the setup visually, adding more butterflies and refining the aesthetics to make the project even more captivating.

Resources

Pulse Island

PulseSensorPlayGround

Week 9 – Reading Reflection

When I read Physical Computing’s Greatest Hits (and misses), I was struck by the balance it finds between tradition and innovation. As someone interested in interactive art, I often struggle with the feeling that certain concepts have been “done to death,” especially in areas like musical interaction or video mirrors. Yet, this reading challenges the idea that originality is a prerequisite for value. It made me realize that even revisited ideas can be unique, with room for personal touches and new approaches. When I reflect on my own creative process, I realize I’ve often dismissed ideas early-on because they felt unoriginal. This piece has shifted my perspective, encouraging me to see repeating themes as opportunities to explore, rather than limitations on creativity. The drum glove and theremin examples showed me how familiar frameworks can offer endless variation, with each design shaped by subtle gestures and intent. This makes me question: how can I bring my own voice to recurring ideas, perhaps through unexpected input methods or reimagining context?

Reading Making Interactive Art: Set the Stage, Then Shut Up and Listen made me rethink people’s approaches to interactive art, especially around how much control artists should exert over the audience’s experience. It suggests that by interpreting or guiding responses too directly, they might be limiting the unique, personal interactions that make interactive art powerful. Instead of letting the work speak for itself, artists might be imposing an idea that restricts the audience’s freedom to find their own meaning. The idea of “setting the stage” but leaving the experience open-ended raises questions about artists’ role as the creator. Can they trust the audience to interpret without their input? How do they balance giving enough context with allowing space for exploration? This reading reframes interactive art as a conversation, where the audience has the final word, and I think it challenges people to think about designing more for discovery than direction.

Assignment 5 – Beany and Snowball

For this assignment, the task was to create an unusual switch (that opens and closes a circuit) using Arduino.

Inspiration

Anyone who knows me knows how much I love plushies! When I started brainstorming project ideas, I wanted to create something cute that included two of my own plushies—Beany the bear and Snowball the penguin. I was also inspired by the movie Inside Out, especially how the character Joy is represented with a warm yellow glow. This gave me the idea to create a scene where one of my plushies says “hi” to the other, sparking a little moment of joy between them.

Concept

The main concept behind my project is to create a glowing yellow heart made of LEDs that lights up when Snowball places his hand on Beany’s shoulder, representing a moment of joy. Using Arduino, I wanted to wire a small circuit with conductive foil attached to each plushie. When the two plushies make contact, the conductive foils complete the circuit, sending a signal through the Arduino to illuminate the LED heart.

Implementation

To bring my idea to life, I started by cutting a piece of cardboard and arranging spots for the LEDs in the shape of a heart. After placing the LEDs, I wired them in parallel, connecting their negative terminals to the ground on the Arduino board. I connected the positive terminals of the LEDs to a 330-ohm resistor, which was then connected to one piece of conductive foil on one of the plushies. The other piece of conductive foil, attached to the second plushie, was connected to the Arduino’s 5V terminal. This setup meant that when the two plushies “touched,” the circuit would complete through the resistor, causing the LED heart to glow.

Here’s a picture of just the LED heart and its wiring:

When I filmed the final product, I got a (teeny-tiny) bit carried away and ended up creating a short, movie-like video with the Inside Out theme song as background music. It really added to the emotion I wanted to capture in this project :))

 

Reflections and Further Improvements

Overall, I’m pretty happy with how this project turned out! The wiring was a bit challenging due to the number of LEDs I used, which sometimes caused the wires to get tangled, but overall, the process went smoothly. As mentioned previously, I also invested some time in filming the final product to give it a more story-like feel.

For future improvements, I might consider adding some sort of sound element that plays whenever the circuit is completed. It would also be nice to make the setup more compact or even fully portable. Lastly, I’d love to explore adding different light patterns or colors to show various emotions, which would create even more expressive interactions between the plushies.

Week 8 – Reading Reflection

Reflecting on Donald Norman’s Emotion and Design: Attractive Things Work Better, an important takeaway was the idea of balancing beauty with usability. For example, in a high-stress setting like healthcare, should a designer prioritize simplicity over aesthetics, or is there value in making things look good too? Norman’s argument that beautiful things “work better” because they make users more forgiving of small flaws is intriguing, but this could also lead designers to mask deeper usability problems with aesthetics. We also see this often in consumer tech, where sleek design sometimes hides complex interfaces. Could a beautifully designed device make users miss important issues if they assume it’s as reliable as it looks? And in general, does an attractive design make people more likely to trust or value a product, even if it has flaws? How do designers keep things both beautiful and easy to use across all types of situations, from routine tasks to high-stress ones?

The article on Margaret Hamilton’s journey was definitely inspiring and got me thinking about how often we overlook the contributions of people who don’t fit the traditional expectations of an industry. What struck me most was her foresight about potential astronaut errors (that had initially been dismissed). She saw risks that others didn’t, not because she knew more, but because she questioned the assumption of perfection in human operators. This made me think about the ways might current technology still rely on assumptions that could lead to serious errors. Are we relying too heavily on “perfect” tech and “infallible” systems, overlooking potential risks just because they seem unlikely?

Overall, I think these readings emphasize the need to balance beauty and function in design while staying aware of human error, showing that technology works best when it adapts to real, everyday challenges.

Midterm Project – Rocky the Pet Rock!

Full Sketch: https://editor.p5js.org/jheel2006/full/zaxMyymUX

Inspiration

When I first started brainstorming this project, I was drawn to the idea of creating something related to pet care. I’ve always thought of pets as a source of joy and responsibility, so I thought a virtual pet could be a fun and meaningful experience. However, I wanted to add a fun twist to the concept. I thought it might be interesting and humorous to make the pet something unexpected, like an inanimate object. This was how Rocky the Pet Rock was born!

I drew and designed Rocky myself, with five distinct mood states: Very Happy, Slightly Happy, Meh, Slightly Sad, and Very Sad. These moods are influenced by the player’s actions, which determine Rocky’s overall well-being. By making Rocky a rock instead of a living creature, I could play with the idea of what it means to care for something that doesn’t typically need attention, and how even the simplest interactions can create an emotional connection. The result is an experience that feels both playful and oddly meaningful.

Concept

The concept behind Rocky the Pet Rock is to create a simple, interactive experience where you take care of a virtual pet rock for a short period of time. Rocky’s mood is initially set to 50 (out of a total of 100). The game has a timer of 100 seconds until which your goal is to keep Rocky as happy as possible.

The game begins with an introduction screen after which  the user can reach the instructions screen by clicking anywhere. Clicking on the instructions screen leads to the main game screen.

After reaching the main game screen, there are three ways to change Rocky’s mood:

Feed: The user can choose from five different types of food to feed Rocky. However, he may or may not like the food that is chosen. If he likes the food, his mood goes up, if he doesn’t like the food, his mood goes down and if he is okay with/indifferent to the food, his mood remains unchanged. These food preferences are randomized in every round of the game. Rocky can only be fed twice during the game.

Play: Here, the user can help Rocky play his favorite game – dodgeball. In this mode, balls keep getting generated from the right side of the screen at Rocky’s current position and the user must use the up and down arrow keys to move Rocky and avoid these balls. A ball hitting Rocky results in a mood decrease and playing with Rocky without hitting any obstacles leads to a consistent mood increase.

Relax: In this mode, the user helps Rocky unwind by choosing between three different music tracks (of different genres) to play. Similar to the feed option, Rocky likes one of these tracks, doesn’t like another and is okay with the third. Choosing any of these track affects his mood depending on his preferences (randomized in every round of the game). The relax option can only be chosen once during the game.

The game ends if the timer reaches 0 or if Rocky’s mood decreases to 0. The game then displays game over screen, allowing the user to begin again by clicking on the screen. The game over screen displays Rocky’s final mood with a message. If the user restarts the game, it resets to the original conditions and begins again.

Implementation:

The implementation of the game is built around the three core modes of interaction: Feed, Play, and Relax. Each one of these had its own set of challenges to implement and integrate into the game as a whole.

  • One aspect of the Feed mode that I’m particularly proud of is the way the food options are arranged around Rocky. I wanted the interaction to be visually appealing, so I placed the five food choices in a circular pattern around Rocky, making the user’s selections feel more natural and connected to the character.
    Randomizing Rocky’s food preferences in each round was another key part of the implementation. Every time the game begins, Rocky likes, dislikes, or feels neutral about different foods, keeping the player on their toes and making each round feel unique. Once the player makes a selection, feedback is immediately displayed on the main screen, showing how Rocky’s mood is affected. It was also essential to disable the feed option after Rocky has been fed twice to prevent over-feeding and keep the game balanced. Implementing this functionality added a strategic element to the game, as players must choose carefully when and what to feed Rocky, knowing their options are limited.

    Here’s some of the feed functionality code!

    function displayFoods() {
      // Display Rocky at the center
      rocky.display();
      
      
      let centerX = rocky.x - rocky.size/8; // Center of the screen (same as Rocky's x)
      let centerY = rocky.y - rocky.size/8; // Center of the screen (same as Rocky's y)
      let radius = 170; // Distance from the center to where the foods will be displayed
    
      let angleStep = TWO_PI / foods.length; // Angle between each food item
    
      for (let i = 0; i < foods.length; i++) {
        let angle = i * angleStep; // Calculate angle for each food item
        let foodX = centerX + radius * cos(angle); // X-coordinate based on angle
        let foodY = centerY + radius * sin(angle); // Y-coordinate based on angle
        
        image(foodImages[i], foodX - 40, foodY - 40, 110, 110); // Display food images
    
        fill(0);
        textAlign(CENTER); // Center the text below each food item
        text(foods[i], foodX+20, foodY + 80); // Display food names under the images
      }
    function setFoodFeedback(food) {
      let moodEffect = foodPreferences[food];
      rocky.mood += moodEffect; // Update Rocky's mood immediately
    
        // Set food feedback based on mood effect
      if (moodEffect > 0) {
        foodFeedback = `Rocky likes ${food}!`;
      } else if (moodEffect < 0) {
        foodFeedback = `Rocky doesn't like ${food}!`;
      } else {
        foodFeedback = `Rocky is okay with ${food}.`;
      }
    
    
      // Clear any previous timeout to avoid overlap
      if (foodFeedbackTimeout) {
        clearTimeout(foodFeedbackTimeout);
      }
    
      // Clear the feedback after 3 seconds
      foodFeedbackTimeout = setTimeout(() => {
        foodFeedback = "";
      }, 3000);
    }
  • The Play mode was an exciting challenge to develop, especially with the random generation of obstacles. I designed the game so that obstacles are generated from the right side of the screen based on Rocky’s current position, ensuring that the game feels dynamic and responsive to the player’s movements. This adds an element of unpredictability, requiring the player to stay alert as they guide Rocky to dodge incoming obstacles.
    I liked the idea of how Rocky’s mood was closely tied to how well the player performs during the game. As long as Rocky successfully avoids the obstacles, his mood gradually increases, rewarding careful play. However, if Rocky gets hit by an obstacle, his mood takes an immediate hit.. This balance between mood progression and obstacle avoidance makes the Play mode both engaging and meaningful, as it directly ties the player’s performance to Rocky’s emotional state.

    Here’s some of the play functionality code!

else if (playing) {
        if (backgroundMusic.isPaused()){
          backgroundMusic.play();
        }

        // Play mode
        timer.display();
        rocky.update();
        rocky.display();
        moodMeter.display(rocky.mood);


        // Generate obstacles
        if (frameCount % 40 == 0) {
          obstacles.push(new Obstacle());
        }

        for (let i = obstacles.length - 1; i >= 0; i--) {
      obstacles[i].update();
      obstacles[i].display();

          // Check for collision
          if (obstacles[i].hits(rocky)) {
            hitSound.play();  // Play hit sound on collision
            rocky.mood -= 10; // Decrease mood on collision
            obstacles.splice(i, 1); // Remove obstacle after collision
            continue; // Skip to the next iteration to avoid calling offscreen() on a spliced element
          }

          // Remove obstacles that go off-screen
          if (obstacles[i].offscreen()) {
            obstacles.splice(i, 1);
          }
    }

        // Check if mood or timer runs out to end the game
        if (rocky.mood <= 0 || timer.isTimeUp()) {
          gameState='gameOver';
          playing = false;
          resetGame();
        }

These are the Rocky and Obstacle classes, which also have elements of the game functionality:

// Rocky class
class Rocky {
  constructor() {
    this.mood = 50;
    this.x = windowWidth / 2;
    this.y = windowHeight / 2 + 100;
    this.size = 100; // The size of Rocky (matches the previous ellipse size)
    this.speed = 5; // Speed for moving up and down
  }

  display() {
    // fill(150);
    // ellipse(this.x, this.y, 100, 100); // Rocky as an ellipse
    if (this.mood >=80){
      rockyImage = rockyImageHappy;
    }
    else if (this.mood >=60){
      rockyImage = rockyImageHappyish;
    }
    else if (this.mood >=40){
      rockyImage = rockyImageMeh;
    }
    else if (this.mood >=20){
      rockyImage = rockyImageSadish;
    }
    else {
      rockyImage = rockyImageSad;
    }
    
    image(rockyImage, this.x - this.size / 2 - 25, this.y - this.size / 2 , this.size+40, this.size); // Display the image with Rocky's coordinates
  }


// Obstacle class for the play mode
class Obstacle {
  constructor() {
    this.x = windowWidth;
    this.size = random(60,75); // Size of obstacle
    // this.y = random(0, windowHeight - this.size); // Random y-position
    this.y = rocky.y; // Spawn the obstacle at Rocky's current y position
  }

  display() {
    // fill(255, 0, 0);
    // rect(this.x, this.y, this.size, this.size);
    image(obstacleImage, this.x, this.y, this.size, this.size);
  }

  update() {
    this.x -= obstacleSpeed; // Move the obstacle to the left
  }

  offscreen() {
    return this.x < -this.size; // Check if the obstacle has moved off-screen
  }

  hits(rocky) {
    let d = dist(this.x, this.y, rocky.x, rocky.y);
    return d < this.size / 2 + 50; // Check for collision with Rocky
  }
}
  • The Relax mode shares similarities with the Feed mode in terms of randomizing Rocky’s music preferences, but the added challenge here was managing the background music. The player can choose from three different music tracks (a lullaby, a funk track and some lo-fi beats), each representing a different genre. Rocky’s mood responds to the track selection—he likes one, dislikes another, and is indifferent to the third, all of which are randomized each round. This randomness keeps the experience fresh and unpredictable, just like the food preferences.
    A technical challenge was ensuring that the background music from the Relax mode would stop properly once the player entered the Relax mode and restarted when the player exited the mode or when the track finished playing. It was also important to return to the main screen after the track ended (after 10 seconds) while displaying the feedback and effect on Rocky’s mood.

    Here’s some of the relax functionality code!

    function randomizeMusicPreferences() {
      let shuffledTracks = shuffle([10, 0, -10]); // One increases, one decreases, one is neutral
      for (let i = 0; i < relaxTracks.length; i++) {
        musicPreferences[i] = shuffledTracks[i]; // Map tracks to mood effects
      }
    }
    function startRelaxMode() {
        relaxing = true;
        playing = false;
        displayTrackButtons(); // Display buttons for selecting tracks
        backButton.display(); // Show back button to return to main screen
      
    }
    
    
    // Function to display track buttons
    function displayTrackButtons() {
      // Display each track button
      for (let trackButton of trackButtons) {
        trackButton.display();
      }
    }
    
    // Function to play the selected track and show feedback
    function playRelaxTrack(trackIndex) {
      if (isRelaxing) {
            return; // If a track is already playing, prevent any other interaction
        }
    
        isRelaxing = true; // Set to true once a track starts playing
        selectedTrack = trackIndex;
        relaxTracks[trackIndex].play(); // Play the selected track
        
        setTimeout(function() {
            relaxTracks[trackIndex].stop(); // Stop the track after 10 seconds
          showRelaxFeedback(trackIndex);
          relaxing = false;      // Return to the main game state
          relaxDisabled = true;       // Disable the Relax button after it's used
          isRelaxing = false;
          rocky.x = windowWidth / 2;
          rocky.y = windowHeight / 2 + 100;
        }, 10000); // Play for 10 seconds
    }
    
    // Function to show feedback based on Rocky's preferences
    function showRelaxFeedback(trackIndex) {
        let moodEffect = musicPreferences[trackIndex]; // Get mood effect for the selected track
        rocky.mood += moodEffect; // Adjust Rocky's mood accordingly
    
        // Set relax feedback based on the mood effect of the track
        if (moodEffect > 0) {
            relaxFeedback = `Rocky likes this track!`;
        } else if (moodEffect === 0) {
            relaxFeedback = `Rocky is okay with this track.`;
        } else {
            relaxFeedback = `Rocky doesn't like this track!`;
        }
    
        // Clear any previous timeout to avoid overlap
      if (relaxFeedbackTimeout) {
        clearTimeout(relaxFeedbackTimeout);
      }
    
      // Clear the feedback after 3 seconds
      relaxFeedbackTimeout = setTimeout(() => {
        relaxFeedback = "";
      }, 3000);
      // Clear any previous timeout to avoid overlap
     
    }

    Reflections and Further Improvements

    Overall I’m quite satisfied with how the game turned out, especially considering the many stages it went through – from my vague, initial idea to its final implementation. Some things that I can consider doing to add to it could be making the Feed mode slightly more interactive by having a drag-and-drop feature (to make the user feel like they’re actually feeding Rocky). I could also add some sound feedback when Rocky’s mood changes (for instance, when he ate a food he liked or listened to a track he didn’t like). Similarly, I could also add sound feedback when the game ends, reflecting Rocky’s current mood state.

All in all, I really enjoyed the experience of bringing my idea to life. There were quite a few bugs as well as some unexpected behavior that came up along the way, but I’m pretty happy with the final result :))