Author: Maryam Emaan Fiyaz

Concept

The Interactive Fruit Catcher Game combines physical hardware inputs with digital visuals to create an engaging and multisensory experience. The goal of the game is to catch falling fruits of various colors into a basket by pressing the corresponding colored buttons. Each button press generates a musical note, adding an auditory layer to the gameplay. The fruits fall faster as the game progresses, making it increasingly challenging. Players have three lives, and the game tracks both the current score and the highest score, fostering competitiveness.

Design

Interaction Design

The interaction design centers around three key elements:

1. User Inputs: Physical colored buttons (Red, Yellow, Green, Blue) corresponding to the fruits.
2. Visual Feedback:
   • Correct button presses result in the fruit being “caught,” and the score increases.
   • Incorrect presses or missed fruits deduct a life and provide visual feedback.
3. Auditory Feedback:
   • Each button press generates a unique musical note, which adds a playful sound layer.

Implementation

Hardware

1. Arduino Components:
   • Four Colored Buttons:
     • Red → Strawberries
     • Yellow → Bananas
     • Green → Green Pear
     • Blue → Blueberries
   • Speaker: Plays musical notes tied to each button.
   • Wiring and Connections:
     • Buttons connect to specific digital pins on the Arduino.
     • Power is supplied through a USB cable.
2. Challenges in Physical Computing:
   • Learning to solder the arcade buttons took time.
   • The wiring was difficult due to loose connections. I tried several approaches like alligator clips and direct connections but ended up using a combination of male-to-male, female-to-female, and male-to-female wires, which I secured using electrical tape.
   • Ensuring stable connections was critical for gameplay.

Schematic

Arduino Code

The Arduino code detects button presses and sends data (letters r, y, g, b) to the p5.js sketch via serial communication. Debouncing logic ensures that a single button press is registered cleanly, and each button press triggers a tone via the buzzer.

Key Features:

1. Reads input from buttons using digitalRead.
2. Sends corresponding data to p5.js.
3. Plays tones through the speaker.

Arduino Code:

// Define button pins
#define BUTTON_R 8
#define BUTTON_Y 2
#define BUTTON_G 4
#define BUTTON_B 7

void setup()
{
  Serial.begin(9600);

  // Configure button pins as INPUT_PULLUP
  pinMode(BUTTON_R, INPUT_PULLUP);
  pinMode(BUTTON_Y, INPUT_PULLUP);
  pinMode(BUTTON_G, INPUT_PULLUP);
  pinMode(BUTTON_B, INPUT_PULLUP);
}

void loop()
{
  // Check each button and print the corresponding letter
  if (digitalRead(BUTTON_R) == LOW) // Button R pressed
  {
    Serial.println("r");
    delay(200); // Debounce delay
  }

  if (digitalRead(BUTTON_Y) == LOW) // Button Y pressed
  {
    Serial.println("y");
    delay(200); // Debounce delay
  }

  if (digitalRead(BUTTON_G) == LOW) // Button G pressed
  {
    Serial.println("g");
    delay(200); // Debounce delay
  }

  if (digitalRead(BUTTON_B) == LOW) // Button B pressed
  {
    Serial.println("b");
    delay(200); // Debounce delay
  }
}

p5.js Sketch

The p5.js sketch manages the game visuals, logic, and interactions. It reads the serial data from Arduino and maps it to fruit colors.

Key Features:

1. Fruit Animation: Fruits of different colors fall from the top of the screen.
2. Game Logic:
   • Correct button presses “catch” the fruits and increase the score.
   • Wrong presses or missed fruits deduct lives.
3. Increasing Difficulty: The fruit speed increases over time.
4. Audio Feedback: Musical notes are played for each button press.

Communication with Arduino:

• Data (r, y, g, b) sent from the Arduino is read using readSerial() and processed in the game logic.

 

Aspects I’m Proud Of

Despite the challenges, I am particularly proud of:

• The Basket and Overall Theme:
  • The game’s design, with colorful fruits falling into a basket, is cheerful and visually appealing.
  • The integration of physical buttons adds an arcade-like feel, making it more interactive.
• Completion of the Physical Setup:
  • Learning how to solder and fixing wiring issues was a huge milestone for me. I finally achieved a stable setup by creatively using multiple types of wires and securing them with electrical tape.
• The Multisensory Experience:
  • The combination of visuals, button presses, and musical notes makes the game engaging and unique.

Challenges and Areas for Improvement

Challenges

1. Physical Computing:
   • Learning soldering took time.
   • Fixing loose connections between the breadboard and buttons was tedious.
2. Serial Communication:
   • Connecting the Arduino to p5.js was challenging due to initial errors. Debugging and ensuring a stable connection required significant effort.
3. Falling Fruit Logic:
   • While the p5.js sketch initially worked, it started bugging out the day before the showcase. Debugging the logic for fruit falling and collision detection caused a lot of stress and worry.

Future Improvements

1. Musical Notes Based on Songs:
   • I initially planned for the chords to align with a specific song. As the player presses buttons, the chords would play in sequence, creating a recognizable melody. Implementing this would add depth to the auditory feedback.
2. Improved Visual Feedback:
   • Add animations for missed fruits or incorrect button presses.
3. Enhanced Stability:
   • Use a more permanent solution for physical wiring, such as a custom PCB or cleaner soldered connections.
4. Gameplay Features:
   • Introduce power-ups or different types of fruits for variety.
   • Add multiple difficulty levels or a multiplayer mode for added fun.

Conclusion

This project successfully combines hardware and software to deliver an engaging and interactive game. The process of designing the hardware, troubleshooting physical connections, and integrating serial communication has been a valuable learning experience. I am proud of the final output and excited about the potential improvements for future iterations.

IMG_3671

IMG_3682

 

For user testing, I connected my p5.js sketch to the Arduino and tested the gameplay with a friend who had a general idea of the game. The user interacted with the arduino buttons and the falling fruits synced to the beats of the song.

The feedback highlighted that the timing between the falling fruits and button presses felt responsive and engaging. The user appreciated the dynamic note generation for each fruit, which added a musical layer to the experience. Minor improvements were suggested for button responsiveness and visual feedback when a fruit was missed.

The next steps include building the box for the basket and integrating the arcade buttons into it, which I have soldered. This test successfully validated the functionality of the game logic, Arduino integration, and user interaction, paving the way for completing the physical setup.

finalProject_UserTesting

Concept 

An interactive game that combines physical hardware inputs with digital visuals to create a fun, engaging experience. The objective of the game is to catch falling fruits of different colors into a basket by pressing the corresponding colored buttons. Each button plays a musical note, adding an auditory layer of feedback to the gameplay. The speed of the falling fruits increases as the game progresses, making it more challenging. Players have three lives, and the game tracks both the current score and the highest score achieved, fostering a sense of competition.

Inspiration 

The concept is inspired by classic arcade-style games like “Fruit Ninja” and “Tetris,” which involve quick reflexes and colorful visuals. I wanted to create a game that not only challenges the player’s reaction time but also provides a multisensory experience by integrating music and physical interaction. The picnic theme stems from the cheerful idea of catching fruits for a basket, evoking a sense of fun and lightheartedness. The addition of musical notes tied to the button presses makes the game playful and unique, while the increasing difficulty ensures sustained engagement.

Implementation Plan 

Hardware:

1. Arduino Components:
   • Four colored buttons: Each button corresponds to a fruit color (e.g., red for apples, yellow for bananas).
   • A piezo buzzer or speaker: Plays a unique note when a button is pressed, creating a musical effect.
2. Connections:
   • Each button will be connected to a specific digital pin on the Arduino.
   • A buzzer will also be connected to a pin to generate sounds.
3. Power Supply: A USB cable will power the Arduino during gameplay.

Software:

1. Arduino Code:
   • The buttons will detect user input and send signals to the p5.js sketch via serial communication.
   • Debounce logic will be implemented to prevent multiple readings from a single press.

1. p5.js Sketch:
   • Fruit Animation:
     • Fruits of various colors will fall from random positions at the top of the screen toward a basket at the bottom.
     • The basket will automatically “catch” fruits when the correct button is pressed.
   • Game Logic:
     • The player scores points for catching fruits correctly.
     • Missing a fruit or pressing the wrong button reduces a life.
     • The game ends when the player loses all three lives.
   • Difficulty:
     • The speed of the fruits will gradually increase to challenge the player as the game progresses.
   • Scoreboard:
     • A live scoreboard will display the current score and the highest score achieved.

Interaction 

• User Inputs: The player uses physical buttons to interact with the game, linking the tactile experience of pressing buttons with the visual and auditory feedback.
• Feedback Mechanisms:
  • Musical notes for correct actions.
  • On-screen effects for catching or missing fruits.
  • A real-time display of lives and scores for progress tracking

Graham Pullin’s Design Meets Disability is an insightful exploration of how design can transcend functionality to create assistive technologies that are inclusive, aesthetically appealing, and empowering for individuals with disabilities. Pullin’s central argument—that assistive devices should prioritize both usability and desirability—challenges conventional approaches that often view disability through a purely medical or technical lens.

What stood out most to me was Pullin’s emphasis on collaboration between designers and users. This participatory approach ensures that devices are not only practical but also reflective of personal identity and style. For instance, the idea of prosthetic limbs being treated as fashion statements rather than something to be concealed feels revolutionary, turning potential stigma into an opportunity for self-expression.

Pullin also highlights a broader societal impact: designing with disability in mind benefits everyone, not just individuals with disabilities. This aligns with the concept of universal design, where solutions developed for specific needs often lead to innovations applicable to all.

The reading left me questioning why more industries don’t adopt this approach. It’s a reminder that inclusive design is not just a moral imperative but a creative opportunity to enrich lives and push the boundaries of innovation.

The following are the p5 codes and the Arduino code for the  exercises that we had done in class. I have used a potentiometer as my sensor.

Exercise 1 :
P5 Code

let serial;
let xPos = 0;

function setup() {
  createCanvas(800, 400);

  // Initialize serial
  serial = new p5.WebSerial();
  serial.open(); // Opens serial connection

  // Listen for incoming data
  serial.on("data", serialEvent);
}

function draw() {
  background(220);
  ellipse(xPos, height / 2, 50, 50);
}

function serialEvent() {
  let data = serial.readLine(); // Read data from Arduino
  if (data.length > 0) {
    xPos = map(Number(data), 0, 1023, 0, width); // Map sensor value to canvas width
  }
}

Arduino Code

int potPin = A0;

void setup() {
  Serial.begin(9600);
}

void loop() {
  int potValue = analogRead(potPin);
  Serial.println(potValue); // Send value to p5.js
  delay(10);
}

Exercise 2:
P5 Code

let serial;
let brightness = 0;

function setup() {
  createCanvas(400, 200);

  // Initialize serial
  serial = new p5.WebSerial();
  serial.open(); // Opens serial connection
}

function draw() {
  background(220);
  brightness = map(mouseX, 0, width, 0, 255); // Map mouse position to brightness
  serial.write(brightness + "\n"); // Send brightness to Arduino
  fill(0);
  text("Brightness: " + brightness, 10, 20);
}

Arduino Code

int ledPin = 9;

void setup() {
  pinMode(ledPin, OUTPUT);
  Serial.begin(9600);
}

void loop() {
  if (Serial.available() > 0) {
    int brightness = Serial.parseInt();
    analogWrite(ledPin, brightness); // Set LED brightness
  }
}

Exercise 3:
P5 Code

let serial;
let balls = [];
let wind = 0;

function setup() {
  createCanvas(400, 400);
  serial = new p5.WebSerial();
  serial.open();

  // Initialize balls
  for (let i = 0; i < 10; i++) {
    balls.push(new Ball(random(width), random(height / 2)));
  }

  serial.on("data", serialEvent);
}

function draw() {
  background(220);
  for (let ball of balls) {
    ball.applyForce(createVector(wind, 0.1)); // Apply wind
    ball.update();
    ball.display();
    if (ball.isBouncing()) {
      serial.write("bounce\n"); // Send bounce signal to Arduino
    }
  }
}

function serialEvent() {
  let data = serial.readLine();
  if (data.length > 0) {
    wind = map(Number(data), 0, 1023, -0.2, 0.2); // Map sensor value to wind
  }
}

class Ball {
  constructor(x, y) {
    this.pos = createVector(x, y);
    this.vel = createVector(0, 0);
    this.acc = createVector(0, 0);
    this.r = 10;
  }

  applyForce(force) {
    this.acc.add(force);
  }

  update() {
    this.vel.add(this.acc);
    this.pos.add(this.vel);
    this.acc.set(0, 0);

    // Check for bounce
    if (this.pos.y > height - this.r) {
      this.pos.y = height - this.r;
      this.vel.y *= -0.8; // Reverse and reduce velocity
    }
  }

  display() {
    ellipse(this.pos.x, this.pos.y, this.r * 2);
  }

  isBouncing() {
    return this.pos.y >= height - this.r;
  }
}

Arduino Code

int sensorPin = A0;
int ledPin = 13;

void setup() {
  pinMode(ledPin, OUTPUT);
  Serial.begin(9600);
}

void loop() {
  int sensorValue = analogRead(sensorPin);
  Serial.println(sensorValue); // Send sensor value to p5.js

  // Toggle LED for bounce feedback
  digitalWrite(ledPin, HIGH);
  delay(100);
  digitalWrite(ledPin, LOW);
}

 

 

Bret Victor’s “A Brief Rant on the Future of Interaction Design” and its follow-up article present an interesting, if not particularly deep, critique of current interaction design trends. Victor’s main argument focuses on the limitations of “Pictures Under Glass” – flat touchscreen interfaces that dominate modern technology. He contends these interfaces fail to fully utilize our hands’ tactile capabilities, which are adept at feeling and manipulating objects.While not groundbreaking, Victor’s observation challenges us to think beyond the status quo of interaction design. He makes a valid point about prioritizing tactile feedback in future interfaces, arguing that our sense of touch is fundamental to how we interact with the world.Victor calls for more research into tangible interfaces, dynamic materials, and haptics, suggesting that truly revolutionary interfaces will come from long-term research efforts rather than incremental improvements to existing technology. This emphasis on pushing boundaries through research is noteworthy, even if not explored in great depth.The follow-up responses highlight that while devices like the iPad are revolutionary, they shouldn’t be the end goal of interface design. They also suggest that dynamic tactile mediums capable of physically representing almost anything are a worthy aspiration for future interfaces.Overall, Victor’s message prompts us to consider the full range of human sensory capabilities in developing new interaction models, encouraging us to imagine more intuitive and expressive interfaces.

Concept

Our project, Ibrahim and I, “Echoes of Light,” emerged from a shared interest in using technology to create expressive, interactive experiences. Inspired by the way sound changes with distance, we aimed to build a musical instrument that would react naturally to light and proximity. By combining a photoresistor and distance sensor, we crafted an instrument that lets users shape sound through simple gestures, turning basic interactions into an engaging sound experience. This project was not only a creative exploration but also a chance for us to refine our Arduino skills together.

Materials Used

• Arduino Uno R3
• Photoresistor: Adjusts volume based on light levels.
• Ultrasonic Distance Sensor (HC-SR04): Modifies pitch according to distance from an object.
• Piezo Buzzer/Speaker: Outputs the sound with controlled pitch and volume.
• LED: Provides an adjustable light source for the photoresistor.
• Switch: Toggles the LED light on and off.
• Resistors: For the photoresistor and LED setup.
• Breadboard and Jumper Wires
  

  Code

The code was designed to control volume and pitch through the analog and digital inputs from the photoresistor and ultrasonic sensor. The complete code, as documented in the previous sections, includes clear mappings and debugging lines for easy tracking.

// Define pins for the components
const int trigPin = 5;               // Trigger pin for distance sensor
const int echoPin = 6;              // Echo pin for distance sensor
const int speakerPin = 10;           // Speaker PWM pin (must be a PWM pin for volume control)
const int ledPin = 2;                // LED pin
const int switchPin = 3;             // Switch pin
const int photoResistorPin = A0;     // Photoresistor analog pin

// Variables for storing sensor values
int photoResistorValue = 0;
long duration;
int distance;

void setup() {
  Serial.begin(9600);                // Initialize serial communication for debugging
  pinMode(trigPin, OUTPUT);          // Set trigger pin as output
  pinMode(echoPin, INPUT);           // Set echo pin as input
  pinMode(speakerPin, OUTPUT);       // Set speaker pin as output (PWM)
  pinMode(ledPin, OUTPUT);           // Set LED pin as output
  pinMode(switchPin, INPUT_PULLUP);  // Set switch pin as input with pull-up resistor
}

void loop() {
  // Check if switch is pressed to toggle LED
  if (digitalRead(switchPin) == LOW) {
    digitalWrite(ledPin, HIGH);      // Turn LED on
  } else {
    digitalWrite(ledPin, LOW);       // Turn LED off
  }

  // Read photoresistor value to adjust volume
  photoResistorValue = analogRead(photoResistorPin);
  
  // Map photoresistor value to a range for volume control (0-255 for PWM)
  // Higher light level (LED on) -> lower photoresistor reading -> higher volume
  int volume = map(photoResistorValue, 1023, 0, 0, 255); // Adjust mapping for your setup

  // Measure distance using the ultrasonic sensor
  digitalWrite(trigPin, LOW);
  delayMicroseconds(2);
  digitalWrite(trigPin, HIGH);
  delayMicroseconds(10);
  digitalWrite(trigPin, LOW);
  
  duration = pulseIn(echoPin, HIGH);
  
  // Calculate distance in cm
  distance = duration * 0.034 / 2;

  // Set frequency based on distance in the range of 2-30 cm
  int frequency = 0;
  if (distance >= 2 && distance <= 30) {
    frequency = map(distance, 1, 100, 20000, 2000); // Closer = higher pitch, farther = lower pitch
    tone(speakerPin, frequency);
    analogWrite(speakerPin, volume);  // Apply the volume based on photoresistor reading
  } else {
    noTone(speakerPin);  // Silence the speaker if the distance is out of range
  }

  // Debugging output
  Serial.print("Photoresistor: ");
  Serial.print(photoResistorValue);
  Serial.print("\tVolume: ");
  Serial.print(volume);
  Serial.print("\tDistance: ");
  Serial.print(distance);
  Serial.print(" cm\tFrequency: ");
  Serial.println(frequency);

  delay(100); // Short delay for sensor readings
}

Video Demonstration

In our video demonstration, we showcase how the instrument responds to changes in light and proximity. We toggle the LED to adjust volume and move a hand closer or farther from the ultrasonic sensor to change pitch, demonstrating the instrument’s sensitivity and interactive potential.

Week10

Reflections

The project successfully combines multiple sensors to create a reactive sound device. The integration of volume and pitch control allows for intuitive, responsive sound modulation, achieving our goal of designing an engaging, interactive instrument.

Improvements:
To improve this instrument, we would enhance the melody range, creating a more refined and versatile sound experience. This could involve using additional sensors or more sophisticated sound generation methods to provide a broader tonal range and a richer melody.

Physical Computing’s Greatest Hits (and misses)

In Physical Computing’s Greatest Hits (and Misses), the author discusses recurring themes in physical computing projects, noting that while some ideas are popular for their simplicity and aesthetic appeal, others provide deeper interactive potential. These themes offer valuable learning opportunities and room for original interpretation, encouraging students not to shy away from them despite their repeated use. The author reviews various common projects, including musical instruments, interactive gloves, video mirrors, and mechanical pixels, each presenting unique interaction possibilities.

One example that stood out to me was the drum gloves. This project is intriguing because it uses a familiar gesture—tapping—to create music, leveraging a natural human rhythm. The gloves transform finger movements into drum sounds, adding a playful and structured interaction that feels intuitive. By applying sensors to each fingertip, this project highlights how everyday gestures can be amplified through physical computing, making it accessible and engaging. The concept demonstrates how physical computing can make technology feel natural, bridging the gap between digital and physical interaction.

Making Interactive Art: Set the Stage, Then Shut Up and Listen

In Making Interactive Art: Set the Stage, Then Shut Up and Listen, Tom Igoe argues that interactive art should let participants interpret and engage on their own terms, without over-explanation from the artist. By giving only minimal cues, artists create an environment for discovery, allowing users to shape their own experiences. I agree with Igoe; interactive media should be intuitive and self-explanatory, engaging participants in a personal way. This approach aligns with our midterm project presentations, where we were encouraged to figure out each other’s games independently before hearing the creator’s explanation, fostering genuine exploration and interaction.

Dynamic Light Play: Interactive LED Control with Potentiometer and Button

Concept

The aim of this project was to use a combination of analog and digital inputs to control two LEDs (one red, one blue) in a creative and functional way. The potentiometer serves as an analog input, allowing us to control the brightness of the red LED, while a button acts as a digital input to control the blue LED. This setup meets the assignment requirement to use at least one analog sensor and one digital sensor, with one LED controlled in an analog fashion and the other in a digital fashion.

Materials Used

• Arduino Uno board
• Breadboard
• Red LED and Blue LED
• Potentiometer (analog sensor)
• Button (digital sensor)
• Resistors:
  • 10kΩ for the button’s pull-up resistor
  • 330Ω for each LED
• Jumper Wires

Arduino Code

 

const int potPin = A0;       
const int buttonPin = 2;    
const int redLEDPin = 9;     
const int blueLEDPin = 8;    

void setup() {
  pinMode(buttonPin, INPUT_PULLUP); 
  pinMode(redLEDPin, OUTPUT);       
  pinMode(blueLEDPin, OUTPUT);      
}

void loop() {
  
  int potValue = analogRead(potPin);

  
  int redLEDIntensity = map(potValue, 0, 1023, 0, 255);

 
  analogWrite(redLEDPin, redLEDIntensity);

  
  int buttonState = digitalRead(buttonPin);

  
  if (buttonState == LOW) {
    digitalWrite(blueLEDPin, HIGH);  
  } else {
    digitalWrite(blueLEDPin, LOW);   
  }

  
  delay(10);
}

Implementation

Video Demonstration

IMG_3370

Reflections

This project was a practical way to understand the difference between analog and digital inputs and outputs. The potentiometer provided smooth, gradual control over the brightness of the red LED, giving me hands-on experience with analog input. The button provided simple on/off control for the blue LED, demonstrating digital input.

Challenges:

• Button Debouncing: Without the slight delay, the button would sometimes register multiple presses in a quick succession due to “bouncing.” Adding a delay(10); solved this issue.

Future Improvements:

• Adding more LEDs or sensors to expand the circuit’s functionality.
• Using more advanced analog sensors, such as a temperature or light sensor, to explore other applications.

Project Overview

This project is a hands-free, light-activated night light designed to automatically turn on an LED when it’s dark and turn it off when it’s light. Using a light-dependent resistor (LDR) as a sensor, the circuit detects ambient light levels and toggles the LED accordingly. This is ideal as a night light or as an indicator light in dark conditions.

Materials Used

• Arduino Uno: The microcontroller board used to control the circuit.
• Jumper Wires: For making connections between components.
• Resistors:
  • 330 Ohm resistor (for the LED, to limit current).
  • 10k Ohm resistor (as a pull-down resistor for the LDR to ensure stable readings).
• LED: Acts as the night light, illuminating when ambient light is low.
• Light-Dependent Resistor (LDR): Detects the light level in the environment and changes its resistance accordingly.
  

  Arduino Code

  The following code reads the light level from the LDR and turns the LED on when the light level drops below a specified threshold (indicating darkness) and off when the light level is above this threshold (indicating light):

int ldrPin = A0;           
int ledPin = 9;           
int threshold = 500;       

void setup() {
  pinMode(ledPin, OUTPUT);    
  Serial.begin(9600);         
}

void loop() {
  int ldrValue = analogRead(ldrPin);  
  Serial.println(ldrValue);           
  
  // Check if the light level is below the threshold
  if (ldrValue < threshold) {
    digitalWrite(ledPin, HIGH);  
  } else {
    digitalWrite(ledPin, LOW);   
  }
  
  delay(100);  
}

Improvements

• • Automatic Brightness Adjustment: Incorporate PWM to adjust the LED brightness gradually based on light levels, creating a dimmer effect in low light.
  • Sensitivity Adjustment: Experiment with different threshold values based on the ambient light conditions in various environments. This ensures accurate and responsive behavior.

Video Demonstration