Category: Spring 2024 – Aaron (Section 001)

CONCEPT:

The Punching Bag Game is an interactive project that combines physical activity with a digital arcade experience. The concept revolves around using a punching bag equipped with a sensor to detect punches and display real-time scores. It’s designed to bring the excitement of a classic arcade game into a physical activity, which inspired me in the first place. The game includes features like score tracking, a fuel bar, sound effects, and a visually engaging gamescreen that responds to the user’s actions. The goal is to create an immersive experience where users can engage physically while enjoying a fun and competitive game.

user testing video

Here is where I placed the sensor on the punching bag:

How Does the Implementation Work?

The project integrates hardware (Arduino) and software (p5.js) to create a seamless interactive system:

Hardware (Arduino):

• An ADXL335 accelerometer attached to the punching bag detects the magnitude of punches.
• An arcade button starts and restarts the game.
• The Arduino processes the sensor data and sends it to p5.js as a formatted packet.

Software (p5.js):

• Displays the title screen, instructions, and gameplay.
• Dynamically updates the score, best score, and fuel bar based on incoming data.
• Plays sound effects and animates the punching bag for a realistic experience.

Description of Interaction Design

1. Title Screen

Features:

• • The game title (“The Punching Bag”) in an arcade-style font.
  • An “Instructions” button leads to a detailed instruction page.
  • A prompt directs users to press the arcade button to start the game.

Design Choices:

• • •  I used bold, retro fonts and a simple overall design to mimic an actual arcade game.

2. Gameplay

• User Interaction:
  • Players punch the bag, and the accelerometer calculates the strength.
  • Scores are displayed dynamically, increasing gradually like in classic arcade games.
  • A fuel bar at the bottom fills based on the user’s performance.
  • Sound effects play after each punch.
• Visual Elements:
  • The punching bag vibrates after each punch, mimicking the actual punching bag
  • A restart button allows players to reset the game and try again.

3. Restart Button

• After the game ends, users can press the restart button to reset the game and clear all scores, unlocking the ability to punch again.

Description of Arduino Code

The Arduino is responsible for detecting punches, processing the sensor data, and communicating with p5.js.

How It Works:

1. Punch Detection:
   • The ADXL335 accelerometer calculates the acceleration magnitude of each punch.
   • If the magnitude exceeds a set threshold, it’s mapped to a score value.
2. Data Transmission:
   • The Arduino sends a data packet in the format ,:<value> to p5.js via serial communication.
3. Button Functionality:
   • The arcade button sends a “START” signal to p5.js to initiate gameplay.

void handleAccelerometer() {
  int xRaw = analogRead(xPin);
  int yRaw = analogRead(yPin);
  int zRaw = analogRead(zPin);

  float magnitudeG = calculateMagnitude(xRaw, yRaw, zRaw);

  if (magnitudeG > impactThreshold) {
    int punchValue = map(magnitudeG, impactThreshold, impactThreshold + 10, 0, 2000);
    Serial.print(",:");
    Serial.println(punchValue);
    delay(200);
  }
}

GITHUB

 

Description of p5.js Code

The p5.js sketch displays the visuals and processes incoming data from the Arduino to create a responsive and interactive experience.

Key Features:

1. Dynamic Visuals:
   • Real-time updates to the score and fuel bar.
   • Vibration animation for the punching bag.
2. Audio Feedback:
   • Plays sound effects after each punch.
3. Serial Data Handling:
   • receives incoming data and updates the game state.

function handleSerialData(data) {
  serialBuffer += data; // Add incoming data to the buffer

  // Check for complete packets
  let startMarker = serialBuffer.indexOf("<");
  let endMarker = serialBuffer.indexOf(">");
  if (startMarker !== -1 && endMarker > startMarker) {
    let packet = serialBuffer.substring(startMarker + 1, endMarker); // Extract packet
    serialBuffer = serialBuffer.substring(endMarker + 1); // Remove processed packet

    // Process the packet
    if (packet.startsWith(",:")) {
      let scoreValue = parseInt(packet.substring(2)); // Extract score value
      if (!isNaN(scoreValue)) {
        console.log("Score received:", scoreValue);
        if (!scoreLocked) {
          score = scoreValue;
          fuel = constrain(fuel + scoreValue / 10, 0, 100);
          bestScore = max(bestScore, score);
          scoreLocked = true;
           isVibrating = true; 
        vibrationTimer = 80; 

        // Play punch sound
        if (roundEndSound.isPlaying()) {
         roundEndSound.stop(); 
        }
      roundEndSound.play(); 
        }
      }
    } else if (packet === "START") {
      gameState = "game";
    }
  }
}

 

COMMUNICATION BETWEEN ARDUINO AND P5.JS:

he communication between Arduino and p5.js is essential for the game to function properly, allowing the physical punches on the bag to reflect accurately in the digital interface. When the punching bag is hit, the ADXL335 sensor attached to it measures the force of the punch in terms of acceleration along the x, y, and z axes. The Arduino processes these raw sensor values and calculates the overall magnitude of the punch. If the punch strength exceeds a preset threshold, the Arduino sends a data packet to p5.js in the format ,:<value>, where <value> is the calculated punch strength mapped to a score range which was 2500. In addition to handling the punches, the Arduino also manages the arcade button. When the button is pressed, it sends a “START” command to p5.js. On the title screen, this command transitions the game to the gameplay state, and during gameplay, it acts as a reset function, clearing the scores and unlocking the ability to punch again. On the p5.js side, the sketch listens for incoming data through serial communication. When data is received, it first checks if it matches the  format to avoid any invalid or incomplete packets. Once validated, p5.js takes the score from the data and updates the game elements, such as the score display, fuel bar, and sound effects. To prevent multiple scores from being processed for a single punch, p5.js locks the score after the first valid data packet and ignores any further updates until the game is restarted.

CHALLENGING ASPECTS AND WHAT IM PROUD OF:
Getting the communication between Arduino and p5.js to work was one of the biggest challenges, but I got it to work in the end woohooo!. At first, the Arduino sent multiple data packets for a single punch, causing issues in p5.js, but I fixed this by adding a debounce delay and structuring the data packets. Soldering was also frustrating at first as it was my first time, and the wires kept coming loose. After several attempts, I managed to secure the connections, ensuring the sensor and button worked properly.

AREAS FOR FUTURE IMPROVEMENT:

I believe I could have maybe made the game more interactive by adding different game modes, maybe like a mode where there’s a timer and the user has to keep punching to fill up the fuel bar within that certain time. So in general, adding more features to the game would have been great.

 

IM SHOW

Catch the horse: In Action

Schematic:

Circuit Design

The circuit design includes:

• Inputs:
  • Buttons for Jump, Rock, Bird, and Choice.
  • Ultrasonic sensor for crouching detection.
• Outputs:
  • Servo motor for mechanical horse movement.

Circuit Setup:

Concept: 

“Catch the Horse” is an interactive, physically engaging game that combines a physical mechanical horse with a virtual environment. The game revolves around a man running after a horse, dodging obstacles, and strategising to catch the runaway animal. Using Arduino for hardware control and p5.js for the game logic, the project merges physical and digital interactivity in a seamless experience.

The game is designed for a single player or multiplayer to interact with both the physical and digital world, utilising buttons, sensors, and a motor to create a truly immersive gameplay environment.

Implementation Overview

The implementation is divided into two main components:

1. Hardware (Arduino): Controls the physical setup, including buttons for gameplay interactions, an ultrasonic sensor for crouching detection, and a motor for the mechanical horse.
2. Software (p5.js): Manages the game logic, animations, audio feedback, and communication with the Arduino.

Interaction Design

The player interacts with the game through a combination of physical actions and button presses:

• Jump Button: Press to make the man jump over obstacles like rocks.
• Rock Button: Press to launch rocks as obstacles for the man.
• Bird Button: Press to release birds, which the man must dodge.
• Lasso Button (Choice Button): Triggers a Choice Screen where the player selects between Skill and Luck to catch the horse.
• Crouching: The ultrasonic sensor detects when the player crouches to avoid fences.

The game starts with a running man chasing a horse. Players must navigate these challenges to achieve victory. Victory conditions include successfully throwing the lasso and either rolling a winning dice number (Luck) or capturing the horse via a precise throw (Skill).

Arduino Code

The Arduino code serves as the bridge between the physical components and the digital game. Key features include:

• Button Mapping: Jump, Rock, Bird, and Choice buttons are connected to Arduino pins, triggering serial communication to p5.js.
• Ultrasonic Sensor Integration: Detects crouching and sends the signal to the game.
• Motor Control: Starts and stops the physical horse motor based on game states.

The final Arduino sketch was robust and efficient, thanks to iterative improvements. One notable challenge was integrating the button connected to pin 5 (Choice Button) to handle multiple game states effectively. Solving this required careful adjustments to both Arduino and p5.js logic.

p5.js Code

The p5.js script handles the game logic, animations, and audio. Key highlights include:

• Game States: Playing, Choice Screen, Game Over, and Game Won states dynamically adjust gameplay.
• Animations: The running man and birds are controlled via sprite animations, providing a polished visual experience.
• Audio Integration: Background music tracks play during specific game states (ninja.mp3 during Playing and horse.mp3 during Game Won), enhancing immersion.
• Obstacle Logic: Rocks and birds have cooldowns to maintain game balance.

The most challenging part was ensuring smooth communication between the Arduino and p5.js. For example, handling simultaneous button presses and sensor inputs required careful synchronisation to avoid bugs.

You can try the game here:

 

Title: Testing Phase of “Catch the Horse” – Insights and Improvements

With the development of “Catch the Horse” completed, I conducted user testing to evaluate the intuitiveness, playability, and overall experience of the game. The goal of this phase was to observe how players interacted with the game, identify areas where they struggled, and determine how the gameplay mechanics could be made more intuitive and accessible.

User Testing Overview

During testing, participants were asked to engage with the game without receiving any prior instructions or guidance. The goal was to simulate the experience of a first-time player encountering the game. I recorded their interactions and noted points of confusion, questions they asked, and their overall feedback.

Observations and Findings

1. What Users Figured Out on Their Own
   • Jump, Bird, and Rock Buttons:
     Players intuitively understood the jump mechanic and the functions of the bird and rock buttons. These actions had immediate visual feedback (e.g., the cowboy jumped, birds appeared, or rocks were thrown), which made the controls feel natural and responsive.
2. What Needed Explanation
   • Lasso Button (Choice Button):
     Players struggled to understand how the lasso button worked. I had to explain that pressing the lasso button initiated the “Choice Screen” and that they could select between Skill or Luck to catch the horse.
     • Skill vs. Luck:
       The difference between the Skill and Luck options was unclear without explanation. Participants were unsure why they would choose one option over the other.

• Interaction with Rocks and Birds:
  Although players understood how to use the rock and bird buttons, they were initially confused about how the cowboy was supposed to interact with these obstacles. For example, they weren’t sure if the rocks could be dodged or destroyed and if the birds required a specific action to avoid.

  What Worked Well

• Physical Integration:
  The physical crouching mechanic, detected by the ultrasonic sensor, added an engaging and immersive element to the game. Users enjoyed the novelty of having to physically move to interact with the game.
• Visual Feedback:
  Immediate visual feedback for the jump, bird, and rock mechanics allowed players to quickly understand these actions without explanation.
• Game Flow and Balance:
  Cooldowns for the horse’s abilities (mud, fences, and booster) were well-received, as they maintained a fair and balanced gameplay experience.

  Lessons Learned

  • Mapping Between Controls and Gameplay:
    Intuitive mapping between controls and gameplay actions is critical. For example, the jump button and crouching were easy to grasp because the controls directly mirrored the in-game actions. However, abstract mechanics like the lasso required additional explanation due to their more complex interactions.
  • The Importance of Feedback:
    Immediate feedback helped players connect their actions to in-game effects. Enhancing feedback for less intuitive mechanics (like the lasso) will likely make the game easier to pick up.
  • Balancing Physical and Digital Gameplay:
    Players found the integration of physical actions (like crouching) and digital gameplay highly engaging. This balance between physical and virtual interaction should remain a cornerstone of the game’s design.

    Next Steps

    1. Add an Instructions Page or Tutorial:
       Include a brief tutorial or instructions page at the beginning of the game to explain the mechanics of the lasso button, Skill vs. Luck, and how to interact with obstacles like birds and rocks.
    2. Enhance In-Game Prompts:
       Add dynamic text prompts or animations during gameplay to guide players through challenging mechanics. For example:
       1. “Press J to Jump Over Rocks!”
       2. “Dodge Birds to Avoid Game Over!”

  Demonstration
  

Reuse and Recycle: A reflection on Design Meets Disability

In Design Meets Disability, the idea of trickle-down effects made a lot of sense to me. The tech market’s priorities often align with majority demand, leaving designs for the disabled marginalized in terms of research and funding. Sectors like fashion or mass-market tech get the lion’s share of attention, while solutions for disabilities often lack the investment they deserve.

However, the book highlights a crucial skill for designers: recognizing the applicability of existing technology to new problems. While creating something original is commendable, repurposing an established design can be equally impactful. Legal hurdles, like patents, may limit financial gains from such adaptations, but progress shouldn’t be confined by a competitive or profit-driven mindset. True innovation lies in recognizing the value of existing tools to accelerate development and as designers or artists, we have the benefit of the sky being the limit to our creativity. Thus, repurposing existing technology is never frowned over by our kind as artists have the ability to bring their own touch to any project.

Title: Catch the Horse
Mechanical Horse.mp4 – Google Drive

Concept:

“Catch the Horse” is an interactive two-player game that blends physical and virtual gameplay to create an engaging experience. A physical mechanical horse and a digital cowboy chase are brought to life through Arduino and p5.js integration. The game revolves around strategy, reflexes, and decision-making, where one player controls a cowboy trying to catch a runaway horse, while the other player actively defends the horse using tools like mud, fences, and boosters. With immersive elements like crouching to dodge obstacles and precise timing challenges to throw a lasso, the game offers an interactive and entertaining experience.

Overview:

In “Catch the Horse,” a virtual horse escapes its stable and leaves the screen, triggering a motor that animates a physical mechanical horse placed beside the monitor. The cowboy avatar must chase after the horse, dodging obstacles and using a lasso to attempt capture. The horse player can make the chase challenging with mud, fences, and boosters, while the cowboy must rely on strategic decision-making and skill to win.

Gameplay Mechanics

The Cowboy’s Role (Player 1)

• Chasing the Horse:
  • The cowboy avatar moves left and right using a joystick to dodge mud and other obstacles.
  • Physical crouching (detected by an ultrasonic sensor) is required to avoid fences thrown by the horse player.
  • At set intervals, the cowboy can attempt to throw a lasso to capture the horse or a bag of treats.
• Lasso Mechanic:
  The cowboy has two options when throwing the lasso:
  1. Catch the Horse (Luck-Based):
     • A meter appears with an oscillating needle that moves slower, making it easier to time the throw.
     • If successful, the player rolls a virtual dice. A roll of 5 or 6 captures the horse; otherwise, the chase continues.
  2. Catch the Treats (Skill-Based):
     • The meter’s needle oscillates faster, requiring greater precision to time the throw.
     • Success guarantees the bag of treats, which stops the horse and ensures a win.
     • Failure results in no capture, and the chase continues. 

The Horse’s Role (Player 2)

• The horse player uses three buttons to evade capture:
  1. Mud Throw: Throws mud obstacles that the cowboy must dodge with the joystick.
  2. Fence Throw: Launches fences that the cowboy must crouch to avoid.
  3. Booster: Temporarily speeds up the horse, preventing the cowboy from throwing a lasso.
• All buttons have cooldown periods to maintain fairness and balance. 

Challenges and Solutions

1. Mechanical Horse Durability:
   • The motor runs at a controlled speed to prevent stress on the fragile body.
2. Button Cooldowns and Fairness:
   • Cooldown timers for the horse player’s buttons ensure balanced gameplay.
3. Accurate Crouch Detection:
   • Calibrate the ultrasonic sensor to detect movement precisely while avoiding false triggers.

Winning Conditions

• The cowboy wins if they:
  • Successfully throw the lasso and roll a 5 or 6.
  • Successfully capture the bag of treats.
• The horse wins if:
  • The cowboy fails all lasso attempts, or the timer runs out.

 

EXCERCISE 1: ARDUINO TO P5.JS COMMUNICATION

RESULT:

TASK 1 VIDEO

P5 CODE:

let sensorValue = 0; // Variable to store sensor data
function setup() {
  createCanvas(640, 480);
  textSize(18);
  if (!serialActive) {
    setUpSerial(); // Start serial communication with Arduino
  }
}
function draw() {
  // Set the background to dark blue and purple hues based on the sensor value
  background(map(sensorValue, 0, 1023, 50, 75), 0, map(sensorValue, 0, 1023, 100, 150));
  // Map the sensor value to control the ellipse's horizontal position
  let ellipseX = map(sensorValue, 0, 1023, 0, width);
  // Draw the ellipse in the middle of the screen
  fill(255); // White ellipse for contrast
  ellipse(ellipseX, height / 2, 50, 50);
  // Display connection status
  fill(255); // White text for readability
  if (!serialActive) {
    text("Press Space Bar to select Serial Port", 20, 30);
  } else {
    text("Connected", 20, 30);
    // Display the current sensor value
    text("Sensor Value = " + str(sensorValue), 20, 50);
  }
}
function keyPressed() {
  if (key === " ") {
    setUpSerial(); // Start the serial connection when the spacebar is pressed
  }
}
// This function is called by the web-serial library with each new line of data
function readSerial(data) {
  if (data != null) {
    // Parse the sensor value from the Arduino
    sensorValue = int(trim(data));
  }
}

ARDUIN CODE:

int sensorPin = A0; // Sensor connected to A0

void setup() {
  Serial.begin(9600); // Start serial communication
}

void loop() {
  int sensorValue = analogRead(sensorPin); // Read sensor value
  Serial.println(sensorValue); // Send the value to p5.js
  delay(10); // Small delay to avoid overwhelming the serial buffer
}

EXCERCISE 2: P5 TO ARDUINO COMMUNICATION

RESULT:

TASK 2 VIDEO

P5 CODE:

let brightness = 0; // Brightness value to send to Arduino

function setup() {
  createCanvas(640, 480);
  textSize(18);

  // Check if serial is active and set it up if not
  if (!serialActive) {
    setUpSerial(); // Initialize serial communication
  }
}

function draw() {
  background(30); // Dark background
  fill(255); // White text
  text("Use the UP and DOWN arrows to control LED brightness", 20, 30);

  // Display the current brightness value
  text("Brightness: " + brightness, 20, 60);
}

function keyPressed() {
  if (keyCode === UP_ARROW) {
    // Increase brightness
    brightness = min(brightness + 10, 255); // Max brightness is 255
    sendBrightness();
  } else if (keyCode === DOWN_ARROW) {
    // Decrease brightness
    brightness = max(brightness - 10, 0); // Min brightness is 0
    sendBrightness();
  } else if (key === " ") {
    // Start serial connection when spacebar is pressed
    setUpSerial();
  }
}

function sendBrightness() {
  if (writer) {
    // Send the brightness value to Arduino
    writer.write(brightness + "\n");
  } else {
    console.error("Writer is not available. Please connect to the serial port.");
  }
}

ARDUINO CODE:

int ledPin = 9; // LED connected to PWM pin 9
int brightness = 0; // Variable to store brightness value from p5.js

void setup() {
  Serial.begin(9600); // Start serial communication
  pinMode(ledPin, OUTPUT); // Set LED pin as an output
}

void loop() {
  // Check if data is available to read
  if (Serial.available() > 0) {
    // Read the brightness value sent from p5.js
    brightness = Serial.parseInt();

    // Constrain the brightness value to 0-255
    brightness = constrain(brightness, 0, 255);

    // Set the LED brightness
    analogWrite(ledPin, brightness);
  }
}

EXCERCISE 3: BI-DIRECTIONAL COMMUNICATION

RESULT:
TASK 3 VIDEO

P5 CODE:

let velocity;
let gravity;
let position;
let acceleration;
let wind;
let drag = 0.99;
let mass = 50;
let sensorValue = 0; // Variable to store wind value from Arduino
let windStrength = 0; // Wind force determined by the sensor

function setup() {
  createCanvas(640, 360);
  noFill();
  position = createVector(width / 2, 0);
  velocity = createVector(0, 0);
  acceleration = createVector(0, 0);
  gravity = createVector(0, 0.5 * mass);
  wind = createVector(0, 0);

  // Initialize serial communication
  if (!serialActive) {
    setUpSerial();
  }
}

function draw() {
  background(255);

  // Apply gravity
  applyForce(gravity);

  // Apply wind (continuously updated from sensor)
  wind.x = map(sensorValue, 0, 1023, -2, 2); // Map sensor value to a stronger wind range
  applyForce(wind);

  // Update position and velocity
  velocity.add(acceleration);
  velocity.mult(drag);
  position.add(velocity);
  acceleration.mult(0);

  // Draw the ball
  ellipse(position.x, position.y, mass, mass);

  // Check for bounce
  if (position.y > height - mass / 2) {
    velocity.y *= -0.9; // A little dampening when hitting the bottom
    position.y = height - mass / 2;

    // Notify Arduino about the bounce
    sendBounce();
  }
}

function applyForce(force) {
  // Newton's 2nd law: F = M * A
  let f = p5.Vector.div(force, mass);
  acceleration.add(f);
}

// Notify Arduino when the ball bounces
function sendBounce() {
  if (writer) {
    writer.write('1\n'); // Send the bounce signal
  }
}

// Read wind control value from Arduino
function readSerial(data) {
  if (data != null) {
    // Parse the sensor value directly into a variable for wind force
    sensorValue = int(trim(data));
  }
}

// Handle serial setup (using the serial.js file)
function keyPressed() {
  if (key === " ") {
    setUpSerial();
  }
}

ARDUINO CODE:

const int ledPin = 9;      // LED connected to pin 9
const int sensorPin = A0;  // Analog sensor for wind control
int sensorValue = 0;       // Variable to store sensor value from analog pin

void setup() {
  Serial.begin(9600);  // Start serial communication
  pinMode(ledPin, OUTPUT);  // Set LED pin as output
}

void loop() {
  // Read the sensor value and send it to p5.js
  sensorValue = analogRead(sensorPin);
  Serial.println(sensorValue);

  // Check if a bounce signal is received
  if (Serial.available() > 0) {
    char command = Serial.read();
    if (command == '1') {
      // Turn on the LED
      digitalWrite(ledPin, HIGH);
      delay(100);  // Keep the LED on briefly
      digitalWrite(ledPin, LOW);  // Turn off the LED
    }
  }
}