Week 13: Infinity Trials and Errors

Backstory: Giving Up, Trying Again and Moving Forward

I decided to work on my midterm project as it will be less time consuming, but unfortunately the project did not work according to my plan. I made this, which is a 3D flying game (well, was supposed to be a 3D flying game). I tried without sensors first to see if the graphics works well or not and showed this to my roommate as an user opinion, who said it might not be one of the best choices to add this as my final game. I still spend some more time on it but no luck!

Moving Forward: 

After digging more on the feasibility of finishing a nicely done project and the limitations of sensors, I decided to change my project. As I am a fan of making something impactful (which I also criticise about myself because making something fun and relaxing can be an amazing accomplishment), I decided to take Sundarban Mangrove Forest as my game idea and the mythology of Bengal Region as my game.

Context of the myth:

In the Sundarbans myth, Bonbibi, the guardian goddess of the forest, defends the local woodcutters and honey gatherers from the malevolent tiger god, Dakshin Rai. The story highlights a young boy, Dukhey, who becomes a victim of a pact between his greedy employer and Dakshin Rai, aiming to sacrifice him to the tiger god. Bonbibi intervenes, rescuing Dukhey and ensuring his safe return, playing her role as a protector and mediator between the natural world and human interests.

Game flow:

Game Flow On Paper

 

 

 

 

 

 

 

User Testing:

Till this point, the game is done only at the initial stage. The design was done and the flex sensor was working correctly to go to first stage of the game. Graphics:

 

  Start Scene
Game Transition Scene

For the testing, people played only the very beginning part, and they liked it because of the idea and the graphics, which gave me some affirmation that yes, I can move forward with it.

I did second part of the game earlier as my flex sensors were working. This is the link to the second part, Bee, here the sensors were able to connect but calibrating two flex sensors, one for moving the bees and another one to collect honey, was not running smoothly. But, as it was somewhat working at least with keyboards, I kept it as my second part.

 

The third of the game while player needs to run away from the tiger was the crucial one where I keep two sonar sensors side by side and player needs to simulate steps like running, like when left sensor detects obstacles, the right sensor does not as the right leg is up. Again when the right sensor detects obstacles, left sensor does not because now as part of running the left leg is up and right leg is on the ground.

I created a dummy version first to implement the logic and some basic visuals, this one I liked as my logic worked with keyboard properly. Link to see the initial one.    and the second draft done one. I was not satisfied with any of them.

Sensor connection on Arduino:

The Arduino Connections

Unfortunately, I did not take any video but when my roommate tried to play the game, she was very confused with the sensors as it was not working properly.

Only the storyline of the experience was satisfying for her and also for my fellow classmates on saw my works on Wednesday.

I needed to explain my game that what I am trying to achieve. Moreover, as the sensor calibration was very troubling and the movements were very poor, I decided to make it easier.

Having different sensors for different stages, made it tough for users to understand what is happening and why does the user needs to use different sensors for different games. It was challenging from the circuit building to the end level of user testing as well.

I added clear instructions on how to play, what is the mythology about and what are the end goals. Moving on, I again needed to drop the plan of tiger and player chasing along with the bee but I did keep my plan with boat and two sensors.

To get rid of any risk, I followed simple mechanism and rules to build my final game. One thing I was sure after user testing that whoever heard the story or saw it, they all liked it. The only problem was how to design the game play.

 

 

Week 13 User Testing and Final Project

Video:

Drive links:https://drive.google.com/file/d/1tj29Zt4eafPmq3sbn2XQxWIdDe19ptf9/view?usp=sharing
https://drive.google.com/file/d/1iaTtnn3k2h35bS9jtLQnl48PngWvzTUW/view?usp=sharing

User Testing Documentation for the Project
To evaluate the user experience of the game, the following steps were conducted:

Participants: Two users were asked to play the game without prior instructions.
Environment: Each participant was given access to the joystick and mouse, along with the visual display of the game.
Recording: Gameplay sessions were recorded, capturing both screen activity and user interactions with the joystick and mouse.
Feedback: After the session, participants were asked to share their thoughts on the experience, including points of confusion and enjoyment.

Observations from User Testing
Most users instinctively tried to use the joystick to control the player.
Mapping joystick movement to player control was understood quickly.
Dying when hitting the wall was unexpected for both players, but they learned to avoid the walls and play more carefully quickly.

The dual control option (mouse click and joystick button) for starting the game worked well.

Powerups:

Participants found the power-up visuals engaging and intuitive.
Some users struggled to understand the effects of power-ups initially (e.g., what happens when picking up a turtle, or a lightning bolt)
But once they passed through the powerups they understood the effects it had.

Game Objectives:

The goal (reaching the endpoint) was clear to all participants.
Participants appreciated the timer and “Lowest Time” feature as a challenge metric.

What Worked Well
Joystick Integration: Smooth player movement with joystick controls was highly praised.
Visual Feedback: Power-up icons and heart-based life indicators were intuitive.
Engagement: Participants were motivated by the timer and the ability to beat their lowest time.
Obstacle Design: The maze structure was well-received for its balance of challenge and simplicity.

 Areas for Improvement:

Power-up Explanation:

Players were unclear about the effects of power-ups until they experienced them.
I think this part does not need changing as it adds to the puzzling aspect of the game and makes further playthroughs more fun.

Collision Feedback:

When colliding with walls or losing a life, the feedback was clear as they could hear the sound effect and can see the heart lost at the top of the screen.

 Lessons Learned
Need for Minimal Guidance: I like the challenge aspect of playing the game for the first time, with the lack of instructions, players are inspired to explore which increases their intrigue in the game.

Engaging Visuals and Sounds: Participants valued intuitive design elements like heart indicators and unique power-up icons.

Changes Implemented Based on Feedback
The speed was decreased slightly as the high speed was leading to many accidental deaths, The volume for the death feedback was increased to more clearly indicate what happens when a player consumes a death powerup or collide with a wall.

 

GAME:

Concept
The project is an interactive maze game that integrates an Arduino joystick controller to navigate a player through obstacles while collecting or avoiding power-ups. The objective is to reach the endpoint in the shortest possible time, with features like power-ups that alter gameplay dynamics (speed boosts, slowdowns, life deductions) and a life-tracking system with visual feedback.

  • Player Movement: Controlled via the Arduino joystick.
  • Game Start/Restart: Triggered by a joystick button press or mouse click.
  • Power-Ups: Randomly spawned collectibles that provide advantages or challenges.
  • Objective: Navigate the maze, avoid obstacles, and reach the goal with the least possible time.

 

The game is implemented using p5.js for rendering visuals and managing game logic, while Arduino provides the physical joystick interface. Serial communication bridges the joystick inputs with the browser-based game.

Design
Joystick Input:

X and Y axes: Control player movement.
Button press: Start or restart the game.

Visuals:

Player represented as a black circle.
Heart icons track lives.
Power-ups visually distinct ( icon-based).

Feedback:

Life loss triggers sound effects and visual feedback.
Timer displays elapsed and lowest times.
Game-over and win screens provide clear prompts.

Arduino Code:

const int buttonPin = 7; // The pin connected to the joystick button
int buttonState = HIGH;  // Assume button is not pressed initially

void setup() {
    Serial.begin(9600);       // Start serial communication
    pinMode(buttonPin, INPUT_PULLUP); // Set the button pin as input with pull-up resistor
}

void loop() {
    int xPos = analogRead(A0); // Joystick X-axis
    int yPos = analogRead(A1); // Joystick Y-axis
    buttonState = digitalRead(buttonPin); // Read the button state

    // Map analog readings (0-1023) to a more usable range if needed
    int mappedX = map(xPos, 0, 1023, 0, 1000); // Normalize to 0-1000
    int mappedY = map(yPos, 0, 1023, 0, 1000); // Normalize to 0-1000

    // Send joystick values and button state as CSV (e.g., "500,750,1")
    Serial.print(mappedX);
    Serial.print(",");
    Serial.print(mappedY);
    Serial.print(",");
    Serial.println(buttonState);

    delay(50); // Adjust delay for data sending frequency
}

The circuit connects the joystick to the Arduino and includes connections for the button and power LEDs (to indicate remaining lives).

  • Joystick:
    • X-axis: A0
    • Y-axis: A1
    • Click (SW) connected to digital pin 7.
    • VCC and GND to power the joystick module.

The p5.js sketch renders the maze, player, and power-ups, while handling game logic and serial communication.

Key features:

  • Player Class: Handles movement, collision detection, and rendering.
  • Power-Up Class: Manages random spawning, effects, and rendering.
  • Obstacles Class: Generates Obstacles, and handles design aspects of them
  • Joystick Input Handling: Updates player movement based on Arduino input.
  • Game Loops: Includes logic for starting, restarting, and completing the game.

Code:

let player; //player variable
let obstacles = []; //list of obstacles
const OBSTACLE_THICKNESS = 18; //thickness of each rectangle
let rectImg, startImg; //maze pattern and start screen
let obstaclesG; // pre rendered obstacle course pattern for performance
let gameStarted = false; //game started flag
let gameEnded = false; //game ended flag
let startTime = 0; //start time 
let elapsedTime = 0; //time passed since start of level
let lowestTime = Infinity; //infinity so the first level completion leads to the new lowest time
let lives = 3; // player starts with 3 lives
let collisionCooldown = false; // Tracks if cooldown is active
let cooldownDuration = 1000; // Cooldown duration in milliseconds
let lastCollisionTime = 0; // Timestamp of the last collision
let heartImg;//live hearts img
let bgMusic;
let lifeLostSound;
let winSound;
let serial; //for arduino connection
let joystickX = 500; // default joystick X position
let joystickY = 500; // default joystick Y position
let powerUps = []; // Array to store power-ups
let powerUpSpawnInterval = 10000; // interval to spawn a new 
let lastPowerUpTime = 0; // time when the last power-up was spawned
let speedUpImg, slowDownImg, loseLifeImg;
let buttonPressed = false;





function preload() {
  rectImg = loadImage('pattern.png'); // Load obstacle pattern
  startImg = loadImage('start.png'); // Load start screen image
  heartImg = loadImage('heart.png');//  load heart image
  bgMusic = loadSound('background_music.mp3'); // background music
  lifeLostSound = loadSound('life_lost.wav');  // Sound for losing a life
  winSound = loadSound('win_sound.wav'); //sound for winning
  speedUpImg = loadImage('speed_up.png'); //icons for powerups
  slowDownImg = loadImage('slow_down.png');
  loseLifeImg = loadImage('lose_life.png');


}

function setup() {
  createCanvas(1450, 900);
  serial = new p5.SerialPort(); // Initialize SerialPort
  serial.open('/dev/tty.usbmodem1101'); //the code for the arduino device being opened
  serial.on('data', handleSerialData);
  player = new Player(30, 220, 15, 5); //maze starting coordinate for player

 //maze background
  obstaclesG = createGraphics(1450, 900);
  obstaclesG.background(220);

  // Add obstacles
  addObstacles(); //adds all the obstacles during setup

  // loops through the list and displays each one
  for (let obs of obstacles) {
    obs.showOnGraphics(obstaclesG);
  }
  bgMusic.loop() //background music starts
}

function spawnPowerUp() {
    let x, y;
    let validPosition = false;

    while (!validPosition) {
        x = random(50, width - 50);
        y = random(50, height - 50);
        //a valid position for a powerup is such that it does not collide with any obstacles
        validPosition = !obstacles.some(obs =>
            collideRectCircle(obs.x, obs.y, obs.w, obs.h, x, y, 30)
        ) && !powerUps.some(pu => dist(pu.x, pu.y, x, y) < 60);
    }

    const types = ["speedUp", "slowDown", "loseLife"];
    const type = random(types); //one random type of powerup

    powerUps.push(new PowerUp(x, y, type)); //adds to powerup array
}


function handlePowerUps() {
  // Spawn a new power-up if the interval has passed
  if (millis() - lastPowerUpTime > powerUpSpawnInterval) {
    spawnPowerUp();
    lastPowerUpTime = millis(); // reset the spawn timer
  }

  // Display and check for player interaction with power-ups
  for (let i = powerUps.length - 1; i >= 0; i--) {
    const powerUp = powerUps[i];
    powerUp.display();
    if (powerUp.collidesWith(player)) {
      powerUp.applyEffect(); // Apply the effect of the power-up
      powerUps.splice(i, 1); // Remove the collected power-up
    }
  }
}

function draw() {
  if (!gameStarted) {
    background(220);
    image(startImg, 0, 0, width, height);
    noFill();
    stroke(0);

    // Start the game with joystick button or mouse click
    if (buttonPressed || (mouseIsPressed && mouseX > 525 && mouseX < 915 && mouseY > 250 && mouseY < 480)) {
      gameStarted = true;
      startTime = millis();
    }
  } else if (!gameEnded) {
    background(220);
    image(obstaclesG, 0, 0);

    player.update(obstacles); // Update player position
    handlePowerUps(); // Manage power-ups
    player.show(); // Display the player

    // Update and display elapsed time, hearts, etc.
    elapsedTime = millis() - startTime;
    serial.write(`L${lives}\n`);
    displayHearts();

    fill(0);
    textSize(22);
    textAlign(LEFT);
    text(`Time: ${(elapsedTime / 1000).toFixed(2)} seconds`, 350, 50);
    textAlign(RIGHT);
    text(
      `Lowest Time: ${lowestTime < Infinity ? (lowestTime / 1000).toFixed(2) : "N/A"}`,
      width - 205,
      50
    );

    if (dist(player.x, player.y, 1440, 674) < player.r) {
      endGame(); // Check if the player reaches the goal
    }
  } else if (gameEnded) {
    // Restart the game with joystick button or mouse click
    if (buttonPressed || mouseIsPressed) {
      restartGame();
    }
  }
}


function handleSerialData() {
    let data = serial.readLine().trim(); // Read and trim incoming data
    if (data.length > 0) {
        let values = data.split(","); // Split data by comma
        if (values.length === 3) {
            joystickX = Number(values[0]); // Update joystick X
            joystickY = Number(values[1]); // Update joystick Y
            buttonPressed = Number(values[2]) === 0; // Update button state (0 = pressed)
        }
    }
}


function displayHearts() { //display lives
  const heartSize = 40; // size of each heart
  const startX = 650; // x position for hearts
  const startY = 40; // y position for hearts
  for (let i = 0; i < lives; i++) { //only displays as many hearts as there are lives left
    image(heartImg, startX + i * (heartSize + 10), startY, heartSize, heartSize);
  }
}

function endGame() {
  gameEnded = true;
  noLoop(); // stop the draw loop
  winSound.play(); //if game ends
  serial.write("END\n");

  // check if the current elapsed time is a new record
  const isNewRecord = elapsedTime < lowestTime;
  if (isNewRecord) {
    lowestTime = elapsedTime; // update lowest time
    
  }

  // Display end screen
  background(220);
  fill(0);
  textSize(36);
  textAlign(CENTER, CENTER);
  text("Congratulations! You reached the goal!", width / 2, height / 2 - 100);
  textSize(24);
  text(`Time: ${(elapsedTime / 1000).toFixed(2)} seconds`, width / 2, height / 2 - 50);

  // Display "New Record!" message if applicable
  if (isNewRecord) {
    textSize(28);
    fill(255, 0, 0); // Red color for emphasis
    text("New Record!", width / 2, height / 2 - 150);
  }

  textSize(24);
  fill(0); // Reset text color
  text("Click anywhere to restart", width / 2, height / 2 + 50);
}


function mouseClicked() {
 
  if (!gameStarted) {
    // start the game if clicked in start button area
    if (mouseX > 525 && mouseX < 915 && mouseY > 250 && mouseY < 480) {
      gameStarted = true;
      startTime = millis();
    }
  } else if (gameEnded) {
    // Restart game
    restartGame();
  }
}
function checkJoystickClick() {
  if (buttonPressed) {
    if (!gameStarted) {
      gameStarted = true;
      startTime = millis();
    } else if (gameEnded) {
      restartGame();
    }
  }
}

function restartGame() {
  gameStarted = true;
  gameEnded = false;
  lives = 3;
  powerUps = []; // Clear all power-ups
  player = new Player(30, 220, 15, 5); // Reset player position
  startTime = millis(); // Reset start time
  loop();
  bgMusic.loop(); // Restart background music
}


function loseGame() {
  gameEnded = true; // End the game
  noLoop(); // Stop the draw loop
  bgMusic.stop();
  serial.write("END\n");

  // Display level lost message
  background(220);
  fill(0);
  textSize(36);
  textAlign(CENTER, CENTER);
  text("Level Lost!", width / 2, height / 2 - 100);
  textSize(24);
  text("You ran out of lives!", width / 2, height / 2 - 50);
  text("Click anywhere to restart", width / 2, height / 2 + 50);
}


function keyPressed() { //key controls
  let k = key.toLowerCase();
  if (k === 'w') player.moveUp(true);
  if (k === 'a') player.moveLeft(true);
  if (k === 's') player.moveDown(true);
  if (k === 'd') player.moveRight(true);
  if (k === 'f') fullscreen(!fullscreen());
}

function keyReleased() { //to stop movement once key is released
  let k = key.toLowerCase();
  if (k === 'w') player.moveUp(false);
  if (k === 'a') player.moveLeft(false);
  if (k === 's') player.moveDown(false);
  if (k === 'd') player.moveRight(false);
}

class Player { //player class
  constructor(x, y, r, speed) {
    this.x = x;
    this.y = y;
    this.r = r;
    this.speed = speed;

    this.movingUp = false;
    this.movingDown = false;
    this.movingLeft = false;
    this.movingRight = false;
  }

update(obsArray) { //update function
  let oldX = this.x;
  let oldY = this.y;

  //joystick-based movement
  if (joystickX < 400) this.x -= this.speed; // move left
  if (joystickX > 600) this.x += this.speed; // move right
  if (joystickY < 400) this.y -= this.speed; // move up
  if (joystickY > 600) this.y += this.speed; // move down

  // constrain to canvas
  this.x = constrain(this.x, this.r, width - this.r);
  this.y = constrain(this.y, this.r, height - this.r);

  //  restrict movement if colliding with obstacles
  if (this.collidesWithObstacles(obsArray)) {
    this.x = oldX; // revert to previous position x and y
    this.y = oldY;

    // Handle life deduction only if not in cooldown to prevent all lives being lost in quick succession
    if (!collisionCooldown) {
      lives--;
      lastCollisionTime = millis(); // record the time of this collision
      collisionCooldown = true; // activate cooldown
      lifeLostSound.play(); // play life lost sound

      if (lives <= 0) {
        loseGame(); // Call loseGame function if lives reach 0
      }
    }
  }

  // Check if cooldown period has elapsed
  if (collisionCooldown && millis() - lastCollisionTime > cooldownDuration) {
    collisionCooldown = false; // reset cooldown
  }
}


  show() { //display function
    fill(0);
    ellipse(this.x, this.y, this.r * 2);
  }

  collidesWithObstacles(obsArray) { //checks collisions in a loop
    for (let obs of obsArray) {
      if (this.collidesWithRect(obs.x, obs.y, obs.w, obs.h)) return true;
    }
    return false;
  }

  collidesWithRect(rx, ry, rw, rh) { //collision detection function checks if distance between player and wall is less than player radius which means a collision occurred
    let closestX = constrain(this.x, rx, rx + rw);
    let closestY = constrain(this.y, ry, ry + rh);
    let distX = this.x - closestX;
    let distY = this.y - closestY;
    return sqrt(distX ** 2 + distY ** 2) < this.r;
  }

  moveUp(state) {
    this.movingUp = state;
  }
  moveDown(state) {
    this.movingDown = state;
  }
  moveLeft(state) {
    this.movingLeft = state;
  }
  moveRight(state) {
    this.movingRight = state;
  }
}

class Obstacle { //obstacle class
  constructor(x, y, length, horizontal) {
    this.x = x;
    this.y = y;
    this.w = horizontal ? length : OBSTACLE_THICKNESS;
    this.h = horizontal ? OBSTACLE_THICKNESS : length;
  }

  showOnGraphics(pg) { //to show the obstacle pattern image repeatedly
    for (let xPos = this.x; xPos < this.x + this.w; xPos += rectImg.width) {
      for (let yPos = this.y; yPos < this.y + this.h; yPos += rectImg.height) {
        pg.image(
          rectImg,
          xPos,
          yPos,
          min(rectImg.width, this.x + this.w - xPos),
          min(rectImg.height, this.y + this.h - yPos)
        );
      }
    }
  }
}

class PowerUp {
    constructor(x, y, type) {
        this.x = x;
        this.y = y;
        this.type = type; // Type of power-up: 'speedUp', 'slowDown', 'loseLife'
        this.size = 30; // Size of the power-up image
    }

    display() {
        let imgToDisplay;
        if (this.type === "speedUp") imgToDisplay = speedUpImg;
        else if (this.type === "slowDown") imgToDisplay = slowDownImg;
        else if (this.type === "loseLife") imgToDisplay = loseLifeImg;

        image(imgToDisplay, this.x - this.size / 2, this.y - this.size / 2, this.size, this.size);
    }

    collidesWith(player) {
        return dist(this.x, this.y, player.x, player.y) < player.r + this.size / 2;
    }

    applyEffect() {
        if (this.type === "speedUp") player.speed += 2;
        else if (this.type === "slowDown") player.speed = max(player.speed - 1, 2);
        else if (this.type === "loseLife") {
            lives--;
            lifeLostSound.play();
            if (lives <= 0) loseGame();
        }
    }
}

function addObstacles() {
  // adding all obstacles so the collision can check all in an array
  
obstacles.push(new Obstacle(0, 0, 1500, true));
obstacles.push(new Obstacle(0, 0, 200, false));
obstacles.push(new Obstacle(0, 250, 600, false));
obstacles.push(new Obstacle(1432, 0, 660, false));
obstacles.push(new Obstacle(1432, 700, 200, false));
obstacles.push(new Obstacle(0, 882, 1500, true));
obstacles.push(new Obstacle(100, 0, 280, false));
obstacles.push(new Obstacle(0, 400, 200, true));
obstacles.push(new Obstacle(200, 90, 328, false));
obstacles.push(new Obstacle(300, 0, 500, false));
obstacles.push(new Obstacle(120, 500, 198, true));
obstacles.push(new Obstacle(0, 590, 220, true));
obstacles.push(new Obstacle(300, 595, 350, false));
obstacles.push(new Obstacle(100, 680, 200, true));
obstacles.push(new Obstacle(0, 770, 220, true));
obstacles.push(new Obstacle(318, 400, 250, true));
obstacles.push(new Obstacle(300, 592, 250, true));
obstacles.push(new Obstacle(420, 510, 85, false));
obstacles.push(new Obstacle(567, 400, 100, false));
obstacles.push(new Obstacle(420, 680, 100, false));
obstacles.push(new Obstacle(567, 750, 150, false));
obstacles.push(new Obstacle(420, 680, 400, true));
obstacles.push(new Obstacle(410, 90, 200, false));
obstacles.push(new Obstacle(410, 90, 110, true));
obstacles.push(new Obstacle(520, 90, 120, false));
obstacles.push(new Obstacle(410, 290, 350, true));
obstacles.push(new Obstacle(660, 90, 710, false));
obstacles.push(new Obstacle(660, 90, 100, true));
obstacles.push(new Obstacle(420, 680, 500, true));
obstacles.push(new Obstacle(410, 290, 315, true));
obstacles.push(new Obstacle(830, 0, 290, false));
obstacles.push(new Obstacle(760, 200, 70, true));
obstacles.push(new Obstacle(742, 200, 90, false));
obstacles.push(new Obstacle(950, 120, 480, false));
obstacles.push(new Obstacle(1050, 0, 200, false));
obstacles.push(new Obstacle(1150, 120, 200, false));
obstacles.push(new Obstacle(1250, 0, 200, false));
obstacles.push(new Obstacle(1350, 120, 200, false));
obstacles.push(new Obstacle(1058, 310, 310, true));
obstacles.push(new Obstacle(760, 390, 300, true));
obstacles.push(new Obstacle(660, 490, 200, true));
obstacles.push(new Obstacle(760, 582, 200, true));
obstacles.push(new Obstacle(920, 680, 130, false));
obstacles.push(new Obstacle(1040, 310, 650, false));
obstacles.push(new Obstacle(790, 760, 200, false));
obstacles.push(new Obstacle(1150, 400, 400, false));
obstacles.push(new Obstacle(1160, 560, 300, true));
obstacles.push(new Obstacle(1325, 440, 200, false));
obstacles.push(new Obstacle(1240, 325, 150, false));
obstacles.push(new Obstacle(1150, 800, 200, true));
obstacles.push(new Obstacle(1432, 850, 130, false));
obstacles.push(new Obstacle(1240, 720, 200, true));

}

What I’m Proud Of
Joystick Integration: Seamless control with physical inputs enhances immersion.
Dynamic Power-Ups: Randomized, interactive power-ups add a strategic layer.
Visual and Auditory Feedback: Engaging effects create a polished gaming experience.
Robust Collision System: Accurate handling of obstacles and player interaction.

Areas for Improvement:

  1. Tutorial/Instructions: Add an in-game tutorial to help new users understand power-ups and controls. This could be a simple maze with all powerups and a wall to check collision.
  2. Level Design: Introduce multiple maze levels with increasing difficulty.
  3. Enhanced Feedback: Add animations for power-up collection and collisions

Conclusion:

I had a lot of fun working on this project, it was a fun experience learning serial communication and especially integrating all the powerup logic. I think with some polishing and more features this could be a project that I could publish one day.

 

 

 

Final Project: Block adventure

Concept

What if instead of moving the hero, you can move the environment instead? In this game, I try to let the user into the environment building process. Where should I put the platform? How high, how low should it be so that the character should jump across? This is a puzzle game, in which player needs to find the right placement for the physical box – representing the in-game platform – to allow the character to jump across. For the design, I decide to use basic shapes to amplify the theme of the game – building the world from the most basic blocks.

Implementation

For the interaction design, I created a simple instruction scene using basic shapes and minimal text, as users often overlook instructions with too many words. For the physical component, I used red tape to clearly indicate the area where the box could be moved. Additionally, I tried to keep the sensors and connecting components inside the cardboard as discreet as possible by using paper clips instead of tape or glue.

Instructions on how to navigate the game

This game project is rather p5js heavy. For the game mechanism, I design one class for the block, and one class for the platforms.

The hero class include:

  • checkGround(): check if the hero is on which platform
  • move(): move the hero
  • display(): display the hero on screen

The box (platform) class include:

  • changeable parameters for x, y, width and height
  • display method

P5js code

Arduino

For the interaction between Arduino and P5js, I use the data read from Arduino photoresistors and send it to P5js.

int right1 = A0;
int right2 = A1; 
int front1 = A2;
int front2 = A3;
int prevFront = 0;
int prevRight = 0;
int front1Read;
int front2Read;
int right1Read;
int right2Read;
int minFront;
int minRight;
void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);
  pinMode(right1,INPUT);
  pinMode(right2,INPUT);
  pinMode(front1,INPUT);
  pinMode(front2,INPUT);
}

void loop() {

  front1Read = map(analogRead(front1),60,100,750,850); //map this value to front2 sensor range
  front2Read = analogRead(front2);
  minFront = min(front1Read,front2Read); //choose the smaller value (the box is infront of this sensor)

  right1Read = map(analogRead(right1),40,60,700,780);
  right2Read = analogRead(right2);
  minRight = min(right1Read,right2Read);

  if(abs(minFront - prevFront) > 40){
    //only update if the difference is bigger than 40(prevent noise)
    Serial.print(minFront);
      prevFront = minFront;
  }else{
    Serial.print(prevFront);
  }
  Serial.print(',');

  if(abs(minRight - prevRight)>40){
    Serial.println(minRight);
      prevRight = minRight;
  }else{
    Serial.println(prevRight);
  }

}

Schematic for Arduino

User interaction video

Struggle

Due to time constraint, I had to cut a lot of parts during the development process. I initially want to incorporate the moving position of the box, however, due to problems with the photoresistors that I could not resolve, I had to do away with this feature in the end. Using photoresistors also cause me multiple problems, mainly due to the different lightings when moving environment. This lead to my having to fix the parameters every time the lightings change. This is particular problematic and the main issue with my project that I hope to improve in the future.

Reflection

For future project, I think I could have more proper planning, including researching the tools (i.e. Arduino sensors) to know which best fit for the project and avoid significant problems. Other than that, I think the user experience design for this final project has improved from my previous midterm. For this final, I try to make the design as intuitive as possible without using a lot of words. Instead, I try to use symbols and colors (red for stop, do not cross the line). I am also invested in the idea of being able to move the environment although it did not turn out as good as expected due to the implementation. In the future, I would love to work on this idea more, particularly the game mechanism.

 

Final Project – User Testing

User design

Specifically for the user interaction, I design an instruction at the start of the game. Learning from my midterm project, where user did not know intuitively which button to press, I wrote a small instruction to let user know they could press any button to continue the experience. For the physical component, because the moveable box could not move out of the limits, I put a red tape to signify which area to move and which not to.

Schematic

The project uses 4 photoresistors for sensing position of the box and its size.


User testing video

User testing feedback

After letting my friends try out the game, most of them understood which button to press and that they could only move the box within the red line. However, the enlargement of the box during movement appeared too jittery and abrupt, making it difficult for users to associate their physical movements with the box on the screen. At this part, I need to step in to explain that the box is controlling the platform on the screen. The primary issue to address is the abrupt interaction, which is mainly caused by the photoresistor generating noisy values.

 

Week 13: User Testing – NYUAD Puzzle Adventure

My game involves solving puzzles of NYU Abu Dhabi images.
The game involves some nice images of the NYU Abu Dhabi campus which are initially is square pieces. I decided to go with square pieces instead of jigsaw pattern because it was easy to  combine move and manipulate square pieces than any other shape.
I invited a friend of mine to test the game and the video can be seen below:

My schematic is as shown here below:

I have four buttons to control game states and actions and the joystick to move image pieces. The speakers are attached to the controller to provide feedback when the pieces move.

Reflection

1. Simplifying Instructions: I realized that users will skip long instructional text. To address this, I need to make my design as intuitive and self-explanatory as possible.

2. Sound Feedback: The sound feedback when pieces attach is effective in making players aware of their actions and creating a sense of progress. This helps avoid confusion or mistaken attachments.

3. Adjusting Final Sound: The final pop-up sound when an image is correctly solved was too loud and needs to be adjusted for a better user experience.

4. Speed of Image Pieces: The speed of the image pieces was too slow, which could make the gameplay feel boring. Increasing the speed would make the experience more engaging.

5. Magnetic Precision: To improve ease of use, I need to reduce the precision required for pieces to attach. Allowing pieces to magnetically snap together would make it easier for players to combine and move them.

6. Background Music: Given that my user testing was quite, and my volunteer was shocked by the celebration sound, I would like to add a background music in order to make the game more interesting.

Week 13: Final Project – User Testing

Hey everyone! 👋

As we’re finishing up our final projects, we also have to conduct user testing to gain valuable feedback and identity potential issues before the showcase.

Here’s one of the short user testing sessions:

 

As you can see, I did not give any prompt or instructions, and they were able to figure it out nearly instantly (that this is a music synthesizing glove), which is a good sign. They were easily able to play around with it and make music, and understood that bending the fingers was controlling the sound.

However, one piece of feedback I got was that while it was obvious which finger controlled which bar, it wasn’t clear what they were mapped to. I had initially expected that the answer would reveal itself upon the glove being played with for a while (especially with the helpful addressable LEDs even providing hints and feedback on their actions), but clearly this wasn’t the case. Most people were able to somewhat guess the action of the first finger (controlling the pitch (actually the base frequency)), but weren’t able to tell more than that. To be honest though, even I wouldn’t have been able to immediately tell the action of the last 2 fingers. So taking this into account, I added little labels at the bottom of the bars, indicating what each finger controlled.

For some curious enough, I also explained how the sound was actually being synthesized. While this does mean there was something to explain, I think this was more the theory behind the scenes, and so it isn’t vitally important to know. In fact, I think people should be able to play around and enjoy the sound, without knowing the actual technical details behind it, which was the case.

Other than that, I’ll polish up the interface, and although I would also like to improve the music generation, I doubt I’ll be able to change it much between now and the showcase.

Final Project – Sense of Music

Concept

It took me some time to come up with the idea for my final project. I could not decide for sure because while most of my peers were either expanding their midterm project or creating a game that would use Arduino as a user-control input, I wanted to create something that correlated with my interests and would be more unique for my developer experience. Thanks to the assignment where we needed to find out the way to control music with the Arduino sensor, and my love for music, I decided to do something associated with music, particularly enhancing its experience.

Since childhood, I have always liked to play with music controls. At some point, I even wanted to become a DJ. While such a dream never came true, a couple of weeks ago I realized that I could utilize Arduino to create something similar to the DJ panel (a simplified version of it). Moreover, I can make images on the computer to interact with the music and visualize it. After thorough research, I found out that there is a built-in p5.js library that allows to work with the sounds. This is exactly what I needed.

My project now consists of the physical part – the box powered by Arduino controls and sensors that allow a user to interact with music and the p5.js, and a digital visualization that is based both on the user actions and the music characteristics (which can also be changed by the user to the certain extent).

Project Demonstration

Implementation

Interaction Design:

A user interacts with the Arduino through box that has a number of input controls.
– 3 buttons on the right side turn on 3 different songs
– 3 buttons on the top provide control over the visualization of music style and effects on the digital screen
The red button in the top left corner is a play/pause button
– White and Black buttons on the left increase and decrease the pitch speed of the music
– Potentiometer controls the volume
– Ultrasonic Distance Sensor controls the High Pass and Low Pass filters

The input from Arduino through Serial Communication allows to influence the music as well as the image that is generated inside p5.js. 

Code:

p5.js: Using the p5.FFT the music characteristics are converted into the image. My code analyses the music, e.g. frequencies, and creates a dynamic visualization. There are 3 main music visualizers that I created – Circle (set by default), Sunburst (can be turned on/off using the button), and Particles (set by default, can be switched off). All of those patterns are based on the same idea of synchronizing with music amplitude. Function fft.getEnergy() measures the bass range in Hz. The value then passed further and is used to control the waveform of the Circle, length of lines of Sunburst, and speed of Particles.

Additionally, I leveraged the knowledge from the previous projects to equip Sense of Music with features like a fullscreen toggle or Rotation effect that can also be activated by pressing the corresponding button. Moreover, as a bonus, I decided to incorporate one of my favorite projects of this semester into Sense of Music. The Lines’ visual effect, although is not influenced by music, can be activated using one of the buttons as a substitute for Particles. Personally, I enjoy combining the Lines with the Rotating effect.

Arduino: Code for Arduino is quite straightforward as well as the connections on the board (see the schematic below). I am really glad I learned how to use and connect potentiometers and ultrasonic distance sensors in class, so I had no problem with intuitively using them for my project.

Link for p5.js code with Arduino code as a comment on the bottom:
https://editor.p5js.org/rbashokov/sketches/SwAigPMVa

Thoughts and Conclusion

I am extremely happy with the final project that I have created. More than that, I am happy and thankful for the Intro to IM class and professor Mang for helping and assisting me through the whole semester. I learned a lot in this class. More specifically, I, in some way, stepped out of the environment I got used to by constantly making myself think outside the box and create something unique, something that can be called interactive art. Other than that, the most difficult part for me was creating the physical box and setting up the buttons and sensors on it. The last time I did something similar was in middle school, so I had literally 0 knowledge of how to approach it. However, I managed to figure out a way to create schematics of the box and print them using the laser cut and connect the wires and buttons using glue. I am still very far away from being close to the results I would like to achieve, but for now, I feel like I made progress and reached the goals I set for this class at the beginning of the semester.

Speaking of my final project, Sense of Music was a great idea, and its implementation was good but not perfect. While the code was not a problem, there is still a lot to build on top of what I have for now to further enhance the user experience. The same goes for the physical parts – the box and the DJ panel. I could make them look more aesthetic and pleasant by using different styles of buttons, perhaps adding more colors to the box, etc. Unfortunately, the time was very limited, especially considering my trip to Shanghai that took the whole National Holiday break. Nevertheless, I am satisfied with the results, and as I see it, the fact that there is a lot of room for project improvement shows that the idea is actually great.

Thank you!

 

Final Project: NYUAD’s Global Dance Party

Concept

From the design to implementation phase I am rather satisfied with how close I was able to stay to my original concept. In the design phase I knew I wanted to create some type of map interaction with different countries and songs for users to dance too. When conceptualizing this project, my mind kept returning to the alphabet/poem rain installation we saw in the first half of the semester. I think it’s the best example we’ve seen of interactive art where there’s a message and a strong interactive component. Although, I was not able produce something of that quality, I am nevertheless pleased with way in which I combine meaning and interaction in my piece.

As a study away student being immersed in NYUAD’s vibrant culture has genuinely changed my perspective of the world. Growing up in New York City, and the US in general I’ve been taught a very specific definition of what diversity looks like and being a student here has completely redefined that for me. Thus, as the semester comes to a close, I thought this project would be a perfect way for me to share a glimpse of what I’ve come to appreciate so much.

Design

As I mentioned previously, throughout the design process I had a pretty clear idea of what I wanted to create.  There were other features that I considered in order to strengthen the interactivity such as body tracking or a point system but I settled on a simpler design to ensure maximum functionality. Nevertheless I took a lot of inspiration from the video game Just Dance, and tried to consider that level of interaction throughout the design and implementation process.

Implementation 

Once I began the ‘build’ process the difficult part was just in mainly getting started. Surprisingly enough, I also had a hard time figuring out what songs/countries would be represented. The hardware, although rather simple, tested a lot of skills in terms of soldering and using tools I’m not super familiar with so I’m glad I was able to use what we learned in class to my advantage. Furthermore, this is the first time I’m working with any kind of hardware components on top of software design so it took a bit of creativity to consider what materials I wanted to use to make my vision come alive.

Caption: I originally had a ton of extra wires because I thought I wanted to extend the Arduino behind my computer but after assessing the material available, I shorted them and added this box to tuck them away. – Phase 2

My prototype consisted of just using the map itself and a few switches to test both the hardware and software. Once that was good to go, my main concern became ‘cleaning up the look’ so that I had a clear, finished project to present. This was rather easy, I just needed to identify areas of concern, and figure out how I could further simplify them.

Prototype video from user testing – no sound

In terms of software implementation, once I decided how I wanted to exchange information between P5 and Arduino, I was able to make significant progress in my code. Basically, when a button on the map is pressed, a flag corresponding to that country in Arduino is sent to P5 which then sets a series of commands off to play it’s respective video. However, a major road block that I hit was the 5 MB upload limit to video files in P5. My solution to this problem is fairly simple, but the route I took to get there was very long and complicated (as is most debugging).  I eventually was able to implement an array based system to cycle through clips of each video so that they could all play as one.

Schematic

Arduino Code

int colombiaButton = 13; // Button connected to pin 13
int indiaButton = 12;
int koreaButton = 11;
int chinaButton = 10;
int italyButton = 9; //black button with whie & blue wire
int prButton = 7; //yellow button with green and blue wire 
int saButton = 5; //green button with white and yellow
int nigeriaButton = 2; //red button with orange and green 
int kazakhButton = 3; //blue button with green wires
int fpButton = 4; //black button yellow and white wire  

void setup() {
  // Start serial communication so we can send data
  Serial.begin(9600);

  // Configure the button pin as input with pullup
  pinMode(colombiaButton, INPUT_PULLUP);
  pinMode(indiaButton, INPUT_PULLUP);
  pinMode(koreaButton, INPUT_PULLUP);
  pinMode(chinaButton, INPUT_PULLUP);
  pinMode(italyButton, INPUT_PULLUP);
  pinMode(kazakhButton, INPUT_PULLUP);
  pinMode(prButton, INPUT_PULLUP);
  pinMode(saButton, INPUT_PULLUP);
  pinMode(nigeriaButton, INPUT_PULLUP);
  pinMode(fpButton, INPUT_PULLUP);
}

void loop() {
  // Check if button is pressed
  if (digitalRead(colombiaButton) == LOW) { // Button pressed (active low)
    Serial.println("colombia"); // Send "push"
  } else if (digitalRead(indiaButton) == LOW) { 
    Serial.println("india"); // Send "push"
  } else if (digitalRead(koreaButton) == LOW) { 
    Serial.println("korea"); // Send "push"
  } else if (digitalRead(chinaButton) == LOW) { 
    Serial.println("china"); // Send "push"
  } else if (digitalRead(italyButton) == LOW){
    Serial.println("italy");
  } else if (digitalRead(prButton) == LOW){
    Serial.println("pr");
  } else if (digitalRead(saButton) == LOW){
    Serial.println("sa");
  } else if (digitalRead(nigeriaButton) == LOW){
    Serial.println("nigeria");
  } else if (digitalRead(fpButton) == LOW){
    Serial.println("fp");
  } else if (digitalRead(kazakhButton) == LOW){
    Serial.println("kazakh");
  } else {
    Serial.println(0);
  }
  delay(150); // Debounce delay
}

P5 Code

https://editor.p5js.org/jajones/sketches/SaANLV2D5

Aspect’s I’m Proud Of

Overall, I am pretty proud of my final product. What I struggled with the most in the midterm was designing realistic plans for my project which left feeling overwhelmed throughout the implementation process. Throughout this process, I gave myself enough space to design new ideas while staying focused on what my original plan was which I think really helped the overall process.

A more concrete component I am proud of this the solution to the file size upload issue I faced earlier. I did a lot of reading on StackOverflow and random P5 forums to help me solve the issue and so even though it isn’t perfect I am really happy with how it turned out.

Future Improvement 

In terms of future improvement, I would love to incorporate more way to encourage people to physically react. Although in my user testing people were active and participating, I feel as though if I wasn’t standing there, watching, they wouldn’t necessarily feel as inclined to do so. To help alleviate this I did try to replicate a dance mirror effect when the videos play so users felt incentivized to dance, but I’m sure I could go even further to record them dancing and then have the program send it to them or add it to some dance database which they might enjoy being a part of.

Progress on Final Project Week 12

Game Title: Maze Craze

Game Image:


Concept Overview: This project is an interactive maze game where players use a joystick connected to an Arduino Uno to navigate a ball through a maze displayed on a p5.js canvas. The game includes challenges like obstacles, a life system (represented by heart icons), and sound effects for feedback. The player wins by reaching a target point in the maze. The game tracks the fastest time for completion and the fastest time is also displayed. The walls are obstacles where the player loses a life when they touch the wall. This is to increase difficulty of the game and make sure players have to be careful while navigating the maze.

 

Arduino Program Design
Inputs:

Joystick (HW 504):

VRX (X-axis): Controls horizontal movement of the ball.
VRY (Y-axis): Controls vertical movement of the ball.
SW (Button): Can be used to reset the game .
Serial Communication:

Sends joystick X and Y axis values to the p5.js program.

Serial Data:
Sends joystick data in the format x,y to p5.js.

 

Arduino Logic:
Read X and Y values from the joystick.
Send the data via serial to p5.js in the format x,y.

Arduino Code:

void setup() {
  Serial.begin(9600); // Initialize serial communication at 9600 baud rate
}

void loop() {
  int xPos = analogRead(A0); // Joystick X-axis
  int yPos = analogRead(A1); // Joystick Y-axis

  // Map analog readings (0-1023) to a more usable range if needed
  int mappedX = map(xPos, 0, 1023, 0, 1000); // Normalize to 0-1000
  int mappedY = map(yPos, 0, 1023, 0, 1000); // Normalize to 0-1000

  // Send joystick values as CSV (e.g., "500,750")
  Serial.print(mappedX);
  Serial.print(",");
  Serial.println(mappedY);

  delay(50); // Adjust delay for data sending frequency
}

P5.js Logic
Game Start:
Display a start screen and wait for mouse click or button press to begin.
Joystick Integration:
Map joystick X and Y data to control the ball’s position on the canvas.
Collision Detection:
Check for collisions with obstacles and deduct a life upon collision.
Game End:
Display a victory or loss message based on game outcomes.

Code for handling serial data:`

function handleSerialData() {
  let data = serial.readLine().trim(); // Read and trim data
  if (data.length > 0) {
    let values = data.split(",");
    if (values.length === 2) {
      joystickX = Number(values[0]);
      joystickY = Number(values[1]);
    }
  }
}

 

 

Week 11 Reading Response

When Design Meets Disability

Reading Graham Pullin’s ‘Design Meets Disability’ made me rethink how we view assistive devices and how much design influences our perception of them. Pullin argues that assistive technology doesn’t have to be just functional—it can also be beautiful, creative, and reflective of individuality. This idea stood out to me because it flips the usual way we think about devices like hearing aids, wheelchairs, or prosthetics. Instead of being tools to hide or blend in, they can be seen as things that people can show off and be proud of, just like any other accessory or piece of technology.

One example Pullin mentions is hearing aids and how they’re often designed to be invisible. I never thought about how strange that is—why do we feel the need to hide something that helps people? What if hearing aids could be stylish, like jewelry, or customized to fit someone’s personality? It’s a simple shift in thinking, but it makes such a big difference. It reminds me of how glasses used to be seen as embarrassing, but now people wear bold frames to express their style. Why can’t assistive devices evolve in the same way? It’s not just about function; it’s about identity and empowerment.

This idea also connects to the bigger issue of how design often caters to an ‘average user, which leaves a lot of people feeling excluded. Pullin’s focus on inclusive design challenges that by showing how products can be more adaptable and personal. It made me imagine what prosthetic limbs could look like if they were designed with personality in mind—like having different patterns, colors, or even glowing lights. A prosthetic arm could be just as much a fashion statement as a designer handbag or a cool pair of sneakers. This would help break down the stigma around disability by celebrating the creativity and individuality of the people using these devices.

Pullin also makes a really interesting point about beauty. He argues that beauty doesn’t have to mean perfection. Instead, it can come from things that are unique, unexpected, or even imperfect. This reminded me of the Japanese concept of wabi-sabi, which finds beauty in imperfection and the natural flow of life. If we applied that to assistive technology, we could design devices that are not only functional but also artistic and meaningful. For example, a wheelchair could have a sleek, futuristic look, or a prosthetic leg could be designed with intricate patterns that make it stand out in a good way. These designs could change how people think about disability, not as something to pity but as something to appreciate and admire.

In the end, Pullin’s book shows that design is never just about solving problems—it’s about making statements and shaping how people see the world. By bringing creativity into assistive technology, we can create a world that’s not only more inclusive but also more exciting and diverse. Design Meets Disability opened my eyes to how much potential there is in rethinking design and how even small changes can make a huge difference in people’s lives.