I am excited to present to you all the final video game I made for this class utilizing p5 js and Arduino which is called The Boy Who Fell:

The Boy Who Fell is a survival game where you must control a boy and keep him above by utilizing the platforms from floating in below. But be careful as some of the platforms are spikes which will decrease the number of lives, but you can replenish those by jumping onto the wooden platforms. Also be wary of the row of spikes on the top so don’t stick onto one platform for too long!

User Testing

Sanjana helped me in the user testing. She was able to figure out how to play the game using just the on-screen instructions. What she did needed help in explaining was the lives system and how it worked which was the spikes take away life points while wood restores them until a maximum of 10. I decided to make wood give life for balancing sake.

Implementation

I started out in p5 js by creating the game with different classes such as Game, Player, and Platforms. The p5 js handles different functions such as the gravity of the player, randomizing which platform will appear and the overall design of the game.

For the Arduino, I added buttons onto the solderless board as to emulate the feeling of holding a controller. The controller is used in order to move the boy left or right. I set it so pressing a specific button sends either 1 or 2, depending on the button, to p5 js through which it detects which input is being pushed the and the boy moves in that direction.

In the end, only the Arduino is sending data to p5 js which is the value of what button is being pressed which is then stored in a variable and then compared to in an IF statement in order to determine which direction the boy moves.

I’m particularly proud of how I implemented the different platforms and they come from the bottom and the interaction between the player and the platform. I had even more platforms ready with different interactions but it proved to be too difficult and too much according to some feedback, so I eased the game down.

For improvements, I can improve the controller used in order to play the game but making a controller out of boxes and using bigger and better buttons. The game graphics can be improved as well as the instructions describing the game.

User Testing:

To conduct user testing, I had Moeez and Sanjana play my game in its initial state, and they provided me with valuable feedback, which I incorporated into my project. Initially, my game required the user to wait for the flower to reach the bottom of the screen and overlap with the corresponding flower image at the bottom. Sanjana suggested changing the game concept to flower catching and increasing the speed, which I agreed to as it adds excitement. I also initially made my game with a fixed speed throughout but Moeez recommended starting with a slower speed and gradually increasing it to allow users to get accustomed to it. To further enhance the user experience, I plan to add on-screen messages, such as “wrong flower pressed” and “good job!” to make the game more interactive.

Video:

https://youtube.com/shorts/7kJ7MyrLNME

Final Project – juicy jackpot

Concept:

The game I have designed is an engaging and interactive experience, incorporating elements from classic games such as piano tiles, paddle catch, and reaction time games. The central theme of the game is simple yet delightful – players are tasked with discovering hidden fruit and must catch the correct fruit to earn points and progress through the levels. As the game proceeds, the speed of play increases, demanding rapid reflexes and quick decision-making from players to succeed. To uncover the hidden fruit, players must stay alert and swiftly respond to left or right signs, then press the correct arcade button to capture the fruit they seek. The game promises a thrilling and immersive experience, testing players’ skills and keeping them captivated as they strive to achieve the highest score possible.

Implementation:

Building and executing the entire setup for my project was a challenging task, but I am thrilled with the results. I decided to use two force sensors to track left and right movements and four arcade buttons to represent the different fruits. These force sensors typically provide values ranging from 0 to 1023, and while I initially mapped them using Arduino code, it became complicated to send so many values over serial communication. So, I resorted to mapping the values in p5js, which proved to be a more effective solution.

Using this setup, I was able to receive values from both force sensors and the arcade buttons over serial communication. While soldering the force sensors, I encountered some issues with the values changing, but I was able to remap them successfully. If the value is above 100 for one sensor, it’s pressed, and for the other, it’s above 500.

To make the game more engaging, I randomly assigned the fruits and direction arrows. Initially, I had planned to implement a piano tiles-style game with increased complexity, but that wouldn’t allow users to play completely. So, I opted for a randomly falling catch game instead. I first tested the collision and mapping of objects with keyboard keys and then proceeded with mapping them with the Arduino reading.

During testing, I faced a lagging issue while storing all mapped values in an array after splitting the string from the Arduino. However, I was able to resolve this issue by using separate variables, simple if-else statements, and increasing the frame rate.

Mapping the values for the force sensors and arcade buttons to their respective fruits and direction arrows was a crucial step, and I am particularly proud of how well it turned out. Overall, I am pleased with the final result and excited to continue improving the game in the future.

p5js code:

link: https://editor.p5js.org/Sanjana-Nambiar/sketches/-JxDz5psg

Arduino code:

//declaring all the port pins as const int
const int yellow = 13;
const int green = 11;
const int blue = 9;
const int red = 7;

const int left = A3;
const int right = A1;

void setup() {
  //serial port activation 
  Serial.begin(9600);

  //declare the switches as digital input devices
  pinMode(yellow, INPUT_PULLUP);
  pinMode(green, INPUT_PULLUP);
  pinMode(blue, INPUT_PULLUP);
  pinMode(red, INPUT_PULLUP);

  pinMode(left, INPUT);
  pinMode(right, INPUT);
}

void loop() {
  String arrow;
  String fruit;
  String message;

//printig the values from the left and right sensors 
  int sensor1 = analogRead(left);
  int sensor2 = analogRead(right);

  Serial.print(sensor1);
  Serial.print(",");
  Serial.print(sensor2);
  Serial.print(",");

//since the arcade buttons are pullup buttons 
//they are high when it isnt pressed.
  if(digitalRead(yellow) == HIGH)
    {Serial.print("0,");}
  else 
    {Serial.print("1,");}

  if(digitalRead(green) == HIGH) 
    {Serial.print("0,");}
  else 
    {Serial.print("1,");}

  if(digitalRead(blue) == HIGH)
    {Serial.print("0,");}
  else 
    {Serial.print("1,");}

  if(digitalRead(red) == HIGH) 
    {Serial.println("0");}
  else 
    {Serial.println("1");}

}

User testing:

During the user testing phase, my game was tested by Saamia and Moeez, who found the game instructions to be simple and straightforward. However, they encountered a minor issue when distinguishing between the fruit colors. The confusion arose from the white berry and guava fruits, which shared a similar color scheme. To address this issue, I took the feedback on the board and included pictures of the fruits at the front of the box. This small change resolved the confusion, and the testers were able to play the game smoothly. Overall, the user testing phase yielded positive results, and the feedback provided by Saamia and Moeez allowed me to make an improvement to the game that enhanced the user experience.

Future improvements and reflections:

Looking toward the future, there are several areas where this game could be improved to make it even more engaging and exciting for players. One possibility is to add more layers of complexity to the game. For example, rather than just catching the fruit, players could be required to perform additional tasks before the fruit is revealed. This could include completing a pattern or sequence of movements or even solving simple puzzles. Additionally, the game could be made into a two-player mode, where players compete against each other to catch the most fruit or avoid the most obstacles. This could be accomplished by adding a second set of sensors and buttons or even incorporating motion-tracking technology to track players’ movements in real-time.

Overall, I am very proud of the mapping part of the game, where the sensors and buttons were effectively mapped to the fruits and arrows for catching them. With further development, this game has the potential to become a popular and entertaining addition to the world of physical gaming.

Concept:

For my final project “Dance Evolution”, I decided to work on a “remake” of a famous Japanese video game called “Dance Revolution.” In this game, instead of being provided the music at the beginning, the user has to press on the tiles in order to generate and construct the music itself track by track. For each round, the user is shown an arrow pointing either to the left or to the right and they have to press on the respective tile in a timely manner, otherwise they lose. Them pressing on the tiles will start slowly “unlocking new tracks,” allowing them to construct the final song.

Implementation:

Arduino:

My Arduino board consisted of 3 force sensors. These would then trigger the work of p5js. However, an issue I faced for some time was that when the sensor was activated, Arduino would register multiple high values of the sensor, which was inconvenient specifically for my project as it would set off a chain of undesirable events. Therefore, when working on my Arduino board and code, I had to make sure I debounced the signal properly so that it only registered the activation only once when the sensor is pressed.

p5.js:

Initially, I thought that the p5.js implementation of the project would be straightforward. However, I encountered a variety of different issues that made me change my approach to making this game. Firstly, I was unable to properly play the sounds of my tracks when the sensors were pressed.  It was very difficult to come up with an algorithm in which the activation of a sensor would lead to the playing of a track in a way that matches the other tracks (that may already be played) rhythmically. Therefore, I decided to play all the tracks in the setup and simply unmute each track when the sensor is triggered.

My p5.js code would take in the values of the sensor, or more specifically “signs” from the Arduino that the sensor was pressed and reached its maximum value, and would use that in order to generate new rounds of the game. My biggest obstacle throughout this process was creating an algorithm that would a) randomly pick between the two different images of arrows, b) make the user go from one round to the other, c) have the pictures be timed so that if the sensor is activated late, the user loses, etc. However, mainly through the usage of the “millis()” function, I was able to tackle these issues.

Code:

p5.js:

let bg;

//variables for all the musical tracks

let bass;

let hat;

let kick;

let snare;

let synth;

let strings;

let vox1;

let vox2;

let now; //variable that trackes the time every time the user presses on a button 

let gameStartTime; //tracks the beginning of each round

let sensorValue;

let middleSensorVal;//value of the sensors that p5js receives from Arduino

let gameState = "start"; // Can be 'start', 'playing', 'end'

let threshold = 5000; //the value for the first arrow that appears

let num = 0; //the value that activates the different cases

let arrowImage = []; //an array that holds the values of the images of the arrows

let randomIndex; //selects a random image from an array 

function preload() {
  vox1 = loadSound("vox_1.mp3");

  kick = loadSound("kick.mp3");

  vox2 = loadSound("vox_2.mp3");

  snare = loadSound("snare.mp3");

  bass = loadSound("hat.mp3");

  strings = loadSound("strings.mp3");

  synth = loadSound("synth.mp3");

  hat = loadSound("hat.mp3");

  bg = loadImage("background_image.png"); 
  
  arrowImage.push(loadImage("rightArrow.png"));
  arrowImage.push(loadImage("leftArrow.png")); //pushing the images into the array 

  bruno = loadFont("BrunoAceSC-Regular.ttf"); //loading the font of the game
}

function setup() {
  createCanvas(windowWidth, windowHeight);
  textAlign(CENTER);
  imageMode(CENTER);

  //playing all the tracks during setup and muting each track

  vox1.setVolume(0);
  vox1.play();
  vox1.loop();

  vox2.play();
  vox2.setVolume(0);
  vox2.loop();

  strings.play();
  strings.setVolume(0);
  strings.loop();

  snare.play();
  snare.setVolume(0);
  snare.loop();

  synth.play();
  synth.setVolume(0);
  synth.loop();

  kick.play();
  kick.setVolume(0);
  kick.loop();

  bass.play();
  bass.setVolume(0);
  bass.loop();

  hat.play();
  hat.setVolume(0);
  hat.loop();
}

function draw() {
  background(220);
  image(bg, width / 2, height / 2, width, height);
  
  //assigning different functions to different states of the game

  if (gameState == "start") {
    drawMainPage();
  } else if (gameState == "instructions") {
    drawInstructions();
  } else if (gameState == "playing") {
    drawGame();
  } else if (gameState == "win") {
    drawWinScreen();
  } else if (gameState == "loss") {
    drawLossScreen();
  } 
}

function windowResized() {
  resizeCanvas(windowWidth, windowHeight);
}

function drawMainPage() {
  textSize(70);
  fill("rgb(255,0,163)");
  stroke(255);
  strokeWeight(5);
  textWrap(WORD);
  textFont(bruno);
  text("Dance Evolution", width / 2, height / 5, width / 13);

  textSize(18);
  fill("white");
  stroke(0);
  strokeWeight(4);
  textFont(bruno);
  text("Press the Space Bar for the Instructions", width / 2, height / 1.2);
}

//setting up the serial connection

function keyPressed() {
  if (key == " ") {
    setUpSerial();
    gameState = "instructions";
  }
}

function drawInstructions() {
  textSize(60);
  fill("rgb(255,0,163)");
  stroke(255);
  strokeWeight(5);
  textWrap(WORD);
  textFont(bruno);
  text("Instructions", width / 2, height / 5);
  
  textWrap(WORD);
  textSize(16);
  stroke(0);
  strokeWeight(3);
  fill(255);
  textFont(bruno);
  text(
    "Once you start the game, you will see an arrow. Wherever the arrow points, that's the tile you should click on. Once you click on the tile, you will hear a track playing. Below the picture of the arrow, you will see the number of seconds you are given for each round. If you don't press the tile in time, you LOSE! If you manage to press on all the arrows on time and play all the tracks, you WIN!",
    0,
    height/2.5,
    width
  );
  
  textSize(18);
  fill("white");
  stroke(0);
  strokeWeight(4);
  textFont(bruno);
  text("Step on the Tile in the Middle to Play!", width / 2, height / 1.2);

  if (middleSensorVal == 1023) {
    gameState = "playing";
    restartRound(); //begins the game when the button is pressed
  }
}

function drawGame() {
  textSize(60);
  fill("rgb(255,0,163)");
  stroke(255);
  strokeWeight(5);
  textWrap(WORD);
  textFont(bruno);
  text("DANCE!", width / 2, height / 6.5);
  
  //tracking to see if the time that passed since the start of the round is less than the time the threshold

  if (millisInGame() < threshold && num < 8) {
    image(arrowImage[randomIndex], width / 2, height / 2, width/2, height/2);

    fill(255);
    strokeWeight(0);
    text(
      round(millisInGame() / 1000) + "/" + round(threshold / 1000) + " s",
      width / 2,
      height / 1.15
    );
  }
  
  //assigning the conditions for loss
  
  if (millisInGame() > threshold && num < 8) {
    gameState = "loss";
  }

  if (sensorValue === 1023 && num < 9) {
    num++;
    console.log(num);
    restartRound();
    threshold = threshold * 0.92;
    console.log(threshold);
    flag = true;
  }

  if (num == 8) {
    drawWinScreen();
  }
  
  switch (num) {
    case 0:
      break;
    case 1:
      vox1.setVolume(0.2);
      break;
    case 2:
      vox2.setVolume(0.2);
      break;
    case 3:
      kick.setVolume(0.2);
      break;
    case 4:
      bass.setVolume(0.2);
      break;
    case 5:
      snare.setVolume(0.2);
      break;
    case 6:
      hat.setVolume(0.2);
      break;
    case 7:
      synth.setVolume(0.2);
      break;
    case 8:
      strings.setVolume(0.2);
      break;
  }
}

function drawWinScreen() {
  background(220);
  image(bg, width / 2, height / 2, width, height);

  textSize(40);
  fill("rgb(255,0,163)");
  stroke(255);
  strokeWeight(5);
  textWrap(WORD);
  textFont(bruno);
  text("CONGRATULATIONS", width / 2, height / 5);
  
  textSize(18);
  fill("white");
  stroke(0);
  strokeWeight(4);
  textFont(bruno);
  text("Step on the tile in the middle to RESTART!", width / 2, height / 1.2);
  
  //restarting the game when the middle sensor is played

  if (middleSensorVal == 1023) {
    gameState = "instructions";
    vox1.setVolume(0);
    vox2.setVolume(0);
    strings.setVolume(0);
    snare.setVolume(0);
    synth.setVolume(0);
    kick.setVolume(0);
    bass.setVolume(0);
    hat.setVolume(0);
    num = 0;
    threshold = 5000;
  }
}

//tracking the start time of each round and generating an index to then randomly generate an image from the array 

function restartRound() {
  now = millis();
  gameStartTime = now;

  randomIndex = floor(random(arrowImage.length));
}

//tracking the difference between the current time and the moment when the game started

function millisInGame() {
  return millis() - gameStartTime;
}

function drawLossScreen() {
  textSize(60);
  fill("rgb(255,0,163)");
  stroke(255);
  strokeWeight(5);
  textWrap(WORD);
  textFont(bruno);
  text("LOSS", width / 2, height / 5);

  if (middleSensorVal == 1023) {
    gameState = "instructions";
    vox1.setVolume(0);
    vox2.setVolume(0);
    strings.setVolume(0);
    snare.setVolume(0);
    synth.setVolume(0);
    kick.setVolume(0);
    bass.setVolume(0);
    hat.setVolume(0);
    num = 0;
    threshold = 5000;
  }
  
  textSize(18);
  fill("white");
  stroke(0);
  strokeWeight(4);
  textFont(bruno);
  text("Step on the tile in the middle to RESTART!", width / 2, height / 1.2);
}

//receiving the information from the Arduino

function readSerial(data) {
  if (data != null) {
    let fromArduino = split(trim(data), ",");

    if (fromArduino.length == 2) {
      
      middleSensorVal = int(fromArduino[0]);
      sensorValue = int(fromArduino[1]);

    }
  }
}

function keyTyped() {
  if (key === 'f') {
    toggleFullscreen();
  }
}
function toggleFullscreen() {
  let fs = fullscreen(); 
  fullscreen(!fs); 
}

Arduino

const int leftPSensor = A0;
const int rightPSensor = A1;
const int middleSensor = A2; 

int lastSensorStateLeft = 0;
int lastSensorStateMiddle = 0;

int threshold = 1022;

bool flag = false;

int rightPressureVal;
int leftPressureVal;
int middleSensorVal;

void setup() {

  pinMode(leftPSensor, INPUT);
  pinMode(rightPSensor, INPUT);
  pinMode(middleSensor, INPUT);
  Serial.begin(4800);
}

void loop() {

  leftPressureVal = analogRead(leftPSensor);
  rightPressureVal = analogRead(rightPSensor);
  middleSensorVal = analogRead(middleSensor);


  if (leftPressureVal < threshold && rightPressureVal < threshold && middleSensorVal < threshold) {
    flag = true;
  } 

  if (flag == true && (leftPressureVal == 1023 || rightPressureVal == 1023)) {
      lastSensorStateLeft = 1023;
      flag = false;
    delay(300);
  } else {
    lastSensorStateLeft = 0;
  }

  if (flag == true && middleSensorVal == 1023) {
      lastSensorStateMiddle = 1023;
      flag = false;
      delay(300);
  } else {
    lastSensorStateMiddle = 0;
  }

  Serial.print(lastSensorStateMiddle);
  Serial.print(", ");
  Serial.println(lastSensorStateLeft);
}

Communication:

The communication between Arduino and p5.js proved to be a difficulty. As mentioned earlier, I tackled the issue of multiple registrations of the activation of the sender by debouncing it and ensuring that the value of the sensor is sent to p5js only once. However, when that was happening, despite p5.js recognizing the value, it would simply ignore it, even though it was clearly instructed to carry out an operation with the value. After some time, I realized that this issue could be resolved by decreasing the baud rate from 9600 to 4800 on both p5js and Arduino, which would allow p5js to recognize the one, small “1023” value that gets lost in a fast stream of values that are being received from Arduino. This was very helpful as it fixed the issue and ensured the smooth operation of the game.

Future improvements:

With this project, I am proud that it ultimately became what I envisioned, albeit with some limitations. I am proud of the algorithm that I used for flipping through rounds using the sensor and playing the different tracks themselves.

In the future I want to make sure there is more randomness to the process in terms of the different tracks that are played because as of now, the tracks are played in the same order. I also want to add different levels of difficulty where the user can choose to make the time given for their step on the tiles shorter. Finally, I want to improve the p5.js – Arduino communication in order to make sure that the user does not have to press on the tiles a few times for the value to be registered, which is an issue I observe currently.

Video:

User Testing:

I conducted user testing on one of my friends, whom I did not have explain the Instructions to. According to him, the instructions were clear. Despite that, he still struggled at times with clicking on the tiles as they would take a few tries for the value to be received by p5.js. All in all, however, he claimed that the game is fairly easy to understand.

Project: Petals and Harmony

Concept

My project aims to provide a simple and engaging activity for elderly individuals who may face difficulty attending physiotherapy sessions. The game is nature-themed and involves a flower-catching game set in a forest or garden, accompanied by a calming fairyland-like musical background. This relaxing environment is designed to calm the nerves, similar to a physiotherapy session.

As the game progresses, flowers appear one at a time from a distance and move closer to the player, creating an enjoyable challenge that encourages individuals to exercise their hand-eye coordination and mobility.

Implementation

For my project, I utilized four pressure sensors as analog sensors to capture input from the user’s four fingers. The sensors provided data within a range of 0 to 1023. However, to ensure that a “press” was distinguished from a mere touch, I set a threshold range for the boolean values. If the sensor reading exceeded 500, the boolean value was set to true.

I created four boolean variables, one for each sensor. These variables were set to true when the sensor reading was above 500, and false otherwise. Only when the user pressed the correct pressure sensor, the moving flower appeared and the next flower appeared on the screen.

To create a realistic flower movement effect, I attempted to simulate the appearance of flowers approaching from afar. At first, I considered having the flowers move in four rows, similar to piano tiles. However, I found that manipulating the size and movement angle of the flowers was more effective in achieving the desired effect. With this approach, I was able to create the impression that the flowers were coming closer from a distance.

This is the code that helps me achieve this effect:

angle = radians(126);
image(img,start_x,start_y,size,size);
start_x += speed * cos(angle);
start_y += speed * sin(angle);
size = map(start_y, 466, height, 10, 110);

I calculated the specific angle value to each flower, allowing me to manipulate the x and y coordinates independently. To adjust the size of the flowers, I utilized the map function. Specifically, when the value of the start_y coordinate (the y-coordinate of the flower) was 466, the size of the flower was set to 10. Similarly, when the y-coordinate reached the end of the canvas (i.e., height), the size of the flower was set to 110 using the same ratio.

My p5.js project features a main page that showcases the game’s title along with a slogan, as well as two buttons: “Start” and “Instructions.” The instructions section provides a comprehensive guide on how to play the game and includes information on how missed flowers will be displayed at the end of the game. Once the user has gone through the instructions, they can then proceed to start the game.

In the top left corner of the game’s interface, there is a fullscreen button that allows the user to enter or exit fullscreen mode. Since my game is in a vertical orientation, I centered the printing of the game’s background images, while covering the rest of the canvas with a plain colored background.

Arduino Code:

const int pressureSensor0 = A0;
const int pressureSensor1 = A1;
const int pressureSensor2 = A2;
const int pressureSensor3 = A3;
// the setup routine runs once when you press reset:
void setup() {
  // initialize serial communication at 9600 bits per second:
  Serial.begin(9600);
  pinMode(pressureSensor0, INPUT);
  pinMode(pressureSensor1, INPUT);
  pinMode(pressureSensor2, INPUT);
  pinMode(pressureSensor3, INPUT);
}

// the loop routine runs over and over again forever:
void loop() {
  // read the input on analog pin 0:
  int sensorValue0 = analogRead(pressureSensor0);
  int sensorValue1 = analogRead(pressureSensor1);
  int sensorValue2 = analogRead(pressureSensor2);
  int sensorValue3 = analogRead(pressureSensor3);

  // print out the value you read:
  //Serial.print("(");
  Serial.print(sensorValue0);
  Serial.print(", ");
  Serial.print(sensorValue1);
  Serial.print(", ");
  Serial.print(sensorValue2);
  Serial.print(", ");
  Serial.println(sensorValue3);
  //Serial.println(")");

}

P5js Implementation:

Areas I am particularly proud of

I am incredibly proud of creating my first interactive game. In the past, I had made games, but they were never as engaging as this one, which uses a glove to take input from the user. This gave a whole new level of interactivity to the project, making it more engaging and immersive. It reminded me of our in-class discussions, where we talked about the importance of direct input from the user. By using not just visual but touch senses to control the success of the game, I was able to create a unique experience for the player. It was exciting to see my vision come to life and to be able to share it with others. I learned a lot during the process, and I’m looking forward to continuing to explore the possibilities of interactive game development in the future.

Future Improvements

In the future to make the game more engaging and interactive, I am considering adding a multiplayer mode. This would provide an opportunity for players, particularly older individuals, to play with their friends rather than playing alone. I believe that a multiplayer game would instill feelings of healthy competition, making the gameplay even more exciting and encouraging players to keep coming back to the game.