My inspiration for this project was one questions I have asked myself really often during the rainy days: “Where do the campus cats go?” and “How do they survive the rain?”. Based on this, I created “CATS SURVIVAL”, inspired also by the classic arcade games where players navigate through obstacles to achieve a high score. In this game, players engage with Arduino push buttons to control the cat attempting to avoid falling water drops while traversing a colorful campus setting.
Final Setup:
IM Showcase:
How it works:
Players start by launching the game, where they are greeted with a vibrant start page featuring the game’s logo. Once the game begins, the cat automatically appears at the center of the screen, and the player’s objective is to keep the cat from being hit by falling water drops.
Using a connected serial input device (Arduino), players can move the cat left or right, dodging incoming obstacles. Each successful dodge increases the player’s score, while collision with a water drop ends the game.
As the game progresses, the speed, and frequency of falling water drops increase, challenging the player’s reflexes and agility. Upon game over, players can restart the game by pressing any key, offering them the opportunity to beat their previous high score and continue the thrilling dodge-and-survive gameplay.
// Constants won't change. They're used here to set pin numbers:
const int buttonPin1 = 2; // The number of the first pushbutton pin
const int buttonPin2 = 3; // The number of the second pushbutton pin
const int ledPin1 = 13; // The number of the first LED pin
const int ledPin2 = 12; // The number of the second LED pin
// Variables will change:
int buttonState1 = 0; // Variable for reading the first pushbutton status
int buttonState2 = 0; // Variable for reading the second pushbutton status
void setup() {
// Initialize the LED pins as outputs:
pinMode(ledPin1, OUTPUT);
pinMode(ledPin2, OUTPUT);
// Initialize the pushbutton pins as inputs:
pinMode(buttonPin1, INPUT_PULLUP); // Changed to INPUT_PULLUP
pinMode(buttonPin2, INPUT_PULLUP); // Changed to INPUT_PULLUP
// Start serial communication:
Serial.begin(9600);
}
void loop() {
// Read the state of the first pushbutton value:
buttonState1 = digitalRead(buttonPin1);
// Check if the first pushbutton is pressed. If it is, the buttonState is LOW:
if (buttonState1 == LOW) {
// Turn the first LED on:
digitalWrite(ledPin1, HIGH);
} else {
// Turn the first LED off:
digitalWrite(ledPin1, LOW);
}
// Read the state of the second pushbutton value:
buttonState2 = digitalRead(buttonPin2);
// Check if the second pushbutton is pressed. If it is, the buttonState is LOW:
if (buttonState2 == LOW) {
// Turn the second LED on:
digitalWrite(ledPin2, HIGH);
} else {
// Turn the second LED off:
digitalWrite(ledPin2, LOW);
}
// Send button states to the p5 sketch
Serial.print(buttonState1);
Serial.print(",");
Serial.println(buttonState2);
delay(100); // Adjust delay as needed
}
p5 snippet code:
Reading serial data
This function reads data from the serial port, interprets it as button states, and updates the cat’s position accordingly. It ensures that the cat remains within the canvas bounds while moving left or right based on the received data.
This snippet demonstrate how the game can interact with an Arduino board via serial communication to control the cat’s movement.
function readSerial(data) {
if (data != null) {
let buttonStates = split(trim(data), ',');
let buttonState1 = int(buttonStates[0]);
let buttonState2 = int(buttonStates[1]);
// Update cat position based on button states
if (buttonState1 == 1) {
catX -= 22; // Move left
}
if (buttonState2 == 1) {
catX += 22; // Move right
}
// Ensure cat stays within canvas bounds
catX = constrain(catX, 0, width - catImg.width);
}
}
Challenges:
The challenge of this game is designing the obstacle mechanics to appropriately balance the game’s difficulty. Since the game operates in full-screen mode, ensuring that the falling obstacles provide a challenging, yet enjoyable experience for players can be tricky. Balancing factors such as the speed, frequency, and size of the obstacles requires careful consideration to prevent the game from becoming too easy or too difficult. Additionally, transitioning from the initial idea of using a potentiometer for input to utilizing two push buttons might pose challenges in terms of code adaptation and player control dynamics.
Future improvements:
Enhance the complexity of the game mechanics and integrating additional features into the circuit in order to elevate the player experience. Adding new gameplay elements such as power-ups, varying obstacle patterns can provide players with more engaging challenges and keep them invested in the game for longer durations.
Incorporating a speaker into the Arduino circuit to synchronize with button presses could add a wider dimension to the gameplay, enhancing immersion and feedback for players. By integrating sound effects or background music that reacts to player actions, such as cat movements and obstacle collisions, the overall gaming experience can be enriched, making it more dynamic and enjoyable.
Pi’s moving Castle is a p5js + Arduino interactive game by Pi.
Pi has a moving castle, but it is broken down by the presence of rust gremelins in the machinery. You have to help Pi eliminate these gremelins so that the castle can walk again.
Here’s a more visually appealing version of the story.
Documentation Video
instagrammable goodies
DeMo & Concept
So the project consists of 2 parts. A p5js computer game, and a castle with walking legs and some user inputs (potentiometer and a switch). You control a cannon on the computer screen, using the potentiometer to rotate the cannon and shoot it with the switch. But there is a catch, Some of the gremelins, you cannot aim at them directly, so you need to make the cannonballs bounce off the walls to deliver justice to these monsters. Finally once you have cleared all the monsters, the castle can start walking and will physically walk. Below is a demo video of me playing the full experience.
Arduino Code
The arduino code is below. It always send back serial data with the potentiometer and switch readings back to the computer, and it will wait for a single serial int. If computer sends a 1, castle walks, and if computer sends a 0, it stops walking. Depending on the game state it changes.
#include // Include the Servo library
Servo myServo; // Create a servo object
Servo myServo2; // Create a servo object
float lastVoltage = -1; // Variable to store the last voltage
// Arduino code for button, which detects the counts
const int buttonPin = 2; // the number of the pushbutton pin
const int ledPin = 3; // the number of the LED pin
// variables will change:
int buttonState = 0; // variable for reading the pushbutton status
int lastButtonState = HIGH; // variable for reading the last pushbutton status
unsigned long lastDebounceTime = 0; // the last time the output pin was toggled
unsigned long debounceDelay = 50; // the debounce time; increase if the output flickers
int pressCount = 0; // count of button presses
//Potentiometer
float floatMap(float x, float in_min, float in_max, float out_min, float out_max) {
return (x - in_min) * (out_max - out_min) / (in_max - in_min) + out_min;
}
void setup() {
pinMode(ledPin, OUTPUT); // initialize the LED pin as an output
pinMode(buttonPin, INPUT_PULLUP); // initialize the pushbutton pin as an input with internal pull-up resistor
myServo.attach(9); // Attach the servo signal pin to digital pin 9
myServo2.attach(10); // Attach the servo signal pin to digital pin 10
Serial.begin(9600); // Initialize serial communication at 9600 bits per second
stopRotation(); // Stop servos by default
}
void loop() {
int reading = digitalRead(buttonPin);
//Output potentiometer
// Read the input on analog pin A0:
int analogValue = analogRead(A0);
// Rescale to potentiometer's voltage (from 0V to 5V):
float voltage = floatMap(analogValue, 0, 1023, 0, 5);
// check if the button state has changed from the last reading
if (reading != lastButtonState) {
// reset the debouncing timer
lastDebounceTime = millis();
}
if ((millis() - lastDebounceTime) > debounceDelay) {
// if the button state has changed:
if (reading != buttonState) {
buttonState = reading;
// only toggle the LED if the new button state is LOW
if (buttonState == LOW) {
digitalWrite(ledPin, HIGH);
pressCount++; // increment the press count
} else {
digitalWrite(ledPin, LOW);
}
}
}
// save the reading. Next time through the loop, it will be the lastButtonState:
lastButtonState = reading;
Serial.print(pressCount); // print the count to the serial monitor
Serial.print(",");
Serial.println(voltage); // Print the distance to the Serial monitor
delay(100); // Short delay before next measurement
if (Serial.available() > 0) { // Check if data has been received
int state = Serial.read() - '0'; // Read the first byte available and convert from ASCII
if (state == 1) {
rotate(); // Rotate servos
} else if (state == 0) {
stopRotation(); // Ensure servos are stopped
}
}
}
void rotate() {
myServo.writeMicroseconds(4000); // Example value for rotation
myServo2.writeMicroseconds(4000); // Adjust if necessary
}
void stopRotation() {
myServo.writeMicroseconds(1500); // 1500 usually represents a stopped servo
myServo2.writeMicroseconds(1500); // Adjust if necessary
}
p5js Sketch
In the game, players help Pi clear rust gremlins from a mechanical castle using a turret that shoots cannonballs, controlled by a physical potentiometer and switch. The game mechanics include obstacles where cannonballs need to be bounced off boundaries to hit some gremlins. The game features a visual and auditory loading sequence with gremlin and turret images, background music, and sound effects for actions like shooting and gremlin deaths. The Arduino setup facilitates interaction by receiving turret control signals from the potentiometer and switch, while sending back movement commands to make the castle walk when the game is completed.
The embedding of the p5js sketch is below (Note that you need the castle to play the game).
Communication between Arduino and p5js
As mentioned above, the communication between p5js and Arduino is serial data. Arduino sends 2 values (a float reading for potentiometer, and an int counting the number of times the button has been clicked). This controls the rotation of the cannon and firing of the cannon in the game.
From the computer (p5), Arduino only receives one number all the time that is either 1 or 0. This dictates whether or not to move the castle and make it walk (it walks when the game is complete.)
What I am proud of
I am particularly very proud of the visual design, the storyline and the walking mechanism. This looks almost unreal to me, I was not expecting that sticking the midjourney textures on an Amazon cardboard box would look sooo good.
Future Improvements
For future improvements, I will integrate what the users have told me during the user tests.
let fsrVal = 0; // Force Sensitive Resistor Value
let smoothfsrVal = 0; // Global Variable to not have jitter for image
let backgroundImage; // Classroom Image
let teachImageHappy; // Play state teacher Image
let teachImageMad; // Win State teacher Image
let teachImageProud // Lose State teacher Image
let gameStarted = false; // Flag for Game Start
let gameOver = false; // Flag for Game Over
let gameWon = false; // Flag for Game won
let gameStartTime; // Variable for timer
let dogBarkSound; // Variable for Dog barking
let barkTimeout; // Variable for when the dog cannot bark
let lastBarkTime = 0; // Variable to hold when dog stopped barking
let winSound; // Variable to hold the win sound
let gameOverSound; // Variable to hold the game over sound
let winSoundPlayed = false; // Variable to track if win sound has been played
let gameOverSoundPlayed = false; // Variable to track if game over sound has been played
let gameMusic; // Variable to hold the game music sound
let gameMusicPlayed = false; // Variable to track if game music has been played
let showingInstructions = false; // Flag to track if we are currently showing instructions
// It is necessary to preload the images in
function preload() {
backgroundImage = loadImage('class.jpeg');
teachImageHappy = loadImage('teacher_woman_happy.png');
teachImageMad = loadImage('teacher_woman_mad.png');
teachImageProud = loadImage("teacher_woman_teaching.png")
dogBarkSound = loadSound('dog_bark.mp3');
winSound = loadSound('win.mp3');
gameOverSound = loadSound('gameover.mp3');
gameMusic = loadSound('gameMusic.mp3');
}
function setup() {
createCanvas(window.innerWidth, window.innerHeight);
textSize(18);
// Serial Point button logic
const connectButton = createButton('Connect to Serial');
connectButton.position(width / 2 - connectButton.width / 2, height / 2 - connectButton.height / 2);
connectButton.mousePressed(setUpSerial);
// Play button logic
const playButton = createButton('Play');
playButton.position(width / 2 - playButton.width / 2-15, height / 2 - playButton.height / 2);
playButton.mousePressed(startGame);
playButton.hide();
styleButton(playButton);
// Instruction button logic
const instructionsButton = createButton('Instructions');
instructionsButton.position(width / 2 - instructionsButton.width / 2 -20, height / 2 +40);
instructionsButton.mousePressed(displayInstructions);
instructionsButton.hide();
styleButton(instructionsButton);
// Restart button logic
const restartButton = createButton('Restart Game');
restartButton.position(width / 2 - restartButton.width / 2 -25, height / 2 +15); // Positioned below the "Play" button
restartButton.mousePressed(restartGame);
restartButton.hide();
styleButton(restartButton);
// Main Menu button logic
const mainMenuButton = createButton('Main Menu');
mainMenuButton.position(width / 2 - mainMenuButton.width / 2 -25, height / 2 + 75);
mainMenuButton.mousePressed(goToMainMenu);
mainMenuButton.hide();
styleButton(mainMenuButton);
// Button branding for further use
window.connectButton = connectButton;
window.playButton = playButton;
window.instructionsButton = instructionsButton;
window.restartButton = restartButton;
window.mainMenuButton = mainMenuButton;
// Background game music was intially too loud
gameMusic.setVolume(0.035);
noLoop(); // Stop drawing until the game starts
}
// Buttons needed to be style
function styleButton(button) {
button.style('background-color', '#FFFFFF'); // White background
button.style('color', '#000000'); // Black text
button.style('border', '2px solid #000000'); // Black border
button.style('padding', '10px 20px'); // Larger padding for bigger size
button.style('font-size', '16px'); // Larger font size
button.style('cursor', 'pointer'); // Cursor pointer on hover
}
function draw() {
if (serialActive) {
// After connection I prefer the button to no longer be present
window.connectButton.hide();
if (showingInstructions) {
// Instruction state needed these buttons hidden
window.playButton.hide();
window.instructionsButton.hide();
} else {
window.playButton.show(); //If not in instruction state the play button can be shown
window.instructionsButton.show(); // And the instructions button can be shown
}
// button logic/visibility during/post-game
if (gameStarted && !gameOver && !gameWon) {
window.playButton.hide();
window.instructionsButton.hide();
updateGame();
} else if (gameOver) {
displayGameOver();
} else if (gameWon) {
displayGameWin();
}
} else {
displayClassCrashOutScreen();
}
}
function updateGame() {
// Tracking time logic (CHATGPT USED TO HELP SET THIS UP)
let elapsedTime = (millis() - gameStartTime) / 1000;
// Start playing game music when the game starts
if (!gameMusicPlayed && gameStarted) {
gameMusic.play();
gameMusic.loop(); // if game music ends early loop it
gameMusicPlayed = true; // Set the flag to true after playing the music
}
// Stop game music when the game ends
if ((gameOver || gameWon) && gameMusic.isPlaying()) {
gameMusic.stop();
}
// Check win condition (CHATGPT WAS USED)
if (fsrVal >= 250 && !gameWon) {
gameWon = true;
dogBarkSound.stop(); // Stop the dog bark sound if it's playing
winTime = elapsedTime; // Record the time taken to win
}
// Check game over condition (CHATGPT WAS USED)
if (elapsedTime >= 45) {
gameOver = true;
dogBarkSound.stop(); // Stop the dog bark sound if it's playing
return;
}
background(backgroundImage); // Set the loaded background image
displayGameElements(elapsedTime);
}
// Code credit to Professor AARON SHERWOOD (Thank you for your help professor)
function displayGameElements(elapsedTime) {
scaleFactor = 1;
smoothfsrVal += (fsrVal - smoothfsrVal) * 0.01;
push();
imageMode(CENTER);
let teachWidth = (teachImageHappy.width / 2) + smoothfsrVal * scaleFactor;
let teachHeight = (teachImageHappy.height / 2) + smoothfsrVal * scaleFactor;
image(teachImageHappy, width / 2, height / 2, teachWidth, teachHeight);
pop();
fill(255);
textStyle(BOLD)
stroke(0)
strokeWeight(4)
text("Connected", 90, 30);
text('Pages = ' + fsrVal, 100, 70 );
textSize(30);
text('Time: ' + elapsedTime.toFixed(2) + 's', width - 150, 30);
}
function displayGameOver() {
stopBarking();
background(backgroundImage);
fill(255);
textSize(27);
text("Game Over", width / 2, height / 2 - 45);
textSize(22);
text("Teacher's Pet :(", width / 2, height / 2 - 5);
//code to indicate which buttons to hide and show
window.restartButton.show();
window.playButton.hide();
window.mainMenuButton.show();
window.instructionsButton.hide();
// Teacher Image scaling and position
let scaledWidth = teachImageProud.width * 0.5;
let scaledHeight = teachImageProud.height * 0.5;
image(teachImageProud, width / 2 + 130, height / 2 - 125, scaledWidth, scaledHeight);
if (!gameOverSoundPlayed && !gameOverSound.isPlaying()) {
gameOverSound.play();
gameOverSoundPlayed = true;
}
if (gameMusic.isPlaying()) {
gameMusic.stop();
}
}
function displayGameWin() {
stopBarking();
background(backgroundImage);
fill(255);
textSize(27);
text("You Got Detention!!!", width / 2, height / 2 - 45);
textSize(22);
// Display the time taken to win
text("Time Taken: " + winTime.toFixed(2) + "s", width / 2, height / 2 - 5);
// Teacher Image scaling and position
let scaledWidth = teachImageMad.width * 0.5;
let scaledHeight = teachImageMad.height * 0.5;
image(teachImageMad, width / 2 + 130, height / 2 - 125, scaledWidth, scaledHeight);
//code to indicate which buttons to hide and show
window.restartButton.show();
window.playButton.hide();
window.mainMenuButton.show();
window.instructionsButton.hide()
if (!winSoundPlayed && !winSound.isPlaying()) {
winSound.play();
winSoundPlayed = true;
}
if (gameMusic.isPlaying()) {
gameMusic.stop();
}
}
function displayClassCrashOutScreen() {
background(backgroundImage);
fill(255);
stroke(0)
strokeWeight(4)
textStyle(BOLD)
textAlign(CENTER);
textSize(27)
text("CLASS CRASH OUT", width / 2, height / 2 - 35);
}
// Play Through Logic (CHATGPT WAS USED)
function startGame() {
gameStarted = true;
gameOver = false;
gameWon = false;
gameStartTime = millis();
gameMusicPlayed = false;
showingInstructions = false;
playDogBark();
}
function restartGame() {
gameStarted = true;
gameOver = false;
gameWon = false;
fsrVal = 0;
smoothfsrVal = 0;
gameStartTime = millis();
window.restartButton.hide();
window.mainMenuButton.hide();
stopBarking();
gameMusicPlayed = false;
winSoundPlayed = false;
gameOverSoundPlayed = false;
playDogBark();
loop();
}
function goToMainMenu() {
stopBarking();
gameStarted = false;
gameOver = false;
gameWon = false;
gameMusic.stop();
gameMusicPlayed = false;
winSoundPlayed = false;
gameOverSoundPlayed = false;
showingInstructions = false;
stopBarking();
loop();
//code to indicate which buttons to hide and show
window.restartButton.hide();
window.mainMenuButton.hide();
window.playButton.show();
window.connectButton.show();
background(backgroundImage);
fill(255);
textStyle(BOLD)
textAlign(CENTER);
textSize(27)
text("CLASS CRASH OUT", width / 2, height / 2 - 35);
}
function displayInstructions() {
background(backgroundImage);
fill(255);
textSize(27);
textAlign(CENTER, CENTER);
text("Your objective: Prank your friend", width / 2, height / 2 -75);
text("Time Limit: 45 seconds", width/2, height/2 -35)
text("Feed the dog 250 HW pages", width/2, height/2)
//code to indicate which buttons to hide and show
window.mainMenuButton.show();
window.playButton.hide();
window.instructionsButton.hide();
window.restartButton.hide();
showingInstructions = true;
}
// Dog bark Logic
function playDogBark() {
dogBarkSound.play();
// Dog barks at random intervals
let nextBarkIn = random(5000, 7500);
barkTimeout = setTimeout(playDogBark, nextBarkIn); // Schedule the next bark
}
function stopBarking() {
clearTimeout(barkTimeout);
}
// This function will be called by the web-serial library
// with each new *line* of data. The serial library reads
// the data until the newline and then gives it to us through
// this callback function
function readSerial(data) {
////////////////////////////////////
//READ FROM ARDUINO HERE
////////////////////////////////////
if (data != null) {
// make sure there is actually a message
// split the message
let fromArduino = split(trim(data), ",");
// if the right length, then proceed
if (fromArduino.length == 1) {
// only store values here
// do everything with those values in the main draw loop
// We take the string we get from Arduino and explicitly
// convert it to a number by using int()
// e.g. "103" becomes 103
fsrVal = int(fromArduino[0]);
}
//////////////////////////////////
//SEND TO ARDUINO HERE (handshake)
//////////////////////////////////
let sendToArduino = 0 + "\n";
writeSerial(sendToArduino);
}
}
Concept:
My final project concept initially centered on creating a controller for my midterm video game project. However, after a discussion with my professor, I was inspired to shift towards a design imbued with deeper thematic meaning and enhanced interactivity. This push led me to thoroughly repackage and rework my midterm project. Through multiple iterations, I developed a concept that stands distinctly apart from its predecessor. In this new version, players engage in a real-world task—feeding a dog—which in turn affects the game by enlarging the teacher on screen. This innovative interaction model is something I am proud to call largely original and distinct from my previous work.”
Include some pictures / video of your project interaction
*Disclaimer had trouble uploading images so I compiled images into a youtube video
How does the implementation work?
In implementing my project concept using Arduino and p5.js, I utilized the lab’s resources to construct a Force Sensitive Resistor (FSR). This involved using Velostat, a folded piece of paper, two strips of copper tape, and ordinary tape. Once assembled, I connected the FSR to the Arduino using crocodile clips attached to jumper cables. For the visual component, I crafted the “dog” from the SparkFun kit box, using three cardboard pieces (two triangles and one rectangle) to form its structure, and added a cartoon dog’s face for a playful touch. The ‘HW’ blocks, integral to the game’s interactivity, were made from wooden blocks wrapped in paper and secured with tape.
Description of interaction design
For the interactivity aspect of my project, under Professor Aaron’s guidance, I established a serial connection enabling the Force Sensitive Resistor (FSR) to communicate with my p5.js sketch. The interface in p5.js features an image of a cartoon teacher that increases in size as the FSR value rises. To address the issue of the image size increasing too rapidly, I introduced a global variable, smoothFsrVal, and applied the formula smoothFsrVal += (fsrVal - smoothFsrVal) * 0.01 to moderate the growth. To ensure the game remained engaging and not overly prolonged, I set a specific FSR value goal of 250, which, when reached, triggers a win state. Additionally, a timer limits the gameplay to 45 seconds, after which a game over state is activated if the goal isn’t met. The p5.js sketch also includes standard interactive elements such as a ‘Connect to Serial’ button, main menu, play, instructions, and restart buttons—all designed with engaging graphics and set against a classroom-themed background
Arduino Code:
void setup() {
// Start serial communication so we can send data
// over the USB connection to our p5js sketch
Serial.begin(9600);
// Blink them so we can check the wiring
// start the handshake
while (Serial.available() <= 0) {
Serial.println("0");
delay(50);
}
}
void loop() {
// wait for data from p5 before doing something
while (Serial.available()) {
int handshakeRead = Serial.parseInt();
if (Serial.read() == '\n') {
int sensor = analogRead(A0);
delay(5);
Serial.println(sensor);
}
}
}
Description of Arduino code:
In the setup function of the Arduino code, serial communication is initiated at 9600 baud to enable data transfer over the USB connection to the p5.js sketch. This setup includes a procedure introduced in class by Professor Aaron called starting a ‘handshake’—a method used to ensure that the connection is established before proceeding. The Arduino sends a zero (‘0’) repeatedly every 50 milliseconds until it receives a response from the p5.js sketch, indicating that the serial connection is ready. In the main loop, the code continuously checks for incoming data from the p5.js sketch. Once data is received, it reads the data to complete the ‘handshake’, ensuring that each transmission begins only after the previous has been fully processed. It then reads the analog value from pin A0, where the Force Sensitive Resistor (FSR) is connected. This sensor value is briefly paused (a delay of 5 milliseconds is introduced for stability), and then sent back over the serial connection to the p5.js sketch, which uses this data to influence the game dynamics, such as adjusting the size of the cartoon teacher’s image based on the FSR readings
Embedded Sketch:
Description of p5.js code:
Initialization and Preloading: Variables are declared for game state management (like gameStarted, gameOver, gameWon), user interface elements (buttons), sounds, and images. The preload() function loads these resources (images and sounds) to ensure they’re available before the game starts.
Setup Configuration: The setup() function creates the game canvas and initializes interface elements such as buttons. Each button is positioned and styled, and their visibility is managed based on game states. Notably, the game music’s volume is adjusted, and the canvas’s draw loop is paused until the game starts.
Game State Management: Buttons trigger changes in game states. For example, the ‘Play’ button starts the game and triggers gameplay logic encapsulated within other functions like startGame(). Buttons like ‘Restart’ and ‘Main Menu’ facilitate game flow by resetting states or navigating the user interface.
Dynamic Content Rendering: The draw() function acts as the central loop where game logic is continuously checked and updated based on the game’s state. It manages what is displayed on the screen, updates gameplay elements like the timer, and reacts to changes in game state (e.g., transitioning to a win or lose screen).
Game Interactivity and Feedback: Interaction with the physical hardware (FSR value) is integrated into the game logic. The value from the sensor influences the gameplay, affecting visual elements like the teacher’s image size based on the smoothed sensor values. Audio cues are played corresponding to game events like winning or losing, and game music loops during gameplay.
Auxiliary Functions: Functions like displayGameOver() and displayGameWin() manage the display elements during these states, showing appropriate messages and images, and managing audio playback. Utility functions like styleButton() apply consistent styling to buttons across the game.
Serial Communication: The readSerial(data) function handles incoming data from the Arduino. It parses this data to update the force sensor value, which in turn affects the game logic and visuals.
Description of communication between Arduino and p5.js:
If the paragraphs mentioned above do not paint a clear enough picture here is the bullet point representation on how my Final project is communicating between p5.js and Arduino
Part 1:
Arduino Setup:
The Arduino initiates serial communication at 9600 baud rate using Serial.begin(9600);. This sets up the Arduino to send and receive data over the USB connection to the computer where the p5.js script runs.
A handshake mechanism is implemented in the setup() function where the Arduino continually sends a zero (‘0’) until it receives any serial data from p5.js, ensuring that both sides are ready to communicate before proceeding.
p5.js Setup:
In p5.js, the serial connection setup is implied within functions like setUpSerial(), which would be responsible for establishing this link, although the specific implementation details aren’t provided in the snippet. The script is prepared to handle incoming data through a callback function that processes each line of data received.
Part 2:
Data Sending (Arduino to p5.js):
Inside the loop() function on the Arduino, there’s a check for any available serial data (Serial.available()). If data is available, it reads the next integer from the serial buffer, which is part of the handshake or command from p5.js.
After the handshake is confirmed (a newline character is detected), the Arduino reads an analog value from pin A0 (connected to the Force Sensitive Resistor) and sends this value back to p5.js using Serial.println(sensor);.
Data Receiving and Sending (p5.js to Arduino):
In p5.js, the received data is handled by the readSerial(data) function. This function parses incoming serial data to update the force sensor value (fsrVal), which is then used within the game logic to modify game elements, such as the size of the teacher image in the interface.
The script also sends data back to Arduino, likely as part of a continual handshake or control commands, maintaining synchronization between the hardware inputs and the software responses.
Part 3:
Game Element Updates: The fsrVal from the Arduino directly impacts game dynamics. For example, an increase in the FSR value causes the teacher image to grow in size, visually representing the game’s progress based on real-world actions (like pressing the FSR).
Dynamic Adjustments: The smoothfsrVal variable in p5.js smooths out the rapid changes in the sensor value to ensure the game’s visual feedback doesn’t appear jittery or overly responsive to noise in sensor readings.
What are some aspects of the project that you’re particularly proud of?
I’m particularly proud of the DIY Force Sensitive Resistor (FSR) sensor that I constructed for this project. Building the primary component of my project from scratch, using only the resources available in the lab and guidance from various YouTube tutorials, was immensely fulfilling. There was no pre-built FSR sensor available that fit my needs, which presented a significant challenge. Tackling this obstacle head-on, I was able to problem-solve and innovate under pressure. This not only enhanced my technical skills but also boosted my confidence in handling and overcoming complex engineering problems on my own. The successful integration of this self-made sensor into the project stands as a testament to the creative and technical prowess that I developed throughout this endeavor.
What are some areas for future improvement?
One of the primary areas for future improvement in my project is the gameplay loop. While it successfully fulfills its basic purpose, it currently lacks sustained entertainment value. Introducing more sound effects and enhanced user feedback could significantly enrich the gaming experience, making it more engaging and dynamic for players. Additionally, the build quality of the interactive controller needs reinforcement. Given that the gameplay involves objects frequently being thrown at the controller, constructing a sturdier framework is essential—especially since the dog’s face component is particularly vulnerable. Another critical area for improvement is the sensitivity of the FSR value. Currently, balancing the sensor’s responsiveness so that it is neither too sensitive nor too unresponsive is a significant challenge. This aspect of the project requires a deeper understanding and more refined coding skills, but I am confident that with time and continued learning, I can develop a more robust and precise response mechanism for a better gameplay experience.
Users understood the relationship between their actions and the game responses well. This was particularly true for tasks involving direct interactions like pressing buttons.
Engagement with Tasks: The distance sensing tasks were well-received, as participants found them to be intuitive and fun.
Sensor Integration: The integration of the potentiometer within the game mechanics worked smoothly, providing a satisfying challenge to the players.
Areas for Improvement
Animation Speed: There was a noticeable slowdown in animation frames as the game progressed, which affected the overall experience. This issue needs to be addressed to ensure smooth gameplay.
Keypad Usage: Although the Adafruit Trellis keypad was a central component of the game, many users needed additional instructions to use it effectively.
Enhancements and Future Steps
To make the game more user-friendly and engaging, here are some steps I plan to take based on the feedback:
Clarify Sensor Usage: Simplify the riddles associated with sensors or provide clearer, step-by-step tutorials that guide the users on how to interact with the game.
Improve Animations: Optimize the code to ensure that animation frames run smoothly throughout the game, enhancing the visual feedback that is crucial for interactive gameplay.
For my project, I did 9 user test in total, with the people who did not know about my game at all. The procedure is such that I invite them, I start the p5js sketch, set up the camera and let them work out without saying anything.
Gladly, out of 9 subjects, only 2 needed instructions from me, and others at one point were able to figure out the entire experience withotu instruction. The average play time/ or time needed for them to enjoy the experience without needing instructions is 3 minutes. Below are 3 sessions of the user testing.
The only parts where I needed to explain for the 2 users are (1) the user did not read the instructions on the screen and did not know they have to touch the castle and (2) The user did not connect to the right serial port.
In general, this is the feedbacks I get for improvement suggestion.
Cover the breadboard of the castle with a ladder texture to make the design more consistent and hide the electronics.
I need to be more specific in telling which serial port to choose (now I only say usbmodem).
Take out all the debugging functionalities I made for myself in the actual game.
Tell the people to stand up and play (It is more comfortable than sitting down).
Sometimes the bullet hit the enemy, but the enemy did not die (I need to adjust the collider radius).
Some users did not get the full background story, so probably have a mini comic printed out so that whoever interested can read.
Special Thanks
Special Thanks to Guglielmo Fonda, Aditya Pandhare, Aneeka Paul, Megan Marzolf, Salma Mansour, Sashank Neupane, Lingfen Ren, Lexie and Nastassja for testing my project.
The journey of transforming a concept into a tangible, functional reality is an exhilarating experience, with challenges and discoveries at every turn. The most enjoyable part of this adventure for me was definitely the user-testing phase of my final project. It’s one thing to nurture an idea in your mind or sketch it on paper, but watching real people interact with your creation adds a layer of excitement and invaluable insights.
A significant hurdle I faced was with the servos intended to be a key component of the project. Despite my efforts, I struggled immensely with integrating and programming them. The complexity proved to be beyond the scope of what could be managed within the time and resource constraints of the project I had to make the difficult decision to pivot and remove the servos entirely from the project. This decision, though tough, was necessary to maintain the project’s viability and ensure a smoother user experience.
Pivoting away from using servos forced me to rethink and simplify the design, focusing on what was most essential and functional. This redesign, though less complex, brought its own set of challenges and learning opportunities.
The insights gained from user-testing were invaluable. Observing how users interacted with the revised version of the project without the servos helped me understand the practicality and user-friendliness of my design. The feedback was instrumental in shaping the final tweaks:
Timer needs to be bigger: Users found the timer too small, highlighting the need for a more visible and accessible design. (Alongside that I believe I will increase the size of all text) Add instructions page: User suggested that there be an instructions page to clarify the objective Fix dog barking glitch: Dog barking would play over itself causing an awkward sound (potential solution is to increase the delay time between sounds
Through user testing, I improved the look of the device itself by covering the wire with insulated heat-shrinkable tubes so that it doesn’t get caught up by anything over the user’s hand. Also, an issue that was resolved after more testing was that, for some odd reason, the code kept looping over the positioning of the UFO, making it glitch after a while of playing the game.
Video of the problem:
After the improvements:
After playing once or twice, people can understand the concept, which made me realize that I needed to create an info tab to finalize the project.
For user testing, I asked two of my friends to come in and try interacting with my piece. I have so far managed to bring everything together: 1) construct the box that would come to enclose the Arduino and whose surface would act as a canvas for the sketch to be projected on, 2) make the butterfly body out of printed laminated paper and attach the wings to the servos, 3) secure the butterfly atop the surface, and 4) attach two wires, shaped like butterfly antennas, beneath the wings to convey touch signals to the p5 sketch and activate the animation and the butterfly flutters. I knew that upon seeing the piece, users might not get the intuition to touch the butterfly. They also might shy away from touching it, out of fear of ruining the butterfly’s delicate structure, or assume that some other automatic process would have to trigger the movement of the piece. To counter that, I will be displaying a name next to my piece that indicates, but does not outright reveal, that a touch signal is necessary. I have not yet settled on a name, but have told my two friends that it would be called “catch a butterfly” or “pet a butterfly” and they immediately figured out that they had to use their hands in some way. I placed the antennas facing them so that they were more likely to come into contact with them and I was glad that they did end up placing their touch close enough to the wires for the signal to be captured.
They both loved seeing the butterfly in action, but gave the following feedback, which was mainly focused on the sketch itself:
add different colors to the butterflies in the sketch
experiment with different movement paths for the projected butterflies (inward movement from outside the sketch bounds or having multiple center points for their emergence, for example) or smoothen/slow down their movement.
I expected the feedback I got from them, as I have been planning to implement those two features after I had the basic version of the project implemented. I will be working on that over the next couple of days, as well as decorating the surface some more (adding flowers to cover the base of the servos and on different parts of the surface).
The project is well on its way, and I have some updates for all of you!
First of all, let’s start with the good things, and that is that the boat structure has been successfully 3d printed and is waterproof. Yaay. The arduino as well as all the batteries and breadboard fit comfortably on the structure so the sizing was correct (for the most part).
There is a few things I need to tackle which I realized during the user testing. Firstly, the fan that I have is not strong enough to move the boat, I will need to find, or make a bigger and better one. The boats construction also needs some styrofoam on the bottom since the boat is too heavy to float, as seen in the video below.
Over the next two days I want to solve the problem with the floating and the movement on the boat so I can leave the controls for the last day.
As I was conducting user testing for my game, an actual storm was hitting Abu Dhabi, echoing the storm scenario in my game and making the experience feel all the more relatable.
I enlisted my sister to test the game without any prior instructions, which was a great decision as the game proved to be intuitive—she knew immediately that she needed to press the button. However, since I haven’t yet built the controller box, I had to hold it for her. For the wooden block, I explained that she should imagine it as a car moving on the screen.
To my surprise, she played better than I did! It was really enjoyable to watch my game perform exactly as intended. She managed to figure everything out smoothly; the only hiccup was the wooden block. If it had actually looked like a car, I wouldn’t have needed to explain anything. I also handled connecting to the serial port since she wasn’t familiar with it. For the actual exhibition, I’ll provide short written instructions for the connection to assist anyone if I’m not around.
Area for improvement:
The game runs smoothly overall, but the collision detection between the hail and the car is glitchy. Sometimes it triggers a game over even when it shouldn’t, which is quite frustrating. I plan to resolve this issue before the exhibition for sure.
