Category: F2024 – Mang

In his Eyeo talk, Casey Reas reflects on how randomness can bring life to a system. At one point, he explains that when there is no variation or unpredictability, everything begins to merge into sameness: “If these lines don’t have any bit of noise or jitter, they gradually tend towards one location. The system becomes homogeneous.” I found this idea so powerful because it mirrors real life. If everything was perfectly controlled, predictable, and identical, the world would feel lifeless. For example, if every tree on my walking path looked exactly the same, going on a walk would be dull and uninspiring. But because there’s always a little difference like a twisted branch, a bird flying by or sudden shift in light ,  my walk is always interesting and dynamic. Reas’ point about adding just a “slight bit of noise” to keep a system alive reminds me that unpredictability is what makes both art and life vibrant. I completely agree with his perspective that randomness doesn’t need to feel chaotic; instead, it can be a natural force that keeps things intriguing and gives them life, much like nature itself.

Reas also talks about the tension between chaos and order in art, saying that historically artists have been expected to impose structure and fight against disorder. His work flips that expectation, showing that embracing randomness doesn’t mean giving up control. He designs rules and systems but leaves certain spaces open for chance, allowing surprising outcomes to emerge. This balance really resonates with me. I’ve noticed in my own creative work that when I try to control every single detail, the result feels rigid and forced, it almost seems as if it traps my flow of creativity. On the other hand, if I let go completely, the work fully loses direction and structure. I think the “optimum balance,” as Reas demonstrates, is to create a strong foundation, like the rules of a game, and then allow randomness to bring it to life. This makes me rethink how I approach creativity. Rather than seeing unpredictability as a threat, I want to treat it as a collaborator, trusting that a little randomness will make my work interesting and lively, just as it makes the world around us more beautiful and alive.

Casey Reas’ talk really resonated with me because I go on walks all the time, and I’ve noticed how much the environment affects how I feel. The first photo, with its perfectly symmetrical path and identical trees, reminds me of walking on a treadmill at the gym,  predictable and boring. But the second photo, with its uneven, colorful flowers and natural variety, captures what makes walking outside so special to me. The flowers aren’t perfectly placed, yet that imperfection gives the path life and personality. It doesn’t feel chaotic, just alive, and that’s what keeps my walks meaningful and exciting.

Overview

For this first assignment, this was actually my very first time coding. Because of that, I wanted to keep things simple, but also try something that I thought was cool and interesting. I decided to make a portrait in a scene that could change from day to night. The feature I liked the most was being able to switch the background from morning to night with a single click, it felt like a big achievement for me and made the project more fun. Overall, I kept the shapes and details simple, but I made sure that I could still change and experiment with the code.

https://editor.p5js.org/hajarr.alkhaja/full/

https://editor.p5js.org/hajarr.alkhaja/sketches/YawBFYuSj

 

Process

We started with a warm-up in class where we learned how to make a simple circle. That gave an idea on how to code for different shapes. We also learned how to make shapes change with a mouse click, and this inspired me to add the day and night feature. I watched the tutorial videos from YouTube, which helped me understand how to use shapes like ellipses, arcs, and triangles. From there, I built my project step by step, first the background, then the grass and sky, and finally the girl. I kept it simple, using a different shape and color for each feature, and added a night variable so everything could change between day and night.

Code Highlights

At first, I really struggled with the code because I didn’t fully understand how the night variable and the if (night) statements worked. It was confusing to me how to make something happen only in the day or only in the night. Slowly, I started to get the hang of it, and once I understood the logic, it became much easier. In the end, most of my code was just repeating the same pattern in different ways, for example, changing colors, showing the sun or the moon, or opening and closing the eyes. 

I found it really interesting how I defined the night variable once and then kept repeating the if/else code for each feature of my self-portrait. I slowly got the hang of it and it helped me understand how coding logic works.

// girl's dress (triangle body)
if (!night) {
  fill(255, 105, 180); // bright pink (day)
} else {
  fill(199, 21, 133);  // darker pink/purple (night)
}
triangle(
  width/2 - 25, height - 70,
  width/2 + 25, height - 70,
  width/2,      height - 190
);

 

Another challenge I faced was figuring out where to place each shape. I had to keep adjusting the positions, running the code, and moving things again until I found the right spots. This took a lot of trial and error, but it also helped me learn how the coordinate system works.

At one point my code even stopped working completely and gave me errors I didn’t understand. I couldn’t figure out why, so I used AI to help me find the mistake. Once I fixed it, everything started working again, and I was able to continue building my portrait. Overall, I feel like I learned a lot about both the logic of coding and the patience it takes to position and test shapes.

Concept:

This game incorporates two key concepts. The first concept, which inspired the creation of this game, is the maze itself—an idea drawn from my midterm project. In the game, the user starts at one end of the maze’s wire frame and navigates through it without touching the wire, aiming to reach the endpoint before time runs out. The maze is uniquely designed to resemble the word “Robinson,” with the first letter forming an “R,” the last letter forming an “N,” and the middle section creatively looped to provide a more engaging and challenging experience for players.

The second concept is inspired by urban design. The maze is mounted on two wooden blocks designed to resemble buildings, giving the entire structure the appearance of a miniature cityscape. This combination of gameplay and aesthetic design enhances the overall experience by integrating storytelling with visually appealing architecture.

Project Interaction:

Do you remember the person I used for my user testing? Well, they’re back again for another round to interact with the program, this time after addressing and resolving any misconceptions about how the game functions. Below is a video of Vladimir testing my game once again:

How Does the Implementation Work?

1. Description of Interaction Design

The interaction design focuses on creating a seamless and engaging user experience. The game begins with a welcome screen that introduces the user to the interface. It includes options to view instructions or start playing the game.

• Instructions Screen: The instructions explain the rules of the game, such as the objective of navigating through the maze without touching the conductive walls and what happens when the wire is touched. A “Back” button is provided to return to the main menu.
• Timer Selection: Players can choose their preferred play duration (30 seconds, 1 minute, or 2 minutes). After selecting a timer, the game transitions to a countdown preparation phase.
• Game Play: During the game, the player must navigate the maze using the loop object without touching the maze walls. Touching the walls triggers a red glow effect and reduces the remaining time by 5 seconds. The player wins by reaching the endpoint (connected to the A3 pin) before the timer runs out. If the timer reaches zero before the endpoint is touched, the player loses.
• Win/Lose Feedback: Winning triggers celebratory fireworks visuals, while losing displays visuals indicating failure. Both states return the user to the main menu after 5 seconds.

2. Description of Arduino Code

The Arduino code handles two primary functions:

• Touch Detection (A0): When the loop object touches the maze walls, the Arduino sends a ‘Touch Detection’ message to p5.js. It also briefly activates the buzzer to provide immediate audio feedback.
• Win Detection (A3): When the loop object reaches the endpoint, the Arduino sends a ‘win’ message to p5.js to indicate success.

The code integrates digital input from the A0 and A3 pins and sends serial messages to the p5.js sketch, allowing the game to react accordingly. The following code is my entire self-written code (including comments) that I used to bring the project to life using the Arduino IDE:

#define TOUCH_PIN A0
#define WIN_PIN A3
#define BUZZER_PIN 8

bool gamePlaying = false; // Flag to track if the game is currently active

void setup() {
  Serial.begin(9600);

  pinMode(TOUCH_PIN, INPUT);  // Touch detection
  pinMode(WIN_PIN, INPUT);    // Win detection
  pinMode(BUZZER_PIN, OUTPUT); // Buzzer feedback

  Serial.println("Setup Complete. Monitoring touch and win states...");
}

void loop() {
  // Check for serial messages from p5.js
  if (Serial.available() > 0) {
    String command = Serial.readStringUntil('\n');
    command.trim();

    if (command == "START") {
      gamePlaying = true; // Start the game
      Serial.println("Game started!");
    } else if (command == "STOP") {
      gamePlaying = false; // Stop the game
      Serial.println("Game stopped!");
    }
  }

  if (gamePlaying) {
    int touchValue = digitalRead(TOUCH_PIN); // Read A0
    int winValue = digitalRead(WIN_PIN);    // Read A3

    // Check if wires are touching
    if (touchValue == HIGH) {
      tone(BUZZER_PIN, 1000); // Play buzzer
      delay(100);
      noTone(BUZZER_PIN);
      Serial.println("Touch detected!"); // Notify touch
    }

    // Check if win condition is met
    if (winValue == HIGH) {
      Serial.println("WIN"); // Notify win
      delay(500);            // Avoid spamming
    }
  } else {
    noTone(BUZZER_PIN); // Ensure the buzzer is off when the game isn't active
  }

  delay(50); // Stability delay
}

3. Schematic

The circuit consists of:

• A0: Connected to the loop object with a 10kΩ pull-down resistor.
• A3: Connected to the maze endpoint for win detection.
• D8: Connected to the buzzer for audio feedback.
• 5V and GND: Power and ground connections.

The schematic visually details how the components are connected to the Arduino board.

4. Description of p5.js Code

The p5.js code manages the visual interface, user interactions, and communication with the Arduino. Key functions include:

• Serial Communication: Establishes a connection with the Arduino and processes messages received from it.
• Visual Design: Displays custom backgrounds for each screen (e.g., welcome, instructions, timer selection, and game states). Buttons are styled and positioned for ease of use.
• Game Logic: Handles the countdown timer, touch detection, win/lose conditions, and visual effects for the game (e.g., the red glow for touches, fireworks for wins).
• Dynamic Transitions: Smoothly transitions between different game states and incorporates a 3-second preparation countdown to ensure the user is ready before gameplay begins.

The following is the p5 embedded code. Be sure to copy the Arduino code into the Arduino IDE, using the schematic connections on the Arduino Uno board if you would like to test this following program

for the full screen link, click here: https://editor.p5js.org/takuthulani/full/H_F1sS4qo

5. Description of Communication Between Arduino and p5.js

The communication between Arduino and p5.js is established through serial data transfer. The Arduino sends the following messages based on the game events:

• “Touch detected!”: Sent when the loop object touches the maze walls (A0 input). p5.js responds by reducing the timer and activating the red glow effect.
• “WIN”: Sent when the loop object reaches the endpoint (A3 input). p5.js displays the “You Won” message and celebratory visuals.

Additionally, p5.js sends a message to the Arduino when the game begins and a stop message when the game ends.

Aspect I am proud of:

If I am being truly honest, I feel really proud of the overall functioning of the game. The game works exactly as I imagined it would, and this is something to be proud of since we know that, from concept to final project, many things can go wrong. One often needs to make exceptions for unforeseen challenges and find creative solutions. In this case, the project works as intended, and I am particularly proud of the fact that when you touch the wire, the small speaker activates, and the screen flashes red to indicate the touch.

The serial communication was one of the trickiest parts of this project. Strangely enough, my game stopped working when the IM showcase began, requiring a few minutes of troubleshooting to get it running again. Beyond that hiccup, I am especially proud of the feature where the timer decreases whenever the user touches the wire. This functionality involved many late nights of debugging, but the result was worth it.

Lastly, I am happy that the program does not react when anyone touches the wire with the conductive loop object while the game is not being played. This demonstrates that the code I wrote provides enough control and that the communication between p5.js and Arduino is solid.

Areas of Future Improvements:

I am someone who enjoys visually stimulating designs, and one thing I would like to improve is the overall visual look of the game. The cover page seems too simple, and I would add enhancements to make it more engaging. Additionally, I would add sounds to the interface buttons so that they produce feedback whenever someone clicks them. More instructions and better visual cues for the game would greatly enhance the user experience.

For the timer, I would make it more visually apparent that the player is running out of time and that they lose time whenever they touch the wire. One improvement could be adding a louder speaker, as the sound of the small speaker was often drowned out by the environmental noise during the IM showcase. Providing users with options to enable or disable background music would also enhance the experience.

Furthermore, the physical structure of the game could use a more polished look. While not many people commented on this during gameplay, a better structure would contribute to the overall mood of the game. Lastly, I would add more engaging animations to the “You Win” screen to make users feel a greater sense of accomplishment. Implementing a high-ranking system could encourage competitiveness among players, as I noticed many users enjoyed challenging one another. Additionally, a more dramatic loss message could motivate users to replay the game more often.

 

Credits:

I would like to take this opportunity to thank Nelson from the scene shop for his immense help in cutting the wood for my project. I would also like to thank the lab TAs for assisting me in finding items in the lab and helping me progress through this project by supplying resources.

A special thanks to Professor Michael Shilo, whom I met in the lab. He helped me find wires in the scene shop, and I am grateful for his suggestions. Lastly, I would like to thank Professor Michael Ang for guiding me through this project from the beginning to the end, as well as for all the knowledge I gained from him throughout the semester.

Disclaimer

I use Grammarly as a browser add-on in Chrome, which assists me in fixing grammar issues while writing this blog.

User Testing

The user testing for this project was both enlightening and entertaining. The first test occurred just a few minutes before the Interactive Media (IM) showcase. Vladimir Sontea, the first person I invited to test the game, provided a unique perspective and valuable feedback. Below is the video recording of the session:

In the video, you can hear me laughing at how quickly Vladimir lost before even starting the game. What stood out wasn’t his loss but the realization of certain faults in my game design. This moment marked the beginning of identifying key areas for improvement.

Observations:

User Experience

• The user quickly figured out the navigation system, including accessing the instructions page, selecting the desired timer, and starting the game. This indicates that the visual structure of the game is intuitive and easy to understand.
• However, there was significant confusion about what happens when the wire is touched. Questions like “How many lives do I have?” and “What are the conditions for losing?” came up frequently.

Game Mechanics

• The key mechanic—reducing the timer by 5 seconds when the wire is touched—was unclear to users. This detail was missing from the instructions, which many users tended to skip.
• Initially, I left this aspect as part of the game’s “discoverability,” where players would learn the rules by playing. However, based on feedback, I made this mechanic more explicit to avoid unnecessary confusion. The following image shows the changes I made to ensure users understood the game instructions:

What Worked Well

• The core mechanics, including the timer countdown and the win/loss conditions, functioned as intended.
• The navigation system, with options for instructions and gameplay, was clear and easy to use.
• The visual appeal of the game effectively drew users’ attention, and the inclusion of the “gold handle” added a premium feel to the design.

Areas for Improvement

1. Instructions:
   • Many users skipped the instructions page, leading to confusion about certain game rules. Adding more visual and interactive cues to emphasize key mechanics (like the 5-second penalty for touching the wire) could help.
2. Design:
   • While functional, the game’s visuals could be enhanced further to improve its overall polish and user experience.
3. Preparation Countdown:
   • The 3-second countdown appeared as a glitch to users unfamiliar with its purpose. Better visual or auditory cues could make it clear that the countdown is intended to give players time to prepare.
4. Safety Concerns:
   • Some users were hesitant to interact with the conductive handle, fearing they might get shocked. Adding a clear safety disclaimer in the instructions would address these concerns.

 

These insights, gathered during the first user test case, informed improvements in the game’s mechanics, design, and user experience. Although these seemed to be the challenges of the game, I strongly believe that it would not take away the experience of the game.

Introduction (week 11)

Coming up with an idea for this project was a challenge. I struggled with creative block and external pressures, which delayed the process. After two weeks of reflection, I decided to revisit and expand on a concept from my midterm project.

In the midterm project, the idea was to guide the main character through a maze. The narrative was that the character (the player) journeys through the maze to finally meet their parents. However, the maze portion was not fully realized. For this project, I wanted to bring this maze concept to life using p5.js and Arduino.

This final project builds on that narrative and integrates digital and physical interaction, creating an engaging and immersive experience.

Game Design and Features (week 12)

Overview

The game combines p5.js visuals with Arduino-based physical interactivity. The player navigates a conductive wire maze using a loop object. The goal is to reach the end of the maze without touching the maze walls and within a set time limit. As a key design element, the maze design spells out the word “ROBINSON,” tying back to the narrative of the midterm project. More on this later.

Arduino Program

The following are the functionalities of the project:

Inputs:

1. A0 (Touch Detection): Detects if the loop touches the maze wire.
   • Behavior: When a touch is detected, it sends a “Touch detected!” message to p5.js.
2. A3 (Win Detection): Detects if the loop reaches the end of the maze.
   • Behavior: When contact is made, it sends “WIN” to p5.js.

Outputs:

1. BUZZER_PIN (Buzzer): Plays a short tone when a touch is detected during gameplay.
   • Behavior: Activates only while the countdown timer is active.

The following is code snippets of how this is brought to life:

void loop() {
  int touchValue = digitalRead(TOUCH_PIN); // Check for touches
  int winValue = digitalRead(WIN_PIN);     // Check for win condition

  if (touchValue == HIGH) {
    tone(BUZZER_PIN, 1000); // Activate buzzer
    delay(100);
    noTone(BUZZER_PIN);
    Serial.println("Touch detected!"); // Notify p5.js
  }

  if (winValue == HIGH) {
    Serial.println("WIN"); // Notify p5.js of win
    delay(500);
  }

  delay(50); // Stability delay
}

p5.js Program

Inputs from Arduino:

1. ‘Touch detection’
   • Deducts 5 seconds from the timer.
   • Triggers a red glow effect on the screen edges.
2. ‘WIN’ detection
   • Displays a “You Won!” message with a fireworks animation.

Outputs to Arduino:

1. START: Sent when the game begins, activating Arduino’s detection logic.
2. STOP: Sent when the game ends, deactivating Arduino outputs.

Key Features:

• Dynamic Timer Countdown: Starts a countdown when a timer button is selected, with a 3-second preparation countdown before the game begins.
• Touch Feedback: Deducts time and triggers a glow effect when the maze walls are touched.
• Win and Lose States: Celebrates a win with fireworks or displays a loss message if time runs out

the following are some key code snippets for how this would work:

function readSerial(data) {
  if (gameState === "playing") {
    const trimmedData = data.trim();

    if (trimmedData === "Touch detected!") {
      glowEffect = true;
      clearTimeout(glowTimer);
      glowTimer = setTimeout(() => (glowEffect = false), 100);
      timeLeft = Math.max(0, timeLeft - 5); // Deduct 5 seconds
    }

    if (trimmedData === "WIN" && timeLeft > 0) {
      clearInterval(timerInterval);
      writeSerial("STOP\n");
      gameState = "win"; // Transition to the win state
    }
  }
}

function startGame(time) {
  timeLeft = time;
  gameState = "countdown";
  let countdownTime = 3;

  let countdownInterval = setInterval(() => {
    countdownTime--;
    if (countdownTime <= 0) {
      clearInterval(countdownInterval);
      gameState = "playing";
      writeSerial("START\n");
      startTimer();
    }
  }, 1000);
}

Execution Process (progress)

Physical Setup

1. Maze Design: The maze spells out “ROBINSON,” aligning with the narrative. This design adds a storytelling element to the game while providing a challenging maze path.
   • Material Used: Conductive wire shaped into the letters of “ROBINSON.”
   • Support Structure: Cut wood panels to create a stable base for the maze.
   • Process: Shaped the conductive wire carefully to form the letters and attached it to the wood base for durability and aesthetics.

the following image is an attempt made to spell the word mentioned above using conductive wire:

The following images show the progress of this project through wires, woodcutting and soldering:

Wiring Schematic

• A0: Connected to the loop object and configured with a pull-down resistor for touch detection.
• A3: Configured for detecting contact at the maze endpoint.
• BUZZER_PIN: Connected to a larger external buzzer to provide audible feedback during gameplay.

The following is the schematic used for this project:

This concludes the progress made for the maze program. After completing this progress, this is when I came up with the name for it: The Robinson Maze.

NYUAD Puzzle Adventure

My final project was inspired by the beauty of the NYU Abu Dhabi campus. I designed a simple game that involved beautiful images of the campus some taken by myself and some from online sources. Initially the game begins with images in 4 square pieces and a player can move and arrange the pieces to make the image whole. The control can be done entirely through a simple controller that I designed. The game keeps track of time in each chapter as the player solves a specific image and the game intentionally designed to loop, Increasing in level of difficulty every time. The level of difficulty increases by increasing need for precision and increasing number of image pieces.

Video Demonstration 

Demonstration 01: Mr. Godbless Osei

Demonstration 02: Mr Aimable Tuyisenge

Demonstration 03: Screen Recording

Implementation

Interaction

Interaction being key in any game design, NYUAD puzzle adventure integrates interaction through a custom controller that is used to move and arrange the image pieces. The controller can be seen below:

Given my game design, I wanted to make a simple controller, So I used four Switches and a Joystick .  I used the switches to change between game states in my game and the joystick to control movement of pieces. The electrical connections the controller can be seen  on a schematic below:

In addition to interaction through the hardware design, I also used sounds that significantly provided cues. One such was a sound that notifies player that two pieces are already attached together. And a celebration rewarding player when an image is completed.

Arduino code

My Arduino code, which was was the heart of the controller involved the following :

1. Reading data from the buttons and the joystick

2. Sending data to P5.js via serial connection.

3. Receiving data from P5.js for the speakers in the controller.

My Arduino code can be see here:

// ------------Buttons pins-------------------------------
int changePin = 3;       
int playPin = 5;   
int instructionsPin = 2;   
int nextPin = 11;   
// ------------Joystick X and Y pins-----------------------
const int xPin = A2; 
const int yPin = A1; 
// ------------Speaker Pin---------------------------------
const int speaker1Pin = 1;
const int speaker2Pin = 4;
// ------------Control Speaker------------------------------
int playOrNot = 0;
const int play1 = 440;
const int play2 = 39;

int xValue = 0;
int yValue = 0;

void setup() 
{
  pinMode(changePin, INPUT);        
  pinMode(instructionsPin, INPUT); 
  pinMode(playPin, INPUT);      
  pinMode(nextPin, INPUT); 
  pinMode(speaker1Pin, OUTPUT);
  pinMode(speaker2Pin, OUTPUT);  
  Serial.begin(9600);           
}

void loop() 
{
  xValue = analogRead(xPin); 
  yValue = analogRead(yPin); 
  int changeState = digitalRead(changePin);
  int playState = digitalRead(playPin);
  int instructionsState = digitalRead(instructionsPin);
  int nextState = digitalRead(nextPin);

  // Serial output in order -> Instructions, Next, Change, Play, X, Y
  Serial.print(instructionsState == HIGH ? "HIGH," : "LOW,");
  Serial.print(nextState == HIGH ? "HIGH," : "LOW,");
  Serial.print(changeState == HIGH ? "HIGH," : "LOW,");
  Serial.print(playState == HIGH ? "HIGH," : "LOW,");
  Serial.print(xValue);
  Serial.print(",");
  Serial.println(yValue);

  if (Serial.available()) 
  {
    playOrNot = Serial.parseInt();
  }
  if (playOrNot == 1) 
  {
    tone(speaker1Pin, play1); 
  } 
  else if (playOrNot == 2)
  {
    tone(speaker2Pin, play2); 
  }
  else
  {
    noTone(speaker1Pin);
    noTone(speaker2Pin);
  }
}

P5.js sketch

My P5.js sketch manages  multiple states such as “WELCOME,” “INSTRUCTIONS,” “PLAY” and uses a serial connection for controller inputs.  Some key features includes

1. Welcome screen: Displays an introductory screen and a short game summary.

2. Controller Integration: Reads serial data for buttons and joystick inputs to navigate states and interact with puzzle pieces.

3. Dynamic Screens and Feedback: Provides visual and audio feedback, including fading hints, a timer, and celebration effects when puzzles are solved.

4. State Management: Transition between screens based on button presses and game progress.

In my code, I also integrated keyboard controls to facilitate debugging.

The P5j.s Sketch can be seen here: -> Fullscreen

Communication 

Data is sent between Arduino and p5.js through a serial connection. Arduino sends data as a string, in a particular order e.g  playButton, value1, value2,…. p5.js listens for this data, splits it into individual pieces, and uses those pieces to control the game states and actions, such as pressing buttons or moving the joystick.

Things I am Proud of

I am proud with how the whole game has turned out. However, some things that really stood out to me are as follows:

1. The controller design:

Designing this controller has helped me understand how complex controllers like those for the Xbox or PS5 work. I could envision my design integrated with many other elements to and become just as complex  as those in market today.

2. The graphics

The graphics used were quite simple, but very appealing in my opinion.

Reflections for Future

Several areas could be improved in this project. Some of them includes:

1. Using different shapes, like jigsaw patterns, instead of simple rectangles. While I initially wanted to use such complex shapes and pattens for my pieces, I failed due to complexity in wrapping images, managing positioning and many other challenges.

2. The controller design could also be improved  by adding sensors like sensors such as touch sensors and adding  to make the controller more interactive. Adding vibration motors would give players a better feel during gameplay.

For my final project, it is an extension of my midterm project onto a physical scale. The project is a game where two players have to work cooperatively to “dance” to input the right sequence to defeat the orbs that appear on the screen moving closer to them. In my final project I initially wanted to make a dance pad which user can control using their feet, but after evaluating the amount of material and time left to complete the project, I decided to scale down my project scope. Additionally, I decided to make a game box which players can press as in an arcade rather than using the button on my laptop.

Schematic

Serial Communication

For my project, the serial communication was mostly reliant on the signal from Arduino rather than p5. Since I didn’t attach any output device (LED, buzzer) on the Arduino, I didn’t need p5 to send anything back to Arduino. On the other hand, Arduino would continuously check if any of the buttons that I had made were pressed and if they were pressed, send a send to p5 so the characters moved, or the game state changed. In the future, I would like to add a buzzer to indicate a life loss or when the player advances to the next round to provide game feedback to the player.

p5 Code (Serial Communication)

function readSerial(data) {
  // Send control values (left, right) to Arduino
  if (data && data.length > 6){
    let fromArduino = split(trim(data), ",");
    redUP_arrowPressed = int(fromArduino[0]);
    redDOWN_arrowPressed = int(fromArduino[1]);
    redRIGHT_arrowPressed = int(fromArduino[2]);
    redLEFT_arrowPressed = int(fromArduino[3]); 
    
    blueUP_arrowPressed = int(fromArduino[4]);
    blueDOWN_arrowPressed = int(fromArduino[5]);
    blueRIGHT_arrowPressed = int(fromArduino[6]);
    blueLEFT_arrowPressed = int(fromArduino[7]); 
    
    proceedButton = int(fromArduino[8]);
    yesButton = int(fromArduino[9]);
    noButton = int(fromArduino[10]);
  }
}

Arduino Serial Communication

void loop() {
  // Read the state of the UP_BUTTON (HIGH = not pressed, LOW = pressed)
  int redUPButtonState = digitalRead(redUP_BUTTON); 
  int redDOWNButtonState = digitalRead(redDOWN_BUTTON); 
  int redRIGHTButtonState = digitalRead(redRIGHT_BUTTON); 
  int redLEFTButtonState = digitalRead(redLEFT_BUTTON); 

  int blueUPButtonState = digitalRead(blueUP_BUTTON); 
  int blueDOWNButtonState = digitalRead(blueDOWN_BUTTON); 
  int blueRIGHTButtonState = digitalRead(blueRIGHT_BUTTON); 
  int blueLEFTButtonState = digitalRead(blueLEFT_BUTTON); 

  int proceedButtonState = digitalRead(proceedButton); 
  int yesButtonState = digitalRead(yesButton);
  int noButtonState = digitalRead(noButton); 

  // Print the value (HIGH or LOW) to the Serial Monitor
  Serial.print(redUPButtonState);
  Serial.print(",");
  Serial.print(redDOWNButtonState);
  Serial.print(",");
  Serial.print(redRIGHTButtonState);
  Serial.print(",");
  Serial.print(redLEFTButtonState);

  Serial.print(",");
  Serial.print(blueUPButtonState);
  Serial.print(",");
  Serial.print(blueDOWNButtonState);
  Serial.print(",");
  Serial.print(blueRIGHTButtonState);
  Serial.print(",");
  Serial.print(blueLEFTButtonState);

  Serial.print(",");
  Serial.print(proceedButtonState);
  Serial.print(",");
  Serial.print(yesButtonState);
  Serial.print(",");
  Serial.println(noButtonState);

}

Integration of Serial Communication into P5

Since my p5 were originally mapped to button on the keyboard, all I had to do was replace the variables mapped to the keyboard to the Arduino through the serial communication. Although that sounds easy enough to change, when I was implementing the buttons to change the game states, I ran into a lot of problems with the communication because I had initialized my buttons with INPUT_PULLUP rather than simply INPUT. By initializing the button to INPUT_PULLUP, the button was more sensitive and would remain on an ON-state longer compared to initializing it to INPUT. As I wanted a quicker and smoother response with the buttons, I had to make the program wouldn’t move on to the next sequence when the ON-signal was still being processed between p5 and the Arduino. As such I did a hybrid catch statements to make sure the signal was in the OFF-state before it read another ON-state signal.

Controlling the ON and OFF state

if (!serialCheck) {
    isCheckSerial();
  }

  else {
      
    if (proceedButton == 1) {
      buttonReleased = true; 
    }
    
    if (!musicPlaying){
      playMusic();
      musicPlaying = true;
    }

    if (gameState == 'start' && buttonReleased){
      startScreen();
    }

    else if(gameState == 'friendship warning' && buttonReleased){
      showFriendShipWarning(); 
    }
.
.
.

Integration

function proceedButtonPressed(){
  console.log('gamestate', gameState, proceedButton)

  if (gameState == 'start' && proceedButton == 0){
    gameState = 'friendship warning';
    buttonReleased = false;
    
  }

  else if(gameState == 'friendship warning' && proceedButton == 0){
    gameState = 'tutorial';
    buttonReleased = false;

  }

As for the player controls, it was much easier to replace the mapping from the keyboard to the buttons and the ON-signal wasn’t as noticeable.

player1Moves() {
    fill(this.playerColor);
    if (redUP_arrowPressed == 0) {

// redUP_arrowPressed being an int variable that holds either 1 (OFF) or 0 (ON)

Reflection

Overall, I am really happy with the out of my project and amount of time I put into making sure the visuals and themes were consistent. I am really glad people enjoyed the little game I made, and I hope in the future I’ll get to work on it more and make it less buggy. For future implements, I want to make the orbs on the screen be actual creatures or monsters which the players will have to defeat. Additionally, I hope to highlight the combination which the players will have to press, so I don’t have to directly remind or tell the player to read from left to right. I also wish to build a smooth storyline/ streamline from the starting page to the game. Some people were telling me there were too many instructions to read.

On the physical side, I want to make my game pad less wobbly. Due to the wires under the game pad, when players were hitting the key, the input may not always be registered and made the experience a bit less enjoyable. I would also like to add more game feedback in terms of sound and LED lights that can help signal which orb was just defeat or when they move onto the next round. Since these were small aspects to the game, I didn’t focus on these tasks within the time constraint, but for future implementations, these are topics I can think about adding.

Project

Full Arduino Code

const int redUP_BUTTON = 12;
const int redDOWN_BUTTON = 11;
const int redRIGHT_BUTTON = 10;
const int redLEFT_BUTTON = 9;

const int blueUP_BUTTON = 7;
const int blueDOWN_BUTTON = 6;
const int blueRIGHT_BUTTON = 5;
const int blueLEFT_BUTTON = 4;

const int proceedButton = 13; 
const int yesButton = 3; 
const int noButton = 2; 

void setup() {
  // Start serial communication at 9600 baud
  Serial.begin(9600);

  // Set button pins as input with internal pull-up resistors
  pinMode(redUP_BUTTON, INPUT_PULLUP); 
  pinMode(redDOWN_BUTTON, INPUT_PULLUP);
  pinMode(redRIGHT_BUTTON, INPUT_PULLUP);
  pinMode(redLEFT_BUTTON, INPUT_PULLUP); 

  pinMode(blueUP_BUTTON, INPUT_PULLUP); 
  pinMode(blueDOWN_BUTTON, INPUT_PULLUP);
  pinMode(blueRIGHT_BUTTON, INPUT_PULLUP);
  pinMode(blueLEFT_BUTTON, INPUT_PULLUP); 

  pinMode(proceedButton, INPUT_PULLUP);
  pinMode(yesButton, INPUT_PULLUP);
  pinMode(noButton, INPUT_PULLUP);
}
void loop() {
  // Read the state of the UP_BUTTON (HIGH = not pressed, LOW = pressed)
  int redUPButtonState = digitalRead(redUP_BUTTON); 
  int redDOWNButtonState = digitalRead(redDOWN_BUTTON); 
  int redRIGHTButtonState = digitalRead(redRIGHT_BUTTON); 
  int redLEFTButtonState = digitalRead(redLEFT_BUTTON); 

  int blueUPButtonState = digitalRead(blueUP_BUTTON); 
  int blueDOWNButtonState = digitalRead(blueDOWN_BUTTON); 
  int blueRIGHTButtonState = digitalRead(blueRIGHT_BUTTON); 
  int blueLEFTButtonState = digitalRead(blueLEFT_BUTTON); 

  int proceedButtonState = digitalRead(proceedButton); 
  int yesButtonState = digitalRead(yesButton);
  int noButtonState = digitalRead(noButton); 

  // Print the value (HIGH or LOW) to the Serial Monitor
  Serial.print(redUPButtonState);
  Serial.print(",");
  Serial.print(redDOWNButtonState);
  Serial.print(",");
  Serial.print(redRIGHTButtonState);
  Serial.print(",");
  Serial.print(redLEFTButtonState);

  Serial.print(",");
  Serial.print(blueUPButtonState);
  Serial.print(",");
  Serial.print(blueDOWNButtonState);
  Serial.print(",");
  Serial.print(blueRIGHTButtonState);
  Serial.print(",");
  Serial.print(blueLEFTButtonState);

  Serial.print(",");
  Serial.print(proceedButtonState);
  Serial.print(",");
  Serial.print(yesButtonState);
  Serial.print(",");
  Serial.println(noButtonState);

}

Link to P5: https://editor.p5js.org/yamdn/sketches/1TavUEyVF