The Grove

Overview

The Grove is a cozy resource management and crafting game built in P5.js for the midterm. The player moves between five locations — a Map, River, Forest, Pottery Studio, and Greenhouse — collecting resources, making pottery, and growing plants. For the final, the project is being expanded into a physical table installation. All mouse and keyboard input is replaced with custom props and sensors, so every in-game action has a corresponding physical one.

What’s Already Built

The full P5.js game is complete from the midterm. This includes:

• • All five scenes with backgrounds, navigation, and per-scene background music
  • A full inventory and resource system (clay, soil, water, seeds, pots)
  • The complete pottery workflow: place clay, shape on wheel, fire in furnace, collect
  • A plant growth system with three growth stages, timers, and harvest logic
  • Sprite sheet animations for plants, pots, and the watering can
  • Sound effects for every interaction and background music per scene
  • A custom cursor, backpack inventory overlay, main menu, pause menu, and instructions screen

Physical Components

Joystick — Navigation

The joystick is the main input device and replaces the mouse entirely. It handles all navigation and selection across every scene.

• • Left/Right on the Map cycles through locations (Studio, Greenhouse, Forest, River). Button enters.
  • In scenes, Up moves focus to the upper HUD (Back to Map / Menu). Down moves to the lower HUD (inventory). Left/Right cycles between options in the current zone. Button confirms.
  • In the Studio, Left/Right switches focus between the pottery wheel and furnace.
  • In the Menu, Up/Down cycles options and Button selects.

Ultrasonic Proximity Sensor — Pottery Wheel

An HC-SR04 sensor is mounted face-up at the Studio zone. To shape a pot, the player holds both hands above it with palms facing down, mimicking cupping clay on a wheel. The closer the hands, the faster the pot shapes. Pulling hands away pauses progress. This replaces the original click-and-hold mechanic.

Potentiometer — Furnace

A rotary dial at the Studio zone controls the furnace. Turning it up starts firing. The player watches the pot sprite on screen and turns the dial down at the right moment to retrieve the pot. Leaving it too long burns or destroys it.

Digging Mechanic — Forest

Five fixed plot positions in the forest scene each map to a physical point on the table. On every spawn and respawn (after a 30-second cooldown), each plot is randomly assigned as clay or soil, readable from the sprite on screen.

The current plan is to use a conductive shovel prop with an aluminum foil tip and five corresponding foil contact points on a flat board (four corners and a center), each wired to its own Arduino pin. Touching the shovel to a point completes the circuit and triggers that plot. Contact must be held for 200ms to avoid false triggers.

An alternative being considered is a keypad, where each key corresponds to one of the five plot positions. This would be simpler to wire and more reliable, but less immersive than the shovel prop given the physical metaphor of the rest of the installation.

Water Sensor — Greenhouse

A moisture sensor sits inside a small cup with drainage holes at the bottom. After placing a seed, the player physically pours water into the cup to water the plant in-game. A high analog threshold distinguishes a real pour from residual moisture, and a short debounce timer prevents repeated triggers while the cup drains.

Arduino

The Arduino reads all sensors and sends a single comma-separated line over serial each loop:

joyX, joyY, joyBtn, proximity, potValue, waterValue, dig0, dig1, dig2, dig3, dig4

All game logic stays in P5. The Arduino only handles reading and transmitting. Debounce and sampling logic is handled on the Arduino side: 150ms for the joystick, 200ms hold for shovel contacts, 3-4 second suppression after a water trigger, and a 5-sample rolling average for the proximity sensor.

P5 Changes

The existing code is being modified, not rewritten. The main changes are:

• • input.js: Web Serial API replaces all mouse and keyboard handlers. Incoming serial data is parsed into a global sensorState object read every frame.
  • globals.js: New sensor state variables added alongside existing ones.
  • screens.js: A focus system is layered into each scene. The joystick moves focus between defined zones per scene. The button triggers whatever is currently focused. Pulsating glow and highlight drawn on the focused element.
  • classes.js: ResourcePlot updated to re-randomise its type on each respawn.
  • sketch.js: Minimal changes.

Table Layout

Each prop sits in a zone on the table that corresponds to its location in the game. The joystick is at the center. The proximity sensor and potentiometer are grouped at the Studio zone. The digging board and shovel are at the Forest zone. The watering cup is at the Greenhouse zone. The idea is that as the player moves between locations in the game, they also shift their focus and hands to a different part of the table.

Progress

• • Done: Full P5.js game from midterm
  • Done: Physical interaction design for all five components
  • Done: Joystick navigation model defined for all scenes
  • In progress: Arduino wiring and serial protocol
  • Pending: Web Serial integration in P5
  • Pending: Focus and highlight system in P5
  • Pending: Sprite changes in P5
  • Pending: Sensor threshold calibration
  • Pending: Physical prop construction and table setup

Finalized Concept

My concept will still follow the opera idea I initially had with a few tweaks. The user will be a composer trying to compose the music needed, if the user composes the wrong pitch, an LED would flicker and the buzzer would make a sound.

I originally had the idea of having the composer’s batons but then I realized that would be extremely difficult to map out for me, so I decided to settle with one glove. the user would move the glove and thats how the music is composed

Arduino:

I will be mapping the glove’s x and y positions for octave and note, and p5 will receive this data from an accelerometer and translate it to audible music.

p5

p5 will be used to run the UI and will send information to the arduino on whether the right note is being played which will then cause the LED light to flicker, I was also thinking of adding ballet dancers in there which will fall from their dance if the music is composed incorrectly.

STIPEND USAGE:

I will be using the stipend to buy this MPU6050 accelerator from amazon

https://www.amazon.ae/AOICRIE-GY-291-ADXL345-Acceleration-Transmission/dp/B09S633YTW/ref=sr_1_1?crid=15F0J7UIXQK35&dib=eyJ2IjoiMSJ9.gDQl-_42Yd4esBPagpxPgPQ1K0jRtEl1-iW409PI-IfKNZ2RDA0hRICe1Ca5CQO83N_NFecaKnxd0M0wb-MUX1e1_oV4-HaxCe8KarHRjisgkRf5YXzLzubvBosNxFh3jF1fxCOIuON14P6KvVqARPChru9DCNpV5LlMncvQg8Ro-_7YjQ8SDpSgTryUpC0bzRT_iYJgE7TdhPJPFwpEfobjkqCHFv6UofjnQWaQvz2MidmSXbQrPRS50HEMsv52y0cumuy0z_MNJD41on7dsMC64UjQqNZ6qDc8AXm-5ME.Jzj2a_U_k_AoU6BZxRHVxhwAFpV0Zcu0PJnDUZjpUM0&dib_tag=se&keywords=GY-291+3-Axis+Accelerometer+ADXL345&qid=1777274584&sprefix=gy-2913-axis+accelerometer+adxl345%2Caps%2C261&sr=8-1 

I also consulted ChatGPT to assess the difficulty of building my project, and it told me its possible but will be a bit tricky. I’ve also used a previous student’s model to understand how to use the accelerometer and map it to my p5. (Shota Matsumoto)

Finalized Concept

The project will be an interactive experience where the users physical body state and actions of caring for themselves will be interpreted to the growth of a digital Bonsai tree. The users breathing are monitored to reflect their calmness, interpreted into the growth of the tree. The act of drinking water will serve as hydration of the tree. The physical light conditions of the user’s environment will be reflected on the tree’s lighting conditions. This experience visualizes meditation and self care, turning it into something tangible and fun.

Arduino programs

Analog Input 1: Breathing. A DIY stretch sensor will be worn by the user as a belt around the chest, monitoring rising and falling movements caused by breathing.

Analog Input 2: Light: A photoresistor will be used to monitor the actual light consitions in the users environment.

Digital Input 1: Hydration: Using the hydrtion switch I designed for week 9, physical drinking will connect a circuit and send a digital input.

Serial communication: The arduino will be constantlu reading the information from the sensors and sending the information in a comma seperated string to p5. Thw arduino will be recieving signals from p5 to trigger LED lights when the value of the different conditions fall under a certain threshold, signaling the user to take corresponding action.

P5 program

P5 will be the actual interface of the experience. Indications of the Bonsais conditions and the Bonsai itself will be shown on the screen. It will also have the function of guiding meditation and breathing. It will receive signals from the arduino and use the data to toggle the trees growth speed, health status, and leaf color.

Serial Communication: P5 will be receiving inputs from arduino, as mentioned above. When hydration, light, or focus of the user (monitored through breathing) falls below 10% of the max value, p5 will send different signals to initiate lighting of different colored LEDs.

Current Progress

I have started building the arduino connections that are possible without the components that are currently in delivery. I also started the initial design of the p5 interface. Serial communication and actual functions hasnt been incorporated because the arduino circuit is not complete yet.

Stipend Breakdown

Conductive rubber cord: ~20RMB

24AWG stranded wire: ~15RMB

Metal LED bezels: ~10RMB

Panel mounted photoresistor: ~10RMB

cardboard: ~15RMB

Shipping for some of the components may cost extra

Finalized Concept

For my final project, I’m making a croissant baking game, where the player goes through the actual steps of making a croissant: pouring the ingredients, rolling the dough, folding in the butter layers, shaping, and baking.

Each step is its own phase in the game. For pouring, the player tilts a little physical prop (I’m thinking a cardboard flour bag) over a bowl on screen. For rolling, you move your hand closer or further from a sensor to control how much pressure you’re applying to the dough. For the butter folding, you flip a switch left or right to match a sequence on screen, which is actually the thing that makes croissants flaky in real life, so I thought it was a fun mechanic to recreate. Then there’s a shaping phase with the mouse, and finally a baking phase where you manage the oven temperature with a potentiometer. At the end you get a croissant tier rating.

Arduino

The Arduino handles all physical input for the project. It reads three sensors and sends their data to p5 over serial communication continuously.

The first sensor is a tilt sensor (SW-520D), which is digital and simply reads HIGH when upright and LOW when tilted. This controls the pouring phase: when the player tilts the sensor, the ingredient starts flowing on screen, and when they straighten it, it stops.

The second sensor is an ultrasonic sensor, which measures the distance of the player’s hand in centimeters. This controls the rolling phase. Close distance maps to heavy pressure, far distance maps to light pressure.

The third component is a slide switch, which reads HIGH or LOW depending on its position, and controls the fold direction in the lamination phase.

The fourth component is a potentiometer, which has a numerical value representing the oven temperature.

The Arduino sends all four values as a single comma-separated line formatted as: tilt state, switch state, distance, pot value.

I am thinking of using LED lights when the croissants are baking in the oven. The red LED light turns on while they are baking, and the green LED turns on when they are done.

p5

p5 is the main visual and game logic system. It reads the serial data from the Arduino, parses the three values, and uses them to update the game state in real time.

Each of the five game phases is managed through a state machine, so the sketch knows which phase is active and which interaction to listen for. During the pouring phase, if the tilt value is LOW, an ingredient fills up on screen at a steady rate. During the rolling phase, the distance value gets mapped to a pressure level that flattens the dough visually. During the lamination phase, the switch value is compared against a generated sequence of prompts and scored accordingly. The shaping phase is handled through mouse and keyboard input directly in p5 without needing the serial data. And then, the baking phase needs the pot value reading from Arduino for the oven temperature.

Visually, each phase has its own screen with animations that respond to the physical input. At the end, a results screen shows the breakdown of each phase score and the final croissant tier.

Current Progress

My biggest progress so far has been finalizing the concept because I really struggled with thinking of something personal and fun.

I asked Gemini to create some visuals for my project. I wanted it to have a Parisian/Ratatouille theme.

Using the Stipend

I found the tilt sensor I need to order on Amazon and will be ordering it today. I also need to a buy a better potentiometer to make it easier to control it. I also want to buy some props such a cutting board and a rolling pin to make a more realistic experience.

While the Infrared senors are on their way, I worked a bit on the P5 part of the project.

Sketch

I created a simple welcome screen and and ambience controller with 9 options. Each options is basically a image which acts as a button as well. Clicking on any will cause the following on P5 and Arduino

• A visual on the screen in correspondence to the chosen ambience
• An Audio played on computer speakers
• RGB tape lights around adjusting according the chosen ambience in terms of there color and blink rates

There is a back button on top left of each ambience to exit from it.

I tried to follow OOP. I made two classes one for the button and one for the main app and but them in the separate files. Button handles representing the images and buttons while app handles the transition between the pages and rendering of animation for each ambience. Then they both are combined in sketch.js.

The communication from P5 to Arduino will be about the light control and the display on the LCD. While the the communication from Arduino to back would be in from of distance sensor readings. I am planning on using it as way to communicate by gestures. The idea is the sensor will be at a fixed place so the reading will be constant. If a hand is placed in front of it for two seconds, everything will pause, and if a hand in waved in front of the P5 will trigger a different ambience. Also something I can add is the heat sensor and have the light take some input form it but I don’t think that I can generate enough of temp difference in the room to see it’s effects.

Here is the part of P5 code I am most proud of

chill() {
    background(0);

    let pulse = 150 + sin(this.t) * 60;

    noStroke();
    fill(180, 220, 255, 80);
    ellipse(width / 2, height / 2, pulse);

    for (let i = 0; i < 8; i++) {
      let x = width / 2 + sin(this.t + i) * (100 + i * 10);
      let y = height / 2 + cos(this.t + i) * (80 + i * 8);

      fill(255, 255, 255, 80);
      circle(x, y, 6);
    }
  }

I use of sign function have a motion that is not exactly repetitive was the most hard but fun part. I had the idea of using sin but how it to make it work was proved to be very tricky

I used AI for coming up with Animations for the ambience on screen. I will refine it later to match the lights, but only when I am working with lights. Other than that I used AI to tidy up the code.

Finalized Concept

For my final project, it will basically be my preliminary concept, where I will create a physically interactive rhythm based system called Recovered Signal, where the user helps reconstruct a corrupted digital story through timed physical interaction. The concept is inspired by rhythm games such as Piano Tiles, but instead of focusing only on scoring, the user will actively uncover fragments of a hidden/broken narrative.

The experience is designed as a mystery signal recovery system. On screen, the user sees falling colored rhythm tiles with music playing, while physically pressing with their hand the right buttons/pads in sync with them. For the visuals, it will initially appear distorted, with glitch effects and broken text to show a broken signal along with the tiles. As the player successfully hits notes on time, the system will respond immediately by reducing the distortion, sharpening the text, and revealing fragments of audio or text of a story. It will gradually form a short narrative, like a corrupted voicemail or like a partial conversation/story. But if the user misses notes, the distortion stays or I might make it so that it temporarily increases.

Arduino

So the Arduino will act as the physical input system for the project.

The inputs will be either an arcade style push buttons or pressure pads, and each pad represents a tile in the rhythm game. When the pad is pressed, it sends a high signal to the Arduino. Within the Arduino, it will continuously read the state of each button, detecting which button was pressed. The Arduino will not handle any game logic or timing evaluation, just detect the input and send that data to p5.

For output and communication, it will send a serial message to p5 in the format of a button id (which will show which pad was pressed), and a timestamp using millis to indicate when the press occurred. I will probably use the numbers 1,2,3,4 in correlation to each pad/button. For example 1 if the red button was pressed or 1, 10230, if it was pressed at a specific time.

UPDATE: the Arduino will send the button information, and then p5 checks if it was the right timing, and p5 sends back whether it was correct or wrong (0 or 1), and correct will turn on the green LED light, and wrong will turn on the red LED light. I am also thinking about making the LEDs flicker if they’re still on the weak signal stage, or have a steady glow when their half way there, and the brightest when it’s strong.

(If time allows, I will also set up an LED that lights up when a button is pressed for feedback/confirmation.)

P5

P5 will function as the main visual and audio system of the project.

It will generate a rhythm game with 4 vertical lanes that displays falling tiles synchronized to a beat. It will detect input from Arduino (which button was pressed) and it will match Arduino input with falling tile positions, evaluate timing accuracy (perfect/miss/late/early), and update the system state based on performance. Everything will be in real time as the button is pressed.

For the audio, it will be handled entirely in p5, so it’s clear. Each correct input will trigger musical elements, and the layered audio starts muffled or distorted, becomes clearer as player progresses, and eventually reveals a full short narrative (voicemail or conversation)

What it will communicate with the Arduino is that it will receive button press data through serial input and use that data to trigger game events and check accuracy, and send back to Arduino if the timing was right and the right button was pressed, in which it will light up a red or green LED light.

Current Progress

I have currently made all of the backgrounds and set up the skeleton code in p5, with the game states and made a signal level bar (as the player taps, a signal bar fills up on the screen. When the signal reaches a certain level, it triggers a story message to appear, like a transmission being decoded).

Here is the link to the sketch and some of the backgrounds:

https://editor.p5js.org/farahshaer/sketches/LJyoiS0O_

I will next make the tiles, stories (using oop), and sounds, glitch effects, and make my Arduino code and circuit…I still have a long way to go.

Stipend Breakdown

I plan to use the $50 stipend to build the pressure pads or buy arcade buttons (but I am not too sure if it will come in time).

Cardboard/foam board for mounting structure
Copper tape and sheets
Colored paper and materials for labeling the pads visually to match the p5 lanes/tiles.
Basic craft supplies (tape, glue, cutting materials)

Concept:
For my final project, I am creating an interactive coffee simulation in which users engage between a digital interface and physical inputs using Arduino. The project allows users to navigate a coffee-making simulation through a p5 sketch displayed on screen, while using tangible tools connected to the Arduino to perform actions.
The goal is to move beyond simple selection-based interaction by allowing users to actively participate in the process of making a cup of coffee. As users perform each step, they receive real-time visual feedback on the screen, along with physical and auditory responses that reflect their actions.
By combining digital visuals with hands-on interaction and realistic sound elements, the project aims to create an engaging and immersive experience that simulates the process of preparing coffee in an intuitive and enjoyable way.

Design and Description of Arduino:
The Arduino program will use inputs and outputs to control the physical interaction. The main input will be the ultrasonic sensor, which reads distance and is used to detect the user’s movements. This data will be interpreted differently depending on the current step, allowing the same sensor to represent actions such as grinding, pressing, steaming, and pouring.
I will also use a push button for confirmation actions, such as starting the system, selecting steps, and moving to the next stage. A joystick module will be used to navigate between tools, where the Arduino reads its values to determine direction. For outputs, I will use three LEDs (red, yellow, and green) to provide feedback. Red indicates a step has not started, yellow shows the process is in progress, and green indicates completion.
The Arduino will send sensor values, joystick input, and button states to p5.js through serial communication. It will also receive information from p5.js about the current step, allowing it to correctly interpret inputs and control the LEDs.

Design and Description of P5:
The p5.js program will manage all visuals and the overall flow of the interaction. It will control the different screens, including the start screen, instructions page, and the main coffee-making interface, as well as display the navigation system using the joystick.
During each step, p5.js will use sprite sheets to display animations that represent the action being performed. These visuals will update in real time based on data received from the Arduino, especially the ultrasonic sensor, creating a direct connection between the user’s physical actions and the digital response.
The p5 program will receive sensor data, joystick movement, and button inputs from Arduino to update visuals and control progression. It will also send the current step information back to Arduino so it can adjust how inputs are interpreted and control the LED feedback.

Materials:
For my project, I will be using components from my Arduino kit, including the Arduino board, breadboard, and jumper wires to build the system. I will use an ultrasonic sensor as the main input to detect the user’s movements and gestures during the coffee-making process. A push button will be used for confirmation actions, such as starting the steps, and a joystick will allow the user to navigate between the different tools or stages. I will also include three LEDs (red, yellow, and green) to provide clear feedback, indicating when the system has not started, is in progress, and when a step is completed. I will be 3D printing or creating physical coffee-making tools such as a grinder, tamper, and cup to make the interaction more realistic. Using the stipend, I will purchase the joystick and build a simple coffee machine-style enclosure using materials like cardboard or foam board to organize the components and hide the wiring. This will improve the overall presentation and create a more immersive and polished experience.

Process:
So far, I have finalized my concept and organized all my ideas, along with doing some research to make sure everything can work the way I imagine it, especially using the ultrasonic sensor with movement and sprite sheet animations.
I have also started working on the visual side by creating images for the layout I plan to follow in p5.js. This includes the start page, the instruction guide, and the main setup with the first step of the process. In addition, I have begun creating the first sprite sheet for the initial movement so that the animation can respond to the user’s interaction.
I have also started preparing the structure of my p5 sketch to integrate these elements. My next step is to import these visuals into the p5 sketch and start coding them to work properly, making sure the interaction functions as expected. After that, I will continue adding the remaining steps using the same structure and process.

Images:

My concept:

I want to do a visualization of Conway’s game of life in real life, but I didn’t want it to be something boring or just simple, so after a long while of searching for inspiration, I landed on infinity mirrors.

The way these work is that there is LED lights sandwiched between 2 mirrors, the back mirror is fully reflective, and the front mirror is a two-way mirror, where its partially reflective and partially transparent. The way it creates this illusion with a single line of LED is:

– LED’s inside emit light.
– Some light goes straight out through the front
– Some light reflects off the back mirror
– The reflected light hits the front mirror again, part of it escapes to your eyes and part of it reflects back inside
– This keeps bouncing back and forth

Each bounce loses a bit of intensity, so you see a chain of dimmer and dimmer reflections that look like they’re going deep into the mirror.

So the main idea is the simulation of the game of life, however the user can decide which cells to start with, or make it random, or add new cells while the simulation is running.

Arduino:

The Arduino is going to be controlling which LED lights turn on, I bought WS2812B LED strip lights which allows me to control each led in a strip individually. I might add a potentiometer and buttons for reset or changing the light brightness or speed of simulation, however I will decide on this depending on how building the project goes.

P5:

This will handle the simulation aspect of it, it will calculate what cells live or die in the next generation and sends these updates to the Arduino so it turns on or off the respective LED’s, if I can’t manage to have an external control panel, then my backup plan will just be having the panel be done in p5js with a UI a user can control.

Progress:

I used the stipend to buy the resources needed for this project which are:

– Black foam boards for the base: https://www.amazon.ae/dp/B0D3Y656M2?ref=ppx_yo2ov_dt_b_fed_asin_title

– 5 Meter WS2812B LED strips: https://www.amazon.ae/dp/B0D1Y995V6?ref=ppx_yo2ov_dt_b_fed_asin_title

– Reflective mirror stickers: https://www.amazon.ae/dp/B0FY2GBZ2S?ref=ppx_yo2ov_dt_b_fed_asin_title

– Clear acrylic sheet: https://www.amazon.ae/dp/B0CK4XFZ8D?ref=ppx_yo2ov_dt_b_fed_asin_title

– One way privacy film: https://www.amazon.ae/dp/B0GCYXYK5L?ref=ppx_yo2ov_dt_b_fed_asin_title

My Concept:

For my final project, I’m making a physically interactive maze game where the player controls minnie mouse using two potentiometers connected to Arduino, one for moving left and right and the other for moving up and down. The goal is to guide minnie through the maze to reach mickey, but the player also has to collect little items scattered around the maze before finishing. I might add like cheese or starts. Arduino reads the knob values and sends them to p5.js, which moves minnie on the screen, checks if she hits a wall, picks up a collectible, or reaches the end. When something happens, p5 sends simple messages back to Arduino so the LED and buzzer can react, for example like blinking when she hits a wall, making a small sound when she collects something, and playing a celebration melody when she reaches mickey at the end. What will be drawn by p5 is the maze, and screens. However, I might use the internet to help me find collectibles and photos of minnie mouse and mickey mouse to use for my game, which will make the experience feel like a real disney game.

P5 Explanation:

On the p5.js side, the sketch will draw the maze, will show minnie and mickey, and handles all the movement and collision detection. It receives the potentiometer values from Arduino and maps them to minnie’s position in the game, it will also check if she hits a wall, collects an item, or reaches the end. When these events happen, p5 sends “hit,” “collect,” or “win” back to Arduino so the LED and buzzer can react to these actions. The sketch also includes a start screen, instructions, and a counter showing how many collectibles the player has picked up.

Arduino Explanation:

For this project, Arduino’s job is to read the two potentiometers using analogRead() and send those values to p5.js through serial communication so minnie’s movement updates to how the player is controlling it. Arduino also listens for messages coming back from p5.js. When p5 detects that minnie hits a wall, collects an item, or reaches mickey, it sends “hit,” “collect,” or “win,” and Arduino responds by blinking the LED or playing sounds on the buzzer. Everything is based on what we practiced in class such as, reading analog inputs, controlling outputs with digitalWrite and PWM, and sending simple serial messages back and forth.

Progress update:

Right now, I’m in the preparation stage of the project. I’ve finalized the concept and planned out how the Arduino and p5.js parts will work together. I ordered the extra materials I need, including the second potentiometer, and while I wait for them to arrive, I’m going to work on the p5.js side. My focus at the moment is sketching the maze layout, setting up the basic structure of the game, and finding the character images, and collectibles. I will also plan out my code for Arduino. Once the parts arrive, I’ll build the physical setup for the potentiometers and begin connecting everything to Arduino so I can combine the full system.

Finalized Concept

My final project is called Spacecraft Mission Control System. It is a fast paced spacecraft control game inspired by Bop It, where the player has to react quickly to different commands. The game will use different physical inputs like colored buttons, a buzzer, a keypad, a light sensor, and hopefully a distance sensor.

The player will receive tasks from the p5.js screen, such as pressing a specific button, entering a code on the keypad, covering the light sensor, or moving their hand near the distance sensor. I want the game to feel like the player is controlling a spacecraft under pressure, where every action matters. If the player gets the task correct, a green LED will light up. If they get it wrong, a red LED will light up. The buzzer will also give sound feedback so the game feels more interactive.

Arduino Program Design

The Arduino will control the physical part of the game. It will read inputs from the colored buttons, keypad, light sensor, and distance sensor. The colored buttons will act as simple commands, like pressing the red, blue, or yellow button when the screen tells the player to. The keypad will be used for code based tasks, where the player has to type a number code to complete the mission. I ordered a keypad from Amazon, and I will try to implement it. If l struggle with the keypad.

For the light sensor I plan on creating a cover, so the player can physically cover and uncover it as part of the game. The distance sensor can be used for tasks like moving the hand closer or farther away. The Arduino will also control the buzzer, red LED, and green LED. It will send the player’s actions to p5.js and receive signals back about whether the answer was correct or incorrect.

P5.js Program Design

The p5.js program will be the digital screen for the game. It will show the title screen, game instructions, current mission, timer, score, and warning messages. The visual style will be a space mission control screen, with a moving star background and a full screen option. p5.js will send the current task to Arduino, such as “press red button, “cover sensor,” or “enter code.” It will also receive information from Arduino about what the player did. For example, if the player presses the red button, Arduino sends that input to p5.js. Then p5.js checks if it matches the current task. If it is correct, the score increases and p5.js sends a correct signal to Arduino so the green LED and success sound can turn on. If it is wrong, p5.js sends an incorrect signal so the red LED and error sound can turn on.

Progress So Far

So far, I finalized the main concept and decided that the project will be a spacecraft themed reaction game. I also planned the main inputs and outputs I want to use. I ordered the keypad and other materials from Amazon, and I will test whether | can connect the keypad properly. My next step is to test each input separately first, then combine them into one working game system.

Breakdown of things I ordered

For my project, I bought a push button kit, keypad module, and an active buzzer to create the main game interactions and feedback. I also purchased a pre wired multicolor LED for clear visual responses, along with jumper wires and a sensor starter kit for connections and extra components. Additionally, I got a display module to show simple messages or tasks. For the physical design, I will use cardboard and colored paper to build the spacecraft control panel.

Generated image of how it would look physically:

Amazon links

https://amzn.eu/d/0hpevbaT 22.6USD

https://amzn.eu/d/00tJPOo5 4.9USD

https://amzn.eu/d/06RK8ekP 4.08USD

https://amzn.eu/d/03ujSIRs 12.52USD

https://amzn.eu/d/05gqbmon 6.26USD

Total 50.37USD

P5js:

I began working on the home screen and instructions page and added the full screen feature and font.

Arduino:

I began setting up the basic inputs and outputs for the Arduino focusing on reading button inputs and triggering LED and buzzer feedback for the correct or incorrect responses.

https://github.com/MouzaAlMheiri/Intro-to-IM/blob/main/draft.ino

Sources so far:

Fullscreen:

https://p5js.org/reference/p5/fullscreen/#:~:text=fullscreen,such%20as%20a%20mouse%20press.

Stars background:

https://editor.p5js.org/pavly/sketches/hzCu0enCy

Game concept:

https://editor.p5js.org/skgmmt/sketches/Sk5VaX2yN

For Keypad:

Using a Keypad with Arduino