I think it was pretty easy for my user to figure out how to use/play the game but I believe that’s mainly because I gave them a glove with foils attached to the fingers, so it was quick to understand what to do. By that, I mean that they understood that touching fingers in the glove would trigger the ballerina to dance.

But they did get confused about the rules of the game. And the idea of losing points and the ballerina falling were not fully working yet, so they were confused on how to win or lose. I didn’t create an instructions page yet so that is part of the reason.

One particular feedback I got was that it was difficult for them to use the glove. Some presses would not work, or they would have to press harder. For that, I will need to find a way to stick my aluminum foils to the glove in a neater more seamless way.

 

User Testing

So I let my siblings test it out and they were able to figure out the basic idea pretty quickly, that they needed to press the colored arcade buttons when the falling tiles reached the hit line on screen. The colors helped a lot because they made the connection between the physical buttons and the tiles more obvious. But they were confused at first about when exactly they were supposed to press, since there were no instructions during gameplay, so they needed a few seconds of trial and error to understand the timing.

I think the mapping between the controls and the experience worked well because each button matched a lane and color on screen, which made the interaction feel natural. Once players understood that each button controlled a specific lane, they were able to continue without much help. The physical arcade buttons also made the experience feel more immersive compared to using a keyboard.

I think the strongest part of the experience was the atmosphere. The background visuals, projector reveal screen, sound design, and glitch effects made the game feel tense and it matched the theme and concept well. They especially liked the reveal screen because it felt like a reward after completing each round and made them curious about the story.

The main area that could be improved is clarity at the beginning. One of them were unsure whether to press buttons immediately or wait for the tiles to reach the bottom hit line. I found myself explaining the timing and the goal of “rebuilding the signal” so they could progress. To make this clearer, I could improve the instruction screen by adding a short visual example or animation showing exactly when to press the buttons (like a quick tutorial).

(Also, I noticed I wrote 4 rounds but its kind of 5 because you have to play again to go to the final message…I am thinking about just reducing it maybe its too long of a game??)

Some users were able to understand how the system worked overall, but there was confusion around the IR sensor. A few people tried to press it like a physical button, and they were unsure how hard to press, how close their hand should be, or how many fingers to use. I think this confusion happened because they interpreted it as a button rather than a proximity sensor, so the mapping between the control and the action wasn’t immediately clear.

Despite that, most users understood the general flow of the experience and were able to move between interacting with the machine and observing the outcome. One part that worked especially well was the portion size. The amount of pet food dispensed felt accurate, and the servo motor’s opening and closing motion was smooth and consistent. However, there was a small issue where some pieces of pet food would fly out from the sides, which could be improved with better containment.

The main part I had to explain was how to use the IR sensor properly. For one user, I also had to point out that they needed to place the bowl under the dispenser, even though there was a visual instruction. This might have been specific to that user, but it suggests that the instructions could be made more noticeable or clearer for first-time users.

Brother 1 User Testing:

Explained info: Rules of con way’s game of life

Project Walk Through:

Bonus – 9 year old brother test subject:

Conclusion:

From my first brother, he didn’t really know how to navigate the website, and I think this is due to the lack of knowledge of how simulations work in general, and I only briefly explained what Conway’s game of life is to him. To be more nitpicking, the paused and running text on the top left of the grid doesn’t seem to be very noticeable by both siblings and I had to tell them both the simulation was paused when they were just clicking the grid hoping to start something. The rest of the buttons aren’t really used either unless I explicitly tell them about it, maybe one part is just them not knowing what to do at all for the video. But what I think the core issue is the fact most people aren’t familiar with how simulations work.

The experience itself works pretty well, it does its purpose in showing something cool entirely made by the user, I however wish I was able to add more external controls, but my potentiometer pins broke while I was trying to add that and the joystick module just would not fit with my design, so I had to resort to controlling the simulation through p5js (I did add some keyboard shortcuts).

For how to solve the issue of the lack of knowledge, I think If I had time I would add some tutorial overlay when the website is first loaded in, so like when you first launch you get a short brief explanation of what every button does in a speech bubble style, have the user do what the tutorial is asking them to do before moving on to get them familiar with the controls. I should also make the pause/running indication more visible, maybe by changing the grid outlines or the background from red -> green or something along the lines, something more “obvious.” A big fat “PAUSED” in the middle of the grid could work too!

The buttons aren’t really intuitive for anyone who isn’t aware of simulations in general, fast, slow, random, clear, presets all were buttons ignored unless I explicitly mentioned, so this should also be fixed with my how to use overlay tutorial type of thing. However the process of clicking on the grid and seeing stuff react when the simulation started seemed to make them understand the mapping pretty instantly! So I think I would call this project a success, just simply lacking some intuitive upgrades to the UI.

PRESSYR – User Testing Reflection

VDO:

https://youtu.be/GeI6QWZyzsk?si=tPqhPQI3fib59SSb

For my final project, I created an interactive piano-inspired instrument called PRESSYR using Arduino, FSR pressure sensors, a CD74HC4067 multiplexer, an I2C LCD display, an arcade power button, and a p5.js visual and sound interface. The project allows users to press pressure-sensitive pads to play musical notes while also generating sound and visuals on a computer screen.

User Testing Process:

I conducted user testing with four participants. During the testing session, I intentionally did not give them detailed instructions at first because I wanted to observe how naturally they could understand and interact with the project on their own.

One thing I noticed immediately was that almost every participant was initially confused about what the device actually was. However, most of them could somewhat guess that it was related to a piano or music instrument because the cardboard enclosure visually resembled piano keys. In addition, my house already contains a real piano, and everyone who participated already knew that I play the piano, so this context may also have influenced their assumptions.

The main confusion at the beginning was understanding what to press first and how to start interacting with the system. Fortunately, the LCD screen helped guide users by displaying “Press RED Button” when the system was off. This instruction helped participants understand that the red arcade button was the actual starting point for interaction.

After turning the system on, some participants were still slightly confused about what to do next. I think this happened for two reasons. First, the instructions on the LCD were written in English, and second, some participants did not immediately understand what the word “pads” referred to. Even so, after a short moment of exploration, every participant eventually understood that the pressure-sensitive pads were musical inputs that played notes like Do, Re, Mi, Fa, Sol, La, Ti, Do.

Observations During Interaction:

One interesting observation from the testing session was that participants rarely looked at the computer screen visualizations while playing. Instead, they focused almost entirely on the physical pads and their hand movements.

I believe this happened mainly because users had to concentrate on pressing the pads accurately and listening to the melody from each key being produced. Since the sound feedback already confirmed that the interaction was working, participants did not feel a strong need to constantly look at the screen. The laptop screen was also positioned slightly farther away from the pads, which may have reduced visual attention even more.

Although it is possible that the visual effects on the screen were not large or attention-grabbing enough, I personally think the stronger reason was that users naturally prioritized the physical interaction and audio feedback over the visual component.

What Worked Well:

Several aspects of the project worked successfully during testing:

– The arcade power button clearly communicated the ON/OFF state of the system.
– The illuminated LED inside the arcade button provided immediate visual feedback.
– The LCD instructions helped guide users through the interaction process.
– The pressure-sensitive pads successfully triggered sound and visuals consistently.
– Users eventually understood the interaction flow without requiring direct explanation.
– The piano-inspired physical design helped communicate the musical concept of the project.

Areas for Improvement:

Some participants commented that the pressure pads felt larger than actual piano keys. This feedback was valuable because it highlighted how the scale of the interaction affected the realism of the experience.

In future versions of the project, I would like to:
– Reduce the size of the pads to better resemble real piano keys.
– Improve the visual prominence of the p5.js animations so users notice them more easily.
– Add clearer visual labels or symbols for first-time users.
– Experiment with different layouts and materials for a more polished interaction experience.
– Explore using velocity-sensitive audio or multiple sound layers for more expressive musical interaction.

Reflection:

This user testing session helped me realize how important clarity and interaction flow are in physical computing projects. Even though the project technically functioned correctly, small design decisions such as wording, placement of instructions, and physical scale strongly affected how users understood the experience.

Overall, the testing process was extremely useful because it revealed areas where the interaction was intuitive and areas where users still needed additional guidance. It also helped me think more carefully about how people naturally approach unfamiliar interactive objects for the first time.

Will be using this blog post to gradually write the blog post instead of all at once:

PROGRESS:

1. Arduino wiring: DONE
2. Arduino code: DONE
3. UI
4. p5 code communication

30/04/26

I tried connecting my accelerometer to my breadboard without soldering. I placed my headers into the breadboard, placed the accelerometer on top, and connected the female end of a jumper wire on top. I downloaded the relevant libraries and wrote the code from this page. Unfortunately, the Arduino showed no connection whatsoever (the code I had was designed to write in the serial “No ADXL345 sensor detected.” And indeed that was what my serial displayed for me. Now, I will be moving on to a different connectivity, building on the unusual switch and musical instrument assignments we made. This idea will still follow the same concept of composing music in an opera; if you compose incorrectly, a buzzer will sound, and the ballet dancers will fall. Instead of using an accelerometer to measure the x-y positions of the user’s hand, I will attach foils and wires to each finger on a glove. The connection between all the wire combinations will control a relevant aspect of the music, such as pitch, speed, note, and beats. This wiring idea was also heavily inspired by the emerging usage of TouchDesigner.

01/05/26

I decided to go on with my glove idea. I originally wanted to involve all 5 fingers, but then I realized that would complicate things. I stuck to using three. One would be in charge of playing the song, the second is in charge of speeding it up, the third is in charge of slowing it down. I tested out my wiring by also connecting an LED to the breadboard to ensure correct wiring, and it worked. I didn’t connect my buzzer today. And honestly, I don’t think I will until the very end because I got a new buzzer and the legs look extremely delicate and if I’m being honest I’m not the most patient or gentle with any electronic device so I don’t want to risk breaking yet another buzzer from the get go.

 

STIPEND USAGE:

1. Accelerometer: 40 AED: https://www.amazon.ae/AOICRIE-GY-291-ADXL345-Acceleration-Transmission/dp/B09S633YTW/ref=sr_1_1?crid=15F0J7UIXQK35&dib=eyJ2IjoiMSJ9.gDQl-_42Yd4esBPagpxPgPQ1K0jRtEl1-iW409PI-IfKNZ2RDA0hRICe1Ca5CQO83N_NFecaKnxd0M0wb-MUX1e1_oV4-HaxCe8KarHRjisgkRf5YXzLzubvBosNxFh3jF1fxCOIuON14P6KvVqARPChru9DCNpV5LlMncvQg8Ro-_7YjQ8SDpSgTryUpC0bzRT_iYJgE7TdhPJPFwpEfobjkqCHFv6UofjnQWaQvz2MidmSXbQrPRS50HEMsv52y0cumuy0z_MNJD41on7dsMC64UjQqNZ6qDc8AXm-5ME.Jzj2a_U_k_AoU6BZxRHVxhwAFpV0Zcu0PJnDUZjpUM0&dib_tag=se&keywords=GY-291+3-Axis+Accelerometer+ADXL345&qid=1777274584&sprefix=gy-2913-axis+accelerometer+adxl345%2Caps%2C261&sr=8-1
2. Jumper wires: 18 AED: https://amzn.eu/d/03P5tjOx

AI USAGE:

I used AI for this project thus far to discuss my ideas and discuss any problems which I may face while building the project.

The Grove

Overview

The Grove is a cozy resource management and crafting game built in P5.js for the midterm. The player moves between five locations — a Map, River, Forest, Pottery Studio, and Greenhouse — collecting resources, making pottery, and growing plants. For the final, the project is being expanded into a physical table installation. All mouse and keyboard input is replaced with custom props and sensors, so every in-game action has a corresponding physical one.

What’s Already Built

The full P5.js game is complete from the midterm. This includes:

• • All five scenes with backgrounds, navigation, and per-scene background music
  • A full inventory and resource system (clay, soil, water, seeds, pots)
  • The complete pottery workflow: place clay, shape on wheel, fire in furnace, collect
  • A plant growth system with three growth stages, timers, and harvest logic
  • Sprite sheet animations for plants, pots, and the watering can
  • Sound effects for every interaction and background music per scene
  • A custom cursor, backpack inventory overlay, main menu, pause menu, and instructions screen

Physical Components

Joystick — Navigation

The joystick is the main input device and replaces the mouse entirely. It handles all navigation and selection across every scene.

• • Left/Right on the Map cycles through locations (Studio, Greenhouse, Forest, River). Button enters.
  • In scenes, Up moves focus to the upper HUD (Back to Map / Menu). Down moves to the lower HUD (inventory). Left/Right cycles between options in the current zone. Button confirms.
  • In the Studio, Left/Right switches focus between the pottery wheel and furnace.
  • In the Menu, Up/Down cycles options and Button selects.

Ultrasonic Proximity Sensor — Pottery Wheel

An HC-SR04 sensor is mounted face-up at the Studio zone. To shape a pot, the player holds both hands above it with palms facing down, mimicking cupping clay on a wheel. The closer the hands, the faster the pot shapes. Pulling hands away pauses progress. This replaces the original click-and-hold mechanic.

Potentiometer — Furnace

A rotary dial at the Studio zone controls the furnace. Turning it up starts firing. The player watches the pot sprite on screen and turns the dial down at the right moment to retrieve the pot. Leaving it too long burns or destroys it.

Digging Mechanic — Forest

Five fixed plot positions in the forest scene each map to a physical point on the table. On every spawn and respawn (after a 30-second cooldown), each plot is randomly assigned as clay or soil, readable from the sprite on screen.

The current plan is to use a conductive shovel prop with an aluminum foil tip and five corresponding foil contact points on a flat board (four corners and a center), each wired to its own Arduino pin. Touching the shovel to a point completes the circuit and triggers that plot. Contact must be held for 200ms to avoid false triggers.

An alternative being considered is a keypad, where each key corresponds to one of the five plot positions. This would be simpler to wire and more reliable, but less immersive than the shovel prop given the physical metaphor of the rest of the installation.

Water Sensor — Greenhouse

A moisture sensor sits inside a small cup with drainage holes at the bottom. After placing a seed, the player physically pours water into the cup to water the plant in-game. A high analog threshold distinguishes a real pour from residual moisture, and a short debounce timer prevents repeated triggers while the cup drains.

Arduino

The Arduino reads all sensors and sends a single comma-separated line over serial each loop:

joyX, joyY, joyBtn, proximity, potValue, waterValue, dig0, dig1, dig2, dig3, dig4

All game logic stays in P5. The Arduino only handles reading and transmitting. Debounce and sampling logic is handled on the Arduino side: 150ms for the joystick, 200ms hold for shovel contacts, 3-4 second suppression after a water trigger, and a 5-sample rolling average for the proximity sensor.

P5 Changes

The existing code is being modified, not rewritten. The main changes are:

• • input.js: Web Serial API replaces all mouse and keyboard handlers. Incoming serial data is parsed into a global sensorState object read every frame.
  • globals.js: New sensor state variables added alongside existing ones.
  • screens.js: A focus system is layered into each scene. The joystick moves focus between defined zones per scene. The button triggers whatever is currently focused. Pulsating glow and highlight drawn on the focused element.
  • classes.js: ResourcePlot updated to re-randomise its type on each respawn.
  • sketch.js: Minimal changes.

Table Layout

Each prop sits in a zone on the table that corresponds to its location in the game. The joystick is at the center. The proximity sensor and potentiometer are grouped at the Studio zone. The digging board and shovel are at the Forest zone. The watering cup is at the Greenhouse zone. The idea is that as the player moves between locations in the game, they also shift their focus and hands to a different part of the table.

Progress

• • Done: Full P5.js game from midterm
  • Done: Physical interaction design for all five components
  • Done: Joystick navigation model defined for all scenes
  • In progress: Arduino wiring and serial protocol
  • Pending: Web Serial integration in P5
  • Pending: Focus and highlight system in P5
  • Pending: Sprite changes in P5
  • Pending: Sensor threshold calibration
  • Pending: Physical prop construction and table setup

Finalized Concept

My concept will still follow the opera idea I initially had with a few tweaks. The user will be a composer trying to compose the music needed, if the user composes the wrong pitch, an LED would flicker and the buzzer would make a sound.

I originally had the idea of having the composer’s batons but then I realized that would be extremely difficult to map out for me, so I decided to settle with one glove. the user would move the glove and thats how the music is composed

Arduino:

I will be mapping the glove’s x and y positions for octave and note, and p5 will receive this data from an accelerometer and translate it to audible music.

p5

p5 will be used to run the UI and will send information to the arduino on whether the right note is being played which will then cause the LED light to flicker, I was also thinking of adding ballet dancers in there which will fall from their dance if the music is composed incorrectly.

STIPEND USAGE:

I will be using the stipend to buy this MPU6050 accelerator from amazon

https://www.amazon.ae/AOICRIE-GY-291-ADXL345-Acceleration-Transmission/dp/B09S633YTW/ref=sr_1_1?crid=15F0J7UIXQK35&dib=eyJ2IjoiMSJ9.gDQl-_42Yd4esBPagpxPgPQ1K0jRtEl1-iW409PI-IfKNZ2RDA0hRICe1Ca5CQO83N_NFecaKnxd0M0wb-MUX1e1_oV4-HaxCe8KarHRjisgkRf5YXzLzubvBosNxFh3jF1fxCOIuON14P6KvVqARPChru9DCNpV5LlMncvQg8Ro-_7YjQ8SDpSgTryUpC0bzRT_iYJgE7TdhPJPFwpEfobjkqCHFv6UofjnQWaQvz2MidmSXbQrPRS50HEMsv52y0cumuy0z_MNJD41on7dsMC64UjQqNZ6qDc8AXm-5ME.Jzj2a_U_k_AoU6BZxRHVxhwAFpV0Zcu0PJnDUZjpUM0&dib_tag=se&keywords=GY-291+3-Axis+Accelerometer+ADXL345&qid=1777274584&sprefix=gy-2913-axis+accelerometer+adxl345%2Caps%2C261&sr=8-1 

I also consulted ChatGPT to assess the difficulty of building my project, and it told me its possible but will be a bit tricky. I’ve also used a previous student’s model to understand how to use the accelerometer and map it to my p5. (Shota Matsumoto)

Finalized Concept

The project will be an interactive experience where the users physical body state and actions of caring for themselves will be interpreted to the growth of a digital Bonsai tree. The users breathing are monitored to reflect their calmness, interpreted into the growth of the tree. The act of drinking water will serve as hydration of the tree. The physical light conditions of the user’s environment will be reflected on the tree’s lighting conditions. This experience visualizes meditation and self care, turning it into something tangible and fun.

Arduino programs

Analog Input 1: Breathing. A DIY stretch sensor will be worn by the user as a belt around the chest, monitoring rising and falling movements caused by breathing.

Analog Input 2: Light: A photoresistor will be used to monitor the actual light consitions in the users environment.

Digital Input 1: Hydration: Using the hydrtion switch I designed for week 9, physical drinking will connect a circuit and send a digital input.

Serial communication: The arduino will be constantlu reading the information from the sensors and sending the information in a comma seperated string to p5. Thw arduino will be recieving signals from p5 to trigger LED lights when the value of the different conditions fall under a certain threshold, signaling the user to take corresponding action.

P5 program

P5 will be the actual interface of the experience. Indications of the Bonsais conditions and the Bonsai itself will be shown on the screen. It will also have the function of guiding meditation and breathing. It will receive signals from the arduino and use the data to toggle the trees growth speed, health status, and leaf color.

Serial Communication: P5 will be receiving inputs from arduino, as mentioned above. When hydration, light, or focus of the user (monitored through breathing) falls below 10% of the max value, p5 will send different signals to initiate lighting of different colored LEDs.

Current Progress

I have started building the arduino connections that are possible without the components that are currently in delivery. I also started the initial design of the p5 interface. Serial communication and actual functions hasnt been incorporated because the arduino circuit is not complete yet.

Stipend Breakdown

Conductive rubber cord: ~20RMB

24AWG stranded wire: ~15RMB

Metal LED bezels: ~10RMB

Panel mounted photoresistor: ~10RMB

cardboard: ~15RMB

Shipping for some of the components may cost extra

Finalized Concept

For my final project, I’m making a croissant baking game, where the player goes through the actual steps of making a croissant: pouring the ingredients, rolling the dough, folding in the butter layers, shaping, and baking.

Each step is its own phase in the game. For pouring, the player tilts a little physical prop (I’m thinking a cardboard flour bag) over a bowl on screen. For rolling, you move your hand closer or further from a sensor to control how much pressure you’re applying to the dough. For the butter folding, you flip a switch left or right to match a sequence on screen, which is actually the thing that makes croissants flaky in real life, so I thought it was a fun mechanic to recreate. Then there’s a shaping phase with the mouse, and finally a baking phase where you manage the oven temperature with a potentiometer. At the end you get a croissant tier rating.

Arduino

The Arduino handles all physical input for the project. It reads three sensors and sends their data to p5 over serial communication continuously.

The first sensor is a tilt sensor (SW-520D), which is digital and simply reads HIGH when upright and LOW when tilted. This controls the pouring phase: when the player tilts the sensor, the ingredient starts flowing on screen, and when they straighten it, it stops.

The second sensor is an ultrasonic sensor, which measures the distance of the player’s hand in centimeters. This controls the rolling phase. Close distance maps to heavy pressure, far distance maps to light pressure.

The third component is a slide switch, which reads HIGH or LOW depending on its position, and controls the fold direction in the lamination phase.

The fourth component is a potentiometer, which has a numerical value representing the oven temperature.

The Arduino sends all four values as a single comma-separated line formatted as: tilt state, switch state, distance, pot value.

I am thinking of using LED lights when the croissants are baking in the oven. The red LED light turns on while they are baking, and the green LED turns on when they are done.

p5

p5 is the main visual and game logic system. It reads the serial data from the Arduino, parses the three values, and uses them to update the game state in real time.

Each of the five game phases is managed through a state machine, so the sketch knows which phase is active and which interaction to listen for. During the pouring phase, if the tilt value is LOW, an ingredient fills up on screen at a steady rate. During the rolling phase, the distance value gets mapped to a pressure level that flattens the dough visually. During the lamination phase, the switch value is compared against a generated sequence of prompts and scored accordingly. The shaping phase is handled through mouse and keyboard input directly in p5 without needing the serial data. And then, the baking phase needs the pot value reading from Arduino for the oven temperature.

Visually, each phase has its own screen with animations that respond to the physical input. At the end, a results screen shows the breakdown of each phase score and the final croissant tier.

Current Progress

My biggest progress so far has been finalizing the concept because I really struggled with thinking of something personal and fun.

I asked Gemini to create some visuals for my project. I wanted it to have a Parisian/Ratatouille theme.

Using the Stipend

I found the tilt sensor I need to order on Amazon and will be ordering it today. I also need to a buy a better potentiometer to make it easier to control it. I also want to buy some props such a cutting board and a rolling pin to make a more realistic experience.