Week 12 – Documentation on Final Project

Finalized Concept: “The Snail’s Journey to School”

Project Summary
The Snail’s Journey to School is an interactive physical–digital storytelling installation.
The player controls a small snail robot navigating a handmade obstacle course, helping it reach “school.” As the player moves the snail using physical buttons, a connected p5.js sketch narrates the adventure, updates animations, and reacts to the snail’s progress in real time through serial communication with Arduino.

1. Finalized Detailed Concept

Physical World (Arduino + Snail Robot)

The snail robot sits on a small motorized base (the base will be the robot that can be built from Arduino Uno kit). The user controls the snail with three buttons:

  • FORWARD
  • LEFT
  • RIGHT

A “finish line” sensor (photoresistor or IR distance sensor) detects when the snail reaches the school.

The obstacle course includes:

  • A puddle (painted blue)
  • Small pebbles
  • A small cardboard ramp
  • A school gate with the sensor hidden inside

Digital World (p5.js Storytelling Screen)

The p5 screen visually tells the story through 4 scenes:

  1. Intro Scene
    “The snail is late for school! Guide it through the obstacle course.”
  2. Instructions Scene
    Shows button directions + images.
  3. Live Story Mode
    Reacts to every button press and displays animations such as:
  • “Snail moves forward!”
  • “Turning left…”
  • Little movement animations or sound effects.
  1. Ending Scene
    When the sensor triggers finish, p5 displays:
    “You made it! The snail is in class now!” with a cute animation.
  2. If the player doesn’t make it on time, which is 2.5 minutes, or fails to cross more than 1 obstacle the screen shows “oh no, you’re late to school!”

Project Interaction Loop (Final Version)

  1. Player presses physical button
  2. Arduino moves snail robot + sends message to p5
  3. p5 receives message and updates story animation
  4. Snail reaches sensor
  5. Arduino sends “finish” to p5 ->  p5 plays ending

Arduino Program Design

Week 12 – Final Project Proposal

My midterm, Pitchy Bird, was a voice-controlled version of the classic Flappy Bird where the user’s vocal pitch determined the bird’s flight height. People leaned into microphones, experimenting with their vocal range to navigate obstacles. It was a fun experience. But for my final project, I want to take what learned to push it into physical computing and psychological testing.

I’m currently actively prototyping Flappy IAT, an endless runner controlled by an Arduino-based accelerometer that gamifies the Implicit Association Test (IAT).

The Concept

Instead of voice, players use a custom handheld controller (powered by an MMA8452Q accelerometer or a joystick). The mechanics require high-speed multitasking to bypass conscious cognitive filtering:

  1. Navigation (“Bump”): Players must physically “bump” the controller upward to flap the bird’s wings and maintain altitude against gravity.
  2. Categorization (“Tilt”): Every pipe obstacle displays a text stimulus (e.g., words like “Happy,” “Sad,” or category pairs like “Female Doctor”). To pass through the pipe safely, the player must tilt the controller Left (Green/Good) or Right (Red/Bad) to correctly categorize the word.

If it works, this game will be an active research tool more than only about survival. By forcing players to make categorization decisions under the pressure of keeping the bird afloat, the system exposes unconscious biases. The game tracks specific metrics via serial communication to p5.js:

  • Reaction Time (RT): The milliseconds between seeing the word and tilting the device.
  • Response Intensity: Measuring the angular velocity and degree of the tilt to detect hesitation or certainty.
  • Cognitive Load: Increasing difficulty across three levels to force players into a “flow state” where implicit biases are harder to suppress.

Technical Implementation

The project utilizes a bidirectional data flow. The Arduino handles raw gesture detection (calibrating “rest” position to detect relative Bumps and Tilts), while p5.js manages the game state, visual feedback (green checks for correct tilts, red flashes for errors), and data logging. At the end of a session, the game exports a CSV file detailing the player’s performance and implicit bias metrics.

Week 12 – Final Project Proposal

Finalized concept

For my final project, I’m building something that sits somewhere between a desk object, a quiet companion, and a tiny archivist of the room it lives in. The idea came from a simple thought: most days pass without leaving much behind, and by the time I get to the end of the night, the whole thing feels like a blur. I’m not someone who journals daily, but I like the idea of having some kind of trace of the day even if it’s abstract, incomplete, or not linguistic at all.

So the artefact is basically a small desk object that listens to the atmosphere of the space throughout the day, and later turns those ambient shifts into a soft, formless visual cloud in p5.js. It’s not interested in what the user did, only in how the day felt. It just records the vibe of a day, no eavesdropping or surveillance (and definitely no productivity tracking). I want the final thing to feel almost poetic, like the object is quietly paying attention on the side while I’m working, and at the end of the day it shows me its version of the memory.

What the Arduino will do

The Arduino will handle all the sensor-stuff during the day. I’m using:

  • a photoresistor to capture light changes,
  • an ultrasonic sensor to sense presence/absence near the desk,
  • a piezo to detect general sound/vibration spikes.

Arduino will collect these readings over time and send them to p5.js through serial. I’m keeping the Arduino’s job simple: sense → store → transmit.
I’ll also have a small physical trigger (most likely a button or dial) that the user presses at the end of the day to “reveal” the visual memory.

What p5.js will do

p5.js will take the day’s data and transform it into an atmospheric, slow-moving cloud. I’m aiming for visuals that sit in between abstract art and environmental “weather.” Light translates to color gradients, presence to density, and sound to softness or sharpness of the shape. I’m also considering a very light ml5.js layer just to classify general movement energy, so the cloud feels a bit more alive.
[Communication is mostly Arduino to p5, but later I might also send a message back to Arduino so the object can react in a small way when the memory is generated]

Early progress

So far, I’ve been sketching a few versions of the cloud visualization to figure out what feels “alive” without being overwhelming. Physically, I’ll keep the build minimal – something that looks more like a desk artefact than a tech box.
I’m hoping the final result feels calm, personal, and a little bit poetic, not a gadget trying to do too much, but an object that simply notices what the day was like and gives it a shape.

Week 12 – Documentation on Final Project

The final project is a Mini DJ Booth, a flat cardboard controller styled like a DJ console with painted “records” and four colored buttons on top. The cardboard box houses the Arduino and wiring, while the laptop screen becomes the virtual stage using p5.js. Each physical button represents a different beat or sound layer (for example: kick, snare, base, drums). When a user presses combinations of buttons, they can live-mix these layers while watching synchronized visuals, mimicking the feel of a compact DJ setup.

Generated with Chat GPT

I haven’t started the actual coding yet, but right now I’m more focused on the physical build, especially because this will be my first time soldering and I want to make sure I do it cleanly and safely. During our lab tour, we saw the LED push buttons, and they’re honestly perfect for this project, they’re big, colorful, satisfying to press, and will make the DJ booth feel much more realistic and professional. My plan is to use scrap cardboard from the back of the lab to construct the box itself, painting it to look like a real DJ setup with turntables and controls. Once the physical build is stable, I’ll move on to connecting the buttons to the Arduino and then writing the p5.js code to bring the whole system to life.

On the Arduino side, the program will continuously read the state of the four input buttons using `digitalRead()`. Each button is wired with a pull-down (or pull-up) resistor so the Arduino can reliably detect presses. In the main loop, the Arduino will encode the button states as a simple serial connection  and send this to the computer via USB using Serial.println(). The Arduino will not generate audio; its main role is to sense button presses quickly and transmit clean, structured data to p5.js so there is minimal latency between touch and feedback.

Here is a schematic example that I could potentially adapt for my final project:

Source: https://www.the-diy-life.com/multiple-push-buttons-on-one-arduino-input/

In the p5.js program, the sketch will receive the serial data, log the four button values, and use them to control both sound playback and visuals. For each button, p5 will toggle/keep looping the beat and  (or optionally stop the sound, depending on the final interaction choice). In the p5 sketch I will download four different audio tracks and will generate distinct color schemes and animations for each active button, for example, different color blocks, pulsing circles over each painted “record,” or waveform-style bars that move with the beat. At this stage, communication is mostly one-way (Arduino → p5), but there is room to extend it later (e.g., p5 sending messages back to Arduino to drive LEDs of each button). I just want to note that I havent started working on the p5 visuals yet in terms of code because I’m still planning the overall mood and ambiance I want to set.

I found royalty free beats I could download for the audio:

Week 11~: Production (Arduino + P5)

Production #1: Potentiometer Moves Ellipse

Link to P5 Sketch #1

Production #2: P5 Controls LEDs

Link to P5 Sketch #2

I modified it to also take “A” and “D” keys as input just like the mouse clicks

week 11 – in class exercises

For this week’s assignment, Zeina and I worked on three different exercises that focused on serial communication.

1- Make something that uses only one sensor  on Arduino and makes the ellipse in p5 move on the horizontal axis, in the middle of the screen, and nothing on arduino is controlled by p5

ARDUINO CODE

void setup() {
  Serial.begin(9600);
}

void loop() {
  int sensorValue = analogRead(A1); // 0–1023
  Serial.println(sensorValue);             // send to p5.js
  delay(50);
}

P5 CODE

let port;
let connectBtn;
let baudrate = 9600;
let lastMessage = "";
let sensorValue = 0;

function setup() {
  createCanvas(400, 400);
  background(220);

  //Setting the global variable port to a new serial port instance inside setup:
  port = createSerial();

  // we can open ports we have used previously without user interaction
  let usedPorts = usedSerialPorts(); //array of used ports
  if (usedPorts.length > 0) {  
    port.open(usedPorts[0], baudrate); //if any used port is in the array, open that port with 9600 baudrate
  }

  // any other ports (new ones) can be opened via a dialog after user interaction (see connectBtnClick below)
  connectBtn = createButton("Connect to Arduino");
  connectBtn.position(width/2, 270);
  connectBtn.mousePressed(connectBtnClick);

}

function draw() {
  background("white");
 
  // Read from the serial port. This is a non-blocking function. If a full line has come in (ending in \n), it returns that text. If the full line is not yet complete, it returns an empty string "" instead.
  let str = port.readUntil("\n");
  if (str.length > 0) {   // if str -a string- has any characters
    // print(str);
    lastMessage = str;
    sensorValue = int(lastMessage);  
  }
 
  //draw ellipse mapped to horizontal axis
  let x = map(sensorValue, 0, 1023, 0, width);  
  ellipse(x, height / 2, 40, 40);
 
  // Display the most recent message
  text("Last message: " + lastMessage, 10, height - 20);

  // change button label based on connection status
  if (!port.opened()) {
    connectBtn.html("Connect to Arduino");
  } else {
    connectBtn.html("Disconnect");
  }
}

function connectBtnClick() {
  if (!port.opened()) {
    port.open("Arduino", baudrate);
  } else {
    port.close();
  }
}

 

 

2-Make something that controls the LED brightness from p5

DEMO

IMG_8392 (2)

ARDUINO CODE

// Week 12 Example of bidirectional serial communication

int leftLedPin = 2;
int rightLedPin = 5;

void setup() {
  Serial.begin(9600);

  pinMode(LED_BUILTIN, OUTPUT);
  pinMode(leftLedPin, OUTPUT);
  pinMode(rightLedPin, OUTPUT);

  digitalWrite(leftLedPin, HIGH);
  digitalWrite(rightLedPin, HIGH);
  delay(200);
  digitalWrite(leftLedPin, LOW);
  digitalWrite(rightLedPin, LOW);

  while (Serial.available() <= 0) {
    digitalWrite(LED_BUILTIN, HIGH);
    Serial.println("0,0");
    delay(300);
    digitalWrite(LED_BUILTIN, LOW);
    delay(50);
  }
}

void loop() {
  while (Serial.available()) {
    digitalWrite(LED_BUILTIN, HIGH);

    int left = Serial.parseInt();
    int right = Serial.parseInt();

    if (Serial.read() == '\n') {

      // -----------------------
      // ONLY CHANGE IS HERE:
      // -----------------------
      digitalWrite(leftLedPin, left);     // left stays ON/OFF
      analogWrite(rightLedPin, right);    // right is now BRIGHTNESS (0–255)
      // -----------------------

      int sensor = analogRead(A0);
      delay(5);
      int sensor2 = analogRead(A1);
      delay(5);
      Serial.print(sensor);
      Serial.print(',');
      Serial.println(sensor2);
    }
  }
  digitalWrite(LED_BUILTIN, LOW);
}

p5 CODE

let port; // making a var to hold the serial port
let baudrate = 9600; // speed for talking to arduino
let brightnessSlider; // slider to pick brightness
let smoothBrightness = 0; //transition into the brightness instead of jumping

function setup() {
  createCanvas(400, 200); // just making a small canvas for ui
  textSize(18); // bigger test

  brightnessSlider = createSlider(0, 255, 0); // slider from 0 to full bright
  brightnessSlider.position(20, 80); // where it shows up on screen
  brightnessSlider.style('width', '200px'); // make it a bit wider

  port = createSerial(); // create a serial object so we can connect to arduino

  let used = usedSerialPorts(); // check if we already used a port before
  if (used.length > 0) {
    port.open(used[0], baudrate); // auto connect to the last used port
  }
}

function setupSerial() {
  if (!port.opened()) { // if no connection yet
    port.open("Arduino", baudrate); // try to open one
  } else {
    port.close(); // if already open then close it (toggle)
  }
}

function draw() {
  background(240); // light grey

  if (!port.opened()) {
    text("Press SPACE to connect", 20, 30); // tell the user what to do
  } else {
    text("Connected!", 20, 30); // connection message
  }

  let target = brightnessSlider.value(); // get the slider value

  // do transitional brightness
  smoothBrightness = lerp(smoothBrightness, target, 0.07);
 

  text("Brightness: " + int(smoothBrightness), 20, 70); // show the number

  // actually send the brightness to the arduino
  if (port.opened()) {
    let sendString = "0," + int(smoothBrightness) + "\n"; // left=0 right=smooth
    port.write(sendString); // send it over serial
  }
}

function keyPressed() {
  if (key === " ") {
    setupSerial(); // hitting space toggles the port connection
  }
}

3-Take the gravity wind example (https://editor.p5js.org/aaronsherwood/sketches/I7iQrNCul) and make it so every time the ball bounces one led lights up and then turns off, and you can control the wind from one analog sensor

ARDUINO CODE

void setup() {
  pinMode(2, OUTPUT);
  Serial.begin(9600);
}

void loop() {
  int sensorValue = analogRead(A1);
  Serial.println(sensorValue);

  //check for bounce
  if (Serial.available() > 0) {
    //if 1, light up led
    if (Serial.parseInt() == 1) {            
      digitalWrite(2, HIGH);
      delay(100);                          
      digitalWrite(2, LOW);
    }
  }
  delay(10);
}

P5.JS CODE

let velocity;
let gravity;
let position;
let acceleration;
let wind;
let drag = 0.99;
let mass = 50;
let port;
let connectBtn;
let baudrate = 9600;
let lastMessage = "";
let sensorValue = 0;

function setup() {
  createCanvas(640, 360);
  noFill();
  position = createVector(width / 2, 0);
  velocity = createVector(0, 0);
  acceleration = createVector(0, 0);
  gravity = createVector(0, 0.5 * mass);
  wind = createVector(0, 0);

  //Setting the global variable port to a new serial port instance inside setup:
  port = createSerial();

 
  // we can open ports we have used previously without user interaction
  let usedPorts = usedSerialPorts(); //array of used ports
  if (usedPorts.length > 0) {  
    port.open(usedPorts[0], baudrate); //if any used port is in the array, open that port with 9600 baudrate
  }
// any other ports (new ones) can be opened via a dialog after user interaction (see connectBtnClick below)
  connectBtn = createButton("Connect to Arduino");
  connectBtn.position(width/2, 270);
  connectBtn.mousePressed(connectBtnClick);
}

function draw() {
  background(255);
 
  // Read from the serial port. This is a non-blocking function. If a full line has come in (ending in \n), it returns that text. If the full line is not yet complete, it returns an empty string "" instead.
  let str = port.readUntil("\n");
  if (str.length > 0) {   // if str -a string- has any characters
    // print(str);
    lastMessage = str.trim();
    sensorValue = int(lastMessage);    
  }
 

  //wind controlled by analog sensor
  wind.x = map(sensorValue, 0, 1023, -1, 1);
  console.log("Sensor value: " + sensorValue);
 

  applyForce(wind);
  applyForce(gravity);

  velocity.add(acceleration);
  velocity.mult(drag);
  position.add(velocity);
  acceleration.mult(0);

  ellipse(position.x, position.y, mass, mass);

  // Bounce detection
  if (position.y > height - mass / 2) {
    velocity.y *= -0.9; // Dampening
    position.y = height - mass / 2;

    //send bounce signal to Arduino
    port.write("1\n");
  }
// Display the most recent message
  text("Last message: " + lastMessage, 10, height - 20);

  // change button label based on connection status
  if (!port.opened()) {
    connectBtn.html("Connect to Arduino");
  } else {
    connectBtn.html("Disconnect");
  }
}

function applyForce(force){
  // Newton's 2nd law: F = M * A
  // or A = F / M
  let f = p5.Vector.div(force, mass);
  acceleration.add(f);
}

function keyPressed(){
  if (keyCode==LEFT_ARROW){
    wind.x=-1;
  }
  if (keyCode==RIGHT_ARROW){
    wind.x=1;
  }
  if (key==' '){
    mass=random(15,80);
    position.y=-mass;
    velocity.mult(0);
  }
}

function connectBtnClick() {
  if (!port.opened()) {
    port.openDialog(); // user selects port
  } else {
    port.close();
  }
}

Final Project Proposal

I was looking for a room in Baraha to get some work done. However, to check some of the rooms there, I had to manually open the doors to see if they were occupied or not. I found the whole process inefficient, since I had to apologize if the room was occupied. The experience was embarrassing, and I was sure there were better ways to avoid it from happening.

So I decided to build my final project based on this idea: a detection machine that lets people know if the room is occupied or not. I found further inspiration from the parking lights installed in some of the malls in Abu Dhabi. There are lights on top of a parking space, and those lights display colors depending on the availability of the space.

At first, I thought using ultrasonic sensors would work. The idea was to have a sensor installed on the door to check if someone walks across the doorway: if they do, activate the light. However, ultrasonic sensors spread in an arc, so they may cause more errors than I expect. Next, I considered using PIR sensors that detect motion and heat. This is a good approach, but since PIR detects only movement, there are issues when people inside do not move: if they sit still and work on their projects, the sensor would assume the room is empty. This defeats the whole purpose of the project.

So for my sensors, I decided to use IR beam-break sensors. It is a single-line sensor that triggers when something gets in the way, so I thought it would be perfect for this project. If I have two of those sensors, I would also be able to determine whether a person enters or exits the room, allowing me to turn the LED on or off. It would be better to have the IR beam sensor installed at the door and a PIR motion sensor installed inside the room for a two-way detection system to ensure fewer errors, but cost-wise, I found that it was not worth having two sensors. I want to buy the sensors so that I can keep the project for my own use without having to return it later.

For the display, I will either order an LED panel or just use a circular LED display to show the status. Universal language such as red light means it is occupied, and green light means that the room is available. I can get more creative and use LED panels to show art installations, which would make it look more like an interactive media project.

Final Project Proposal: Plant Care Station

Final Project Proposal: Plant Care Station

The final project idea I have in mind is to make a connection between the Arduino and p5.js to take care of plants. Through buttons the user can fill the meter for water (through a blue buttons), then add fertilizer (yellow button), to trim (red button), to move sunlight closeness(yellow buttons one for arrow up one for arrow down).

You can even get a score or a rating based on how well you took care of the plant. Checking each metric. The Arduino part would be multiple buttons and a box that covers these buttons with larger buttons on the wooden box (design is inspired by the beat hoven project displayed in class).

ARDUINO

p5

Final Project Idea

My project idea is to make a game or maybe some kind of art tool that uses a tilt controller I’m going to build myself. I might use an Adafruit Circuit Playground to be the controller, since it has an accelerometer inside to sense which way you’re tilting it. I plan to use the Arduino IDE to program the Circuit Playground and get the accelerometer data. Then, I want to send that data to my computer and use p5.js to make the actual game or art program. I’m not totally sure what the final thing will be, but it could be a maze game where you tilt to move the character, a simple pilot game, or a cool art program where the tilt moves the brush around instead of using arrow keys.

Week 11: Final Project Concept

Main Concept

The main concept for my final project is the “Lock In Buddy.” It basically never lets you stop studying. Your laptop acts as the monitor, and the system uses Arduino sensors and p5 to track your behavior while you are studying. If it detects your phone, it triggers a super loud alarm. If it senses that you’re leaving the table using a distance sensor, it activates the speaker and says, “Come back to the desk. You gotta lock in!” I also want to incorporate the Pomodoro technique, where you study for 25 minutes and take a 5-minute break. A student can press a physical button to start the timer, and press it again to stop it. At the end, the system will give you a score based on how distracted you were by environmental variables, like looking at people walking by, eating, or any other interruptions. Essentially, it becomes your study buddy until you graduate.

 

Lock In Buddy will have these features: 

    • Phone detection alarm 

It will detect when the user picks up a phone, uses, or brings their phone to a nearby desk. The monitor is always watching the user to make sure that they don’t use their phone. 

    • Leaving the desk detection 

The distance sensor detects when the user moves away from the study area. It plays a warning voice message saying “you gotta lock in buddy. Or you will become homeless”. 

    • Real time distraction detection 

The system detects whether you’re being distracted by environmental conditions. Those include looking at people passing by, eating, chair movement, or noise.

    • Pomodoro timer with physical buttons 

Users can press the button to start, pause and stop the Pomodoro session. Also, it will record how long you have been studying for, which is going to be recorded for the scoring system as well. 

    • Focus score generation 

P5 will calculate the number of distractions, phone usage count, time away from desk, noise levels, and the number of successful Pomodoro sessions.

 

How to integrate p5 and Arduino

Phone detection + alarm: 

    • p5.js + Webcam + ML5 COCO-SSD 

Or 

    • Arduino light sensor + phone jail box
    • Start playing the warning alarm saying “Lock in” in p5

 

Eating or talking: 

    • Sound sensor

 

Pomodoro Timer with physical buttons: 

    • Buttons with arduino

 

Leaving desk: 

    • Ultrasonic distance sensor 

 

Focus Score generation: 

    • Arduino continuously reads 
      • Light sensor → phone removed 
      • Ultrasonic sensor → left desk 
      • Sound sensor → eating/talking 
      • Button → start/stop Pomodoro
    • Send those data to p5 and calculate the score