Final Project Concept – SpideySense

Spidey-Sense: An Interactive Web Swing

For my final project, I’m making something inspired by my favorite superhero, Spider-Man. The idea is that you can “swing” across a virtual city and shoot webs using hand movements. The system reacts in real time, so when you move your hand left or right, Spider-Man moves with you, and a web shoots when you do the classic Spider-Man hand gesture.

I’m using a distance sensor (or flex sensor) with Arduino to read the hand movements and send the data to P5.js. P5 handles all the graphics, like Spider-Man swinging across a city skyline with sprites and visuals inspired by  and possibly taken from the Spider-Man PS5 game. And it’s bidirectional, when you hit a target, P5 can send a signal back to Arduino to light up an LED or vibrate a tiny motor, so you get physical feedback too. Maybe I could use a motor to create a fan-like effect, so the user feels the wind from swinging around the city, too.

Week 11 Reading Reflection

I read this text with my brother in mind, he has autism, and I often notice how the design of everyday objects and spaces affects him. Small details like lighting, textures, and how objects feel to the touch can make a big difference in his comfort and understanding of the world. When he broke his leg, for example, he struggled to make sense of the cast, and the experience reminded me how design interacts with perception, comprehension, and well-being.

This reading made me reflect on my own assumptions about design. I realized that when I work on projects, I often focus on what makes sense to me, without considering the diverse needs of other users. Thinking about my brother made me ask: how could I design with someone else’s perspective in mind? How could I make objects and environments more inclusive, without losing creativity or aesthetic appeal?

It also made me notice the subtle ways society treats differences. Glasses, once stigmatized, are now fashion statements, yet many other assistive tools are still hidden or minimized. This made me think about how design can either reinforce stigma or remove it. For my brother, thoughtful design could mean the difference between feeling overwhelmed and feeling calm in his daily life.

Finally, the reading helped me see inclusion in a broader sense. Simple, thoughtful design isn’t just functional; it can enhance comfort, independence, and confidence. I noticed how many spaces we use aren’t made with accessibility in mind, though some places are beginning to shift. There’s a mall in Dubai now that was designed with Autism friendliness in mind. It made me think about my own role as a designer and how I should approach projects with attention to diverse experiences and an awareness of cognitive and sensory differences.

week 11: reading response

This reading gave me a new perspective on the intersection between disability and technology. I didn’t know that eyeglasses were initially made for the medical purposes only, so it was really interesting to learn how societal perceptions and stigma can change so dramatically over the course of time. I realized how powerful designers are in shaping these perceptions, as discussed in the reading. This is definitely something that engineers cannot achieve by themselves. As a prospective software engineer, I learned that my goal should not be just to make things useful, but to make them as simple and usable as possible for everyone in the world, like AirPods, irrespective of nationality and background. I’ve personally experienced how complexity can mess up the development process and even the essential purpose of a product.

 

For instance, I’m currently developing my mobile app that turns TikTok into studying. Initially, I wanted to integrate a social media feature where users can compete with each other in terms of how many questions they can answer in a row. But that feature turned out to be super complex, and spending too much time on implementing that single feature prevented the essential purpose of my app, which is to make studying addictive for students in the world. That is the reason why I decided to focus sorely on the scrolling system, since it is the simplest and most essential feature of TikTok that makes it so addictive. Through this reading, I was able to understand how important it is to make a product as simple and usable as possible for all the users in the world.

Week 11 – Serial Communication With Shota

With Shota Matsumoto
EXERCISE 01: ARDUINO TO P5 COMMUNICATION

Concept

For the first exercise we used a photoresistor as an analog sensor on Arduino to make our ellipse shape move on the horizontal axis in p5. For our code, we adjust it so that with every touch on the photoresistor, the ellipse would move on the canvas.

VIDEO:

https://drive.google.com/file/d/12b1KzwTlo1IQLtDfleAJNJccZhL7J0Tb/view?usp=sharing

Code we’re proud of:

When it comes to the first exercise, the part we are most proud of is the block of code that controls the x-position of the ellipse using the light sensor value from the Arduino. We read the sensor value through the serial port that is connected to the Arduino and map it from 0 to the width of the canvas so that it never goes out of the dimension of the screen.

 

//read the data coming to the port until the newline character
   let data = port.readUntil("\n");
   //if there is a data
   if (data.length > 0) {
     //convert the clean data into integer
     rValue = int(trim(data));
     //store data of left and right into msg
     let msg = left + "," + right + "\n";
     //send the msg back to the Arduino
     port.write(msg);
   }
   text("rValue = " + rValue, 20, 50);
 }
 //move xpos depending on the sensor value (max = 640)
 let xpos = map(rValue, 550, 900, 0, width);
 fill(0, 0, 255);
 ellipse(xpos, height / 2, 100, 50);

 

EXERCISE 02: P5 TO ARDUINO COMMUNICATION
Make something that controls the LED brightness from p5.

Concept

For the second exercise we used the createSlide function to create a slider on the canvas of p5. This way, every time we moved the slider to the right, the brightness of the LED on the Arduino would increase, and every time the slider was moved to the left, the brightness would decrease.

VIDEO:

https://drive.google.com/file/d/12b1KzwTlo1IQLtDfleAJNJccZhL7J0Tb/view?usp=sharing

Code we’re proud of:

For the second exercise, this block of code combines what we learned during the class about p5.js (createSlider) and Arduino. Since we remembered using a slider to control a value or game score, we decided to use it to control the brightness of the LED. We take the slider value, store it in a variable called brightness, and send it to the Arduino through the serial port, as shown in the block of the code below.

//read the slider value
 brightnessValue = brightnessSlider.value();
 //if the port is open
 if (port.opened()) {
   //store the brightness value from the slider
   let msg = brightnessValue + "\n";
   //send it to Arduino
   port.write(msg);
 }

 

EXERCISE 03: BI-DIRECTIONAL COMMUNICATION

Concept

For the third exercise, we made use of a potentiometer ( analog sensor ) on Arduino and a LED light, connecting it on the corresponding pins and then connecting Arduino UNO with P5. After adding the necessary codes on P5, we adjusted our “ball” so that every time it bounced against the floor, the LED would light up, and once it stopped bouncing the LED turned off. With the potentiometer, we were able to control the “wind”, causing the ball to be pushed from side to side, depending on the direction in which we turned the potentiometer.

VIDEO:

https://drive.google.com/file/d/1YA0d_xkwM2zxTco6mqnZNbEIr9Ou60oq/view?usp=sharing

Code we’re proud of:

For the third exercise, we’re proud of the block of code below. It basically captures the main objective of this exercise. When the ball reaches the ground and the LED state is 0, meaning the LED is currently off, we turn it on by sending a 1 to the Arduino through the serial port. It is simple but we needed to understand the basics of what is going on in the vectors.

//if the ball reached the floor and the led is currently off
 if (isBounced && ledStatus === 0) {
   //turn the LED on
   port.write("1");
   //set the state to 1
   ledStatus = 1;
 }
 else if (!isBounced && ledStatus === 1) {
   //turn the led off
   port.write("0");
   ledState = 0;
 }

 

Github

Links:

exercise1 Arduino

exercise1 p5

exercise2 arduino

exercise2 p5

exercise3 arduino

exercise3 p5

Challenges and Further Improvements

Fortunately, we were able to complete all the assignments successfully, and managed to achieve all the necessary responses of P5 and Arduino. For each exercise, we started by planning out which tools we would use, how to arrange them on our breadboard, and once this was done, our greatest challenges consisted of arranging and adding the codes required to make each response function. On the third assignment we especially struggled writing the codes required to make the potentiometer affect the “wind” and consequently the direction in which the ball was pushed. For future improvements, we would like to grasp a better understanding of the exchanges between P5 and Arduino in order to make more complex and interactive responses that can also satisfy the assignment’s requirements.

 

References:

We used ChatGPT to help us understand the bouncing ball template code since we hadn’t yet learned about the vector library in class. After understanding the foundation of it, we were able to integrate the isBounced logic into the template code to detect whether the ball was bouncing or not. Thus, we’re glad we were able to learn a lot about vectors library. We also used ChatGPT to learn about DOM library methods such as .position() and .style(). Although we had covered the basics of createSlider, we didn’t delve deeper into it during the class, so we needed to learn how to adjust its position and CSS styling of the ball.

 

Week 11: Exercises – Shota and Isabella

EXERCISE 01: ARDUINO TO P5 COMMUNICATION

Concept:

For the first exercise we used a photoresistor as an analog sensor on Arduino to make our ellipse shape move on the horizontal axis in p5. For our code, we adjust it so that with every touch on the photoresistor, the ellipse would move on the canvas.

VIDEO:

video1

Code we’re proud of:

When it comes to the first exercise, the part we are most proud of is the block of code that controls the x-position of the ellipse using the light sensor value from the Arduino. We read the sensor value through the serial port that is connected to the Arduino and map it from 0 to the width of the canvas so that it never goes out of the dimension of the screen.

//read the data coming to the port until the newline character
   let data = port.readUntil("\n");
   //if there is a data
   if (data.length > 0) {
     //convert the clean data into integer
     rValue = int(trim(data));


     //store data of left and right into msg
     let msg = left + "," + right + "\n";
     //send the msg back to the Arduino
     port.write(msg);
   }


   text("rValue = " + rValue, 20, 50);
 }


 //move xpos depending on the sensor value (max = 640)
 let xpos = map(rValue, 550, 900, 0, width);
 fill(0, 0, 255);
 ellipse(xpos, height / 2, 100, 50);

 

EXERCISE 02: P5 TO ARDUINO COMMUNICATION
Make something that controls the LED brightness from p5

Concept:

For the second exercise we used the createSlide function to create a slider on the canvas of p5. This way, every time we moved the slider to the right, the brightness of the LED on the Arduino would increase, and every time the slider was moved to the left, the brightness would decrease.

VIDEO:

video2

Code we’re proud of:

For the second exercise, this block of code combines what we learned during the class about p5.js (createSlider) and Arduino. Since we remembered using a slider to control a value or game score, we decided to use it to control the brightness of the LED. We take the slider value, store it in a variable called brightness, and send it to the Arduino through the serial port, as shown in the block of the code below.

//read the slider value
 brightnessValue = brightnessSlider.value();


 //if the port is open
 if (port.opened()) {
   //store the brightness value from the slider
   let msg = brightnessValue + "\n";
   //send it to Arduino
   port.write(msg);
 }

 

EXERCISE 03: BI-DIRECTIONAL COMMUNICATION

Concept:

For the third exercise, we made use of a potentiometer ( analog sensor ) on Arduino and a LED light, connecting it on the corresponding pins and then connecting Arduino UNO with P5. After adding the necessary codes on P5, we adjusted our “ball” so that every time it bounced against the floor, the LED would light up, and once it stopped bouncing the LED turned off. With the potentiometer, we were able to control the “wind”, causing the ball to be pushed from side to side, depending on the direction in which we turned the potentiometer.

VIDEO:

video3

Code we’re proud of:

For the third exercise, we’re proud of the block of code below. It basically captures the main objective of this exercise. When the ball reaches the ground and the LED state is 0, meaning the LED is currently off, we turn it on by sending a 1 to the Arduino through the serial port. It is simple but we needed to understand the basics of what is going on in the vectors. 

//if the ball reached the floor and the led is currently off
 if (isBounced && ledStatus === 0) {
   //turn the LED on
   port.write("1");
   //set the state to 1
   ledStatus = 1;
 }
 else if (!isBounced && ledStatus === 1) {
   //turn the led off
   port.write("0");
   ledState = 0;
 }

 

Challenges and Further Improvements:

Fortunately, we were able to complete all the assignments successfully, and managed to achieve all the necessary responses of P5 and Arduino. For each exercise, we started by planning out which tools we would use, how to arrange them on our breadboard, and once this was done, our greatest challenges consisted of arranging and adding the codes required to make each response function. On the third assignment we especially struggled writing the codes required to make the potentiometer affect the “wind” and consequently the direction in which the ball was pushed. For future improvements, we would like to grasp a better understanding of the exchanges between P5 and Arduino in order to make more complex and interactive responses that can also satisfy the assignment’s requirements.

 

Github Link:

exercise1 Arduino

exercise1 p5

exercise2 arduino

exercise2 p5

exercise3 arduino

exercise3 p5

Reference:

We used ChatGPT to help us understand the bouncing ball template code since we hadn’t yet learned about the vector library in class. After understanding the foundation of it, we were able to integrate the isBounced logic into the template code to detect whether the ball was bouncing or not. Thus, we’re glad we were able to learn a lot about vectors library. We also used ChatGPT to learn about DOM library methods such as .position() and .style(). Although we had covered the basics of createSlider, we didn’t delve deeper into it during the class, so we needed to learn how to adjust its position and CSS styling of the ball.

Week 11 Reading Reflection

This week’s article made me see things in a new way. It used eyeglasses as its main example. Glasses are a medical tool, but they are also fashion items. The article asks why other aids, like hearing aids, can’t be the same. I had never really thought about it before, but it’s true. It feels like a failure that we treat these items so differently.

This made me think about how design makes people feel. When a device looks cold and medical, it can make a person feel like they are just a “patient” or that their needs are something to hide. The article shows that making these items beautiful or cool is not silly. It’s actually very important for giving people dignity and choice. It changes the focus from “fixing a problem” to just “living a life.”

Finally, the most interesting point was that this is a two-way street. Designing for disability can actually spark amazing new ideas for everyone. When designers have to solve a specific problem, it can lead to a brand new way of thinking. This means design for disability isn’t a small or separate thing; it’s a source of creativity that can make all design better.

Preliminary concept for Final Project

For my final project, I will design an interactive healthy-choice game using both Arduino and p5.js. The game features a Kazakh girl character positioned at the bottom of the screen while different items fall from the top. The healthy items, apples, dates, milk, and water are foods and drinks that people commonly consume in Kazakhstan, making the game culturally relevant and grounded in everyday life. These are contrasted with less healthy options such as burgers and Coca-Cola.

The game progresses in two alternating phases. In the first phase, apples, dates, and burgers fall from the sky. The player must collect apples and dates while avoiding burgers. After a set period, the game switches to a second phase, where milk, water, and Coca-Cola fall. In this phase, the player should collect milk and water while avoiding Coca-Cola. Each healthy item increases the score: apples give 20 points, dates 10, milk 20, and water 10. The player starts with three lives and loses one whenever they collide with an unhealthy item. When all lives are lost, the game ends, and p5 displays the final score.

Arduino provides the physical interaction through three buttons: one for start/restart and two for moving left and right. Arduino continuously senses button presses and sends serial messages to p5.js, which updates movement, animations, and game states.

Week 11 – Reading Reflection

This reading made me rethink how much design pretends to care about “everyone” while actually designing for some imaginary default person. The whole debate about hiding versus showing assistive devices hit me the most. The hearing aid example frustrated me. It reminded me of how often people, including me at times, feel pressured to tone themselves down just to blend in. Seeing designers hide assistive devices made that pressure feel even more obvious. If a device supports someone’s life, why shouldn’t it be allowed to exist proudly?

I liked how the reading returned to the idea that objects carry identity. Even in my IM projects, I can feel that tension between making something perfectly sleek or letting it look like something I actually touched. My work always ends up somewhere in the middle: functional, but still a little sentimental, a little messy, a little me.

The idea of bringing artists and fashion designers into accessibility design made complete sense. It made assistive tech feel less like “equipment” and more like something that can match someone’s personality. A prosthetic can be a tool, but it can also be a statement. A hearing aid can be medical, but it can also be stylish. That’s the kind of design I actually care about: things that work, but also let people feel like themselves.

Week 11 – 3 Exercises

The 3 tasks we worked on were practice on how to make both Arduino and P5j work together.

Group: Deema Al Zoubi and Rawan Al Ali 

Exercise 1 :

let connectButton;
let port;
let reader;
let sensorValue = 0;
let keepReading = false;

function setup() {
  createCanvas(600, 400);

  // Create a connect button 
  connectButton = createButton('Connect to Arduino');
  connectButton.position(10, 10);
  connectButton.mousePressed(connectSerial);

  textAlign(CENTER, CENTER);
  textSize(14);
}

async function connectSerial() {
  // If already connected, close first
  if (port && port.readable) {
    await closeSerial();
    connectButton.html('Connect to Arduino');
    return;
  }

  try {
  
    port = await navigator.serial.requestPort();
    await port.open({ baudRate: 9600 });

    console.log('Port opened', port);
    connectButton.html('Disconnect');

    const decoder = new TextDecoderStream();
    // Pipe the readable stream from the port to the decoder
    port.readable.pipeTo(decoder.writable);
    reader = decoder.readable.getReader();

    keepReading = true;
    readLoop(); // start read loop
  } catch (err) {
    console.error('Error opening serial port:', err);
    alert('Could not open serial port. Make sure your device is connected and try again.');
  }
}

async function closeSerial() {
  keepReading = false;
  try {
    if (reader) {
      await reader.cancel();
      reader.releaseLock();
      reader = null;
    }
    if (port && port.close) {
      await port.close();
      console.log('Port closed');
    }
    port = null;
  } catch (err) {
    console.warn('Error closing port:', err);
  }
}

async function readLoop() {
  let partial = '';
  try {
    while (keepReading && reader) {
      const { value, done } = await reader.read();
      if (done) {
        console.log('Reader closed');
        break;
      }
      if (!value) continue;

      // split by newline
      partial += value;
      let lines = partial.split(/\r?\n/);
      // Keep the last partial line in buffer
      partial = lines.pop();

      for (let line of lines) {
        line = line.trim();
        if (line === '') continue;        
        const num = parseInt(line, 10);
        if (!Number.isNaN(num)) {
          // clamp to expected range in case of weird data
          sensorValue = Math.max(0, Math.min(1023, num));
          console.log('sensorValue:', sensorValue);
        } else {
          console.log('non-numeric line ignored:', line);
        }
      }
    }
  } catch (err) {
    console.error('Read loop error:', err);
  }
}

function draw() {
  background(240);

  let x = map(sensorValue, 0, 1023, 25, width - 25);
  fill(100, 150, 240);
  ellipse(x, height / 2, 50, 50);
}
window.addEventListener('beforeunload', async (e) => {
  if (port && port.readable) {
    await closeSerial();
  }
});

We used one sensor (a potentiometer) on the Arduino to control the horizontal position of an ellipse in p5.js. The ellipse stayed in the middle of the screen vertically, and all movement was controlled by the Arduino sensor, p5 didn’t send any commands to Arduino.

Video 1: IMG_1012

Schematic 1:

 

Exercise 2:

let connectButton;
let port;
let writer;
let slider;

let targetBrightness = 0;
let currentBrightness = 0;

function setup() {
  createCanvas(400, 200);

  connectButton = createButton("Connect to Arduino");
  connectButton.position(10, 10);
  connectButton.mousePressed(connectSerial);

  slider = createSlider(0, 255, 0);
  slider.position(10, 60);
  slider.style('width', '300px');

  textSize(16);
}

async function connectSerial() {
  try {
    port = await navigator.serial.requestPort();
    await port.open({ baudRate: 9600 });
    writer = port.writable.getWriter();
    connectButton.html("Connected");
  } catch (err) {
    console.error("Connection error:", err);
  }
}

function draw() {
  background(230);

  // Slider sets the target brightness
  targetBrightness = slider.value();

  // Gradually move toward target (both up and down)
currentBrightness = lerp(currentBrightness, targetBrightness, 0.15);

  // Snap to 0/255 when very close so it truly turns off/on
  if (abs(currentBrightness - targetBrightness) < 1) {
    currentBrightness = targetBrightness;
  }

  fill(0);
  text("LED Brightness: " + int(currentBrightness), 10, 110);

  // Send brightness
  if (writer) {
    writer.write(new Uint8Array([int(currentBrightness)]));
  }
}

We created a slider in p5.js to control the brightness of an LED connected to the Arduino. Moving the slider to the right gradually increased the LED brightness, and moving it back to the left gradually turned it off.

Video 2: https://intro.nyuadim.com/wp-content/uploads/2025/11/IMG_1030.mov

Schematic 2:

 

Exercise 3:

let velocity;
let gravity;
let position;
let acceleration;
let drag = 0.99;
let mass = 50;

let brightnessValue = 512; // potentiometer value (0–1023)
let ballDropped = false;
let ledOn = false;

let port, writer, reader;
let serialActive = false;
let serialBuffer = '';

function setup() {
  createCanvas(640, 360);
  textSize(18);

  position = createVector(width / 2, 0);
  velocity = createVector(0, 0);
  acceleration = createVector(0, 0);
  gravity = createVector(0, 0.2 * mass);

  // Connect button
  let btn = createButton("Connect to Arduino");
  btn.position(10, 10);
  btn.mousePressed(connectAndStart);
}

function draw() {
  background(255);

  // Show instructions before dropping ball
  fill(0);
  if (!ballDropped) {
    text("Press B to drop the ball", 20, 30);
    return; // stop here until ball is dropped
  }

  // Show serial status
  if (serialActive) {
    text("Connected", 20, 30);
    text(`Potentiometer: ${brightnessValue}`, 20, 50);
  } else {
    text("Serial Port Not Connected", 20, 30);
  }

  // Apply gravity
  velocity.y += gravity;
  velocity.y *= drag;

  // Horizontal control from potentiometer
  let windX = map(brightnessValue, 0, 1023, -2, 2);
  velocity.x = windX;

  position.add(velocity);

  // Bounce on floor
  if (position.y >= height - mass / 2) {
    position.y = height - mass / 2;
    velocity.y *= -0.9;

    // Flash LED
    if (serialActive && !ledOn) {
      writeSerial("F");
      ledOn = true;
    }
  } else if (ledOn) {
    ledOn = false;
  }

  // Bounce on top
  if (position.y - mass / 2 < 0) {
    position.y = mass / 2;
    velocity.y *= -0.9;
  }

  // Keep ball inside canvas horizontally
  position.x = constrain(position.x, mass / 2, width - mass / 2);

  // Draw ball
  fill(100, 150, 240);
  ellipse(position.x, position.y, mass, mass);
}

function applyForce(force) {
  let f = p5.Vector.div(force, mass);
  acceleration.add(f);
}

function keyPressed() {
  if (key == "B" || key == "b") dropBall();
}

function dropBall() {
  position.set(width / 2, 0);
  velocity.set(0, 0);
  mass = 50;
  gravity = 0.05 * mass;
  ballDropped = true;
}

// Serial functions
async function connectAndStart() {
  try {
    port = await navigator.serial.requestPort();
    await port.open({ baudRate: 9600 });

    writer = port.writable.getWriter();

    const decoder = new TextDecoderStream();
    port.readable.pipeTo(decoder.writable);
    reader = decoder.readable.getReader();

    serialActive = true;

    // Start reading
    readLoop();
  } catch (err) {
    console.error("Serial error:", err);
  }
}

async function readLoop() {
  while (serialActive && reader) {
    const { value, done } = await reader.read();
    if (done) break;
    if (value) parseSerial(value);
  }
}

function writeSerial(msg) {
  if (writer) writer.write(new TextEncoder().encode(msg + "\n"));
}

function parseSerial(data) {
  serialBuffer += data;
  let lines = serialBuffer.split('\n');
  serialBuffer = lines.pop();

  for (let line of lines) {
    let val = parseInt(line.trim());
    if (!isNaN(val)) brightnessValue = val;
  }
}

We made the ball bounce continuously up and down in p5.js, and connected an LED on the Arduino that lights up briefly every time the ball hits the floor. A potentiometer on the Arduino was used to control the horizontal movement of the ball: turning the knob to higher values moves the ball to the right, and lower values move it to the left. The potentiometer’s outer pins were connected to 5V and GND, and the middle pin to A0. Using Web Serial, we read the potentiometer values in p5 and mapped them to the horizontal position of the ball while it keeps bouncing.

Video 3: IMG_1030

Schematic 3:

Reflection: 

These exercises helped us understand how p5.js and Arduino can work together. We saw that Arduino can send real-world sensor data, like potentiometer values, to p5 to control visuals or simulations, and that p5 can also send commands back to Arduino, like turning an LED on or off. This gave us a clear idea of how much influence p5 can have on Arduino outputs, and how Arduino inputs can drive digital interactions in p5. Practicing this will be really useful for our final project, as it shows how to combine physical sensors, real-time controls, and visual feedback in a simple interactive system.

 

3 exercises – Week 11

The 3 tasks we worked on were practice on how to make both Arduino and P5j work together.

Group: Deema Al Zoubi and Rawan Al Ali 

Exercise 1 :

let connectButton;
let port;
let reader;
let sensorValue = 0;
let keepReading = false;

function setup() {
  createCanvas(600, 400);

  // Create a connect button 
  connectButton = createButton('Connect to Arduino');
  connectButton.position(10, 10);
  connectButton.mousePressed(connectSerial);

  textAlign(CENTER, CENTER);
  textSize(14);
}

async function connectSerial() {
  // If already connected, close first
  if (port && port.readable) {
    await closeSerial();
    connectButton.html('Connect to Arduino');
    return;
  }

  try {
  
    port = await navigator.serial.requestPort();
    await port.open({ baudRate: 9600 });

    console.log('Port opened', port);
    connectButton.html('Disconnect');

    const decoder = new TextDecoderStream();
    // Pipe the readable stream from the port to the decoder
    port.readable.pipeTo(decoder.writable);
    reader = decoder.readable.getReader();

    keepReading = true;
    readLoop(); // start read loop
  } catch (err) {
    console.error('Error opening serial port:', err);
    alert('Could not open serial port. Make sure your device is connected and try again.');
  }
}

async function closeSerial() {
  keepReading = false;
  try {
    if (reader) {
      await reader.cancel();
      reader.releaseLock();
      reader = null;
    }
    if (port && port.close) {
      await port.close();
      console.log('Port closed');
    }
    port = null;
  } catch (err) {
    console.warn('Error closing port:', err);
  }
}

async function readLoop() {
  let partial = '';
  try {
    while (keepReading && reader) {
      const { value, done } = await reader.read();
      if (done) {
        console.log('Reader closed');
        break;
      }
      if (!value) continue;

      // split by newline
      partial += value;
      let lines = partial.split(/\r?\n/);
      // Keep the last partial line in buffer
      partial = lines.pop();

      for (let line of lines) {
        line = line.trim();
        if (line === '') continue;        
        const num = parseInt(line, 10);
        if (!Number.isNaN(num)) {
          // clamp to expected range in case of weird data
          sensorValue = Math.max(0, Math.min(1023, num));
          console.log('sensorValue:', sensorValue);
        } else {
          console.log('non-numeric line ignored:', line);
        }
      }
    }
  } catch (err) {
    console.error('Read loop error:', err);
  }
}

function draw() {
  background(240);

  let x = map(sensorValue, 0, 1023, 25, width - 25);
  fill(100, 150, 240);
  ellipse(x, height / 2, 50, 50);
}
window.addEventListener('beforeunload', async (e) => {
  if (port && port.readable) {
    await closeSerial();
  }
});
void setup() {
  Serial.begin(9600);
}

void loop() {
  int sensorValue = analogRead(A0);
  Serial.println(sensorValue);
  delay(50); // ~20Hz
}

We used one sensor (a potentiometer) on the Arduino to control the horizontal position of an ellipse in p5.js. The ellipse stayed in the middle of the screen vertically, and all movement was controlled by the Arduino sensor, p5 didn’t send any commands to Arduino.

Video 1: IMG_1012

Schematic 1:

Exercise 2:

let connectButton;
let port;
let writer;
let slider;

let targetBrightness = 0;
let currentBrightness = 0;

function setup() {
  createCanvas(400, 200);

  connectButton = createButton("Connect to Arduino");
  connectButton.position(10, 10);
  connectButton.mousePressed(connectSerial);

  slider = createSlider(0, 255, 0);
  slider.position(10, 60);
  slider.style('width', '300px');

  textSize(16);
}

async function connectSerial() {
  try {
    port = await navigator.serial.requestPort();
    await port.open({ baudRate: 9600 });
    writer = port.writable.getWriter();
    connectButton.html("Connected");
  } catch (err) {
    console.error("Connection error:", err);
  }
}

function draw() {
  background(230);

  // Slider sets the target brightness
  targetBrightness = slider.value();

  // Gradually move toward target (both up and down)
currentBrightness = lerp(currentBrightness, targetBrightness, 0.15);

  // Snap to 0/255 when very close so it truly turns off/on
  if (abs(currentBrightness - targetBrightness) < 1) {
    currentBrightness = targetBrightness;
  }

  fill(0);
  text("LED Brightness: " + int(currentBrightness), 10, 110);

  // Send brightness
  if (writer) {
    writer.write(new Uint8Array([int(currentBrightness)]));
  }
}
int ledPin = 9; // LED connected to PWM pin 9
int value = 0;  // variable to store incoming brightness

void setup() {
  pinMode(ledPin, OUTPUT);
  Serial.begin(9600); // match the baud rate in p5
}

void loop() {
  if (Serial.available() > 0) {
    // read incoming string until newline
    String data = Serial.readStringUntil('\n');
    data.trim(); // remove whitespace
    if (data.length() > 0) {
      value = data.toInt();          // convert to integer
      value = constrain(value, 0, 255); // keep within PWM range
      analogWrite(ledPin, value);   // set LED brightness
    }
  }
}

We created a slider in p5.js to control the brightness of an LED connected to the Arduino. Moving the slider to the right gradually increased the LED brightness, and moving it back to the left gradually turned it off.

Video 2: https://intro.nyuadim.com/wp-content/uploads/2025/11/IMG_1030.mov

Schematic 2:

Exercise 3:

let velocity;
let gravity;
let position;
let acceleration;
let drag = 0.99;
let mass = 50;

let brightnessValue = 512; // potentiometer value (0–1023)
let ballDropped = false;
let ledOn = false;

let port, writer, reader;
let serialActive = false;
let serialBuffer = '';

function setup() {
  createCanvas(640, 360);
  textSize(18);

  position = createVector(width / 2, 0);
  velocity = createVector(0, 0);
  acceleration = createVector(0, 0);
  gravity = createVector(0, 0.2 * mass);

  // Connect button
  let btn = createButton("Connect to Arduino");
  btn.position(10, 10);
  btn.mousePressed(connectAndStart);
}

function draw() {
  background(255);

  // Show instructions before dropping ball
  fill(0);
  if (!ballDropped) {
    text("Press B to drop the ball", 20, 30);
    return; // stop here until ball is dropped
  }

  // Show serial status
  if (serialActive) {
    text("Connected", 20, 30);
    text(`Potentiometer: ${brightnessValue}`, 20, 50);
  } else {
    text("Serial Port Not Connected", 20, 30);
  }

  // Apply gravity
  velocity.y += gravity;
  velocity.y *= drag;

  // Horizontal control from potentiometer
  let windX = map(brightnessValue, 0, 1023, -2, 2);
  velocity.x = windX;

  position.add(velocity);

  // Bounce on floor
  if (position.y >= height - mass / 2) {
    position.y = height - mass / 2;
    velocity.y *= -0.9;

    // Flash LED
    if (serialActive && !ledOn) {
      writeSerial("F");
      ledOn = true;
    }
  } else if (ledOn) {
    ledOn = false;
  }

  // Bounce on top
  if (position.y - mass / 2 < 0) {
    position.y = mass / 2;
    velocity.y *= -0.9;
  }

  // Keep ball inside canvas horizontally
  position.x = constrain(position.x, mass / 2, width - mass / 2);

  // Draw ball
  fill(100, 150, 240);
  ellipse(position.x, position.y, mass, mass);
}

function applyForce(force) {
  let f = p5.Vector.div(force, mass);
  acceleration.add(f);
}

function keyPressed() {
  if (key == "B" || key == "b") dropBall();
}

function dropBall() {
  position.set(width / 2, 0);
  velocity.set(0, 0);
  mass = 50;
  gravity = 0.05 * mass;
  ballDropped = true;
}

// Serial functions
async function connectAndStart() {
  try {
    port = await navigator.serial.requestPort();
    await port.open({ baudRate: 9600 });

    writer = port.writable.getWriter();

    const decoder = new TextDecoderStream();
    port.readable.pipeTo(decoder.writable);
    reader = decoder.readable.getReader();

    serialActive = true;

    // Start reading
    readLoop();
  } catch (err) {
    console.error("Serial error:", err);
  }
}

async function readLoop() {
  while (serialActive && reader) {
    const { value, done } = await reader.read();
    if (done) break;
    if (value) parseSerial(value);
  }
}

function writeSerial(msg) {
  if (writer) writer.write(new TextEncoder().encode(msg + "\n"));
}

function parseSerial(data) {
  serialBuffer += data;
  let lines = serialBuffer.split('\n');
  serialBuffer = lines.pop();

  for (let line of lines) {
    let val = parseInt(line.trim());
    if (!isNaN(val)) brightnessValue = val;
  }
}
int ledPin = 9; // LED connected to pin 9
int potPin = A0; // potentiometer connected to A0
int potValue = 0; 

void setup() {
  pinMode(ledPin, OUTPUT);
  Serial.begin(9600); // match baud rate with p5.js
}

void loop() {
  // Read potentiometer (optional if p5 reads directly)
  potValue = analogRead(potPin);
  Serial.println(potValue); // send to p5 for horizontal control

  // Check for serial input from p5
  if (Serial.available() > 0) {
    char command = Serial.read(); // read single character
    if (command == 'F') {        // 'F' = flash LED
      digitalWrite(ledPin, HIGH);
      delay(100);                // keep LED on briefly
      digitalWrite(ledPin, LOW);
    }
  }
  
  delay(10); // small delay for stability
}

We made the ball bounce continuously up and down in p5.js, and connected an LED on the Arduino that lights up briefly every time the ball hits the floor. A potentiometer on the Arduino was used to control the horizontal movement of the ball: turning the knob to higher values moves the ball to the right, and lower values move it to the left. The potentiometer’s outer pins were connected to 5V and GND, and the middle pin to A0. Using Web Serial, we read the potentiometer values in p5 and mapped them to the horizontal position of the ball while it keeps bouncing.

Video 3: IMG_1030

Schematic 3:

Reflection: 

These exercises helped us understand how p5.js and Arduino can work together. We saw that Arduino can send real-world sensor data, like potentiometer values, to p5 to control visuals or simulations, and that p5 can also send commands back to Arduino, like turning an LED on or off. This gave us a clear idea of how much influence p5 can have on Arduino outputs, and how Arduino inputs can drive digital interactions in p5. Practicing this will be really useful for our final project, as it shows how to combine physical sensors, real-time controls, and visual feedback in a simple interactive system.