Author: Zeina Khadem

Finalized Concept

For my final project, I’m building an interactive Spider-Man experience where you can “swing” around a simplified version of the NYUAD campus using a glove with a sensor in it. The glove basically becomes your controller, but in a way that feels way more like Spiderman. When you do the signature web shooting gesture, Spider-Man reacts instantly on screen. I Instead of clicking buttons or pressing keys, you just move your hand, and the game turns that motion into a web-shooting movement across the campus. The whole idea is to make it feel intuitive and fun, like you’re actually guiding him through the space instead of controlling him from a distance.

Hardware

I’m using a capacitive touch sensor to detect the Spider-Man web-shooting gesture. The sensor is placed on the palm of the glove, and the fingertips are left exposed so that when the player touches the sensor with the correct fingers, mimicking Spider-Man’s iconic “thwip” pose, the system registers a web shot. When the hand relaxes and the fingers release the sensor, the web disappears in p5.

The challenge is integrating the sensor onto the glove in a way that keeps it comfortable and responsive. 

Arduino

The Arduino will:

• • Continuously read values from the capacitive touch sensor on the glove’s palm
    
  • Detect when the correct fingertips touch the sensor to register the Spider-Man web-shooting gesture
    
  • Send a signal to p5.js via Serial whenever the web-shooting gesture is made or released

This creates a one-way connection from the glove to the game, letting the hand gesture directly control Spider-Man’s web-shooting action in p5.js.

P5.js

• • Receive the readings from Arduino
  • Detect the web-shooting gesture to attach/release the web
  • Draw Spider-Man as a sprite moving across a simplified NYUAD campus
  • Apply physics for swinging: gravity, momentum, web forces

Currently, the p5 prototype is in progress. I plan to make campus look like the setting of a playstation game, by putting pictures of campus and asking AI to game-ify.

Progress So Far

I have the basic structure of the Spider-Man glove game implemented in p5.js. This includes multiple screens (intro, instructions, game), a building with target points, and the beginning logic for shooting a web. Right now, the web-shooting gesture is simulated with the SPACE key, which triggers a web from Spider-Man to a random target for a short duration. The game is set up so that the Arduino input from the capacitive touch sensor can be integrated later to replace the key press.

 

 

For this week’s assignment, Zeina and I worked on three different exercises that focused on serial communication.

1- Make something that uses only one sensor  on Arduino and makes the ellipse in p5 move on the horizontal axis, in the middle of the screen, and nothing on arduino is controlled by p5

ARDUINO CODE

void setup() {
  Serial.begin(9600);
}

void loop() {
  int sensorValue = analogRead(A1); // 0–1023
  Serial.println(sensorValue);             // send to p5.js
  delay(50);
}

P5 CODE

let port;
let connectBtn;
let baudrate = 9600;
let lastMessage = "";
let sensorValue = 0;

function setup() {
  createCanvas(400, 400);
  background(220);

  //Setting the global variable port to a new serial port instance inside setup:
  port = createSerial();

  // we can open ports we have used previously without user interaction
  let usedPorts = usedSerialPorts(); //array of used ports
  if (usedPorts.length > 0) {  
    port.open(usedPorts[0], baudrate); //if any used port is in the array, open that port with 9600 baudrate
  }

  // any other ports (new ones) can be opened via a dialog after user interaction (see connectBtnClick below)
  connectBtn = createButton("Connect to Arduino");
  connectBtn.position(width/2, 270);
  connectBtn.mousePressed(connectBtnClick);

}

function draw() {
  background("white");
 
  // Read from the serial port. This is a non-blocking function. If a full line has come in (ending in \n), it returns that text. If the full line is not yet complete, it returns an empty string "" instead.
  let str = port.readUntil("\n");
  if (str.length > 0) {   // if str -a string- has any characters
    // print(str);
    lastMessage = str;
    sensorValue = int(lastMessage);  
  }
 
  //draw ellipse mapped to horizontal axis
  let x = map(sensorValue, 0, 1023, 0, width);  
  ellipse(x, height / 2, 40, 40);
 
  // Display the most recent message
  text("Last message: " + lastMessage, 10, height - 20);

  // change button label based on connection status
  if (!port.opened()) {
    connectBtn.html("Connect to Arduino");
  } else {
    connectBtn.html("Disconnect");
  }
}

function connectBtnClick() {
  if (!port.opened()) {
    port.open("Arduino", baudrate);
  } else {
    port.close();
  }
}

 

 

2-Make something that controls the LED brightness from p5

DEMO

IMG_8392 (2)

ARDUINO CODE

// Week 12 Example of bidirectional serial communication

int leftLedPin = 2;
int rightLedPin = 5;

void setup() {
  Serial.begin(9600);

  pinMode(LED_BUILTIN, OUTPUT);
  pinMode(leftLedPin, OUTPUT);
  pinMode(rightLedPin, OUTPUT);

  digitalWrite(leftLedPin, HIGH);
  digitalWrite(rightLedPin, HIGH);
  delay(200);
  digitalWrite(leftLedPin, LOW);
  digitalWrite(rightLedPin, LOW);

  while (Serial.available() <= 0) {
    digitalWrite(LED_BUILTIN, HIGH);
    Serial.println("0,0");
    delay(300);
    digitalWrite(LED_BUILTIN, LOW);
    delay(50);
  }
}

void loop() {
  while (Serial.available()) {
    digitalWrite(LED_BUILTIN, HIGH);

    int left = Serial.parseInt();
    int right = Serial.parseInt();

    if (Serial.read() == '\n') {

      // -----------------------
      // ONLY CHANGE IS HERE:
      // -----------------------
      digitalWrite(leftLedPin, left);     // left stays ON/OFF
      analogWrite(rightLedPin, right);    // right is now BRIGHTNESS (0–255)
      // -----------------------

      int sensor = analogRead(A0);
      delay(5);
      int sensor2 = analogRead(A1);
      delay(5);
      Serial.print(sensor);
      Serial.print(',');
      Serial.println(sensor2);
    }
  }
  digitalWrite(LED_BUILTIN, LOW);
}

p5 CODE

let port; // making a var to hold the serial port
let baudrate = 9600; // speed for talking to arduino
let brightnessSlider; // slider to pick brightness
let smoothBrightness = 0; //transition into the brightness instead of jumping

function setup() {
  createCanvas(400, 200); // just making a small canvas for ui
  textSize(18); // bigger test

  brightnessSlider = createSlider(0, 255, 0); // slider from 0 to full bright
  brightnessSlider.position(20, 80); // where it shows up on screen
  brightnessSlider.style('width', '200px'); // make it a bit wider

  port = createSerial(); // create a serial object so we can connect to arduino

  let used = usedSerialPorts(); // check if we already used a port before
  if (used.length > 0) {
    port.open(used[0], baudrate); // auto connect to the last used port
  }
}

function setupSerial() {
  if (!port.opened()) { // if no connection yet
    port.open("Arduino", baudrate); // try to open one
  } else {
    port.close(); // if already open then close it (toggle)
  }
}

function draw() {
  background(240); // light grey

  if (!port.opened()) {
    text("Press SPACE to connect", 20, 30); // tell the user what to do
  } else {
    text("Connected!", 20, 30); // connection message
  }

  let target = brightnessSlider.value(); // get the slider value

  // do transitional brightness
  smoothBrightness = lerp(smoothBrightness, target, 0.07);
 

  text("Brightness: " + int(smoothBrightness), 20, 70); // show the number

  // actually send the brightness to the arduino
  if (port.opened()) {
    let sendString = "0," + int(smoothBrightness) + "\n"; // left=0 right=smooth
    port.write(sendString); // send it over serial
  }
}

function keyPressed() {
  if (key === " ") {
    setupSerial(); // hitting space toggles the port connection
  }
}

3-Take the gravity wind example (https://editor.p5js.org/aaronsherwood/sketches/I7iQrNCul) and make it so every time the ball bounces one led lights up and then turns off, and you can control the wind from one analog sensor

ARDUINO CODE

void setup() {
  pinMode(2, OUTPUT);
  Serial.begin(9600);
}

void loop() {
  int sensorValue = analogRead(A1);
  Serial.println(sensorValue);

  //check for bounce
  if (Serial.available() > 0) {
    //if 1, light up led
    if (Serial.parseInt() == 1) {            
      digitalWrite(2, HIGH);
      delay(100);                          
      digitalWrite(2, LOW);
    }
  }
  delay(10);
}

P5.JS CODE

let velocity;
let gravity;
let position;
let acceleration;
let wind;
let drag = 0.99;
let mass = 50;
let port;
let connectBtn;
let baudrate = 9600;
let lastMessage = "";
let sensorValue = 0;

function setup() {
  createCanvas(640, 360);
  noFill();
  position = createVector(width / 2, 0);
  velocity = createVector(0, 0);
  acceleration = createVector(0, 0);
  gravity = createVector(0, 0.5 * mass);
  wind = createVector(0, 0);

  //Setting the global variable port to a new serial port instance inside setup:
  port = createSerial();

 
  // we can open ports we have used previously without user interaction
  let usedPorts = usedSerialPorts(); //array of used ports
  if (usedPorts.length > 0) {  
    port.open(usedPorts[0], baudrate); //if any used port is in the array, open that port with 9600 baudrate
  }
// any other ports (new ones) can be opened via a dialog after user interaction (see connectBtnClick below)
  connectBtn = createButton("Connect to Arduino");
  connectBtn.position(width/2, 270);
  connectBtn.mousePressed(connectBtnClick);
}

function draw() {
  background(255);
 
  // Read from the serial port. This is a non-blocking function. If a full line has come in (ending in \n), it returns that text. If the full line is not yet complete, it returns an empty string "" instead.
  let str = port.readUntil("\n");
  if (str.length > 0) {   // if str -a string- has any characters
    // print(str);
    lastMessage = str.trim();
    sensorValue = int(lastMessage);    
  }
 

  //wind controlled by analog sensor
  wind.x = map(sensorValue, 0, 1023, -1, 1);
  console.log("Sensor value: " + sensorValue);
 

  applyForce(wind);
  applyForce(gravity);

  velocity.add(acceleration);
  velocity.mult(drag);
  position.add(velocity);
  acceleration.mult(0);

  ellipse(position.x, position.y, mass, mass);

  // Bounce detection
  if (position.y > height - mass / 2) {
    velocity.y *= -0.9; // Dampening
    position.y = height - mass / 2;

    //send bounce signal to Arduino
    port.write("1\n");
  }
// Display the most recent message
  text("Last message: " + lastMessage, 10, height - 20);

  // change button label based on connection status
  if (!port.opened()) {
    connectBtn.html("Connect to Arduino");
  } else {
    connectBtn.html("Disconnect");
  }
}

function applyForce(force){
  // Newton's 2nd law: F = M * A
  // or A = F / M
  let f = p5.Vector.div(force, mass);
  acceleration.add(f);
}

function keyPressed(){
  if (keyCode==LEFT_ARROW){
    wind.x=-1;
  }
  if (keyCode==RIGHT_ARROW){
    wind.x=1;
  }
  if (key==' '){
    mass=random(15,80);
    position.y=-mass;
    velocity.mult(0);
  }
}

function connectBtnClick() {
  if (!port.opened()) {
    port.openDialog(); // user selects port
  } else {
    port.close();
  }
}

Spidey-Sense: An Interactive Web Swing

For my final project, I’m making something inspired by my favorite superhero, Spider-Man. The idea is that you can “swing” across a virtual city and shoot webs using hand movements. The system reacts in real time, so when you move your hand left or right, Spider-Man moves with you, and a web shoots when you do the classic Spider-Man hand gesture.

I’m using a distance sensor (or flex sensor) with Arduino to read the hand movements and send the data to P5.js. P5 handles all the graphics, like Spider-Man swinging across a city skyline with sprites and visuals inspired by  and possibly taken from the Spider-Man PS5 game. And it’s bidirectional, when you hit a target, P5 can send a signal back to Arduino to light up an LED or vibrate a tiny motor, so you get physical feedback too. Maybe I could use a motor to create a fan-like effect, so the user feels the wind from swinging around the city, too.

I read this text with my brother in mind, he has autism, and I often notice how the design of everyday objects and spaces affects him. Small details like lighting, textures, and how objects feel to the touch can make a big difference in his comfort and understanding of the world. When he broke his leg, for example, he struggled to make sense of the cast, and the experience reminded me how design interacts with perception, comprehension, and well-being.

This reading made me reflect on my own assumptions about design. I realized that when I work on projects, I often focus on what makes sense to me, without considering the diverse needs of other users. Thinking about my brother made me ask: how could I design with someone else’s perspective in mind? How could I make objects and environments more inclusive, without losing creativity or aesthetic appeal?

It also made me notice the subtle ways society treats differences. Glasses, once stigmatized, are now fashion statements, yet many other assistive tools are still hidden or minimized. This made me think about how design can either reinforce stigma or remove it. For my brother, thoughtful design could mean the difference between feeling overwhelmed and feeling calm in his daily life.

Finally, the reading helped me see inclusion in a broader sense. Simple, thoughtful design isn’t just functional; it can enhance comfort, independence, and confidence. I noticed how many spaces we use aren’t made with accessibility in mind, though some places are beginning to shift. There’s a mall in Dubai now that was designed with Autism friendliness in mind. It made me think about my own role as a designer and how I should approach projects with attention to diverse experiences and an awareness of cognitive and sensory differences.

I really agreed with what Bret Victor was saying in “A Brief Rant on the Future of Interaction Design.” The piece did exactly what it was meant to do; it made me aware of something I never really saw as a problem before. When he talked about “Pictures Under Glass,” I honestly felt a bit called out. I’ve never questioned how flat and disconnected our interactions with screens have become; I just accepted them as normal.

Working with Arduino this semester made his point hit even harder. For some reason, it feels so much more rewarding than coding in p5, maybe because I can actually touch and see what I’m creating. It’s not just visuals on a screen; it exists in the real world. Even our first project, where we had to design creative switches that didn’t rely on hands, felt like a step toward the kind of thinking Victor wants designers to embrace.

I don’t know enough about the field to say whether designs nowadays are really “timid,” but I get where he’s coming from. The black-and-white photography analogy stuck with me; it shows how something can be revolutionary for its time but still just a transition toward something better. This reading made me rethink what “interaction” really means and imagine a future for technology that feels more connected to the body, not just the screen.

Inspiration:

For this week’s project, the main inspo for our instrument was Stormae’s Song “Alors on Danse”. We were mainly inspired by the way that the songs main notes are split into 3 notes of varying pitches, with one sound constantly playing in the background. For that reason we varied the pitches of the three sounds our project produces with a 4th note that is constantly playing when the button is pressed.

Concept:

For this week’s project, we used 3 light sensors to play sounds on the piezo speaker, with one note being played constantly when a button is clicked. With the light sensor, once the user’s finger covers the sensor that is when the note is played. Furthermore, we have three sensors each of which plays a different pitch on the piezo speaker. The condition that allows for this is the reading of the sensor in comparison to the threshold we defined. An additional component we added was the button that allows for the sounds to be played on the piezo speaker and then stopped once the button is pressed again.

Code:

int photoPins[3] = {A0, A1, A2};// first we define a list of integers holding the analog pins
int buttonPin = 2; // digi pin 2 for the buttons
int piezzoPin = 8; //digi pin 8 for the piezzo speaker

int threshold = 700; //this is the threshold fo rte light/no light intensity that worked wit our light sensors in our environment/lighting

bool prevPhoto[3] = {false, false, false}; //keeping track of whether the light sebsir was interacted with or not false initially
bool prevButton = false; //initially false
bool buttonState = false;//initially false

void setup() {
  pinMode(buttonPin, INPUT_PULLUP); //for the button pint as an input for the arduino
  pinMode(piezzoPin, OUTPUT); //setting the buzzor pin as output so the arduino sneds the sound signal
  Serial.begin(9600); // serial monitor for debugging
}

void loop() {
  for (int i = 0; i < 3; i++) { //looping over the 3 sensors to reasd their analog value
    int value = analogRead(photoPins[i]);
    bool tapped = value < threshold; //comparing the value captured by the sensor and the defined threshold
    if (tapped && !prevPhoto[i]) { //checking for tap in the current state compared to prev
      if (i == 0) tone(piezzoPin, 440, 200); // translates to A0
      if (i == 1) tone(piezzoPin, 523, 200); // translates to A1
      if (i == 2) tone(piezzoPin, 659, 200); // translates to A2
    }
    prevPhoto[i] = tapped; //MAKING SURE TO NOTE it as tapped to have a singular tap rather than looping

    Serial.print(value); //serial print
    Serial.print(",");
  }

  bool pressed = digitalRead(buttonPin) == LOW; //setting the reading of the button to low meaning the button is pressed
  if (pressed && !prevButton) { //when the button is pressed state changes from not pressed(false) to presssed(true)
    buttonState = !buttonState;
    if (buttonState) tone(piezzoPin, 784);// if toggled on play a continuoue G5 tone
    else noTone(piezzoPin); //otherwise stop the buzzer
  }
  prevButton = pressed;

  Serial.println(pressed ? "1" : "0"); //for monitoring purposes

  delay(50);//short delay
}

Disclaimer: Some AI/ChatGPT was used to help with debugging and allowing multiple elements to work cohesively.

More Specifically:

1- When trying to debug, to check if button is pressed is true on the serial monitor (this line: Serial.println(pressed ? “1” : “0”); //for monitoring purposes)
2- Recomended values for frequency in hertz to mimic Alors on Danse (if (i == 0) tone(piezzoPin, 440, 200); // translates to A0 if (i == 1) tone(piezzoPin, 523, 200); // translates to A1 if (i == 2) tone(piezzoPin, 659, 200); // translates to A2) The SECOND parameter

Schematic: Demo:

IMG_8360

Future Improvements:

As for the future improvements, one main thing we wanted to capture in this project is being to overlap the sounds, but since we were working with one piezo speaker, we were not able to do that. To address this we aim to learn more about how we can maybe start playing the sounds from our laptops instead of the physical speaker we connect to the Arduino. Other improvements could be exploring how we can incorporate different instrument sounds maybe and create an orchestra like instrument.

Concept

The idea behind this project is to create a simple mood lamp. You can switch between different colors using buttons and control the brightness with a potentiometer.

An important feature is that when you set a certain brightness for one color, that same brightness carries over when you switch to a different color. This makes the lamp feel intuitive to use. For example, if the lamp is set to 50% brightness on blue, switching to red keeps it at 50% instead of jumping to full brightness. This allows smooth color transitions without having to constantly readjust brightness.

The project combines digital inputs (buttons) and an analog input (potentiometer) to control analog outputs (LED brightness) in a simple but effective way.

https://github.com/kzeina/Intro-To-IM

week9 circuit demo

Code I’m Proud of

//use internal pull-up resistor (button connects to 5v when pressed)

   //default state is high, pressed = low

   pinMode(buttonPins[i], INPUT_PULLUP);

I used INPUT_PULLUP, which is a built-in resistor in Arduino that helps stabilize button inputs. It works by keeping the pin in a known state , HIGH when unpressed and LOW when pressed, preventing it from randomly floating between readings. I discovered it through ChatGPT while debugging an issue where my LEDs were turning on unexpectedly. I learned that because my setup only used buttons as inputs (without any pull-up resistors), the readings were unstable, causing the LEDs to light up when they shouldn’t.

 

void loop() {

 //loop through each button to check if it’s pressed

 for (int i = 0; i < 3; i++) {

   //button pressed (low because of input_pullup)

   if (digitalRead(buttonPins[i]) == LOW) { 

     //set this led as the active one

     activeLED = i;                         

     //turn off all other leds

     for (int j = 0; j < 3; j++) {

       if (j != i) analogWrite(ledPins[j], 0);

     }

   }

 }

 //if an led is active, adjust its brightness based on the potentiometer

 if (activeLED != -1) {

   //read analog value from potentiometer (0–1023)

   int sensorValue = analogRead(A0);            

   //control brightness of the active led

   analogWrite(ledPins[activeLED], sensorValue/4);  

 }

}

Frankly, I’m proud of almost all my code for this project. My background in computer science definitely helped me figure out the logic more easily. I used a nested loop that goes through each button (which I stored in an array, along with the LED pins) and checks its state. When a button is pressed, its corresponding LED turns on while all the others turn off. To let the potentiometer control the brightness of the active LED, I created a variable called activeLED. As long as it’s not set to -1 (meaning no LED is active), the code reads the potentiometer’s value and uses it to adjust the LED’s brightness.

Future Improvements

For future improvements, I’d like to make the interaction feel smoother and more dynamic. One idea is to add a toggle feature so pressing the same button again turns its LED off instead of just switching between them. I’d also love to make the LEDs fade in and out when changing colors to create a softer transition effect. It could be interesting to display which LED is currently active on an LCD screen or even through the Serial Monitor. Finally, adding sound feedback, like a small beep when an LED is switched, would make the experience more interactive and responsive.

Physical Computing Themes & Making Interactive Art

Reading Tom Igoe’s “Physical Computing’s Greatest Hits (and Misses)” and “Making Interactive Art: Set the Stage, Then Shut Up and Listen” really resonated with me because it reminded me of my own experience designing interactive projects. My first big interactive project was the midterm photobooth, and I remember feeling overwhelmed by all the features I wanted to implement. I was worried that what I thought was straightforward and easy to use might not actually be intuitive for other people. This made me realize how important it is to balance functionality, design, and user experience, something both readings emphasize in different ways.

From the first reading, I was particularly drawn to mechanical pixels, because it reminded me of the Al Bahr Towers in Abu Dhabi, what I like to call the “pineapple towers”, whose scaled windows open and close in response to sunlight. It made me think about how even simple, repeating elements can be visually engaging while also responding to a system or environment. I started to see that originality doesn’t always come from inventing something completely new, but from how you execute and contextualize it.

Igoe’s second reading really made me reflect on my tendency to over-explain. I often want to make sure people ‘get it,’ but the article reminded me that interactive art is a conversation, not a lecture. It’s about providing context, affordances, and suggestions, then letting the audience explore, discover, and even surprise you with unexpected interactions. I like that this perspective treats the audience as active participants rather than passive observers, which matches my belief that both aesthetics and engagement are equally important. If a project doesn’t look appealing, people won’t approach it. If it doesn’t engage them, it loses its purpose.

What stood out most to me is how much trust you have to put in the design and in the audience. Seeing a project unfold in unexpected ways isn’t a failure; it’s part of the collaborative experience. I also realized that while I enjoy seeing people interact in ways I hadn’t anticipated, it only works as long as the interaction isn’t harmful.

Reading about Margaret Hamilton alongside Don Norman’s ideas made me reflect a lot on my own experiences as an IM major and CS minor. Hamilton’s work is astounding, not just in its technical brilliance, but in the stakes involved. I can’t imagine the pressure of knowing that a single error in your code could be catastrophic. That hit home for me when I thought back to my summer internship: I often felt anxious during functional testing, making sure that a feature actually worked before even thinking about optimizing it. Like Hamilton, I had to prioritize reliability over aesthetics at first, knowing that structure and efficiency could come later. Similarly, when working with Professor Moore on designing a robot extension, our creative vision for the robot was limited by its movements and what was functional, which was frustrating but also helped me come to the realization that function always had to come first.
Norman’s ideas about aesthetics and human experience made me notice the other side of this balance. I’ve realized that I learn better and retain information more effectively when it’s presented aesthetically, which is why I rewrite notes or design presentations carefully. I genuinely think it affects focus and engagement. Reflecting on Hamilton through Norman’s lens, I see that even in high-stakes work, there can be an “internal aesthetics” for the creator, like the satisfaction of elegantly structured, reliable code. This reading also made me think about how much balance is a part of my own work. I often want to implement my vision exactly, but my capabilities, time, and technical constraints sometimes get in the way. It can be frustrating, but I see now that navigating this tension, between function, reliability, and human experience, is a universal part of creating, whether you’re sending astronauts to the Moon or designing a robot extension. Hamilton’s example pushes me to aim for excellence under pressure, while Norman reminds me that design is also about how people, whether users or creators, experience the work.