HAILSTORM HAIVOC

My game concept simulates driving safely through a hailstorm in AbuDhabi. As a player, you control a toy car, displayed on the screen, by physically moving it left or right to dodge falling hailstones. This game is designed to reflect the unpredictable nature of real hailstorms and incorporates real-time physical interaction through a toy car and Arduino sensors, offering a unique and engaging gameplay experience, also symbolizing the hailstorm in Abu Dhabi months ago.

Implementation Overview

I built the game using p5.js for the visual components and game logic, and Arduino for the physical interaction aspects. The Arduino setup uses a pushbutton to start or restart the game and an HC-SRO ultrasonic sensor to determine the position of the toy car, which translates into the car’s movement on the screen. These inputs are sent to the p5.js application through serial communication, allowing the player’s physical movements to directly influence the gameplay.

Interaction Design Description

The interaction design focuses on tangible interaction, where the physical movement of the toy car (left or right) translates to the movement of the car on the screen. This method fosters more engaging and intuitive gameplay. An arcade button connected to the Arduino allows players to start or restart the game easily, making the interface user-friendly and accessible.

Arduino Code Explanation

In my Arduino code, I manage inputs from the ultrasonic sensor and the button. The ultrasonic sensor measures the distance of an object (the toy car) from the sensor, and this measurement is used to control the car’s position on the p5.js screen. The button input is debounced to avoid processing multiple unintended signals, used to start or restart the game, and toggles an LED for visual feedback. Serial communication sends the button press count and distance measurement to the p5.js application.

p5.js Code Explanation

My p5.js code is responsible for creating the visual representation of the game—rendering the car, hailstones, and other visual elements on the screen. It also handles the game logic, such as detecting collisions between the car and hailstones, updating the game state based on Arduino inputs, and managing game timers and scores.

The game has 3 screens:

The Main menu where the player is told the directions to play, the winning screen which comes up after 10 seconds of the user playing, and the game over screen which pops up if the car collides with the hailstones.

Communication Between Arduino and p5.js

I achieve communication between the Arduino and p5.js through serial communication. The Arduino continuously sends data from the button and the ultrasonic sensor to the p5.js application, which reads this data to update the game state accordingly. The p5.js listens for serial data, parses it, and uses these inputs to control the car’s movements and manage game controls like start and restart.

Important Images

Aspects I’m Particularly Proud Of

I am proud of several key accomplishments in this project:

• • Successfully integrating physical components with a digital interface, which enhanced the interactive gaming experience.
  • Overcoming the challenges associated with serial communication between Arduino and p5.js, a complex aspect of hardware-software integration.
  • Completing the project within a limited timeframe and being able to innovate with a unique approach to game design and interaction.
  • As well as my setup with the street and box; I really enjoyed making the box and the street, adding an extra layer of creativity

Future Improvement Areas

For future enhancements, I could consider:

• • Implementing sound and music for the game, I initially had music in the game but I decided to remove it, since the exhibition is already quite chaotic and the music won’t be heard
  • Enhancing the game’s visual and sound effects to create a more immersive experience.
  • Implementing additional gameplay features, such as different levels of difficulty or various weather conditions affecting gameplay.
  • Exploring different sensors or refining the calibration of the HC-SRO4 sensor. As it was slightly glitchy at first, but I managed to fix it.

Here is my Arduino code:

const int trigPin = 9;
const int echoPin = 10;

// Arduino code for button, which detects the counts
const int buttonPin = 2;  // the number of the pushbutton pin
const int ledPin = 3;    // the number of the LED pin

// variables will change:
int buttonState = 0;         // variable for reading the pushbutton status
int lastButtonState = HIGH;  // variable for reading the last pushbutton status
unsigned long lastDebounceTime = 0;  // the last time the output pin was toggled
unsigned long debounceDelay = 50;    // the debounce time; increase if the output flickers
int pressCount = 0;  // count of button presses


void setup() {
  pinMode(trigPin, OUTPUT);
  pinMode(echoPin, INPUT);
   pinMode(ledPin, OUTPUT);   // initialize the LED pin as an output
  pinMode(buttonPin, INPUT_PULLUP);  // initialize the pushbutton pin as an input with internal pull-up resistor
  Serial.begin(9600);
}

void loop() {
  float distance = getDistanceCm();  // Get the distance in cm
  int reading = digitalRead(buttonPin);

  // check if the button state has changed from the last reading
  if (reading != lastButtonState) {
    // reset the debouncing timer
    lastDebounceTime = millis();
  }

  if ((millis() - lastDebounceTime) > debounceDelay) {
    // if the button state has changed:
    if (reading != buttonState) {
      buttonState = reading;

      // only toggle the LED if the new button state is LOW
      if (buttonState == LOW) {
        digitalWrite(ledPin, HIGH);
        pressCount++;  // increment the press count
      } else {
        digitalWrite(ledPin, LOW);
      }
    }
  }

  // save the reading. Next time through the loop, it will be the lastButtonState:
  lastButtonState = reading;
  
  Serial.print(pressCount);  // print the count to the serial monitor
  Serial.print(",");
  Serial.println(distance);          // Print the distance to the Serial monitor
  delay(100);                        // Short delay before next measurement
}

float getDistanceCm() {
  // Trigger the measurement
  digitalWrite(trigPin, LOW);
  delayMicroseconds(2);
  digitalWrite(trigPin, HIGH);
  delayMicroseconds(10);
  digitalWrite(trigPin, LOW);

  // Calculate the distance based on the time of echo
  float duration = pulseIn(echoPin, HIGH);
  float distance = (duration * 0.0343) / 2;

  return distance;
}

And this is the code from P5:

//Add this in index.html
//   <!-- Load the web-serial library -->
//     <script src="p5.web-serial.js"></script>
   
// Go download the web serial at https://github.com/Pi-31415/Intro-To-IM/blob/main/p5.web-serial.js

// Declare a variable to hold the smoothed value
let smoothedDistance = 0;
let gameMode = 0; // Variable to store the current game mode
var landscape; // Variable to store the landscape graphics
var car_diameter = 15; // Diameter of the ball
var bomb_diameter = 10; // Diameter of the bombs
var cardistancex;
var ypoint;
var zapperwidth = 6; // Width of the zapper
var numofbombs = 3; // Number of bombs
var bombposX = []; // Array to store X positions of bombs
var bombposY = []; // Array to store Y positions of bombs
var bombacceleration = []; // Array to store acceleration of each bomb
var bombvelocity = []; // Array to store velocity of each bomb
var time = 0; // Variable to track time, usage context not provided
var timeperiod = 0; // Variable to store a time period, usage not clear without further context
//var score = 0; // Variable to store the current score
var posX; // X position, usage context not provided
var inMainMenu = true; // Boolean to check if the game is in the main menu
//var prevScore = 0; // Variable to store the previous score
let font; // Variable to store font, usage context not provided
let introgif;
let gameovergif;
let gif3;
let survivedgif;
let countdownTimer = 10; // Countdown timer starting from 30 seconds
let serial; // Declare a serial port object
let latestData = "waiting for data"; // Latest data received
let gameovernow = false;

//CONNECTION

let clickCount = 0;
let previousClickCount = 0; // Store the previous click count
let distanceReal = 255;

let ignorefirstclick = false;

let gameovergifLarge;

function preload() {
  introgif = createImg(
    "https://media.giphy.com/media/v1.Y2lkPTc5MGI3NjExNTA4MG42MWlhdWV3Y2cyZ3U1cTFqZHhpbHp1amcweDhjYzhkcHBkYyZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/35cSlj5ELlzGSg0ZYM/giphy.gif"
  );
  introgif.hide();
  survivedgif = createImg(
    "https://media.giphy.com/media/v1.Y2lkPTc5MGI3NjExdm1yeWt3aGlteWZkcHB3czk3Ym81YWtrZTVtb29pMng2NW83bnF4bCZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/LY3JbLuhmjDVfCcCxh/giphy.gif"
  );
  survivedgif.hide();
  gameovergif = createImg(
    "https://media.giphy.com/media/v1.Y2lkPTc5MGI3NjExZHo1dmNrdzQ5NnYycWdvMjBqOGt1Zmg0MTdxZHQ4eHAyZGZrMDZtbCZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9cw/hp9wzCTbGeGfIkXE6a/giphy.gif"
  );
  gameovergif.hide();

  gameovergifLarge = createImg("https://intro-to-im.vercel.app/afra/gameoer.gif");
  gameovergifLarge.hide();
  //Temp
  gif3 = loadImage("https://intro-to-im.vercel.app/afra/bggif.gif");

  font = loadFont("fonts/Inconsolata_Condensed-Light.ttf");
  car = loadImage("car.png");
  car2 = loadImage("car2.png");
}

// Cloud class starts
class Cloud {
  constructor(x, y, speed, stepSpeed, scale) {
    this.x = x;
    this.y = y;
    this.scale = scale; // Add scale property
    this.speed = speed;
    this.stepSpeed = stepSpeed;
    this.step = 0;
    this.facingRight = false; // Initially moving to the left
    this.animationTimer = null;
  }

  move() {
    if (this.facingRight) {
      this.x += this.speed;
    }
  }

  display() {
    push();
    if (!this.facingRight) {
      scale(-this.scale, this.scale); // Apply scale with horizontal flip
      image(oneDimensionarray[this.step], -this.x, this.y);
    } else {
      scale(this.scale, this.scale); // Apply scale
      image(oneDimensionarray[this.step], this.x, this.y);
    }
    pop();
  }

  advanceStep() {
    this.step = (this.step + 1) % 8;
  }

  startAnimation() {
    this.facingRight = true;
    clearInterval(this.animationTimer);
    this.animationTimer = setInterval(() => this.advanceStep(), this.stepSpeed);
  }

  stopAnimation() {
    this.facingRight = false;
    clearInterval(this.animationTimer);
  }
}

let clouds = [];
// Cloud class ends


// Define a maximum boundary for the distance
const maxDistance = 600;  // Set this to whatever maximum value makes sense in your context

function mapDistance(distanceReal) {
  // Define the smoothing factor (alpha). Smaller values make the motion smoother but less responsive.
  const alpha = 0.2;

  // Calculate the target position without smoothing
  const targetPosition = (640 * (distanceReal - 3)) / 17;

  // Apply exponential smoothing
  smoothedDistance = alpha * targetPosition + (1 - alpha) * smoothedDistance;

  // Ensure the smoothed distance does not exceed the maximum allowed distance
  if (smoothedDistance > maxDistance) {
    smoothedDistance = maxDistance;
  }

  return smoothedDistance;
}


function setup() {
  createCanvas(640, 480);
  textAlign(CENTER);

  gif3.resize(640 * 2, 480 * 2);

  var temp00 = 0,
    temp01 = -20;

  // A while loop that increments temp01 based on temp00 until temp01 is less than the canvas height
  while (temp01 < height) {
    temp00 += 0.02; // Increment temp00 by 0.02 in each loop iteration
    temp01 += temp00; // Increment temp01 by the current value of temp00
    timeperiod++; // Increment timeperiod in each iteration
  }

  // Calculate the initial position of posX based on zapperwidth and car_diameter
  posX = zapperwidth + 0.5 * car_diameter - 2;

  // Set cardistancex and ypoint relative to the width and height of the canvas
  cardistancex = 0.7 * width; // Set cardistancex to 70% of the canvas width
  ypoint = height - 0.5 * car_diameter + 1; // Set ypoint based on the canvas height and car_diameter

  initbombpos(); // Call the initbombpos function (presumably initializes bomb positions)

  imageMode(CENTER); // Set the image mode to CENTER for drawing images centered at coordinates

  // Initialize variables for width and height based on

  // Create 3 clouds with horizontal offsets, different speeds and scales
  clouds.push(new Cloud(width / 8, height / 9, 0, 100, 0.9)); // First cloud
  clouds.push(new Cloud((2 * width) / 5, height / 9, 0, 100, 1.2)); // Second cloud
  clouds.push(new Cloud((2 * width) / 2, height / 9, 0, 200, 1.0)); // Third cloud
}

//Serial Read
function readSerial(data) {
  ////////////////////////////////////
  //READ FROM ARDUINO HERE
  ////////////////////////////////////

  if (data != null) {
    // make sure there is actually a message
    // split the message
    let fromArduino = split(trim(data), ",");
    // if the right length, then proceed
    if (fromArduino.length == 2) {
      // only store values here
      // do everything with those values in the main draw loop

      // We take the string we get from Arduino and explicitly
      // convert it to a number by using int()
      // e.g. "103" becomes 103
      clickCount = int(fromArduino[0]);
      distanceReal = parseFloat(fromArduino[1]);
    }
  }
}

function draw() {
  clear();
  //Establish Serial
  if (!serialActive) {
  } else {
    text("Connected", 20, 30);
    // Print the current values
    console.log("clickCount = " + str(clickCount), 20, 50);
    text("distanceReal = " + str(distanceReal), 20, 70);
  }

  // Check if clickCount has increased
  if (clickCount > previousClickCount) {
    if (ignorefirstclick) {
      simulateMouseClick(); // Call your function that simulates a mouse click
    }
    ignorefirstclick = true;
  }
  previousClickCount = clickCount; // Update previousClickCount

  // background(gif3);
  // displayTimer();
  // updateTimer();

  if (gameMode == 0) {
    //Main Menu
    textFont(font);

    textSize(50); // Larger text size for the game title
    textAlign(CENTER, CENTER); // Align text to be centered
    text("HAILSTORM HAVOC", width / 2, height / 2 - 40);
    textSize(16); // Smaller text size for the directions
    text(
      "DIRECTIONS:\n click mouse to dodge hail\n the longer the press the further right\n the car will go\n\n AVOID the red line - crossing it means game over",
      width / 2,
      height / 2 + 50
    );
    textSize(20);
    text("Click to start!", width / 2, height / 2 + 140);
    introgif.show();
    introgif.position(0, 0);
    introgif.size(width, height);
  } else if (gameMode == 1) {
    //Actual game
    // gif3.show();
    image(gif3, 20, 20);
    displayTimer();
    updateTimer();

    introgif.hide();
    survivedgif.hide();
    gameovergifLarge.hide();

    // fill(239, 58, 38);
    // rect(0, 0, zapperwidth, height);
    //scoreUpdate();

    fill(255);
    noStroke();
    for (var i = 0; i < numofbombs; i++) {
      ellipse(bombposX[i], bombposY[i], bomb_diameter, bomb_diameter);
    }

    updatebombpos();
    // ellipse(cardistancex, ypoint, car_diameter, car_diameter);
    //Betwen 0 and 640
    let cardistancex = mapDistance(distanceReal);
    image(car, cardistancex, ypoint - 30, car_diameter * 5, car_diameter * 5);

    if (cardistancex <= posX || bombCollistonTest()) {
      //gameover(); // Call the gameover function if either condition is true
      gameMode = 3;
    }

    time += 1;
    // if (frameCount % 60 == 0) {
    //   score++; // Increase score by 1
    // }
    checkGameOver();
    // gif3.show();
    // gif3.position(0, 0);
    // gif3.size(width, height);
  } else if (gameMode == 2) {
    //Survive
    survivedgif.show();
    survivedgif.position(0, 0);
    survivedgif.size(width, height);
    restartGame();
    // displayWin();
  } else if (gameMode ==3){
    //GameOver
    gameovergifLarge.show();
    gameovergifLarge.position(0, 0);
    gameovergifLarge.size(width, height);
    restartGame();
  }
  
}

function displayTimer() {
  if (font) {
    textFont(font); // Set the loaded font for displaying text
  }
  fill(255, 255, 0); // Set the text color to white for visibility
  textSize(30); // Set the text size
  textAlign(RIGHT, TOP); // Align text to the center top
  textStyle(BOLD);
  text("Time: " + countdownTimer, width - 10, 10); // Display the timer on the canvas
}

function updateTimer() {
  if (frameCount % 60 == 0 && countdownTimer > 0) {
    countdownTimer--; // Decrease timer by 1 each second
  }
}
function updatebombpos() {
  // Iterate over each bomb
  for (var i = 0; i < numofbombs; i++) {
    bombvelocity[i] += bombacceleration[i]; // Update the velocity of the bomb by adding its acceleration
    bombposY[i] += bombvelocity[i]; // Update the Y position of the bomb based on its velocity
  }

  if (time > timeperiod) {
    initbombpos(); // Reinitialize the positions of the bombs by calling the initbombpos function
    time = 0;
  }
}

function initbombpos() {
  for (var i = 0; i < numofbombs; i++) {
    bombacceleration[i] = random(0.02, 0.03); // Randomize the acceleration
    bombvelocity[i] = random(0, 5); // Randomize the initial velocity
    bombposX[i] = random(zapperwidth + 0.5 * car_diameter, width); // Randomize the X position within playable area
    bombposY[i] = -bomb_diameter; // Start bombs just above the top of the canvas
  }
}

function bombCollistonTest() {
  // Define the car's bounding box
  let carLeft = cardistancex - car_diameter * 2.5;
  let carRight = cardistancex + car_diameter * 2.5;
  let carTop = ypoint - 20 - car_diameter * 2.5;
  let carBottom = ypoint - 20 + car_diameter * 2.5;

  // Iterate over each bomb to check for a collision
  for (var i = 0; i < numofbombs; i++) {
    // Check if bomb is within the bounding box of the car
    if (
      bombposX[i] >= carLeft &&
      bombposX[i] <= carRight &&
      bombposY[i] >= carTop &&
      bombposY[i] <= carBottom
    ) {
      return true; // Collision detected
    }
  }
  return false; // No collision
} //This function checks for collisions between the player and each bomb by comparing the distance between them to a threshold. If any bomb is too close (within the threshold), it returns true (collision detected). Otherwise, it returns false.

function gameover() {
  gameovernow = true;
  let cardistancex = mapDistance(distanceReal);
  image(car2, cardistancex, ypoint - 30, car_diameter * 5, car_diameter * 5);
  gameovergif.show();
  gameovergif.position(0, 0);
  gameovergif.size(width, height);
}

function keyPressed() {
  if (key == "a") {
    // important to have in order to start the serial con nection!!
    setUpSerial();
  }
}

function simulateMouseClick() {
  console.log("Mouse clicked via Arduino"); // Log or perform actions here
  // You can call any functions here that you would have called in mouseClicked()
  if (gameMode == 0 || gameMode == 2 || gameMode == 3) {
    gameMode = 1;
  }

  //just flipping between modes 0 and 1

  clouds.forEach((cloud) => cloud.startAnimation());
}

function mousePressed() {
  //No mouse press
}

function mouseReleased() {
  clouds.forEach((cloud) => cloud.stopAnimation());
}
function checkGameOver() {
  if (countdownTimer <= 0) {
    gameMode = 2;
  }
}
function restartGame() {
  // Reset all game variables to their initial values
  gameovernow = false;
  gameovergif.hide();
  time = 0;
  //score = 0;
  countdownTimer = 5;
  posX = zapperwidth + 0.5 * car_diameter - 2;
  cardistancex = 0.5 * width;
  ypoint = height - 0.5 * car_diameter + 1;
  initbombpos();
  // Restart the game loop
  loop();
}
//This function resets the game environment and variables to their initial state, essentially restarting the game. It resumes background music, pauses any game over , resets score and time, repositions the player and bombs, and restarts the game loop.

Overall, I’m very proud of my project and I was very happy to see users play my game in the exhibition.

As I was conducting user testing for my game, an actual storm was hitting Abu Dhabi, echoing the storm scenario in my game and making the experience feel all the more relatable.

I enlisted my sister to test the game without any prior instructions, which was a great decision as the game proved to be intuitive—she knew immediately that she needed to press the button. However, since I haven’t yet built the controller box, I had to hold it for her. For the wooden block, I explained that she should imagine it as a car moving on the screen.

To my surprise, she played better than I did! It was really enjoyable to watch my game perform exactly as intended. She managed to figure everything out smoothly; the only hiccup was the wooden block. If it had actually looked like a car, I wouldn’t have needed to explain anything. I also handled connecting to the serial port since she wasn’t familiar with it. For the actual exhibition, I’ll provide short written instructions for the connection to assist anyone if I’m not around.

Area for improvement:

The game runs smoothly overall, but the collision detection between the hail and the car is glitchy. Sometimes it triggers a game over even when it shouldn’t, which is quite frustrating. I plan to resolve this issue before the exhibition for sure.

HAILSTORM HAIVOC CONTINUED

Concept for the project

For my final project, I am building on my midterm project, which involves a game where players dodge falling hail. Initially, I had envisioned this game with a car maneuvering to avoid the hail, but the concept became too complex. Through trial and error, I simplified the gameplay to where players now use the mouse click to move the car away from the hail. This feature unexpectedly also allows control over another sprite, adding a unique twist to the gameplay. Taking this project to the next level, I plan to integrate an arcade button to start and re-start the game and a toy car that players can manipulate/move to control the game car on the screen through a sensor. Differing from the midterm which was consistently playing, my game has an end after a couple seconds of playing where the player would have won and arrived home safely from the storm, and this would enhance the interactive experience and depth of the game.

Description of what your Arduino program will do with each input and output and what it will send to and/or receive from P5:

1. Inputs:
   • HC-SR04 Ultrasonic Sensor: This sensor will measure the distance of an object from it. It uses two pins, TRIG and ECHO, to send out ultrasonic waves and measure the time it takes for the echo to return.
   • Arcade Button: The button will act as a simple digital input, where pressing the button changes its state from LOW to HIGH.
2. Outputs:
   • Serial Data to P5: The Arduino program will continuously send two pieces of data to the P5 sketch via the serial port:
     • Distance measured by the ultrasonic sensor, translated into a value that indicates the position of the car on the screen, hopefully when the toy car is moved left (close to the sensor it will move the car on screen left) and vise versa.
     • The state of the arcade button to control game states (start and restart).

Description of what P5 program will do and what it will send to and/or receive from Arduino:

As of my game, I don’t think P5 is going to send anything to Arduino other than:  Serial Data from Arduino: The P5 sketch receives the distance and button state as a serialized string. Game Controls: Based on the received data, the P5 sketch adjusts the position of the car on the screen and manages the game state (starting and restarting).

I initially gave the idea to professor to make this reading optional since we have a lot on our plate with the final being due, but since its extra credit, it automatically became crucial for me to complete 🙂

So while reading “Design Meets Disability” by Graham Pullin, it has profoundly shifted my view on the design of assistive devices. Pullin advocates for merging functionality with aesthetic appeal, an approach that resonates deeply with current trends in consumer technology. For example, the transformation of eyeglasses from functional items to fashion statements illustrates the potential for assistive devices to follow a similar trajectory.

However, the reality in the market shows a lag in adopting this inclusive design philosophy. Many devices, especially those for less visible disabilities, remain primarily functional, highlighting a gap between Pullin’s ideal and the industry’s current practices. This discrepancy might be due to cost constraints and a lack of awareness among designers about how to integrate aesthetics without compromising functionality.

Reflecting on this, I am more aware of the biases that might exist in the design community and society’s perception of disability. Pullin’s optimism is inspiring, yet it also brings to light the practical challenges of such a vision. It raises crucial questions about balancing cost and style and how the industry can be motivated to embrace this more holistic approach to design.

With Amal and Stefania

Initial connection and wiring from the schematic:

Exercise 1:

This was probably the easiest exercise, we just had to edit the code that was in the Week 12 Lecture Notes.

P5.js Code:

let potentiometer = 0;

function setup() {
  createCanvas(640, 480);
  textSize(18);
}

function draw() {
  background(255);
 
  if (!serialActive) {
    text("Press Space Bar to select Serial Port", 20, 30);
  } else {
    text("Connected", 20, 30);
    text('Potentiometer = ' + potentiometer, 20, 70);
  }

 
  let ellipseX = map(potentiometer, 0, 1023, 0, width);
  fill(0);
  ellipse(ellipseX, height / 2, 50, 50);

function keyPressed() {
  if (key === ' ') {
    setUpSerial();
  }
}

function readSerial(data) {
  if (data != null) {
    let trimmedData = trim(data);
    if (!isNaN(trimmedData)) {
      potentiometer = int(trimmedData);
    }
  }
}

Arduino Code: 

void setup() {
  Serial.begin(9600);

  pinMode(LED_BUILTIN, OUTPUT);
  digitalWrite(LED_BUILTIN, HIGH);
  delay(200);
  digitalWrite(LED_BUILTIN, LOW);
}

void loop() {
  int sensorValue = analogRead(A1);

  Serial.println(sensorValue);

  delay(10);
}

According to the other exercises, we tried multiple times with the code, and tried manipulating it to fit the requirments but we kept getting errors

After multiple trials we gave up and decided to dedicate all our time for our final project, however here is the code that was in progress with multiple errors:

the P5 code was the one giving us errors:

let serial;  
let rVal = 0;
let alpha = 255;
let left = 0;
let right = 0;
let slider;

function setup() {
  createCanvas(640, 480);
  textSize(18);
  
  function setup() {
  console.log('Setting up...');

  serial = new p5.SerialPort();
  serial.on('data', readSerial);
  serial.on('open', onSerialOpen);
  serial.on('error', onSerialError);
  serial.open('COM3');  // Update this to your port
}


  slider = createSlider(0, 255, 127);
  slider.position(10, 10);
  slider.style('width', '380px');

  // Initialize the serial port
  serial = new p5.SerialPort();
  serial.open('COM3'); // change this to your Arduino port
  serial.on('data', readSerial); // function to call when new data arrives
  serial.on('open', onSerialOpen); // function to call when the serial port opens
  serial.on('error', onSerialError); // function to call on error
}

function draw() {
  background(map(rVal, 0, 1023, 0, 255), 255, 255);
  fill(255, 0, 255, map(alpha, 0, 1023, 0, 255));

  if (!serial.isOpen()) {
    text("Press Space Bar to select Serial Port", 20, 30);
  } else {
    text("Connected", 20, 30);
    text('rVal = ' + str(rVal), 20, 50);
    text('alpha = ' + str(alpha), 20, 70);

    if (slider.value() > 127) {
      left = 1;
      right = 0;
    } else {
      left = 0;
      right = 1;
    }

    let sendToArduino = left + "," + right + "\n";
    serial.write(sendToArduino);
  }
}

function keyPressed() {
  if (key === ' ') {
    serial.open('/dev/tty.usbmodem1411'); // open a serial port
  }
}

function readSerial() {
  let data = serial.readLine(); // read the incoming string
  trim(data); // trim off trailing whitespace
  if (!data) return;
  let fromArduino = split(data, ',');
  if (fromArduino.length >= 2) {
    rVal = int(fromArduino[0]);
    alpha = int(fromArduino[1]);
  }
}

function onSerialOpen() {
  console.log('Serial Port Opened');
}

function onSerialError(err) {
  console.log('Serial Port Error: ' + err);
}

Arduino Piano

🌟 Inspiration: 

Music possesses a remarkable capacity to bridge cultural and linguistic divides and unite individuals from diverse backgrounds. Motivated by the notion of promoting interactive experiences and democratizing music creation, we set out to build a one-of-a-kind musical instrument out of buttons and Arduino. Our intention was to enable people to use sound as a creative medium for self-expression, irrespective of their experience with music. We chose the piano as our main inspiration because we could recreate the chords using buttons.

💡 Process:

Using Arduino Uno we wired up buttons to serve as interactive triggers for different musical notes. Each button was assigned a specific pitch or sound, allowing users to create melodies by pressing combinations of buttons. We leveraged our programming skills to code the logic behind the instrument, ensuring seamless functionality and an intuitive user interface.

🚀 Difficulties: 

Although our journey was exciting and innovative, there were certain difficulties. A major challenge we encountered was guaranteeing the reliability and consistency of button pushes. We have to use precise calibration and filtering techniques to get beyond problems like noise interference and debounce.

There were additional difficulties in creating an intuitive user interface. To ensure that users could simply grasp how to interact with the instrument while still having the freedom to explore and experiment with various musical compositions, we had to find a balance between simplicity and utility.

Arduino illustration:

The code:

const int speakerPin = 9;  // Speaker connected to pin 9
int buttonPins[] = {2, 3, 4, 5};  // Button pins for C, D, E, F
int notes[] = {262, 294, 330, 349};  // Frequencies for C4, D4, E4, F4
int potPin = A0;  // Potentiometer connected to A0
int volume;
int melody[] = {262, 262, 349, 349, 330, 330, 294};  // Melody for Twinkle Twinkle
int noteDurations[] = {500, 500, 500, 500, 500, 500, 500};  // Duration of each note


void setup() {
 for (int i = 0; i < 4; i++) {
   pinMode(buttonPins[i], INPUT_PULLUP);
 }
 pinMode(speakerPin, OUTPUT);
}


void loop() {
 volume = analogRead(potPin) / 4;  // Read volume level from potentiometer


 if (digitalRead(buttonPins[0]) == LOW) {  // Check if the first button is pressed
   playSong();
 }
 for (int i = 0; i < 4; i++) {
   if (digitalRead(buttonPins[i]) == LOW) {  // Check if any button is pressed
     tone(speakerPin, notes[i], volume);  // Play note with dynamic volume
     delay(200);  // A short delay to help debounce the button
     while (digitalRead(buttonPins[i]) == LOW);  // Wait for button release
     noTone(speakerPin);  // Stop the note
   }
 }
}


void playSong() {
 for (int thisNote = 0; thisNote < 7; thisNote++) {
   int noteDuration = 1000 / noteDurations[thisNote];
   tone(speakerPin, melody[thisNote], noteDuration);
   int pauseBetweenNotes = noteDuration * 1.30;
   delay(pauseBetweenNotes);
   noTone(speakerPin);
 }

The video:

Mood lamp

My idea for the final project is the Interactive Mood Lamp, a user-responsive system that combines hardware interaction through Arduino and software interfacing via P5.js to create an emotionally adaptive environment. This system utilizes a webcam to capture the user’s facial expressions, which are then processed through P5.js to analyze and interpret emotions. Following this analysis, the detected emotions are communicated to an Arduino-controlled RGB LED lamp, which changes its colors accordingly. The primary goal of this project is to provide visual feedback based on the user’s emotional state.

Sensing: The project begins with the webcam capturing live video of the user’s face. This video feed is processed by a P5.js application running on a connected computer. The application utilizes a facial recognition algorithm, potentially enhanced by machine learning libraries

Processing: Upon identifying an emotion, the P5.js application assigns a specific color to represent each mood: blue might indicate sadness, green happiness, red anger, and yellow surprise. This color mapping will be based on common psychological associations between colors and emotions

Responding: After processing the emotional data, the P5.js application sends the corresponding color command to the Arduino via a serial communication link. The Arduino then adjusts the RGB LED’s color to match the detected emotional state.

Reading 1:

I found great resonance in reading the assessment of the shortcomings of the interfaces used by modern technology, particularly the widespread “Pictures Under Glass” method. It caused me to reevaluate how I use technology and how these regular interactions lack the tactile depth. I became dissatisfied with the devices that rule my life after reading the author’s claim that our future interfaces will not be visionary; instead, they will only perpetuate the problems with the current system.
I was forced to consider how my devices, which are primarily touchscreens, feel impersonal and distant after reading the criticism. My laptop, phone, and even the smart devices at home operate with simple, repetitive motions that don’t have the complex feedback loop that comes with more tactile interactions. This insight made me uncomfortable since it brought to light a sensory deprivation that I was experiencing when interacting with the digital world. Compared to handling tangible objects like books or tools, I’m now more conscious of the shallowness of the experience and the flatness of the screens I swipe across on a regular basis.

Reading 2:

In reflecting on “A Brief Rant on the Future of Interaction Design,” a key question emerges: How can designers and technologists create interactive technologies that truly enhance human capabilities and foster a deeper connection with our physical environment, rather than simply replicating or replacing traditional physical interactions?

This reading essentially aligns with an argument for a more deliberate approach to technology, one that honors and strengthens rather than ignores our cognitive and physical abilities. It serves as a reminder that, whether we are using, designing, or developing technology, our relationship with it should not be passive. Rather, we should support and push for advancements that genuinely increase human potential in terms of interacting with the digital world. This mode of thinking encourages us to imagine a future where technology supports and enhances human talents in more comprehensive and meaningful ways, which is in line with a larger philosophical dilemma concerning the role technology should play in our lives.

Physical Computing’s Greatest Hits (and misses)

As I think back on this reading, the idea that creativity in physical computing—or any other creative field—comes from personalizing and reinterpreting existing concepts rather than from creating anything completely new encourages me. The theremin, for example, is a deceptively simple instrument that may develop into a very personal undertaking that expresses the creative and technical sensitivities of its maker in addition to being functional.

Furthermore, the discourse surrounding initiatives such as “Multitouch Interfaces” and “Body-as-cursor” emphasizes the need of human-centered design in physical computing. The way these initiatives connect human-machine interaction is remarkable, since physical gestures are translated into digital answers that are subsequently translated into tactile or visual outputs. The elegance of physical computing lies in its capacity to enhance technology’s intuitiveness and responsiveness to human presence and movements, as demonstrated by this smooth integration.

Making Interactive Art: Set the Stage, Then Shut Up and Listen

I found the emphasis on avoiding from dictating interpretation to be especially compelling while reading about the process of making interactive artworks. The reading  encourages artists to avoid assuming too much control over the meaning of their creations and the appropriate ways in which viewers should respond to them. Rather, artists are urged to see their works as interactive platforms that allow viewers to delve deeper and find meaning on their own. Furthermore, After establishing the interactive environment, the instruction to “shut up” is dramatic because it subverts conventional ideas of artistic power and control. It suggests faith in the audience’s capacity to interact with and participate in the creative process. This informal attitude enhances the viewing experience for the audience while simultaneously giving the artist insights about how viewers engage with the piece—what they do, what they overlook, and how they understand the tools and surroundings that are made available to them.

In order to demonstrate both digital and analog input processing, my project uses an Arduino Uno to control two LEDs depending on input from an analog photo resistor and a digital pushbutton. The circuit consists of a pushbutton that activates the system when pressed and a photoresistor that controls the state of a red and a green LED by measuring the amount of ambient light. Pressing the button causes the system to come alive, activating a flag that maintains the LEDs’ responsiveness to light variations. The green LED shines in brighter lighting, while the red LED flashes in low light to signify darkness.

This is my code:

// Define pin numbers
const int buttonPin = 2;     // Digital pin connected to the button
const int redLEDPin = 3;     // Digital pin connected to the red LED
const int greenLEDPin = 4;   // Digital pin connected to the green LED
const int photoResistorPin = A0; // Analog pin connected to the photoresistor

bool ledsActive = false;  // State variable to keep track of LEDs' activity

void setup() {
  pinMode(buttonPin, INPUT);
  pinMode(redLEDPin, OUTPUT);
  pinMode(greenLEDPin, OUTPUT);
}

void loop() {
  int buttonState = digitalRead(buttonPin);
  int lightLevel;

  // Check if the button is pressed
  if (buttonState == HIGH) {
    ledsActive = true;  // Remember that LEDs should be active
  }

  // Control LEDs based on the 'ledsActive' state
  if (ledsActive) {
    // Read light level from the photoresistor
    lightLevel = analogRead(photoResistorPin);
    
    // Determine which LED to keep on based on light level
    if (lightLevel < 512) {  // Threshold for light level can be adjusted
      digitalWrite(redLEDPin, HIGH);  // Turn on red LED if dark
      digitalWrite(greenLEDPin, LOW); // Turn off green LED
    } else {
      digitalWrite(redLEDPin, LOW);   // Turn off red LED if bright
      digitalWrite(greenLEDPin, HIGH); // Turn on green LED
    }
  } else {
    // Turn off both LEDs if the LEDs should not be active
    digitalWrite(redLEDPin, LOW);
    digitalWrite(greenLEDPin, LOW);
  }
}

And this is the setup of my Arduino:

I really enjoyed playing around with the photoresistor in this project and seeing how variations in the surrounding light dynamically affected the LEDs’ behavior, providing a concrete example of how electronic components interact with their surroundings. In addition to adding another level of interest, this investigation into light sensitivity helped me gain a deeper comprehension of analog sensors. Nevertheless, I did struggle with a couple of things. At first, I had trouble wiring the circuit correctly and dealing with connections that were wrongly placed in incorrect places on the breadboard. Even though they were annoying, these errors taught me a great deal about circuit design and debugging and made me realize how important it is to pay close attention to detail when working with electronics.

Please view the video to visualize how my project works: