Final project final concept

Concept:

Drum pad/ machine, that takes inputs in the form of buttons an doutputs sound.

P5.js:

Responsible for the output of sound, the selection of sounds, and shows a digital layout of the drumpad, with the names of each drum on top of the corresponding pads.

Arduino:

Will send serial data for each button press. Each button will also light up when it is pressed. 

Progress: 

The p5 sketch has the general layout of the drum pad, loaded with 2 sets of drums. On the Arduino side, I have programmed button presses and the light up sequence for each corresponding button.

A CAD of the case has also been made to be 3D printed.

Final Project Progress

Concept

The game is inspired by the history of Taiwanese rock, which developed not as a direct product of American cultural imperialism but as a manifestation of our people’s desire to preserve our cultural identity through these modern forms of music; many Taiwanese rock bands write their lyrics in Taiwanese Hokkien and are very involved in social movements outside of their music. These are elements of Taiwanese culture that were suppressed during the martial law era, and the vibrant pop punk/punk rock scene embodies the resilience of our people. Taiwanese rock is an amalgamation of local Taiwanese, Chinese, American, and Japanese influences that reflects our complex history as a colonial subject of many but also our present and future as a country.

I intend for the game to be a light-hearted introduction to this aspect of Taiwan. In the spirit of the wacky, energetic aesthetic of the Scott Pilgrim franchise, the player controls a guitarist who is running late for a gig and has to get there as soon as possible, evading cops and picking up pirated music tapes to gain experience and recharge. My final project will be a game centered around a guitar repurposed as a gamepad: each string, when strummed, triggers a specific movement for the protagonist.

Progress

Assets from itch.io

As of now, I have a rough configuration of the guitar controller; 4 of the 6 strings are connected to the breadboard, on which a circuit will send input if the strings are strum by the metal guitar pick.

In my previous assignments, I wrapped a regular plastic guitar pick in tin foil and taped jumper wires to the foil, but it was rather flimsy. This time, I decided to use a flat metal nut as a guitar pick so that I don’t have to wrap additional material on it to make it conductive. Right now, the wires are temporarily taped to the nut, but I am considering soldering them directly so that they don’t shift around and affect connectivity.
As for the game itself, I have set up the moving background, as the game will be a side-scroller.

Final Project: Preliminary Idea

Armenian Culture Navigator.

For my final project, I have an idea of making something related to Armenian culture. As of now, I imagine the project as a digital map displayed on p5.js with tactile and sensory inputs facilitated by Arduino. Users can explore iconic Armenian symbols by clicking on them on the map, like the pomegranate, lavash (Armenian bread), and Mount Ararat through engaging tasks which I still have to develop. For instance, the pomegranate can be colored by touching a conductive pad connected to an Arduino sensor. Similarly, users might virtually “stretch” the lavash by pulling on an elastic sensor. To add another layer of interactivity, blowing into a microphone sensor might simulate wind, and make digitally drawn clouds to drift across Mount Ararat on the map.

Areas for development:

I’m thinking about other objects that represent Armenian culture and symbols like the duduk (Armenian musical instrument), or recognisable carpets. I will also need to think about the ways I will build meaningful p5 and Arduino interactions with these symbols. 

Final Project | Some ideas

I’ve always wanted to create something related to gaze tracking—mapping the intangible act of looking into something tangible that showcases the power of staring. It’s about turning private intention into public expression.

Idea 1:
Develop a human body figure in p5.js and use p5 to detect and record gaze (eyeball) movements as the user observes the figure. Then, create a physical representation of the body figure and embed LED pins in different sections. p5.js would send gaze data to an Arduino, causing the LEDs to blink in the areas the user’s gaze travels to. Simultaneously, a speaker could play audio, creating an interactive experience that visualizes and vocalizes the gaze.

Idea 2:
This concept shifts from exploring the male gaze and fetishization to a more practical application: an eye-blink sensor for preventing drowsy driving. My cousin got us into a car accident last year because he fell asleep while driving, which inspired this idea. Although less creative, it’s highly practical. The system would include a speaker that plays a reminder whenever the user closes their eyes, encouraging them to stay alert. This device would be designed to be portable, so it can easily attach to any pair of glasses.

Reflective Response

Design Meets Disability

The reading helped me understand the intersection go aesthetics and utility. Its emphasis on glasses changing from medical necessity to fashion accessory made me realise how deep the idea of “hiding” disability is in our culture. We tend to prioritise discretion in disability design over self-expression. This showed how good designs should not just accommodate but enhance and adapt to human experiences without bringing attention to differences. I now see how disability is more of a limitation by environments which aren’t designed inclusively rather than a limitation of a person.
I was particularly struck by the contrast between traditional medical design and the bold, unapologetic aesthetics introduced by people like Aimee Mullins. Her idea that a prosthetic leg can be glamorous and even a fashion statement reforms disability not as a limitation but as an opportunity for individuality. Another interesting point was the way simple designs such as the Muji CD player were framed as essentially inclusive. It made me question the products I use daily ,such as elevators without braille and phones with complex interfaces, and how they often don’t accommodate diverse needs. Some designs exclude certain groups by default by catering only to “average” users. In my opinion many innovations could be made if designers embraced the tension between accessibility and aesthetic appeal.

This reading changed my perspective on inclusivity in design, it’s not just about solving problems but embracing differences.

 

Final Project Ideas

For my final project I had a couple of ideas of possible project. I am yet to narrow them down into one which I will work with as my project. I hope to do this as I progress to the next step of the assignment.

Idea 1

My first idea was to create a car with parking assistant simulator. My inspiration for this was trying to emulate the modern car parking assistance systems. My idea is to use distance sensors to detect how close the car is to surrounding objects. I would then use p5 to give a visual presentation of the parking scenario giving the user a visual feedback. I would also like to have alerts such as LED and buzzer warnings to give a better experience to the user

Idea 2

My second idea was to create a simple robot that can be controlled to move and possible wave and give data such as distance from objects around it. I plan to equip the robot with basic motion capabilities and a sensor to measure its distance from nearby objects. I intend to have the user interact with the robot through a controller or the keypad depending on what will be more convenient

WEEK 11 – EXCERSISE

EXCERCISE 1: ARDUINO TO P5.JS COMMUNICATION

RESULT:

IMG_1537

P5.JS:

CODE (P5.JS):

let sensorValue = 0; // Variable to store sensor data

function setup() {
  createCanvas(640, 480);
  textSize(18);
  if (!serialActive) {
    setUpSerial(); // Start serial communication with Arduino
  }
}

function draw() {
  // Set the background to dark blue and purple hues based on the sensor value
  background(map(sensorValue, 0, 1023, 50, 75), 0, map(sensorValue, 0, 1023, 100, 150));

  // Map the sensor value to control the ellipse's horizontal position
  let ellipseX = map(sensorValue, 0, 1023, 0, width);

  // Draw the ellipse in the middle of the screen
  fill(255); // White ellipse for contrast
  ellipse(ellipseX, height / 2, 50, 50);

  // Display connection status
  fill(255); // White text for readability
  if (!serialActive) {
    text("Press Space Bar to select Serial Port", 20, 30);
  } else {
    text("Connected", 20, 30);
    // Display the current sensor value
    text("Sensor Value = " + str(sensorValue), 20, 50);
  }
}

function keyPressed() {
  if (key === " ") {
    setUpSerial(); // Start the serial connection when the spacebar is pressed
  }
}

// This function is called by the web-serial library with each new line of data
function readSerial(data) {
  if (data != null) {
    // Parse the sensor value from the Arduino
    sensorValue = int(trim(data));
  }
}

 

CODE (ARDUINO): https://github.com/aaa10159/IntroToIM/blob/a6dc11112996470e161e0ef812409366ebd219d4/week%2011%20-%20exercise%201

EXCERCISE 2: P5 TO ARDUINO COMMUNICATION

RESULT:

IMG_1541

P5.JS:

CODE (P5.JS):

let brightness = 0; // Brightness value to send to Arduino

function setup() {
  createCanvas(640, 480);
  textSize(18);

  // Check if serial is active and set it up if not
  if (!serialActive) {
    setUpSerial(); // Initialize serial communication
  }
}

function draw() {
  background(30); // Dark background
  fill(255); // White text
  text("Use the UP and DOWN arrows to control LED brightness", 20, 30);

  // Display the current brightness value
  text("Brightness: " + brightness, 20, 60);
}

function keyPressed() {
  if (keyCode === UP_ARROW) {
    // Increase brightness
    brightness = min(brightness + 10, 255); // Max brightness is 255
    sendBrightness();
  } else if (keyCode === DOWN_ARROW) {
    // Decrease brightness
    brightness = max(brightness - 10, 0); // Min brightness is 0
    sendBrightness();
  } else if (key === " ") {
    // Start serial connection when spacebar is pressed
    setUpSerial();
  }
}

function sendBrightness() {
  if (writer) {
    // Send the brightness value to Arduino
    writer.write(brightness + "\n");
  } else {
    console.error("Writer is not available. Please connect to the serial port.");
  }
}

CODE (ARDUINO):

https://github.com/aaa10159/IntroToIM/blob/64b87caff1ef02ad4c3254acbd5fcd12b89e00f0/exercise%202

EXCERCISE 3: BI-DIRECTIONAL COMMUNICATION

RESULT:

EF08D8F3-8CC4-4F3D-939A-E8996AC8E21A

P5.JS:

CODE (P5.JS):

let velocity;
let gravity;
let position;
let acceleration;
let wind;
let drag = 0.99;
let mass = 50;
let sensorValue = 0; // Variable to store wind value from Arduino
let windStrength = 0; // Wind force determined by the sensor

function setup() {
  createCanvas(640, 360);
  noFill();
  position = createVector(width / 2, 0);
  velocity = createVector(0, 0);
  acceleration = createVector(0, 0);
  gravity = createVector(0, 0.5 * mass);
  wind = createVector(0, 0);

  // Initialize serial communication
  if (!serialActive) {
    setUpSerial();
  }
}

function draw() {
  background(255);

  // Apply gravity
  applyForce(gravity);

  // Apply wind (continuously updated from sensor)
  wind.x = map(sensorValue, 0, 1023, -2, 2); // Map sensor value to a stronger wind range
  applyForce(wind);

  // Update position and velocity
  velocity.add(acceleration);
  velocity.mult(drag);
  position.add(velocity);
  acceleration.mult(0);

  // Draw the ball
  ellipse(position.x, position.y, mass, mass);

  // Check for bounce
  if (position.y > height - mass / 2) {
    velocity.y *= -0.9; // A little dampening when hitting the bottom
    position.y = height - mass / 2;

    // Notify Arduino about the bounce
    sendBounce();
  }
}

function applyForce(force) {
  // Newton's 2nd law: F = M * A
  let f = p5.Vector.div(force, mass);
  acceleration.add(f);
}

// Notify Arduino when the ball bounces
function sendBounce() {
  if (writer) {
    writer.write('1\n'); // Send the bounce signal
  }
}

// Read wind control value from Arduino
function readSerial(data) {
  if (data != null) {
    // Parse the sensor value directly into a variable for wind force
    sensorValue = int(trim(data));
  }
}

// Handle serial setup (using the serial.js file)
function keyPressed() {
  if (key === " ") {
    setUpSerial();
  }
}

 

CODE (ARDUINO):

https://github.com/aaa10159/IntroToIM/blob/b181568724f29fd0d2bf9f290ca3b29949e18740/exercise%203

Assignment 8 | 3 exercises of Arduino & P5 (Jheel & Linda)

EXERCISE 01: ARDUINO TO P5 COMMUNICATION 

Make something that uses only one sensor on arduino and makes the ellipse in p5 move on the horizontal axis, in the middle of the screen, and nothing on arduino is controlled by p5.

The ball moves to the left when it’s dark; to the right when it’s bright.

  • Please find the demonstration video here

Exercise 1

  • Please find the p5.js sketch here 
let port, reader, writer;
let serialActive = false;

async function getPort(baud = 9600) {
  let port = await navigator.serial.requestPort();
  await port.open({ baudRate: baud });

  // create read & write streams
  textDecoder = new TextDecoderStream();
  textEncoder = new TextEncoderStream();
  readableStreamClosed = port.readable.pipeTo(textDecoder.writable);
  writableStreamClosed = textEncoder.readable.pipeTo(port.writable);

  reader = textDecoder.readable
    .pipeThrough(new TransformStream(new LineBreakTransformer()))
    .getReader();
  writer = textEncoder.writable.getWriter();

  return { port, reader, writer };
}

class LineBreakTransformer {
  constructor() {
    this.chunks = "";
  }

  transform(chunk, controller) {
    this.chunks += chunk;
    const lines = this.chunks.split("\r\n");
    this.chunks = lines.pop();
    lines.forEach((line) => controller.enqueue(line));
  }

  flush(controller) {
    controller.enqueue(this.chunks);
  }
}

async function setUpSerial() {
  noLoop();
  ({ port, reader, writer } = await getPort());
  serialActive = true;
  runSerial();
  loop();
}

async function runSerial() {
  try {
    while (true) {
      const { value, done } = await reader.read();
      if (done) {
        reader.releaseLock();
        break;
      }
      readSerial(value);
    }
  } catch (e) {
    console.error(e);
  }
}

let rVal = 0; // Value from photoresistor

function setup() {
  createCanvas(640, 480);
  textSize(18);
}

function draw() {
  background(245, 245, 200);
  fill(0);

  if (!serialActive) {
    text("Press Space Bar to select Serial Port", 20, 30);
  } else {
    text("Connected", 20, 30);
    text('Photoresistor Value = ' + str(rVal), 20, 50);

    // Map the sensor value to the x-position of the ellipse
    let xPos = map(rVal, 0, 1023, 0, width);
    fill(100, 123, 158);
    ellipse(xPos, height / 2, 50, 50);
  }
}

function keyPressed() {
  if (key == " ") {
    setUpSerial();
  }
}

function readSerial(data) {
  if (data != null) {
    let fromArduino = trim(data); // Remove any whitespace
    rVal = int(fromArduino); // Convert the string to an integer
  }
}
void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);
}


void loop() {
  // put your main code here, to run repeatedly:
  int sensorValue = analogRead (A2);
  Serial.println (sensorValue);
  Serial.write(sensorValue);
  delay (1);
}

 

EXERCISE 02: P5 TO ARDUINO COMMUNICATION 

Make something that controls the LED brightness from p5.

  • LED brightness changes according to the slider.

  • Please find the demonstration video here

Exercise 2

  • Please find the p5.js sketch here 
let brightness = 0; 
let slider;

function setup() {
  createCanvas(400, 200);
  //make the slider
  slider = createSlider(0, 255, 127); 
  slider.position(10, 10);
  slider.style('width', '300px');

  let serialButton = createButton("Connect to Arduino");
  serialButton.position(10, 50);
  serialButton.mousePressed(setUpSerial);
}

//troubleshoot
function readSerial(data) {
  console.log("Received data:", data); // Log the received data to the console
}

function draw() {
  background(220);
  brightness = slider.value(); 
  fill(0);
  textSize(16);
  text(`LED Brightness: ${brightness}`, 10, 100);

  // Send brightness value to Arduino via Serial
  if (serialActive) {
    writeSerial(brightness + "\n"); // Append a newline character
  }
}
int ledPin = 9; // PWM pin connected to LED

void setup() {
  Serial.begin(9600);
  pinMode(ledPin, OUTPUT);
}

void loop() {
  int bright;

  if (Serial.available() > 0) {
    bright = Serial.parseInt();
    analogWrite(ledPin, bright);
  }
}

EXERCISE 03: BI-DIRECTIONAL COMMUNICATION

Every time the ball bounces one led lights up and then turns off, and you can control the wind from one analog sensor

Please find the demonstration video here

Please find the p5

let port, reader, writer;
let serialActive = false;

async function getPort(baud = 9600) {
  let port = await navigator.serial.requestPort();
  await port.open({ baudRate: baud });

  // create read & write streams
  textDecoder = new TextDecoderStream();
  textEncoder = new TextEncoderStream();
  readableStreamClosed = port.readable.pipeTo(textDecoder.writable);
  writableStreamClosed = textEncoder.readable.pipeTo(port.writable);

  reader = textDecoder.readable
    .pipeThrough(new TransformStream(new LineBreakTransformer()))
    .getReader();
  writer = textEncoder.writable.getWriter();

  return { port, reader, writer };
}

class LineBreakTransformer {
  constructor() {
    this.chunks = "";
  }

  transform(chunk, controller) {
    this.chunks += chunk;
    const lines = this.chunks.split("\r\n");
    this.chunks = lines.pop();
    lines.forEach((line) => controller.enqueue(line));
  }

  flush(controller) {
    controller.enqueue(this.chunks);
  }
}

async function setupSerial() {
  noLoop();
  ({ port, reader, writer } = await getPort());
  serialActive = true;
  runSerial();
  loop();
}

async function runSerial() {
  try {
    while (true) {
      const { value, done } = await reader.read();
      if (done) {
        reader.releaseLock();
        break;
      }
      readSerial(value);
    }
  } catch (e) {
    console.error(e);
  }
}


////////////////////////////////////////////////////////////

let velocity;
let gravity;
let position;
let acceleration;
let wind;
let drag = 0.99;
let mass = 50;
let windSpeed;


function setup() {
  createCanvas(640, 360);
  noFill();
  position = createVector(width / 2, 0);
  velocity = createVector(0, 0);
  acceleration = createVector(0, 0);
  gravity = createVector(0, 0.5 * mass);
  wind = createVector(0, 0);
  textSize(16);
}

function draw() {
  background(0);
  
  if (serialActive){
    wind.x = map(windSpeed, 0, 1023, -1, 1);
  }
  applyForce(wind);
  applyForce(gravity);
  velocity.add(acceleration);
  velocity.mult(drag);
  position.add(velocity); 
  acceleration.mult(0);
  ellipse(position.x, position.y, mass, mass);
  console.log("Wind Speed: " + windSpeed); // For debugging

  if (position.y > height - mass / 2) {
    velocity.y *= -0.9; // A little dampening when hitting the bottom
    position.y = height - mass / 2;

    // Send signal to Arduino on bounce
    if (serialActive) {
      sendBounceSignal();
    }
  }

  if (!serialActive) {
    fill(255);
    text("Press SPACE to connect to Serial Port", 20, 30);
  } else {
    fill(0, 255, 0);
    text("Connected to Serial Port", 20, 30);
  }
}

function applyForce(force) {
  // Newton's 2nd law: F = M * A or A = F / M
  let f = p5.Vector.div(force, mass);
  acceleration.add(f);
}

function keyPressed() {
  if (keyCode == LEFT_ARROW) {
    wind.x = -1;
  }
  if (keyCode == RIGHT_ARROW) {
    wind.x = 1;
  }
  if (key == " ") {
    if (!serialActive) {
      setupSerial();
    } else {
      mass = 50;
      position.y = -mass;
      velocity.mult(0);
    }
  } 
}

async function setupSerial() {
  try {
    noLoop();
    // ({ port, reader, writer } = await getPort());
    
    port = await navigator.serial.requestPort();
    await port.open({ baudRate: 9600 });
    
    
    // Create a TextDecoderStream to decode incoming bytes to text
    const textDecoder = new TextDecoderStream();
    const readableStreamClosed = port.readable.pipeTo(textDecoder.writable);

    // Create the reader to read from the decoded text stream
    reader = textDecoder.readable
    .pipeThrough(new TransformStream(new LineBreakTransformer())) // Optional: split data by lines
    .getReader();
    
    writer = port.writable.getWriter();
    serialActive = true;
    // Start reading data after successfully opening the port
    runSerial();
    loop();
  } catch (err) {
    console.error("Serial connection failed:", err);
    serialActive = false;
  }
}

function readSerial(data) {
  if (data != null) {
    let fromArduino = trim(data); // Remove any whitespace
    console.log(data);
    if (fromArduino !== "") {
      fromArduino = parseInt(fromArduino, 10);
      windSpeed = int(fromArduino); // Convert the string to an integer
      
    }
  }
}


async function sendBounceSignal() {
  try {
    if (writer) {
      await writer.write(new TextEncoder().encode("bounce\n"));
    }
  } catch (err) {
    console.error("Failed to send bounce signal:", err);
  }
}

Please find the Arduino

let brightness = 0; 
let slider;

function setup() {
  createCanvas(400, 200);
  //make the slider
  slider = createSlider(0, 255, 127); 
  slider.position(10, 10);
  slider.style('width', '300px');

  let serialButton = createButton("Connect to Arduino");
  serialButton.position(10, 50);
  serialButton.mousePressed(setUpSerial);
}

//troubleshoot
function readSerial(data) {
  console.log("Received data:", data); // Log the received data to the console
}

function draw() {
  background(220);
  brightness = slider.value(); 
  fill(0);
  textSize(16);
  text(`LED Brightness: ${brightness}`, 10, 100);

  // Send brightness value to Arduino via Serial
  if (serialActive) {
    writeSerial(brightness + "\n"); // Append a newline character
  }
}

 

Final Project Proposal

Inspiration

I’ve always tried making most of my projects personal/unique to me in some way,  and really I wanted to do the same for the final project! When I tried thinking about things I love, the first thing that came to mind was books and reading in general. Reading has always been one of my favorite activities and for this project, I tried to think of creating something that could make the experience of reading even more enjoyable and interactive.

I then remembered how a few years ago, I had stumbled upon a video of an automatic page turning mechanism and while it was cool, I couldn’t help but think: What if this was more interactive, more human-like? That’s when the concept of a reading buddy robot began to take shape in my mind. I envisioned a small, friendly robot sitting across from me, holding my book as I read. Not just flipping pages mechanically, but responding to my actions—like looking at the left or right page when I’m reading—and maybe even bringing some humor or companionship to the reading experience. The idea of combining robotics with interactivity felt like a perfect challenge that merges both technical and creative elements.

Concept

The reading buddy robot will be designed to face me while holding a book flat out on a  table, almost like a tiny librarian or companion. Its main features will include:

  1. Page Flipping: With a simple gesture or button press, the robot will flip the book’s pages one at a time. I want this mechanism to be smooth and reliable, even for thinner pages.
  2. Interactive Movements: The robot will “look” at the left page when I’m reading on the left side and the right page when I’m reading on the right. This can be achieved using a combination of servo motors and sensors or even a webcam paired with eye-tracking software. It could also give some sort of indication to about my distance from the book, so that I don’t strain my eyes too much.
  3. Personality Through Interaction: The robot won’t just be functional; it will have a personality! For example, if I take too long on one page, it could nod or tilt its “head” as if it’s waiting for me to move on. I’m thinking of adding some sound effects or p5.js animations to make it feel like the robot is alive.

To add a modern twist, I want to connect the robot to a digital interface using p5.js, allowing it to display fun visuals or stats about my reading progress. Maybe it could track how long I’ve been reading and cheer me on, or even throw a playful tantrum if I ignore it for too long!

Parts I’m Uncertain About

While I’m excited about this project, there are a few areas that I’m still figuring out:

  1. The Page Flipping Mechanism:
    I’ve seen examples of flat arm page-turners, but adapting this mechanism to hold the book upright and flip pages smoothly is going to be tricky.  Thin book pages might make this more challenging, so I’ll need to experiment with different designs and materials.
  2. Making It Human-Like:
    The goal is to make the robot feel interactive and expressive, but that’s easier said than done. I can imagine using servos to give it a moving “head,” but what if it feels too mechanical? Adding personality through sounds, lights, or p5.js animations could help, but I’m still brainstorming the best way to tie all of this together.

Final Thoughts

This project is still in its early stages, but I’m really excited about where it’s heading. The idea of combining robotics, interactivity, and reading feels like a perfect way to push myself creatively and technically. I know there will be challenge but I look forward to working on it and seeing how it turns out!

Assignment 11: In Class Exercises (Serial Communication)

Exercise 1 – Moving Ellipse

Prompt: Make something that uses only one sensor  on Arduino and makes the ellipse in p5 move on the horizontal axis, in the middle of the screen, and nothing on arduino is controlled by p5

Arduino Code

P5.js Sketch

Exercise 2 – LED Brightness

Prompt: Make something that controls the LED brightness from p5

Arduino Code

P5.js Sketch

Exercise 3 – Bouncing Ball

Prompt: Take the gravity wind example (https://editor.p5js.org/aaronsherwood/sketches/I7iQrNCul) and make it so every time the ball bounces one led lights up and then turns off, and you can control the wind from one analog sensor

Arduino Code

P5.js Sketch