Category: F2023 – Mang

Concept:

For this assignment, I decided to create a gender reveal project. I used a red LED and a button to represent “girl,” and a blue LED and button to represent “boy.”

Implementation:

When the red button is pressed, the red LED (representing “girl”) lights up. Similarly, when the blue button is pressed, the blue LED (representing “boy”) turns on.

int yellowLED = 13; // yellow LED
int blueLED = 9;    //  blue LED
int yellowButton = A0; // yellow button 
int blueButton = A2;   // blue button 
void setup() {  
  pinMode(yellowLED, OUTPUT);    
  pinMode(blueLED, OUTPUT);        
  pinMode(yellowButton, INPUT_PULLUP); 
  pinMode(blueButton, INPUT_PULLUP);   
}

void loop() {
  int switchPositionYellow = digitalRead(yellowButton);  
  int switchPositionBlue = digitalRead(blueButton);     

  // Check if both buttons are pressed at the same time
  if (switchPositionYellow == LOW && switchPositionBlue == LOW) {
    digitalWrite(yellowLED, LOW);  
    digitalWrite(blueLED, LOW);     
    delay(500);               

    digitalWrite(yellowLED, HIGH);  
    delay(500);
    digitalWrite(yellowLED, LOW);

    digitalWrite(blueLED, HIGH);    
    delay(500);
    digitalWrite(blueLED, LOW);
  }
  
  // If only the yellow button is pressed
  else if (switchPositionYellow == LOW) { 
    digitalWrite(yellowLED, HIGH);   // Turn ON yellow LED
    digitalWrite(blueLED, LOW);      // Turn OFF blue LED
  }

  // If only the blue button is pressed
  else if (switchPositionBlue == LOW) { 
    digitalWrite(yellowLED, LOW);    // Turn OFF yellow LED
    digitalWrite(blueLED, HIGH);     // Turn ON blue LED
  }

  // If neither button is pressed
  else {
    digitalWrite(yellowLED, LOW);    // Turn OFF both
    digitalWrite(blueLED, LOW);
  }
}

IMG_8148

USER TESTING:
My deliverable was still at a premature stage so I was unable to film a user-testing video, however I was able to get verbal feedback on my p5 program design (and so have adjusted my code accordingly). Initially, I had no text to indicate any sort of instructions or context, as I had thought the arrow buttons were enough to prompt users into action. However, my sister advised against this and suggested I include a phrase or 2 to provide basic information as to what my project is about – even more so since I did not have a start/ introductory screen. Another feedback I got was regarding the separate display screen for when the recommended playlist sounds – which was mentioned in the previous documentation. I was initially just planning to display the user’s chosen personalised cassette however my sister thought it to be too static, commenting that it was lacking flair. I starting brainstorming other potential display screens I could have but the one that resonated most with me was actually animating a rolling cassette tape, of course this would mean I had to create animations for all possible cassette tape designs.

Final project video: https://youtu.be/t_wIKjY5s1o

My final project concept is based off of my midterm project. In the previous midterm, I made a digital radio that played 2 meaningful songs. Each song displayed (what was supposed to be) a dynamic background of a personal memory associated with the songs. My final project builds off on this idea but with an added twist. You are able to customise your own cassette tape and based on your choices, it assembles a recommended playlist. There are 3 choices in each stage of customizing for the personalisation of your cassette and when finished, it plays a recommended playlist of 4 songs, each similar in genre. You can adjust the volume, use a skip function and reset the entire experience through physical means (buttons and potentiometer). Whilst my midterm involved a sense of personal intimacy, I tried to make this project evoke a more shared intimacy. Music is very personal to me and by sharing it with others, I am able to show a partial extension of my own identity which can be reciprocated by external users. Speaking from personal experience, it cements a bond quicker.

//arduino code:

int button1Pin = A2; 
int button2Pin = A3; 
int potentiometerPin = A0;

void setup() {
  // Start serial communication so we can send data
  // over the USB connection to our p5js sketch
  Serial.begin(9600);

  // We'll use the built-in LED as a status output.
  pinMode(LED_BUILTIN, OUTPUT);
}

void loop() {
  // Read button states and potentiometer value
  int button1State = digitalRead(button1Pin);
  int button2State = digitalRead(button2Pin);
  int potValue = analogRead(potentiometerPin);

  // Send data to p5.js
  Serial.print(button1State);
  Serial.print(",");
  Serial.print(button2State);
  Serial.print(",");
  Serial.println(potValue);

  delay(100); // Adjust delay as needed
}

This final project was so so painstakingly challenging and was such an arduous experience in general it took my soul and a chunk of hair by the time I was finished. Every portion of code had a bug and debugging it took a minimum of 1.5 hours (rarely as I wasn’t so lucky) and a maximum of 7 hours+. I will most likely never forget such an experience. The first portion – p5.js, was doable, yes there were countless debugging that was really frustrating but the cherry on the cake was the second portion – serial communication of the final project. The process of this entire final was tedious overall:

PROCESSES – ART STAGE:
I first created 3 playlists of 4 songs. Then using pinterest inspirations of vintage cassette tapes, I drew each stage: cassette base, cassette sticker, cassette detail using procreate. I illustrated specific combinations of these cassettes to equate to a certain playlist and I drew the details to correspond with the overall music vibe and aesthetic: (see below: 1) result1, 2) result2, 3) result3). As mentioned in my user-testing documentation section,  I wanted to create an animation of the cassette tape rolling when users entered the final stage: music playing. The only plausible way was to create a gif file containing such animation. Because there are 3 choices for each 3 stages and 3 different combinations users could select, it meant I had to create animations for a total of 27 cassettes, hence why it was so time consuming.

PROCESSES – P5.JS:
Essentially both coding experiences were one I do not want to remember… the endless bug fixes, the endless error messages on the console, it was just incredibly stressful. However the code that evoked the most stress and hence, I’m most proud of was attributing the corresponding gif files to every possible indices user could end up with – likewise with creating the cassette animations, there were 27 different combinations. This meant that the program had to store the index chosen at each stage and use this information to call upon a gif file with the corresponding index. This was one of those sections that took 7+ hours to debug and code. I didn’t know where to start and how, so, like I always did with previous assignments, I began researching and looking for codes that fulfilled similar instances on google. Then came the experimentation and checking using console.log. Through this I was able to learn syntax I had never encountered before and this acted as a sort of revelation for me. Here is the relevant code section:

//
const gifFilenameMap = { //attributing gif file pathway to user selected indices
//for cassetteBase[0]
  "0_0_0": "gifs/result1_prpl1.gif",
  "0_0_1": "gifs/result1_prpl2.gif",
  "0_0_2": "gifs/result1_prpl3.gif",
  "0_1_0": "gifs/result1_green1.gif",
  "0_1_1": "gifs/result1_green2.gif",
  "0_1_2": "gifs/result1_green3.gif",
  "0_2_0": "gifs/result1_grey1.gif",
  "0_2_1": "gifs/result1_grey2.gif",
  "0_2_2": "gifs/result1_grey3.gif",

//for cassetteBase[1]
  "1_0_0": "gifs/result2_prpl1.gif",
  "1_0_1": "gifs/result2_prpl2.gif",
  "1_0_2": "gifs/result2_prpl3.gif",
  "1_1_0": "gifs/result2_green1.gif",
  "1_1_1": "gifs/result2_green2.gif",
  "1_1_2": "gifs/result2_green3.gif",
  "1_2_0": "gifs/result2_grey1.gif",
  "1_2_1": "gifs/result2_grey2.gif",
  "1_2_2": "gifs/result2_grey3.gif",

//for cassetteBase[2]
  "2_0_0": "gifs/result3_prpl1.gif",
  "2_0_1": "gifs/result3_prpl2.gif",
  "2_0_2": "gifs/result3_prpl3.gif",
  "2_1_0": "gifs/result3_green1.gif",
  "2_1_1": "gifs/result3_green2.gif",
  "2_1_2": "gifs/result3_green3.gif",
  "2_2_0": "gifs/result3_grey1.gif",
  "2_2_1": "gifs/result3_grey2.gif",
  "2_2_2": "gifs/result3_grey3.gif",
};

//generates gif filename based on indices of selected cassette components
function generateGifFilename(baseIndex, stickerIndex, detailIndex) {
  return gifFilenameMap[`${baseIndex}_${stickerIndex}_${detailIndex}`]; //generating filename using map => e.g., 2_1_0
}

function determineResult() {
...
//generating filename (e.g., "1_2_3") based on indices of selected components
  const gifFilename = generateGifFilename(selectedBaseIndex, selectedStickerIndex, selectedDetailIndex);
  gifElement = createImg(gifFilename, "selectedGif"); // displaying selected gif on canvas
  gifElement.size(imageWidth, imageHeight);
  gifElement.position(imagePosition.x, imagePosition.y);
}

PROCESSES – ARDUINO + SERIAL COMMUNICATION:
Serial communication was one I had the most issues with. I used the existing serial communication code (in mang’s lecture notes) for both arduino and p5.js and altered it around my main piece of code however, problem 1) there seemed to be issues with p5.js and arduino exchanging data, hence it was impossible to know whether the physical wiring of the components on the breadboard was the problem or whether it was the code itself that was causing issues. 2) I continually experienced error messages stating that there was a network error hence I was unable to connect to a serial port. Both cases required patience, calmness and perseverance and through this it was engrained into me again the importance of console logging when debugging faulty code. At the start, I wasn’t able to understand the serial communication code that was provided but after the completion of my final project, everything kind of clicked into place.

Regarding attributing functions to the physical components: 2 push buttons and a potentiometer, I was also having major problems with applying my desired functions: play/pause, skip forward, skip backward, to the push buttons. Mapping the volume to the potentiometer value was really easy as something like it had already been done for the serial communication assignment. For the rest, it was a nightmare. I think it was the structure of the code and the specific manner in which I coded that caused so many breakdowns and errors. In the end I was incredibly short for time and so was forced to compensate and only code 1) resetToInitialState, 2) skip forward on loop. when coding for the function: resetToInitialState, 2 problems occurred: 1) gif image appearing over initial state, 2) sound continuing to play regardless of being set to its initial state. With extensive experimentation, I realised that creating new variables to keep track of the states of both the gif and sound was the most simplest and most rational solution – here is the relevant code:

let gifElement; (ADDED) 

//within function determineResult() 
if (gifElement) { (ADDED) 
  gifElement.remove(); //remove existing gifElement if it exists 
} 

const gifFilename = generateGifFilename(selectedBaseIndex, selectedStickerIndex, selectedDetailIndex); 
gifElement = createImg(gifFilename, "selectedGif"); // displaying selected gif on canvas (ADDED) 
gifElement.size(imageWidth, imageHeight); (ADDED) 
gifElement.position(imagePosition.x, imagePosition.y); (ADDED) 

/////////////////////////////////////////////////////////////////////////////////////
let shouldPlayNextSound = true; //(ADDED) 

//within function playNextSound()
if (shouldPlayNextSound) { //(ADDED)
  currentSoundIndex++; // increment sound index
  if (currentStage === 4 && currentSoundIndex >= result1.length) {
    determineResult();
    currentSoundIndex = 0; //reset to the beginning if end is reached
  }
}

//within function resetToInitialState()
shouldPlayNextSound = false; //disable skip function (ADDED)

//stopping all currently playing sounds
for (let i = 0; i < result1.length; i++) {
result1[i].stop();
}
for (let i = 0; i < result2.length; i++) {
result2[i].stop();
}
for (let i = 0; i < result3.length; i++) {
result3[i].stop();
}
shouldPlayNextSound = true; //enable skip function (ADDED)

FINAL REFLECTIONS + FUTURE IMPROVEMENTS:
Whilst it was the most nerve wrecking, anxiety inducing overall experience, since persisting bugs were fixed the day of the IM show, I was quite proud of what I have completed. Whilst the coding aspect of this project was beyond challenging, I can’t deny that it was lowkey fun at the same time – creating a project that involves my passion. To me, it certainly felt like a large leap in the level of difficulty, compared to my midterm project, and this was more so why I am proud of the finished result. For future improvements on the project, perhaps there could be a personality test which based on your selected answers allocates you to a specific design for each stage of the cassette customisation. This way the experience maintains for longer. I also think it builds more excitement and anticipation as to what cassette you’ll end up with. Improvements for the physical aspect of the project would be to build a radio with more extensive functions, like originally planned.

Regarding improvements for future IM projects, I am incredibly motivated to put thought into the building of an exterior because that, at the end of the day, is what can elevate user experience. Since it was my first time both showcasing and attending an IM show, I experienced somewhat of an epiphanous moment. In future classes I will be more mindful in creating a more immersive user experience that is able to appeal to a wider body of people, because whilst mine did have some sort of user experience, it was more so stagnant with limited interaction compared to the other projects that were showcased. Overall I think it was an excellent opportunity to understand the fundamentals of what Interactive Media embodies and it has further propelled my motivation to learn in depth creative coding.

PRELIMINARY CONCEPT:
I wanted to do some brainstorming regarding the concept of my final project. I wanted to do something that wasn’t too ambitious (considering the myriad of deadlines as a result of finals week and the limited time constraints). I think because I couldn’t think of a theme, ideas were very vast which didn’t help. After some major considerations and scrapping of multiple ideas, I came to the conclusion that I would want to build off on my midterm project to create a physical radio that is controllable through i.e., buttons, potentiometer, power switch. Output would be digital and input would be both digital and physical.

FINALIZED CONCEPT:
I thought creating a purely physical radio with a digital output would not be enough interactivity or at least a meaningful interaction, hence I wanted to perpetuate intimacy. I love music and (to me) what is more intimate than sharing your favourite playlists/ artists with others? I decided to make my final project a 2-part: 1) determine corresponding mood/ personality through personalizing their own cassette tape, 2) according to results, a playlist of 4 songs would be curated – this way there is more intimate and meaningful back and forth. Essentially it would work like those buzzfeed quizzes except physical input via push buttons are possible.

Total physical components would include 3 push buttons and a potentiometer for volume control. My arduino program will read the potentiometer value and buttons’/switch’s state and send the information to p5.js via serial communication. When p5.js receives information about the button’s push state, it will call upon its corresponding function, i.e., pause/play, skip forward/ backward. I will use the mapping function to map the potentiometer value to the volume so that whenever the potentiometer is toggled, p5.js will continuously receive its p value and adjust the volume accordingly. With regards to my p5 program design, I intend to have a cassette displayed in the middle with “select” and arrows buttons. Since users will select designs for each of the 3 stages, the “select” button will store user choices in a separate variable so as to assign a “result1/2/3” playlist. Although undecided, I intend to have a separate display screen for when the recommended playlist sounds to indicate the change of focus from p5 to the actual physical components.

Before completing the project, I did some user testing with a small sample of participants. Here is a video of Zion using my project:

User testing stage gave me valuable insight that was difficult to see from my perspective as the creator of the project. Here are the tips that I implemented in the final version:

• Specifying how many stages (color inputs) there will be, helps the participant to understand the structure of the experience from the start.
• Starting the instructions by saying what is the main thing that will be required (color input based on prompts).
• Introducing an additional screen in between the last color input and generated gradient to build some suspension.

Are they able to figure it out? Where do they get confused and why? Do they understand the mapping between the controls and what happens in the experience?

– Everyone was able to figure it out, sometimes there would be a little confusion in the beginning due to multiple potentiometers though.

What parts of the experience are working well? What areas could be improved?

– The generative gradient tends to produce a pleasant reaction, so I would say that works well. The button perhaps is the one that is not too reliable.

What parts of your project did you feel the need to explain? How could you make these areas more clear to someone that is experiencing your project for the first time?

-I would often receive questions as to how many stages there will be in total, therefore I decided to add this information to the instructions page.

User testing video:

I gave a user a try of the project after I finished the pairing and connection, and he was able to figure out what everything meant. He was very smart and was able to figure out what should be done even with my hints being somewhat not quite directional. Also, my intention for this project is to let the user figure out what everything does because one thing about cats is that it is hard to get what they mean, and finding out what everything is is part of the process. Eventually, my tester understood everything and had fun playing around with it.

During the testing, basically, everything turned out well. The only thing that might be improved is that sometimes when you are obviously mean towards the cat, she becomes more affectionate and happier, which should not be happening. That is also why I added to the notes that the cat’s change in emotions can be quite unpredictable. It is the problem with the AI model, so I don’t have a fix for it. Another thing is that the API only accepts three inputs per minute. therefore if the user input too fast the program might crash because the parseInt function can get nothing and the mood becomes corrupted. I couldn’t think of a way that could fix this.

For now, I think I don’t need to explain anything more because the exploration is part of the process. but I might find the need to do further explaining when giving this to a wider audience. I could add a page to the “help” button to explain the whole mechanism but I truly believe that would decrease the fun and make it really a simulator instead of an experience.

User Testing:

After finalizing my project and creating a prototype of the box, I asked my friend to test out my code and this is a video of her trying it out.

*I am still in the process of creating a nice box for my project so it would look nicer than what is shown in the video.

• Are they able to figure it out? Where do they get confused and why? Do they understand the mapping between the controls and what happens in the experience?

Yes, She was able to figure it out pretty easily. My project is pretty easy to understand and link the

• What parts of the experience are working well? What areas could be improved?

Mostly, everything is working well, the only thing that might need some improvement is the sensitivity of the button because sometimes it loops through multiples rooms according to the force of the button press.

• What parts of your project did you feel the need to explain? How could you make these areas more clear to someone that is experiencing your project for the first time?

The only part I felt I needed to explain in the beginning was that each room has a different sound file, but she managed to figure it out pretty quickly. The description on the front page explains everything and the labels on the buttons also provide some info on what each button is for.

Crack the Code!

In this project, I’ve developed a “Crack the Code” puzzle to unlock a wooden box. The user receives four sets of hints to guess the three-digit code required to open the safe. Using the knob and button on the box, the user inputs each digit of the code. When the correct code is entered, the box unlocks, and the user wins the chocolate candies inside! Until the correct code is guessed, the box remains locked.

Video Demonstration of the Project

Interaction Design: The user is initially provided instructions on how to interact with the box through the computer screen. The red button on the box not only enters digits but also serves as a “start” and “restart” button for the game, functioning as a navigation button.

Arduino Sketch: My implementation involves using a servo motor, potentiometer, and button. The servo motor locks and unlocks the box by turning to 90 degrees when locked and 0 degrees when unlocked. The potentiometer changes the digits when the user inputs the code, mapping different resistances to digits from 0 to 10 (mapped to 0-9 initially, but because of display issues, extended to 10). The button navigates the game or inputs digits based on the game state. These data are sent to p5.js, which in turn determines whether to open the box or not.

const int servoPin = 9;
const int buttonPin = 3;
const int potentioPin = A1;
const int ledPin = 11;

#include <Servo.h>
Servo myservo;

int potValue;
bool locked = true;
int digit;

void setup() {
  pinMode(ledPin, OUTPUT);
  digitalWrite(ledPin, HIGH);
  delay(1000);
  digitalWrite(ledPin, LOW);

  myservo.attach(servoPin);
  pinMode(buttonPin, INPUT_PULLUP); 

  Serial.begin(9600);

  // Always lock the safe at the beginning of the program
  unlock();
  delay(2000);
  lock();
}

void loop() {
  // Check if data is available to read from serial
  if (Serial.available() > 0) {
    // Read the incoming data
    String receivedData = Serial.readStringUntil('\n');
    
    // Print received data to the Serial Monitor
    Serial.println(receivedData);
    
    // Check if the received data matches 'true' (to unlock)
    if (receivedData == "true") {
      unlock(); // Unlock the safe
    } else if (receivedData == "false") {
       lock(); // Unlock the safe
    }
  }

  int buttonState = digitalRead(buttonPin);
  potValue = analogRead(potentioPin);
  digit = map(potValue, 0, 1023, 0, 10);

  // Print values to Serial Monitor in a single line
  Serial.print(digit);
  Serial.print(", ");
  Serial.print(buttonState == LOW ? "false" : "true"); // Check if button is pressed
  Serial.print(", ");
  Serial.println(locked ? "true" : "false");
}

void lock() {
  myservo.write(0); // Set the servo to lock the safe (position 0)
  locked = true; // Update the locked status
}

void unlock() {
  myservo.write(90); // Set the servo to unlock the safe (position 90)
  locked = false; // Update the locked status
}

P5.js Sketch: The primary instructions and hints are presented to the user in the p5 sketch. There are three different code sets, one of which is chosen as the code to open the box at the game’s start. I was initially planning to code different sets of hints and it was a difficult process, I changed it to three sets of codes for each game round. For wrong inputs, there’s sound and visual feedback. I aimed for fun and engaging screen slides, including real box images for clear user instructions.

The p5 sketch receives data from the Arduino potentiometer to display corresponding digits. When the game state ends, it sends a signal to Arduino to open the box, and the servo motor complies.

 

In this project, I’m particularly proud of the box I’ve created. Despite the door hinges don’t function perfectly as intended, the box maintains a good overall shape. The box is made out of ply wood. Creating the box involved learning Adobe Illustrator to sketch precise and accurate measurements for the wooden parts, which was a challenging but rewarding process.

For future improvements, placing the Arduino board inside the box is important thing to do. It was part of the initial plan, but due to incorrect box measurements the Arduino couldn’t fit in the box. Moreover, improving the box door for sturdiness is crucial. I am also considering to add background sounds for screen feedback interactions and that would enhance the overall experience. 

Overall, the project was an enjoyable and rewarding process, and I learned a lot more about merging creative design, technical use of Arduino and P5.js, and problem-solving to deliver an engaging and interactive experience.

Now that it’s *actually* completed, I can post the actual documentation! Introducing my final project: “StarScape”.

Concept:

For my midterm project, I made a retro game, so this time I wanted to make something more artsy, i.e. focused more on the display and visuals, rather than having a linear progression of events that have an end-goal. Personally, my favorite type of interactive art installations are the ones that make use of lights (some very primitive part of my brain still goes “Wow” like a child when I see lights turn on and off). Especially inspired by N O W I S W H E N W E A R E (the stars), I wanted to make a piece that simulates the stars and has a corresponding display of lights.

The interaction is relatively straightforward: the user presses the Arcade buttons, which will trigger a corresponding movement on the screen and a light sequence on the RGB strip.

Implementation:

Hardware:

Four arcade LED buttons, and a NeoPixel RGB Strip

Software:

For this project, I used p5.js and the Arduino IDE.

In the p5.js code, there is a Galaxy object that contains all the particles (and has the methods that trigger the changes). Everytime the code detects that the user has pressed a button on the Arduino, the repective method for that button is called. Each method (

The p5.js code is scattered across three files:

particle.js:

// Reference: https://p5js.org/examples/simulate-particles.html


// this class describes the properties of a single particle.
class Particle {
// setting the co-ordinates, radius and the
// speed of a particle in both the co-ordinates axes.
  constructor(x, y, particleColor, strokeColor){
    // this.x = random(-rangeBuffer, width + rangeBuffer);
    // this.y = random(-rangeBuffer, height + rangeBuffer);
    this.particleColor = particleColor;
    this.strokeColor = strokeColor;
    this.x = x;
    this.y = y;
    this.r = random(1,5);
    this.xSpeed = random(-2,2);
    this.ySpeed = random(-1,1.5);
  }

// creation of a particle.
  createParticle() {
    noStroke();
    fill(this.particleColor);
    circle(this.x,this.y,this.r);
  }

// setting the particle in motion.
  moveParticle() {
    if(this.x < 0 || this.x > width)
      this.xSpeed*=-1;
    if(this.y < 0 || this.y > height)
      this.ySpeed*=-1;
    this.x+=this.xSpeed;
    this.y+=this.ySpeed;
  }

// this function creates the connections(lines)
// between particles which are less than a certain distance apart
  joinParticles(particles) {
    particles.forEach(element =>{
      let dis = dist(this.x,this.y,element.x,element.y);
      if(dis<85) { 
        stroke(this.strokeColor);
        line(this.x,this.y,element.x,element.y);
      }
    });
  }
  
  //make the particle move in a specific direction
  moveInDirection(xMovement, yMovement) {
    this.x += xMovement;
    this.y += yMovement;
  }

}

galaxy.js:

let jitterAmount = 5; //the distance particles jitter
let jitterIterations = 5; // Number of jitter iterations

//variables for the particles colors
let galaxyColor = "rgba(200,169,169,0.5)"; //White
let galaxyStroke = "rgba(255,255,255,0.04)";
let heartColor = "rgba(255, 100, 100, 0.5)"; //Pink
let heartStroke = "rgba(255, 150, 150, 0.04)";
let spiralColor = "rgba(184, 134, 11, 0.5)"; // Dark Golden 
let spiralStroke = "rgba(255, 215, 0, 0.04)";
let convergenceColor = "rgba(100, 149, 237, 0.5)"; // Blue
let convergenceStroke = "rgba(173, 216, 230, 0.04)";
let scatterColor = "rgba(60, 179, 113, 0.5)"; // Green color
let scatterStroke = "rgba(173, 255, 47, 0.04)";

// function that calculates the center of the particle cluster
function calculateCenter(particleArray) {
  let centerX = 0;
  let centerY = 0;
  for (let i = 0; i < particleArray.length; i++) {
    centerX += particleArray[i].x;
    centerY += particleArray[i].y;
  }

  centerX /= particleArray.length;
  centerY /= particleArray.length;

  return [centerX, centerY];
}

class Galaxy {
  constructor() {
    this.num = 350; //number of particles in the galaxy
    this.rotationSpeed = 0.002;
    this.radius = max(width / 2, height / 2);
    this.particles = [];
    this.centerX = 0;
    this.centerY = 0;
    this.beforeMovement = [];

    // initialize the particles to be scattered across the canvas in a circular distribution
    for (let i = 0; i < this.num; i++) {
      let angle = random(TWO_PI); //generate a random angle btwn 0 and 2π radians
      let r = sqrt(random()) * this.radius; // random radius (limited to the MAXRADIUS of the distribution)
      //calculate the x and y coordinates for the particle based on polar coordinates (angle and radius), converting them to Cartesian coordinates
      let particleX = width / 2 + r * cos(angle);
      let particleY = height / 2 + r * sin(angle);
      //add the particle to the array
      this.particles.push(
        new Particle(
          particleX,
          particleY,
          galaxyColor,
          galaxyStroke,
          "rgba(255,255,255,0.04)"
        )
      );
    }

    let center = calculateCenter(this.particles);
    this.centerX = center[0];
    this.centerY = center[1];
  }

  //move the entire cluster in a specific direction
  moveGalaxy(xMovement, yMovement) {
    for (let i = 0; i < this.particles.length; i++) {
      this.particles[i].moveInDirection(xMovement, yMovement);
    }
  }

  moveGalaxyRandom() {
    for (let i = 0; i < this.particles.length; i++) {
      // Generate random movement for each particle
      let xMovement = random(-2, 2); // Random movement along the x-axis
      let yMovement = random(-2, 2); // Random movement along the y-axis
      this.particles[i].moveInDirection(xMovement, yMovement);
    }
  }

  //move the entire galaxy downwards
  moveGalaxyDownwards() {
    let iterations = 0;
    let prevPositions = [];
    function moveDown(particleArray) {
      if (iterations < 30) {
        // Adjust the number of iterations as needed
        for (let i = 0; i < particleArray.length; i++) {
          if (iterations == 0) {
            prevPositions.push([particleArray[i].x, particleArray[i].y]);
          }
          particleArray[i].y += 2; // Move particles downwards
        }
        iterations++;

        setTimeout(() => moveDown(particleArray), 10);
      }
    }

    moveDown(this.particles);
    this.returnToOriginalPositions(prevPositions);
  }

  //return the particles to the position that they were in before a certain position was enacted
  returnToOriginalPositions() {
    for (let i = 0; i < this.particles.length; i++) {
      let direction = createVector(
        this.beforeMovement[i][0] - this.particles[i].x,
        this.beforeMovement[i][1] - this.particles[i].y
      );
      this.particles[i].x += direction.x * 0.1;
      this.particles[i].y += direction.y * 0.1;
    }
  }

  //rotate the galaxy
  // Function to rotate the galaxy by a given rotation speed
  rotateGalaxy(rotationSpeed) {
    // Loop through all particles in the galaxy
    for (let i = 0; i < this.particles.length; i++) {
      // Calculate the angle between the particle's position and the center of the canvas
      let angle = atan2(
        this.particles[i].y - height / 2, // Y-component distance from the center of the canvas
        this.particles[i].x - width / 2 // X-component distance from the center of the canvas
      );

      // Add the rotation speed to the angle to rotate the particle
      angle += rotationSpeed;

      // Calculate the distance from the particle to the center of the canvas
      let radius = dist(
        width / 2, // X-coordinate of the center of the canvas
        height / 2, // Y-coordinate of the center of the canvas
        this.particles[i].x, // X-coordinate of the particle
        this.particles[i].y // Y-coordinate of the particle
      );

      // Update the particle's position based on the new angle and radius
      this.particles[i].x = width / 2 + radius * cos(angle);
      this.particles[i].y = height / 2 + radius * sin(angle);
    }

    // Calculate the new center of the galaxy based on the updated particle positions
    let center = calculateCenter(this.particles);

    // Update the center X and Y coordinates of the galaxy
    this.centerX = center[0];
    this.centerY = center[1];
  }

  // Function to jitter (move randomly) particles within a given limit
  jitterParticles() {
    let iterations = 0; // Variable to track the number of iterations for jittering
    let prevPositions = []; // Array to store previous positions of particles

    // Inner function to perform the actual jittering of particle positions recursively
    function jitter(particleArray) {
      if (iterations < 10) {
        // Perform jittering for 10 iterations
        for (let i = 0; i < particleArray.length; i++) {
          // Store the previous positions of particles before jittering
          prevPositions.push([particleArray[i].x, particleArray[i].y]);

          // Move particles randomly within a specific range (jitterAmount)
          particleArray[i].x += random(-jitterAmount, jitterAmount) * 4;
          particleArray[i].y += random(-jitterAmount, jitterAmount) * 4;

          // On the first iteration, randomly change the color of some particles
          if (iterations == 0) {
            let changeColor = random(0, 1);
            if (changeColor > 0.5) {
              particleArray[i].particleColor = scatterColor; // Change particle color
              particleArray[i].strokeColor = scatterStroke; // Change stroke color
            } else if (changeColor < 0.3) {
              particleArray[i].particleColor = galaxyColor; // Restore particle color
              particleArray[i].strokeColor = galaxyStroke; // Restore stroke color
            }
          }
        }

        iterations++; // Increment the iteration count
        // Use setTimeout to call the jitter function recursively after a delay of 10 milliseconds
        setTimeout(() => jitter(particleArray), 10);
      }
    }

    // Start the jittering process for the current set of particles
    jitter(this.particles);

    // Save the positions of particles before the movement for reference
    this.beforeMovement = prevPositions;

    // Calculate the new center of the particle set after jittering
    let center = calculateCenter(this.particles);

    // Update the center X and Y coordinates of the particle set
    this.centerX = center[0];
    this.centerY = center[1];
  }

  // Function to jitter particles upwards within a given limit
  jitterParticlesUpwards() {
    let iterations = 0; // Variable to track the number of iterations for jittering upwards

    // Inner function to perform the upward jittering of particle positions recursively
    function jitterUpwards(particleArray) {
      if (iterations < jitterIterations) {
        // Perform upward jittering for a specified number of iterations
        for (let i = 0; i < particleArray.length; i++) {
          // Move particles randomly within a specific range horizontally (x-axis)
          // Move particles upwards by adjusting the y-coordinate (subtracting from y-axis)
          particleArray[i].x += random(-jitterAmount, jitterAmount) * 4;
          particleArray[i].y -= random(0, jitterAmount) * 4; // Adjusting y coordinate to move particles upwards
        }

        iterations++; // Increment the iteration count
        // Use setTimeout to call the jitterUpwards function recursively after a delay of 10 milliseconds (adjustable)
        setTimeout(() => jitterUpwards(particleArray), 10); // Adjust timeout as needed for speed
      }
    }

    // Start the upward jittering process for the current set of particles
    jitterUpwards(this.particles);
  }

  jitterParticlesCloser() {
    let iterations = 0;
    let prevPositions = [];
    function jitterCloser(particleArray) {
      if (iterations < jitterIterations) {
        for (let i = 0; i < particleArray.length; i++) {
          prevPositions.push([particleArray[i].x, particleArray[i].y]);
          let xOffset = random(-jitterAmount, jitterAmount) * 0.4;
          let yOffset = random(-jitterAmount, jitterAmount) * 0.4;

          particleArray[i].x += xOffset * 0.5; // Adjust x-coordinate to bring particles closer
          particleArray[i].y += yOffset * 0.5; // Adjust y-coordinate to bring particles closer
        }
        iterations++;
        setTimeout(() => jitterCloser(particleArray), 10); // Adjust timeout for speed of jittering
      }
    }

    jitterCloser(this.particles);
    this.beforeMovement = prevPositions;
  }

  jitterParticlesTowardsCenter() {
    let iterations = 0;
    let totalIterations = 7;
    let jitterAmnt = 5;
    const convergenceRate = 0.05; // Rate at which particles converge towards the center
    // console.log("woooo big function");
    function jitter(particleArray, centralY, centralX) {
      // console.log("woooo function");
      if (iterations < totalIterations) {
        // console.log("woooo iterations");
        for (let i = 0; i < particleArray.length; i++) {
          // Calculate distance to the center
          const distanceX = centralX - particleArray[i].x;
          const distanceY = centralY - particleArray[i].y;

          // Move particles closer together
          particleArray[i].x += random(-jitterAmnt, jitterAmnt) * 6;
          particleArray[i].y += random(-jitterAmnt, jitterAmnt) * 6;

          // Move particles towards the center
          particleArray[i].x += distanceX * convergenceRate;
          particleArray[i].y += distanceY * convergenceRate;

          if (iterations == 0) {
            let changeColor = random(0, 1);
            if (changeColor > 0.5) {
              particleArray[i].particleColor = convergenceColor;
              particleArray[i].strokeColor = convergenceStroke;
            }
          }
        }
        iterations++;
        setTimeout(() => jitter(particleArray, centralX, centralY), 0.5); // Adjust timeout as needed for speed
      }
    }

    jitter(this.particles, this.centerX, this.centerY);
  }

  explodeParticles() {
    let iterations = 0;
    const explodeIterations = 30; // Adjust the number of iterations for the explosion

    function explode(particleArray) {
      if (iterations < explodeIterations) {
        // Calculate the center of the galaxy (average position of all particles)
        let centerX = 0;
        let centerY = 0;

        for (let i = 0; i < particleArray.length; i++) {
          centerX += particleArray[i].x;
          centerY += particleArray[i].y;
        }

        centerX /= particleArray.length;
        centerY /= particleArray.length;

        for (let i = 0; i < particleArray.length; i++) {
          // Move particles away from the center of the galaxy
          let deltaX = (particleArray[i].x - centerX) / 15;
          let deltaY = (particleArray[i].y - centerY) / 15;

          // Adjust the particles' positions
          particleArray[i].x += deltaX * 0.1; // Adjust the factor to control the speed of explosion
          particleArray[i].y += deltaY * 0.1; // Adjust the factor to control the speed of explosion
        }

        iterations++;
        setTimeout(() => explode(particleArray), 10); // Adjust timeout as needed
      }
    }

    explode(this.particles);
  }

  moveParticlesInHeart() {
    let iterations = 0;
    let prevPositions = [];
    let heartIterations = 30;

    function moveTowardsHeart(particleArray, heartParticles) {
      if (iterations < heartIterations) {
        for (let i = 0; i < heartParticles.length; i++) {
          prevPositions.push([particleArray[i].x, particleArray[i].y]);

          // Calculate the movement towards the heart shape
          let targetX = heartParticles[i].x;
          let targetY = heartParticles[i].y;

          // Update particle positions towards the heart shape
          particleArray[i].x += (targetX - particleArray[i].x) * 0.4; // Adjust the animation speed as needed
          particleArray[i].y += (targetY - particleArray[i].y) * 0.4; // Adjust the animation speed as needed

          particleArray[i].particleColor = heartColor; // Change the color to red during animation
          particleArray[i].strokeColor = heartStroke; // Change the stroke color to blue during animation
        }

        iterations++;
        setTimeout(() => moveTowardsHeart(particleArray, heartParticles), 10); // Adjust timeout as needed for speed
      }
    }

    let heartParticles = []; // Define heart shape particles here

    // Calculate heart shape particles as before
    let spacing = 15; // Adjust this for the heart shape

    for (let angle = 0; angle < TWO_PI; angle += 0.1) {
      // Calculate x and y coordinates for the heart shape using mathematical functions
      let x = 16 * pow(sin(angle), 3);
      let y = -(
        (
          13 * cos(angle) - // First circular pattern
          5 * cos(2 * angle) - // Second circular pattern with twice the frequency
          2 * cos(3 * angle) - // Third circular pattern with three times the frequency
          cos(4 * angle)
        ) // Fourth circular pattern with four times the frequency
      );

      // Scale the coordinates by a spacing factor
      x *= spacing;
      y *= spacing;

      // Shift the heart shape to the center of the canvas
      x += width / 2;
      y += height / 2;

      // Store the calculated x and y coordinates as a vector in the heartParticles array
      heartParticles.push(createVector(x, y));
    }

    moveTowardsHeart(this.particles, heartParticles);

    this.beforeMovement = prevPositions;
    let center = calculateCenter(this.particles);
    this.centerX = center[0];
    this.centerY = center[1];
  }

  moveParticlesInSpiral() {
    let iterations = 0;
    let prevPositions = [];
    let spiralIterations = 35;

    function moveTowardsSpiral(particleArray, spiralParticles) {
      if (iterations < spiralIterations) {
        for (let i = 0; i < spiralParticles.length; i++) {
          prevPositions.push([particleArray[i].x, particleArray[i].y]);

          // Calculate the movement towards the spiral shape
          let targetX = spiralParticles[i].x;
          let targetY = spiralParticles[i].y;

          // Update particle positions towards the spiral shape
          particleArray[i].x += (targetX - particleArray[i].x) * 0.4; // Adjust the animation speed as needed
          particleArray[i].y += (targetY - particleArray[i].y) * 0.4; // Adjust the animation speed as needed

          particleArray[i].particleColor = spiralColor; // Change the color to red during animation
          particleArray[i].strokeColor = spiralStroke;
        }

        iterations++;
        setTimeout(() => moveTowardsSpiral(particleArray, spiralParticles), 10); // Adjust timeout as needed for speed
      }
    }

    let spiralParticles = []; // Define spiral shape particles here

    // Calculate spiral shape particles
    let spacing = 10; // Adjust this for the spiral shape
    for (let angle = 0; angle < 6 * PI; angle += 0.1) {
       // Calculate x and y coordinates for the spiral shape using trigonometric functions
      let x = angle * cos(angle) * spacing + width / 2;
      let y = angle * sin(angle) * spacing + height / 2;
      spiralParticles.push(createVector(x, y));
    }

    moveTowardsSpiral(this.particles, spiralParticles);

    this.beforeMovement = prevPositions;
    let center = calculateCenter(this.particles);
    this.centerX = center[0];
    this.centerY = center[1];
  }
  
  //function to actually draw the galaxy
  drawGalaxy() {
    for (let i = 0; i < this.particles.length; i++) {
      this.particles[i].createParticle(); // Create a single particle
      this.particles[i].joinParticles(this.particles.slice(i)); // Join the created particle with others
    }
  }
}

sketch.js:

//background music source: https://freesound.org/people/Seth_Makes_Sounds/sounds/701610/

let numParticles = 200;
let nightParticles;
let portConnected = false;
let showStartScreen = true;
let font, font2;
let bgMusic;

//variables to store data from Arduino board
let redAction = 0;
let blueAction = 0;
let yellowAction = 0;
let greenAction = 0;

//flags to check if the action was performed
let redActionPerformed = false;
let yellowActionPerformed = false;
let greenActionPerformed = false;
let blueActionPerformed = false;

let lastInteractionTime = 0;
const idleTimeThreshold = 60000;

let amp;


//start screen for user
function startScreen() {
  textFont(font);
  textSize(80);
  fill("white");
  let message = "StarScape";
  // console.log(width, height);
  let textW = textWidth(message);
  let textH = textAscent() + textDescent();


  let centerX = width / 2;
  let centerY = height / 2;

  // Set text alignment to center and display the text
  textAlign(CENTER, CENTER);
  text(message, centerX, centerY);

  textFont(font2);
  textSize(20);

  let captionX = centerX;
  let captionY = centerY + 80;

  text("Click anywhere to begin. Click again to restart.", captionX, captionY);
}

function preload() {
  soundFormats("mp3", "ogg");
  bgMusic = loadSound("/sounds/bgMusic2.mp3");
  font = loadFont("fonts/NewYork.otf");
  font2 = loadFont("fonts/AppleGaramond-LightItalic.ttf");
}

function setup() {
  //responsive canvas set to the dimensions of the window
  createCanvas(windowWidth, windowHeight);
  //initialize particles
  nightParticles = new Galaxy();
  //loop the music
  bgMusic.loop();
  amp = new p5.Amplitude();
}

function draw() {
  background("#0f0f0f");
  
  if (showStartScreen) {
    startScreen();
  } else {
    //if red button pressed, particles jitter + scatter
    if (redAction && !redActionPerformed) {
      nightParticles.jitterParticles();
      redActionPerformed = true;
    //if yellow button is pressed, particles cluster together
    } else if (yellowAction && !yellowActionPerformed) {
      nightParticles.jitterParticlesTowardsCenter();
      yellowActionPerformed = true;
    //if the green button is pressed, particles form a heart
    } else if (greenAction && !greenActionPerformed) {
      nightParticles.moveParticlesInHeart();
      greenActionPerformed = true;
    // if blue button, the particles form a spiral
    } else if (blueAction && !blueActionPerformed) {
      nightParticles.moveParticlesInSpiral();
      blueActionPerformed = true;
    }
    
    //the particles are continuously rotating, with a speed proportional to the amplitude of the music
    let vol = amp.getLevel();
    let pSpeed = map(vol, 0, 1, 0, 0.005);
    nightParticles.rotateGalaxy(pSpeed);
    nightParticles.drawGalaxy();
  }
}

function readSerial(data) {
  ////////////////////////////////////
  //READ FROM ARDUINO HERE
  ////////////////////////////////////

  if (data != null) {
    // make sure there is actually a message
    // split the message
    let fromArduino = split(trim(data), ",");
    // console.log(fromArduino);
    // if the right length, then proceed
    if (fromArduino.length == 4) {
      // only store values here
      // do everything with those values in the main draw loop

      // We take the string we get from Arduino and explicitly
      // convert it to a number by using int()
      redAction = int(fromArduino[0]);
      yellowAction = int(fromArduino[1]);
      greenAction = int(fromArduino[2]);
      blueAction = int(fromArduino[3]);
      
      //reset the actionsPerformed to false everytime data is read from the arduino
      redActionPerformed = false;
      yellowActionPerformed = false;
      greenActionPerformed = false;
      blueActionPerformed = false;
    }

    //////////////////////////////////
    //SEND TO ARDUINO HERE (handshake)
    //////////////////////////////////
    let sendToArduino = "\n";
    writeSerial(sendToArduino);
  }
}

function mousePressed() {
  //first, connect the port
  if (!portConnected) {
    setUpSerial();
    portConnected = true;
  } else {
    //after port is connected, use mouse press to start (and restart)
    showStartScreen = !showStartScreen;
    //if the game is restarting, create a new instance of Galaxy class
    if (showStartScreen) {
      nightParticles = new Galaxy();
    }
  }
}

The Arduino code sets up the serial connection with the p5.js sketch, and then reads the data from the buttons. After detecting a button press, it sends it to the sketch. After sending it to the sketch, it triggers a light sequence on the strip. The code in the Arduino IDE: