week13&14 – final project documentation

USER TESTING:
My deliverable was still at a premature stage so I was unable to film a user-testing video, however I was able to get verbal feedback on my p5 program design (and so have adjusted my code accordingly). Initially, I had no text to indicate any sort of instructions or context, as I had thought the arrow buttons were enough to prompt users into action. However, my sister advised against this and suggested I include a phrase or 2 to provide basic information as to what my project is about – even more so since I did not have a start/ introductory screen. Another feedback I got was regarding the separate display screen for when the recommended playlist sounds – which was mentioned in the previous documentation. I was initially just planning to display the user’s chosen personalised cassette however my sister thought it to be too static, commenting that it was lacking flair. I starting brainstorming other potential display screens I could have but the one that resonated most with me was actually animating a rolling cassette tape, of course this would mean I had to create animations for all possible cassette tape designs.

Final project video: https://youtu.be/t_wIKjY5s1o

My final project concept is based off of my midterm project. In the previous midterm, I made a digital radio that played 2 meaningful songs. Each song displayed (what was supposed to be) a dynamic background of a personal memory associated with the songs. My final project builds off on this idea but with an added twist. You are able to customise your own cassette tape and based on your choices, it assembles a recommended playlist. There are 3 choices in each stage of customizing for the personalisation of your cassette and when finished, it plays a recommended playlist of 4 songs, each similar in genre. You can adjust the volume, use a skip function and reset the entire experience through physical means (buttons and potentiometer). Whilst my midterm involved a sense of personal intimacy, I tried to make this project evoke a more shared intimacy. Music is very personal to me and by sharing it with others, I am able to show a partial extension of my own identity which can be reciprocated by external users. Speaking from personal experience, it cements a bond quicker.

//arduino code:

int button1Pin = A2; 
int button2Pin = A3; 
int potentiometerPin = A0;

void setup() {
  // Start serial communication so we can send data
  // over the USB connection to our p5js sketch
  Serial.begin(9600);

  // We'll use the built-in LED as a status output.
  pinMode(LED_BUILTIN, OUTPUT);
}

void loop() {
  // Read button states and potentiometer value
  int button1State = digitalRead(button1Pin);
  int button2State = digitalRead(button2Pin);
  int potValue = analogRead(potentiometerPin);

  // Send data to p5.js
  Serial.print(button1State);
  Serial.print(",");
  Serial.print(button2State);
  Serial.print(",");
  Serial.println(potValue);

  delay(100); // Adjust delay as needed
}

This final project was so so painstakingly challenging and was such an arduous experience in general it took my soul and a chunk of hair by the time I was finished. Every portion of code had a bug and debugging it took a minimum of 1.5 hours (rarely as I wasn’t so lucky) and a maximum of 7 hours+. I will most likely never forget such an experience. The first portion – p5.js, was doable, yes there were countless debugging that was really frustrating but the cherry on the cake was the second portion – serial communication of the final project. The process of this entire final was tedious overall:

PROCESSES – ART STAGE:
I first created 3 playlists of 4 songs. Then using pinterest inspirations of vintage cassette tapes, I drew each stage: cassette base, cassette sticker, cassette detail using procreate. I illustrated specific combinations of these cassettes to equate to a certain playlist and I drew the details to correspond with the overall music vibe and aesthetic: (see below: 1) result1, 2) result2, 3) result3). As mentioned in my user-testing documentation section,  I wanted to create an animation of the cassette tape rolling when users entered the final stage: music playing. The only plausible way was to create a gif file containing such animation. Because there are 3 choices for each 3 stages and 3 different combinations users could select, it meant I had to create animations for a total of 27 cassettes, hence why it was so time consuming.

PROCESSES – P5.JS:
Essentially both coding experiences were one I do not want to remember… the endless bug fixes, the endless error messages on the console, it was just incredibly stressful. However the code that evoked the most stress and hence, I’m most proud of was attributing the corresponding gif files to every possible indices user could end up with – likewise with creating the cassette animations, there were 27 different combinations. This meant that the program had to store the index chosen at each stage and use this information to call upon a gif file with the corresponding index. This was one of those sections that took 7+ hours to debug and code. I didn’t know where to start and how, so, like I always did with previous assignments, I began researching and looking for codes that fulfilled similar instances on google. Then came the experimentation and checking using console.log. Through this I was able to learn syntax I had never encountered before and this acted as a sort of revelation for me. Here is the relevant code section:

//
const gifFilenameMap = { //attributing gif file pathway to user selected indices
//for cassetteBase[0]
  "0_0_0": "gifs/result1_prpl1.gif",
  "0_0_1": "gifs/result1_prpl2.gif",
  "0_0_2": "gifs/result1_prpl3.gif",
  "0_1_0": "gifs/result1_green1.gif",
  "0_1_1": "gifs/result1_green2.gif",
  "0_1_2": "gifs/result1_green3.gif",
  "0_2_0": "gifs/result1_grey1.gif",
  "0_2_1": "gifs/result1_grey2.gif",
  "0_2_2": "gifs/result1_grey3.gif",

//for cassetteBase[1]
  "1_0_0": "gifs/result2_prpl1.gif",
  "1_0_1": "gifs/result2_prpl2.gif",
  "1_0_2": "gifs/result2_prpl3.gif",
  "1_1_0": "gifs/result2_green1.gif",
  "1_1_1": "gifs/result2_green2.gif",
  "1_1_2": "gifs/result2_green3.gif",
  "1_2_0": "gifs/result2_grey1.gif",
  "1_2_1": "gifs/result2_grey2.gif",
  "1_2_2": "gifs/result2_grey3.gif",

//for cassetteBase[2]
  "2_0_0": "gifs/result3_prpl1.gif",
  "2_0_1": "gifs/result3_prpl2.gif",
  "2_0_2": "gifs/result3_prpl3.gif",
  "2_1_0": "gifs/result3_green1.gif",
  "2_1_1": "gifs/result3_green2.gif",
  "2_1_2": "gifs/result3_green3.gif",
  "2_2_0": "gifs/result3_grey1.gif",
  "2_2_1": "gifs/result3_grey2.gif",
  "2_2_2": "gifs/result3_grey3.gif",
};

//generates gif filename based on indices of selected cassette components
function generateGifFilename(baseIndex, stickerIndex, detailIndex) {
  return gifFilenameMap[`${baseIndex}_${stickerIndex}_${detailIndex}`]; //generating filename using map => e.g., 2_1_0
}

function determineResult() {
...
//generating filename (e.g., "1_2_3") based on indices of selected components
  const gifFilename = generateGifFilename(selectedBaseIndex, selectedStickerIndex, selectedDetailIndex);
  gifElement = createImg(gifFilename, "selectedGif"); // displaying selected gif on canvas
  gifElement.size(imageWidth, imageHeight);
  gifElement.position(imagePosition.x, imagePosition.y);
}

PROCESSES – ARDUINO + SERIAL COMMUNICATION:
Serial communication was one I had the most issues with. I used the existing serial communication code (in mang’s lecture notes) for both arduino and p5.js and altered it around my main piece of code however, problem 1) there seemed to be issues with p5.js and arduino exchanging data, hence it was impossible to know whether the physical wiring of the components on the breadboard was the problem or whether it was the code itself that was causing issues. 2) I continually experienced error messages stating that there was a network error hence I was unable to connect to a serial port. Both cases required patience, calmness and perseverance and through this it was engrained into me again the importance of console logging when debugging faulty code. At the start, I wasn’t able to understand the serial communication code that was provided but after the completion of my final project, everything kind of clicked into place.

Regarding attributing functions to the physical components: 2 push buttons and a potentiometer, I was also having major problems with applying my desired functions: play/pause, skip forward, skip backward, to the push buttons. Mapping the volume to the potentiometer value was really easy as something like it had already been done for the serial communication assignment. For the rest, it was a nightmare. I think it was the structure of the code and the specific manner in which I coded that caused so many breakdowns and errors. In the end I was incredibly short for time and so was forced to compensate and only code 1) resetToInitialState, 2) skip forward on loop. when coding for the function: resetToInitialState, 2 problems occurred: 1) gif image appearing over initial state, 2) sound continuing to play regardless of being set to its initial state. With extensive experimentation, I realised that creating new variables to keep track of the states of both the gif and sound was the most simplest and most rational solution – here is the relevant code:

let gifElement; (ADDED) 

//within function determineResult() 
if (gifElement) { (ADDED) 
  gifElement.remove(); //remove existing gifElement if it exists 
} 

const gifFilename = generateGifFilename(selectedBaseIndex, selectedStickerIndex, selectedDetailIndex); 
gifElement = createImg(gifFilename, "selectedGif"); // displaying selected gif on canvas (ADDED) 
gifElement.size(imageWidth, imageHeight); (ADDED) 
gifElement.position(imagePosition.x, imagePosition.y); (ADDED) 

/////////////////////////////////////////////////////////////////////////////////////
let shouldPlayNextSound = true; //(ADDED) 

//within function playNextSound()
if (shouldPlayNextSound) { //(ADDED)
  currentSoundIndex++; // increment sound index
  if (currentStage === 4 && currentSoundIndex >= result1.length) {
    determineResult();
    currentSoundIndex = 0; //reset to the beginning if end is reached
  }
}

//within function resetToInitialState()
shouldPlayNextSound = false; //disable skip function (ADDED)

//stopping all currently playing sounds
for (let i = 0; i < result1.length; i++) {
result1[i].stop();
}
for (let i = 0; i < result2.length; i++) {
result2[i].stop();
}
for (let i = 0; i < result3.length; i++) {
result3[i].stop();
}
shouldPlayNextSound = true; //enable skip function (ADDED)

FINAL REFLECTIONS + FUTURE IMPROVEMENTS:
Whilst it was the most nerve wrecking, anxiety inducing overall experience, since persisting bugs were fixed the day of the IM show, I was quite proud of what I have completed. Whilst the coding aspect of this project was beyond challenging, I can’t deny that it was lowkey fun at the same time – creating a project that involves my passion. To me, it certainly felt like a large leap in the level of difficulty, compared to my midterm project, and this was more so why I am proud of the finished result. For future improvements on the project, perhaps there could be a personality test which based on your selected answers allocates you to a specific design for each stage of the cassette customisation. This way the experience maintains for longer. I also think it builds more excitement and anticipation as to what cassette you’ll end up with. Improvements for the physical aspect of the project would be to build a radio with more extensive functions, like originally planned.

Regarding improvements for future IM projects, I am incredibly motivated to put thought into the building of an exterior because that, at the end of the day, is what can elevate user experience. Since it was my first time both showcasing and attending an IM show, I experienced somewhat of an epiphanous moment. In future classes I will be more mindful in creating a more immersive user experience that is able to appeal to a wider body of people, because whilst mine did have some sort of user experience, it was more so stagnant with limited interaction compared to the other projects that were showcased. Overall I think it was an excellent opportunity to understand the fundamentals of what Interactive Media embodies and it has further propelled my motivation to learn in depth creative coding.

week 11&12 – (final project) preliminary concept | idea finalization | user testing

PRELIMINARY CONCEPT:
I wanted to do some brainstorming regarding the concept of my final project. I wanted to do something that wasn’t too ambitious (considering the myriad of deadlines as a result of finals week and the limited time constraints). I think because I couldn’t think of a theme, ideas were very vast which didn’t help. After some major considerations and scrapping of multiple ideas, I came to the conclusion that I would want to build off on my midterm project to create a physical radio that is controllable through i.e., buttons, potentiometer, power switch. Output would be digital and input would be both digital and physical.

FINALIZED CONCEPT:
I thought creating a purely physical radio with a digital output would not be enough interactivity or at least a meaningful interaction, hence I wanted to perpetuate intimacy. I love music and (to me) what is more intimate than sharing your favourite playlists/ artists with others? I decided to make my final project a 2-part: 1) determine corresponding mood/ personality through personalizing their own cassette tape, 2) according to results, a playlist of 4 songs would be curated – this way there is more intimate and meaningful back and forth. Essentially it would work like those buzzfeed quizzes except physical input via push buttons are possible.

Total physical components would include 3 push buttons and a potentiometer for volume control. My arduino program will read the potentiometer value and buttons’/switch’s state and send the information to p5.js via serial communication. When p5.js receives information about the button’s push state, it will call upon its corresponding function, i.e., pause/play, skip forward/ backward. I will use the mapping function to map the potentiometer value to the volume so that whenever the potentiometer is toggled, p5.js will continuously receive its p value and adjust the volume accordingly. With regards to my p5 program design, I intend to have a cassette displayed in the middle with “select” and arrows buttons. Since users will select designs for each of the 3 stages, the “select” button will store user choices in a separate variable so as to assign a “result1/2/3” playlist. Although undecided, I intend to have a separate display screen for when the recommended playlist sounds to indicate the change of focus from p5 to the actual physical components.

User Testing

Before completing the project, I did some user testing with a small sample of participants. Here is a video of Zion using my project:

User testing stage gave me valuable insight that was difficult to see from my perspective as the creator of the project. Here are the tips that I implemented in the final version:

  • Specifying how many stages (color inputs) there will be, helps the participant to understand the structure of the experience from the start.
  • Starting the instructions by saying what is the main thing that will be required (color input based on prompts).
  • Introducing an additional screen in between the last color input and generated gradient to build some suspension.

Are they able to figure it out? Where do they get confused and why? Do they understand the mapping between the controls and what happens in the experience?

– Everyone was able to figure it out, sometimes there would be a little confusion in the beginning due to multiple potentiometers though.

What parts of the experience are working well? What areas could be improved?

– The generative gradient tends to produce a pleasant reaction, so I would say that works well. The button perhaps is the one that is not too reliable.

What parts of your project did you feel the need to explain? How could you make these areas more clear to someone that is experiencing your project for the first time?

-I would often receive questions as to how many stages there will be in total, therefore I decided to add this information to the instructions page.

Final project: user testing

User testing video:

I gave a user a try of the project after I finished the pairing and connection, and he was able to figure out what everything meant. He was very smart and was able to figure out what should be done even with my hints being somewhat not quite directional. Also, my intention for this project is to let the user figure out what everything does because one thing about cats is that it is hard to get what they mean, and finding out what everything is is part of the process. Eventually, my tester understood everything and had fun playing around with it.

During the testing, basically, everything turned out well. The only thing that might be improved is that sometimes when you are obviously mean towards the cat, she becomes more affectionate and happier, which should not be happening. That is also why I added to the notes that the cat’s change in emotions can be quite unpredictable. It is the problem with the AI model, so I don’t have a fix for it. Another thing is that the API only accepts three inputs per minute. therefore if the user input too fast the program might crash because the parseInt function can get nothing and the mood becomes corrupted. I couldn’t think of a way that could fix this.

For now, I think I don’t need to explain anything more because the exploration is part of the process. but I might find the need to do further explaining when giving this to a wider audience. I could add a page to the “help” button to explain the whole mechanism but I truly believe that would decrease the fun and make it really a simulator instead of an experience.

Final Project User Testing

User Testing:

After finalizing my project and creating a prototype of the box, I asked my friend to test out my code and this is a video of her trying it out.

 

*I am still in the process of creating a nice box for my project so it would look nicer than what is shown in the video.

  • Are they able to figure it out? Where do they get confused and why? Do they understand the mapping between the controls and what happens in the experience?

Yes, She was able to figure it out pretty easily. My project is pretty easy to understand and link the

  • What parts of the experience are working well? What areas could be improved?

Mostly, everything is working well, the only thing that might need some improvement is the sensitivity of the button because sometimes it loops through multiples rooms according to the force of the button press.

  • What parts of your project did you feel the need to explain? How could you make these areas more clear to someone that is experiencing your project for the first time?

The only part I felt I needed to explain in the beginning was that each room has a different sound file, but she managed to figure it out pretty quickly. The description on the front page explains everything and the labels on the buttons also provide some info on what each button is for.

Week 13: Final Project

Crack the Code!

In this project, I’ve developed a “Crack the Code” puzzle to unlock a wooden box. The user receives four sets of hints to guess the three-digit code required to open the safe. Using the knob and button on the box, the user inputs each digit of the code. When the correct code is entered, the box unlocks, and the user wins the chocolate candies inside! Until the correct code is guessed, the box remains locked.

Video Demonstration of the Project

Interaction Design: The user is initially provided instructions on how to interact with the box through the computer screen. The red button on the box not only enters digits but also serves as a “start” and “restart” button for the game, functioning as a navigation button.

 

Beginning of the game. Box is locked and instruction to start is given on the screen.
Playing the game. Entering the digits of the code.
End of the game. Box is open and the candies are inside!!!

Arduino Sketch: My implementation involves using a servo motor, potentiometer, and button. The servo motor locks and unlocks the box by turning to 90 degrees when locked and 0 degrees when unlocked. The potentiometer changes the digits when the user inputs the code, mapping different resistances to digits from 0 to 10 (mapped to 0-9 initially, but because of display issues, extended to 10). The button navigates the game or inputs digits based on the game state. These data are sent to p5.js, which in turn determines whether to open the box or not.

const int servoPin = 9;
const int buttonPin = 3;
const int potentioPin = A1;
const int ledPin = 11;

#include <Servo.h>
Servo myservo;

int potValue;
bool locked = true;
int digit;

void setup() {
  pinMode(ledPin, OUTPUT);
  digitalWrite(ledPin, HIGH);
  delay(1000);
  digitalWrite(ledPin, LOW);

  myservo.attach(servoPin);
  pinMode(buttonPin, INPUT_PULLUP); 

  Serial.begin(9600);

  // Always lock the safe at the beginning of the program
  unlock();
  delay(2000);
  lock();
}

void loop() {
  // Check if data is available to read from serial
  if (Serial.available() > 0) {
    // Read the incoming data
    String receivedData = Serial.readStringUntil('\n');
    
    // Print received data to the Serial Monitor
    Serial.println(receivedData);
    
    // Check if the received data matches 'true' (to unlock)
    if (receivedData == "true") {
      unlock(); // Unlock the safe
    } else if (receivedData == "false") {
       lock(); // Unlock the safe
    }
  }

  int buttonState = digitalRead(buttonPin);
  potValue = analogRead(potentioPin);
  digit = map(potValue, 0, 1023, 0, 10);

  // Print values to Serial Monitor in a single line
  Serial.print(digit);
  Serial.print(", ");
  Serial.print(buttonState == LOW ? "false" : "true"); // Check if button is pressed
  Serial.print(", ");
  Serial.println(locked ? "true" : "false");
}

void lock() {
  myservo.write(0); // Set the servo to lock the safe (position 0)
  locked = true; // Update the locked status
}

void unlock() {
  myservo.write(90); // Set the servo to unlock the safe (position 90)
  locked = false; // Update the locked status
}

P5.js Sketch: The primary instructions and hints are presented to the user in the p5 sketch. There are three different code sets, one of which is chosen as the code to open the box at the game’s start. I was initially planning to code different sets of hints and it was a difficult process, I changed it to three sets of codes for each game round. For wrong inputs, there’s sound and visual feedback. I aimed for fun and engaging screen slides, including real box images for clear user instructions.

The p5 sketch receives data from the Arduino potentiometer to display corresponding digits. When the game state ends, it sends a signal to Arduino to open the box, and the servo motor complies.

 

In this project, I’m particularly proud of the box I’ve created. Despite the door hinges don’t function perfectly as intended, the box maintains a good overall shape. The box is made out of ply wood. Creating the box involved learning Adobe Illustrator to sketch precise and accurate measurements for the wooden parts, which was a challenging but rewarding process.

Cut out frame sketch of the box

For future improvements, placing the Arduino board inside the box is important thing to do. It was part of the initial plan, but due to incorrect box measurements the Arduino couldn’t fit in the box. Moreover, improving the box door for sturdiness is crucial. I am also considering to add background sounds for screen feedback interactions and that would enhance the overall experience. 

Overall, the project was an enjoyable and rewarding process, and I learned a lot more about merging creative design, technical use of Arduino and P5.js, and problem-solving to deliver an engaging and interactive experience.

(Final) Final Project: Documentation

Now that it’s *actually* completed, I can post the actual documentation! Introducing my final project: “StarScape”.

Concept:

For my midterm project, I made a retro game, so this time I wanted to make something more artsy, i.e. focused more on the display and visuals, rather than having a linear progression of events that have an end-goal. Personally, my favorite type of interactive art installations are the ones that make use of lights (some very primitive part of my brain still goes “Wow” like a child when I see lights turn on and off). Especially inspired by N O W I S W H E N W E A R E (the stars), I wanted to make a piece that simulates the stars and has a corresponding display of lights.

The interaction is relatively straightforward: the user presses the Arcade buttons, which will trigger a corresponding movement on the screen and a light sequence on the RGB strip.

Implementation:

Hardware:

Four arcade LED buttons, and a NeoPixel RGB Strip

Software:

For this project, I used p5.js and the Arduino IDE.

In the p5.js code, there is a Galaxy object that contains all the particles (and has the methods that trigger the changes). Everytime the code detects that the user has pressed a button on the Arduino, the repective method for that button is called. Each method (jitterParticles, jitterParticlesCloser, moveParticlesInHeart, moveParticlesInSpiral) does two things: move the particles around, and change their colors.

The p5.js code is scattered across three files:

particle.js:

// Reference: https://p5js.org/examples/simulate-particles.html


// this class describes the properties of a single particle.
class Particle {
// setting the co-ordinates, radius and the
// speed of a particle in both the co-ordinates axes.
  constructor(x, y, particleColor, strokeColor){
    // this.x = random(-rangeBuffer, width + rangeBuffer);
    // this.y = random(-rangeBuffer, height + rangeBuffer);
    this.particleColor = particleColor;
    this.strokeColor = strokeColor;
    this.x = x;
    this.y = y;
    this.r = random(1,5);
    this.xSpeed = random(-2,2);
    this.ySpeed = random(-1,1.5);
  }

// creation of a particle.
  createParticle() {
    noStroke();
    fill(this.particleColor);
    circle(this.x,this.y,this.r);
  }

// setting the particle in motion.
  moveParticle() {
    if(this.x < 0 || this.x > width)
      this.xSpeed*=-1;
    if(this.y < 0 || this.y > height)
      this.ySpeed*=-1;
    this.x+=this.xSpeed;
    this.y+=this.ySpeed;
  }

// this function creates the connections(lines)
// between particles which are less than a certain distance apart
  joinParticles(particles) {
    particles.forEach(element =>{
      let dis = dist(this.x,this.y,element.x,element.y);
      if(dis<85) { 
        stroke(this.strokeColor);
        line(this.x,this.y,element.x,element.y);
      }
    });
  }
  
  //make the particle move in a specific direction
  moveInDirection(xMovement, yMovement) {
    this.x += xMovement;
    this.y += yMovement;
  }

}


galaxy.js:

let jitterAmount = 5; //the distance particles jitter
let jitterIterations = 5; // Number of jitter iterations

//variables for the particles colors
let galaxyColor = "rgba(200,169,169,0.5)"; //White
let galaxyStroke = "rgba(255,255,255,0.04)";
let heartColor = "rgba(255, 100, 100, 0.5)"; //Pink
let heartStroke = "rgba(255, 150, 150, 0.04)";
let spiralColor = "rgba(184, 134, 11, 0.5)"; // Dark Golden 
let spiralStroke = "rgba(255, 215, 0, 0.04)";
let convergenceColor = "rgba(100, 149, 237, 0.5)"; // Blue
let convergenceStroke = "rgba(173, 216, 230, 0.04)";
let scatterColor = "rgba(60, 179, 113, 0.5)"; // Green color
let scatterStroke = "rgba(173, 255, 47, 0.04)";

// function that calculates the center of the particle cluster
function calculateCenter(particleArray) {
  let centerX = 0;
  let centerY = 0;
  for (let i = 0; i < particleArray.length; i++) {
    centerX += particleArray[i].x;
    centerY += particleArray[i].y;
  }

  centerX /= particleArray.length;
  centerY /= particleArray.length;

  return [centerX, centerY];
}

class Galaxy {
  constructor() {
    this.num = 350; //number of particles in the galaxy
    this.rotationSpeed = 0.002;
    this.radius = max(width / 2, height / 2);
    this.particles = [];
    this.centerX = 0;
    this.centerY = 0;
    this.beforeMovement = [];

    // initialize the particles to be scattered across the canvas in a circular distribution
    for (let i = 0; i < this.num; i++) {
      let angle = random(TWO_PI); //generate a random angle btwn 0 and 2π radians
      let r = sqrt(random()) * this.radius; // random radius (limited to the MAXRADIUS of the distribution)
      //calculate the x and y coordinates for the particle based on polar coordinates (angle and radius), converting them to Cartesian coordinates
      let particleX = width / 2 + r * cos(angle);
      let particleY = height / 2 + r * sin(angle);
      //add the particle to the array
      this.particles.push(
        new Particle(
          particleX,
          particleY,
          galaxyColor,
          galaxyStroke,
          "rgba(255,255,255,0.04)"
        )
      );
    }

    let center = calculateCenter(this.particles);
    this.centerX = center[0];
    this.centerY = center[1];
  }

  //move the entire cluster in a specific direction
  moveGalaxy(xMovement, yMovement) {
    for (let i = 0; i < this.particles.length; i++) {
      this.particles[i].moveInDirection(xMovement, yMovement);
    }
  }

  moveGalaxyRandom() {
    for (let i = 0; i < this.particles.length; i++) {
      // Generate random movement for each particle
      let xMovement = random(-2, 2); // Random movement along the x-axis
      let yMovement = random(-2, 2); // Random movement along the y-axis
      this.particles[i].moveInDirection(xMovement, yMovement);
    }
  }

  //move the entire galaxy downwards
  moveGalaxyDownwards() {
    let iterations = 0;
    let prevPositions = [];
    function moveDown(particleArray) {
      if (iterations < 30) {
        // Adjust the number of iterations as needed
        for (let i = 0; i < particleArray.length; i++) {
          if (iterations == 0) {
            prevPositions.push([particleArray[i].x, particleArray[i].y]);
          }
          particleArray[i].y += 2; // Move particles downwards
        }
        iterations++;

        setTimeout(() => moveDown(particleArray), 10);
      }
    }

    moveDown(this.particles);
    this.returnToOriginalPositions(prevPositions);
  }

  //return the particles to the position that they were in before a certain position was enacted
  returnToOriginalPositions() {
    for (let i = 0; i < this.particles.length; i++) {
      let direction = createVector(
        this.beforeMovement[i][0] - this.particles[i].x,
        this.beforeMovement[i][1] - this.particles[i].y
      );
      this.particles[i].x += direction.x * 0.1;
      this.particles[i].y += direction.y * 0.1;
    }
  }

  //rotate the galaxy
  // Function to rotate the galaxy by a given rotation speed
  rotateGalaxy(rotationSpeed) {
    // Loop through all particles in the galaxy
    for (let i = 0; i < this.particles.length; i++) {
      // Calculate the angle between the particle's position and the center of the canvas
      let angle = atan2(
        this.particles[i].y - height / 2, // Y-component distance from the center of the canvas
        this.particles[i].x - width / 2 // X-component distance from the center of the canvas
      );

      // Add the rotation speed to the angle to rotate the particle
      angle += rotationSpeed;

      // Calculate the distance from the particle to the center of the canvas
      let radius = dist(
        width / 2, // X-coordinate of the center of the canvas
        height / 2, // Y-coordinate of the center of the canvas
        this.particles[i].x, // X-coordinate of the particle
        this.particles[i].y // Y-coordinate of the particle
      );

      // Update the particle's position based on the new angle and radius
      this.particles[i].x = width / 2 + radius * cos(angle);
      this.particles[i].y = height / 2 + radius * sin(angle);
    }

    // Calculate the new center of the galaxy based on the updated particle positions
    let center = calculateCenter(this.particles);

    // Update the center X and Y coordinates of the galaxy
    this.centerX = center[0];
    this.centerY = center[1];
  }

  // Function to jitter (move randomly) particles within a given limit
  jitterParticles() {
    let iterations = 0; // Variable to track the number of iterations for jittering
    let prevPositions = []; // Array to store previous positions of particles

    // Inner function to perform the actual jittering of particle positions recursively
    function jitter(particleArray) {
      if (iterations < 10) {
        // Perform jittering for 10 iterations
        for (let i = 0; i < particleArray.length; i++) {
          // Store the previous positions of particles before jittering
          prevPositions.push([particleArray[i].x, particleArray[i].y]);

          // Move particles randomly within a specific range (jitterAmount)
          particleArray[i].x += random(-jitterAmount, jitterAmount) * 4;
          particleArray[i].y += random(-jitterAmount, jitterAmount) * 4;

          // On the first iteration, randomly change the color of some particles
          if (iterations == 0) {
            let changeColor = random(0, 1);
            if (changeColor > 0.5) {
              particleArray[i].particleColor = scatterColor; // Change particle color
              particleArray[i].strokeColor = scatterStroke; // Change stroke color
            } else if (changeColor < 0.3) {
              particleArray[i].particleColor = galaxyColor; // Restore particle color
              particleArray[i].strokeColor = galaxyStroke; // Restore stroke color
            }
          }
        }

        iterations++; // Increment the iteration count
        // Use setTimeout to call the jitter function recursively after a delay of 10 milliseconds
        setTimeout(() => jitter(particleArray), 10);
      }
    }

    // Start the jittering process for the current set of particles
    jitter(this.particles);

    // Save the positions of particles before the movement for reference
    this.beforeMovement = prevPositions;

    // Calculate the new center of the particle set after jittering
    let center = calculateCenter(this.particles);

    // Update the center X and Y coordinates of the particle set
    this.centerX = center[0];
    this.centerY = center[1];
  }

  // Function to jitter particles upwards within a given limit
  jitterParticlesUpwards() {
    let iterations = 0; // Variable to track the number of iterations for jittering upwards

    // Inner function to perform the upward jittering of particle positions recursively
    function jitterUpwards(particleArray) {
      if (iterations < jitterIterations) {
        // Perform upward jittering for a specified number of iterations
        for (let i = 0; i < particleArray.length; i++) {
          // Move particles randomly within a specific range horizontally (x-axis)
          // Move particles upwards by adjusting the y-coordinate (subtracting from y-axis)
          particleArray[i].x += random(-jitterAmount, jitterAmount) * 4;
          particleArray[i].y -= random(0, jitterAmount) * 4; // Adjusting y coordinate to move particles upwards
        }

        iterations++; // Increment the iteration count
        // Use setTimeout to call the jitterUpwards function recursively after a delay of 10 milliseconds (adjustable)
        setTimeout(() => jitterUpwards(particleArray), 10); // Adjust timeout as needed for speed
      }
    }

    // Start the upward jittering process for the current set of particles
    jitterUpwards(this.particles);
  }

  jitterParticlesCloser() {
    let iterations = 0;
    let prevPositions = [];
    function jitterCloser(particleArray) {
      if (iterations < jitterIterations) {
        for (let i = 0; i < particleArray.length; i++) {
          prevPositions.push([particleArray[i].x, particleArray[i].y]);
          let xOffset = random(-jitterAmount, jitterAmount) * 0.4;
          let yOffset = random(-jitterAmount, jitterAmount) * 0.4;

          particleArray[i].x += xOffset * 0.5; // Adjust x-coordinate to bring particles closer
          particleArray[i].y += yOffset * 0.5; // Adjust y-coordinate to bring particles closer
        }
        iterations++;
        setTimeout(() => jitterCloser(particleArray), 10); // Adjust timeout for speed of jittering
      }
    }

    jitterCloser(this.particles);
    this.beforeMovement = prevPositions;
  }

  jitterParticlesTowardsCenter() {
    let iterations = 0;
    let totalIterations = 7;
    let jitterAmnt = 5;
    const convergenceRate = 0.05; // Rate at which particles converge towards the center
    // console.log("woooo big function");
    function jitter(particleArray, centralY, centralX) {
      // console.log("woooo function");
      if (iterations < totalIterations) {
        // console.log("woooo iterations");
        for (let i = 0; i < particleArray.length; i++) {
          // Calculate distance to the center
          const distanceX = centralX - particleArray[i].x;
          const distanceY = centralY - particleArray[i].y;

          // Move particles closer together
          particleArray[i].x += random(-jitterAmnt, jitterAmnt) * 6;
          particleArray[i].y += random(-jitterAmnt, jitterAmnt) * 6;

          // Move particles towards the center
          particleArray[i].x += distanceX * convergenceRate;
          particleArray[i].y += distanceY * convergenceRate;

          if (iterations == 0) {
            let changeColor = random(0, 1);
            if (changeColor > 0.5) {
              particleArray[i].particleColor = convergenceColor;
              particleArray[i].strokeColor = convergenceStroke;
            }
          }
        }
        iterations++;
        setTimeout(() => jitter(particleArray, centralX, centralY), 0.5); // Adjust timeout as needed for speed
      }
    }

    jitter(this.particles, this.centerX, this.centerY);
  }

  explodeParticles() {
    let iterations = 0;
    const explodeIterations = 30; // Adjust the number of iterations for the explosion

    function explode(particleArray) {
      if (iterations < explodeIterations) {
        // Calculate the center of the galaxy (average position of all particles)
        let centerX = 0;
        let centerY = 0;

        for (let i = 0; i < particleArray.length; i++) {
          centerX += particleArray[i].x;
          centerY += particleArray[i].y;
        }

        centerX /= particleArray.length;
        centerY /= particleArray.length;

        for (let i = 0; i < particleArray.length; i++) {
          // Move particles away from the center of the galaxy
          let deltaX = (particleArray[i].x - centerX) / 15;
          let deltaY = (particleArray[i].y - centerY) / 15;

          // Adjust the particles' positions
          particleArray[i].x += deltaX * 0.1; // Adjust the factor to control the speed of explosion
          particleArray[i].y += deltaY * 0.1; // Adjust the factor to control the speed of explosion
        }

        iterations++;
        setTimeout(() => explode(particleArray), 10); // Adjust timeout as needed
      }
    }

    explode(this.particles);
  }

  moveParticlesInHeart() {
    let iterations = 0;
    let prevPositions = [];
    let heartIterations = 30;

    function moveTowardsHeart(particleArray, heartParticles) {
      if (iterations < heartIterations) {
        for (let i = 0; i < heartParticles.length; i++) {
          prevPositions.push([particleArray[i].x, particleArray[i].y]);

          // Calculate the movement towards the heart shape
          let targetX = heartParticles[i].x;
          let targetY = heartParticles[i].y;

          // Update particle positions towards the heart shape
          particleArray[i].x += (targetX - particleArray[i].x) * 0.4; // Adjust the animation speed as needed
          particleArray[i].y += (targetY - particleArray[i].y) * 0.4; // Adjust the animation speed as needed

          particleArray[i].particleColor = heartColor; // Change the color to red during animation
          particleArray[i].strokeColor = heartStroke; // Change the stroke color to blue during animation
        }

        iterations++;
        setTimeout(() => moveTowardsHeart(particleArray, heartParticles), 10); // Adjust timeout as needed for speed
      }
    }

    let heartParticles = []; // Define heart shape particles here

    // Calculate heart shape particles as before
    let spacing = 15; // Adjust this for the heart shape

    for (let angle = 0; angle < TWO_PI; angle += 0.1) {
      // Calculate x and y coordinates for the heart shape using mathematical functions
      let x = 16 * pow(sin(angle), 3);
      let y = -(
        (
          13 * cos(angle) - // First circular pattern
          5 * cos(2 * angle) - // Second circular pattern with twice the frequency
          2 * cos(3 * angle) - // Third circular pattern with three times the frequency
          cos(4 * angle)
        ) // Fourth circular pattern with four times the frequency
      );

      // Scale the coordinates by a spacing factor
      x *= spacing;
      y *= spacing;

      // Shift the heart shape to the center of the canvas
      x += width / 2;
      y += height / 2;

      // Store the calculated x and y coordinates as a vector in the heartParticles array
      heartParticles.push(createVector(x, y));
    }

    moveTowardsHeart(this.particles, heartParticles);

    this.beforeMovement = prevPositions;
    let center = calculateCenter(this.particles);
    this.centerX = center[0];
    this.centerY = center[1];
  }

  moveParticlesInSpiral() {
    let iterations = 0;
    let prevPositions = [];
    let spiralIterations = 35;

    function moveTowardsSpiral(particleArray, spiralParticles) {
      if (iterations < spiralIterations) {
        for (let i = 0; i < spiralParticles.length; i++) {
          prevPositions.push([particleArray[i].x, particleArray[i].y]);

          // Calculate the movement towards the spiral shape
          let targetX = spiralParticles[i].x;
          let targetY = spiralParticles[i].y;

          // Update particle positions towards the spiral shape
          particleArray[i].x += (targetX - particleArray[i].x) * 0.4; // Adjust the animation speed as needed
          particleArray[i].y += (targetY - particleArray[i].y) * 0.4; // Adjust the animation speed as needed

          particleArray[i].particleColor = spiralColor; // Change the color to red during animation
          particleArray[i].strokeColor = spiralStroke;
        }

        iterations++;
        setTimeout(() => moveTowardsSpiral(particleArray, spiralParticles), 10); // Adjust timeout as needed for speed
      }
    }

    let spiralParticles = []; // Define spiral shape particles here

    // Calculate spiral shape particles
    let spacing = 10; // Adjust this for the spiral shape
    for (let angle = 0; angle < 6 * PI; angle += 0.1) {
       // Calculate x and y coordinates for the spiral shape using trigonometric functions
      let x = angle * cos(angle) * spacing + width / 2;
      let y = angle * sin(angle) * spacing + height / 2;
      spiralParticles.push(createVector(x, y));
    }

    moveTowardsSpiral(this.particles, spiralParticles);

    this.beforeMovement = prevPositions;
    let center = calculateCenter(this.particles);
    this.centerX = center[0];
    this.centerY = center[1];
  }
  
  //function to actually draw the galaxy
  drawGalaxy() {
    for (let i = 0; i < this.particles.length; i++) {
      this.particles[i].createParticle(); // Create a single particle
      this.particles[i].joinParticles(this.particles.slice(i)); // Join the created particle with others
    }
  }
}

sketch.js:

//background music source: https://freesound.org/people/Seth_Makes_Sounds/sounds/701610/

let numParticles = 200;
let nightParticles;
let portConnected = false;
let showStartScreen = true;
let font, font2;
let bgMusic;

//variables to store data from Arduino board
let redAction = 0;
let blueAction = 0;
let yellowAction = 0;
let greenAction = 0;

//flags to check if the action was performed
let redActionPerformed = false;
let yellowActionPerformed = false;
let greenActionPerformed = false;
let blueActionPerformed = false;

let lastInteractionTime = 0;
const idleTimeThreshold = 60000;

let amp;


//start screen for user
function startScreen() {
  textFont(font);
  textSize(80);
  fill("white");
  let message = "StarScape";
  // console.log(width, height);
  let textW = textWidth(message);
  let textH = textAscent() + textDescent();


  let centerX = width / 2;
  let centerY = height / 2;

  // Set text alignment to center and display the text
  textAlign(CENTER, CENTER);
  text(message, centerX, centerY);

  textFont(font2);
  textSize(20);

  let captionX = centerX;
  let captionY = centerY + 80;

  text("Click anywhere to begin. Click again to restart.", captionX, captionY);
}

function preload() {
  soundFormats("mp3", "ogg");
  bgMusic = loadSound("/sounds/bgMusic2.mp3");
  font = loadFont("fonts/NewYork.otf");
  font2 = loadFont("fonts/AppleGaramond-LightItalic.ttf");
}

function setup() {
  //responsive canvas set to the dimensions of the window
  createCanvas(windowWidth, windowHeight);
  //initialize particles
  nightParticles = new Galaxy();
  //loop the music
  bgMusic.loop();
  amp = new p5.Amplitude();
}

function draw() {
  background("#0f0f0f");
  
  if (showStartScreen) {
    startScreen();
  } else {
    //if red button pressed, particles jitter + scatter
    if (redAction && !redActionPerformed) {
      nightParticles.jitterParticles();
      redActionPerformed = true;
    //if yellow button is pressed, particles cluster together
    } else if (yellowAction && !yellowActionPerformed) {
      nightParticles.jitterParticlesTowardsCenter();
      yellowActionPerformed = true;
    //if the green button is pressed, particles form a heart
    } else if (greenAction && !greenActionPerformed) {
      nightParticles.moveParticlesInHeart();
      greenActionPerformed = true;
    // if blue button, the particles form a spiral
    } else if (blueAction && !blueActionPerformed) {
      nightParticles.moveParticlesInSpiral();
      blueActionPerformed = true;
    }
    
    //the particles are continuously rotating, with a speed proportional to the amplitude of the music
    let vol = amp.getLevel();
    let pSpeed = map(vol, 0, 1, 0, 0.005);
    nightParticles.rotateGalaxy(pSpeed);
    nightParticles.drawGalaxy();
  }
}

function readSerial(data) {
  ////////////////////////////////////
  //READ FROM ARDUINO HERE
  ////////////////////////////////////

  if (data != null) {
    // make sure there is actually a message
    // split the message
    let fromArduino = split(trim(data), ",");
    // console.log(fromArduino);
    // if the right length, then proceed
    if (fromArduino.length == 4) {
      // only store values here
      // do everything with those values in the main draw loop

      // We take the string we get from Arduino and explicitly
      // convert it to a number by using int()
      redAction = int(fromArduino[0]);
      yellowAction = int(fromArduino[1]);
      greenAction = int(fromArduino[2]);
      blueAction = int(fromArduino[3]);
      
      //reset the actionsPerformed to false everytime data is read from the arduino
      redActionPerformed = false;
      yellowActionPerformed = false;
      greenActionPerformed = false;
      blueActionPerformed = false;
    }

    //////////////////////////////////
    //SEND TO ARDUINO HERE (handshake)
    //////////////////////////////////
    let sendToArduino = "\n";
    writeSerial(sendToArduino);
  }
}

function mousePressed() {
  //first, connect the port
  if (!portConnected) {
    setUpSerial();
    portConnected = true;
  } else {
    //after port is connected, use mouse press to start (and restart)
    showStartScreen = !showStartScreen;
    //if the game is restarting, create a new instance of Galaxy class
    if (showStartScreen) {
      nightParticles = new Galaxy();
    }
  }
}

The Arduino code sets up the serial connection with the p5.js sketch, and then reads the data from the buttons. After detecting a button press, it sends it to the sketch. After sending it to the sketch, it triggers a light sequence on the strip. The code in the Arduino IDE:

#include <Adafruit_NeoPixel.h>
#include <avr/power.h>


unsigned long previousMillis = 0;
const long interval =250; 



#define PIN_NEO_PIXEL 10  // Arduino pin that connects to NeoPixel
#define NUM_PIXELS 13    // The number of LEDs (pixels) on NeoPixel

#define DELAY_INTERVAL 250  // 250ms pause between each pixel

int arcadeBtnRed = 5;
// int arcadeLEDRed = 13;
int arcadeBtnYellow = 4;
// int arcadeLEDRed = 13;
int arcadeBtnGreen = 3;
int arcadeBtnBlue = 2;

int neoPixelPin = 10;

int redAction = 0;
int blueAction = 0;
int yellowAction = 0;
int greenAction = 0;



Adafruit_NeoPixel NeoPixel(NUM_PIXELS, PIN_NEO_PIXEL, NEO_GRB + NEO_KHZ800);

// Adafruit_NeoPixel strip = Adafruit_NeoPixel(24, neoPixelPin, NEO_RGBW + NEO_KHZ800);

void heart(){
  NeoPixel.clear();

  // unsigned long currentMillis = millis();
  // turn pixels to green one by one with delay between each pixel
  for (int pixel = 0; pixel < NUM_PIXELS; pixel++) {           // for each pixel
    NeoPixel.setPixelColor(pixel, NeoPixel.Color(255, 50, 150));  // Pink color: Red=255, Green=50, Blue=150
    NeoPixel.show();                                           // send the updated pixel colors to the NeoPixel hardware.

    delay(60);  // pause between each pixel
  }
}


void spiral() {
  NeoPixel.clear();
  int centerPixel = NUM_PIXELS / 2;  // Center of the LED strip
  int startPixel = 0;  // Starting pixel

  for (int i = 0; i < NUM_PIXELS / 2; i++) {
    NeoPixel.setPixelColor(startPixel + i, NeoPixel.Color(255, 165, 0)); // Set color to golden (RGB: 255, 215, 0)
    NeoPixel.setPixelColor(NUM_PIXELS - 1 - i, NeoPixel.Color(255, 165, 0)); // Set color to golden (RGB: 255, 215, 0)

    NeoPixel.show();
    delay(100); // Adjust the delay to control the speed of the spiral

    // Fade out the previously lit LEDs
    NeoPixel.setPixelColor(startPixel + i, NeoPixel.Color(0, 0, 0)); // Turn off the LED
    NeoPixel.setPixelColor(NUM_PIXELS - 1 - i, NeoPixel.Color(0, 0, 0)); // Turn off the LED
  }
}


void jitterIllusion() {
  NeoPixel.clear();
  for (int i = 0; i < 50; i++) {  // Repeat the jitter effect multiple times
    for (int pixel = 0; pixel < NUM_PIXELS; pixel++) { // Loop through each pixel
      // Set a random shade of green for each pixel
      NeoPixel.setPixelColor(pixel, NeoPixel.Color(0, random(256), 0)); // Random green shades (RGB: 0, random value, 0)
    }

    NeoPixel.show(); // Show the updated colors
    delay(50); // Adjust the delay to control the speed of the jitter illusion
  }
}

void gentleWaves() {
  NeoPixel.clear();
  int duration = 5000; // Duration of the wave effect in milliseconds
  int waves = 5; // Number of waves to display

  for (int i = 0; i < duration; i += 50) { // Time loop
    float phaseShift = (float)i / duration * 2 * PI * waves; // Phase shift based on time for wave effect

    for (int pixel = 0; pixel < NUM_PIXELS; pixel++) { // Loop through each pixel
      // Calculate a blue value based on a sine wave to create a gentle wave effect
      int blueValue = (sin(phaseShift + (float)pixel / NUM_PIXELS * 2 * PI) + 1) * 128; // Adjust the amplitude and offset as needed

      NeoPixel.setPixelColor(pixel, NeoPixel.Color(0, 0, blueValue)); // Set the pixel color with varying shades of blue
    }

    NeoPixel.show(); // Show the updated colors
    delay(50); // Adjust the delay to control the speed of the gentle waves
  }
}

void allWhite() {
  NeoPixel.clear();
  for (int pixel = 0; pixel < NUM_PIXELS; pixel++) {
    NeoPixel.setPixelColor(pixel, NeoPixel.Color(40, 40, 40));  // Set all pixels to white (RGB: 255, 255, 255)
  }
  NeoPixel.show();  // Show the updated pixel colors
}





void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);
  // pinMode(LED_BUILTIN, OUTPUT);
  pinMode(arcadeBtnRed, INPUT);
  pinMode(arcadeBtnYellow, INPUT);
  pinMode(arcadeBtnBlue, INPUT);
  pinMode(arcadeBtnGreen, INPUT);
  // pinMode(arcadeLEDRed, OUTPUT);

  NeoPixel.begin();  // INITIALIZE NeoPixel strip object (REQUIRED)
  // NeoPixel.clear();
  // strip.begin();
  // strip.show();
  // start the handshake
  while (Serial.available() <= 0) {
    // digitalWrite(LED_BUILTIN, HIGH);  // on/blink while waiting for serial data
    Serial.println("0,0,0,0");        // send a starting message
    delay(300);                       // wait 1/3 second
    // digitalWrite(LED_BUILTIN, LOW);
    // delay(50);
  }
}

void loop() {
  
  while (Serial.available()) {
    if (Serial.read() == '\n') {
      redAction = digitalRead(arcadeBtnRed);
      delay(5);
      blueAction = digitalRead(arcadeBtnBlue);
      delay(5);
      greenAction = digitalRead(arcadeBtnGreen);
      delay(5);
      yellowAction = digitalRead(arcadeBtnYellow);

      Serial.print(redAction);
      Serial.print(',');
      Serial.print(yellowAction);
      Serial.print(',');
      Serial.print(greenAction);
      Serial.print(',');
      Serial.println(blueAction);
    }
  }

  if (redAction) {
    jitterIllusion();
  } else if (yellowAction) {
    gentleWaves();
  } else if (greenAction) {
    heart();
  } else if (blueAction) {
    spiral();
  } else {
    allWhite();
  }
}

What I’m particularly proud of

I’m proud of the methods in the galaxy class… it took me a while to figure out how to code the movement of the particles in a way that looks satisfying. I’m also proud of the overall aesthetics (most people’s first reactions were to comment on how the sketch *looked* before they even interacted with it).

Reflections and Future Improvements

I love the idea of my piece, but, in its current scope I don’t think its full potential has been realized. I think the concept would be much more impactful if there were much more lights, or if it was designed to be within an art installation where the lights are all around the user (similar to Andrew’s piece). If I were given the opportunity to expand the piece to be as such, it would be closer to the original thing I envisioned. Regardless, I’m happy with how it turned out and though its small, I think it still captures the essence of what I wanted to do.

Final Project: Md studios

Final Project concept:

An Interactive drum kit with multiple rooms, each room has a different theme and different drum kit sounds. some of the themes are unusual everyday sounds. I have implemented a regular drum kit, an arabic percussion kit,  a kitchen kit and a sci-fi kit. I wanted to give my users a new experience that they would not be able to get using traditional musical instruments, a small glimpse of the world of electronic music you may say.

Implementation design:

I used 3 force sensors for the drum plates and I used 2 buttons to loop between the themes and go back to home screen. I created screenshots of the themes and added a creative touch to each theme to make it more relevant to the theme.

Interaction between Arduino and P5js:

The only interaction between Arduino and p5js is reading the force sensors and buttons on Arduino and sending it to p5js to change the themes/sounds accordingly. To make my buttons and force sensors work as smooth as possible I have added a delay on reading certain inputs without using the delay() function so that other inputs work simultaneously

Arduino Code:

for the arduino code, I read the values from the sensors and sent them to the p5js through serial.print() by using commas to separate the values. to make the interaction smoother I used “delay without delay()” to ensure that the button is only read every few seconds whereas t

he force sensors are read on a continuous basis.

#define FORCE_SENSOR_PIN1 A0 // the FSR and 10K pulldown are connected to A0
#define FORCE_SENSOR_PIN2 A1 // the FSR and 10K pulldown are connected to A1
#define FORCE_SENSOR_PIN3 A2 // the FSR and 10K pulldown are connected to A2
#define Button1 7 //the switch in connected to D7 (Red button)
#define Button2 6 //the switch in connected to D6 (Yellow button)
#define Button3 5 //the switch in connected to D5 (Blue button)

unsigned long previousMillis = 0;
const long interval = 200; 

void setup() {
  Serial.begin(9600);
}

void loop() {
    unsigned long currentMillis = millis();
    int analogReading1 = analogRead(FORCE_SENSOR_PIN1);
    int analogReading2 = analogRead(FORCE_SENSOR_PIN2);
    int analogReading3 = analogRead(FORCE_SENSOR_PIN3); 
    int digitalReading1 = 1;
    int digitalReading2 = digitalRead(Button2);

    if (currentMillis - previousMillis >= interval) {
    // save the last time you read from the button
      previousMillis = currentMillis;
      digitalReading1 = digitalRead(Button1);
    }
    // print the raw analog readings seperated by commas

    Serial.print(analogReading1); //read without delay
    Serial.print(",");
    Serial.print(analogReading2); //read without delay
    Serial.print(",");
    Serial.print(analogReading3); //read without delay
    Serial.print(",");    
    Serial.print(digitalReading1); //read with delay!!!!!!
    Serial.print(",");    
    Serial.print(digitalReading2); // read without delay
    Serial.println();

}

p5js:

for my p5js code, I read the values from Arduino and split them at the comma and stored each value in a variable that I later used to create conditions on to make sure that the right sounds and the right themes are displayed depending on the user’s inputs.

The aspects of the project I am proud of: 

I am proud that I managed to create the force sensors responsive to the touch so that the sounds are played right when the user presses on the force sensor, to make it smooth. I am also proud of the the

me designs I created, I put in extra thought to make sure the user can understand what the theme is without having to explain it.

Pictures of my project:

first prototype:

 

 

 

 

 

 

 

Final Version:

 

 

 

 

 

 

 

 

 

 

 

 

Future improvements:

I initially wanted to create a record button that records users’ jams and adds them all to one file to have a collective file of many users’ artistic touch, but the p5js sound. library was not working as expected and was causing so many bugs in my program so I decided to skip it for now, but I would love to add later on because I believe it makes the project much more interesting.

Sources:

https://soundpacks.com/free-sound-packs/sci-fi-sound-fx-pack/

https://www.reddit.com/r/edmproduction/comments/3lh4t0/free_kitchen_sounds_sample_pack/

https://docs.arduino.cc/built-in-examples/digital/BlinkWithoutDelay

https://www.clipartmax.com/

 

 

 

Final Project: Emotions in Color

Concept

With my project I am attempting to translate human emotions into color. It is a personal experience during which a participant is asked to input 5 colors for 5 different emotions. The colors are then used to generate a personal generative artwork that portrays the unique color gradient of the participant.

In addition to its primary goal, the project carries an educational aspect. I aim to provide participants with a deeper understanding of the interplay between RGB values and the diverse outcomes arising from various color mixtures. This dual-purpose project invites individuals to explore their emotions creatively while gaining insights into the fascinating world of color.

Pictures and Videos

Interaction Design

Regarding the interaction design, I wanted the setup to be simple and minimalistic: 3 potentiometers and one button. The color input happens through the potentiometers each of which represent Red, Green and Blue color channel values (RGB). The button is used to progress through the experience.

The visual design in P5 is also quite simple, following a simple color palette. It is structured and clean, guiding the participant through the experience. The feedback on the color input is provided instantly, ensuring a seamless and responsive interaction that enhances the overall engagement.

Arduino code

My Arduino code reads values from three potentiometers and a button and sends it to P5. The potentiometers provide analog input representing different color channels (Red, Green, and Blue) by measuring voltage. The button is used as a digital input.

const int potPin1 = A0;  // Potentiometer connected to A0
const int potPin2 = A1;  // Potentiometer connected to A1
const int potPin3 = A2;  // Potentiometer connected to A2
const int buttonPin = 4; // Button connected to digital pin 4

void setup() {
  Serial.begin(9600);  // Initialize serial communication at 9600 bps
  pinMode(buttonPin, INPUT_PULLUP); // Set the button pin as input with pull-up resistor
}

void loop() {
  // Read values from potentiometers
  int value1 = analogRead(potPin1);
  int value2 = analogRead(potPin2);
  int value3 = analogRead(potPin3);

  // Read the state of the button
  int buttonState = digitalRead(buttonPin);

  // If the button is pressed, send a button message
  if (buttonState == LOW) {
    Serial.println("BUTTON_PRESSED");
  } else {
    // Send potentiometer values as usual
    Serial.print(value1);
    Serial.print(',');
    Serial.print(value2);
    Serial.print(',');
    Serial.println(value3);
  }

  delay(500);  // Add a delay for stability, adjust as needed
}
P5 code

The P5 code controls the different stages of the experience. The project unfolds through different stages, each focusing on a specific emotion, such as nostalgia, serenity, courage, curiosity, and resilience. P5 is receiving the data from the Arduino, and the combined color choices are stored in an array, which is used to create a generative artwork based on the personalized color gradients.

One of the more challenging parts was the storing of the final input colors in an array. I had to create additional flag variables or else the storing would be continuous and the array would expand super quickly. However, with the flags I was able to store the last value before the press of the button:

hasPushedColors = false;
if (buttonPressed && !hasPushedColors) {
  // Add the last values of R, G, and B to the array
  p_colors.push([R, G, B]);
  console.log("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
  console.log("color" + p_colors);
  hasPushedColors = true; // Set the flag to true to prevent multiple pushes
  buttonPressed = false;
Parts I am proud of

I am particularly proud of my concept. I find it to be very simple, yet it has a sense of discoverability, anticipation and surprise, especially with the reveal of the gradient. Even after completing the experience multiple times, each time I am curious to put it different colors to see what final gradient will be generated. I am also happy that it has a deeper psychological context and how unconsciously we do tend to connect certain colors to certain feelings. I already made some observations, for instance, a big part of my participants tended to associate red with resilience.

I am also quite satisfied with my interaction design. It was a great learning experience for me to better understand color through color combinations, and the potentiometers are quite an accurate choice for input. They are easy to control and provide good precision.

Future Improvements

An improvement I would be most keen to explore would be to provide an interpretation for the color choices of participants. Either at every stage, or at with the final result. It would be interesting to perhaps incorporate machine learning into analysing the color selections and generating insights into the potential emotional or psychological significance behind each choice. This could involve training a machine learning model on a dataset of color-emotion associations, allowing the system to recognize patterns and correlations. By providing participants with personalized feedback on their color selections, the project could become a more insightful and introspective experience.

Week 13 – Final Project

Båige – Horse Race!

User testing is in a separate blog post. Here is the video of my final project-> FINAL PROJECT VIDEO

My Concept: The project is an interactive game about the horse race, one of the traditional games in Kazakh culture. As horse riding was a part of our nomadic lifestyle, we have a lot of games connected with horse riding. One of these games is called Båige as the name of this game, where the equestrians compete with each other while riding the horses and that might include overcoming some obstacles. I wanted to incorporate that into my game: the user is an equestrian, riding the horse and avoiding obstacles in the form of snakes while collecting the stars that add up extra points to the score. As the game progresses, the snakes move faster, making the game harder for the user. 

The implementation of the idea: To implement the idea four main buttons were used, which are responsible for each of the lines in the racing game. When the button is pressed, the horse moves to the corresponding line. To make the experience of racing, the obstacles and the awards are added to the game, so the user should avoid the former and collect the latter, adding up to the score. 

Description of interaction design: The user interacts with the game by pressing the physical buttons on the box, each of which is responsible for the corresponding line as shown in Fig 1. For instance, when the user presses the red button on Arduino, the equestrian on the p5.js moves to the red line. Hence, when the obstacles appear in the way of the user, the user will try to press the buttons of the lines with no obstacles. More obstacles are avoided, and more score is given to the user. Furthermore, the stars appear on the p5.js and by pressing the buttons of the lines with the stars, the user will be able to collect these stars and add extra points to the score. When the user collides with the snakes, the game is over and the score collected by the user shows up. 


Fig. 1. P5.js and Arduino set up sketch

Description of Arduino code: The Arduino code declares the buttons as digital inputs with internal pull-up resistors. It checks if the buttons are pressed and sends the corresponding number through serial if they are pressed. For instance, if the white button is pressed, the number 1 is sent to the serial communication. The delay was added, so when the button is pressed once, only one number is printed and sent. 

Link to full Arduino sketch:

// Intro To IM - Fall 2023
// Michael Ang
// Final Project - Baige- Horse Race! 
// Diana Alibekova

// assigning pins to the buttons and declaring them as constant int, so their values will not be accidentally altered
const int greenbutton = 2;
const int yellowbutton = 3;
const int redbutton = 4;
const int whitebutton = 5;
const int startbutton = 6;

void setup() {
  //activating the serial communication
  Serial.begin(9600);
  //declaring the buttons as digital inputs with internal pull up resistors (the external resistors are not used)
  pinMode(greenbutton, INPUT_PULLUP);
  pinMode(yellowbutton, INPUT_PULLUP);
  pinMode(redbutton, INPUT_PULLUP);
  pinMode(whitebutton, INPUT_PULLUP);
  pinMode(startbutton, INPUT_PULLUP);
}

void loop() {
  // checking if the buttons are pressed and sending the corresponding number through serial if they are pressed
  if (digitalRead(whitebutton) == LOW) {
    Serial.println(1);
    // delay is needed because it pauses the program for a half a second, so only one press of the button is registered.
    delay(500); 
  } else if (digitalRead(redbutton) == LOW) {
    Serial.println(2);
    delay(500);
  } else if (digitalRead(yellowbutton) == LOW) {
    Serial.println(3);
    delay(500);
  }
  else if (digitalRead(greenbutton) == LOW) {
    Serial.println(4);
    delay(500);
  }
  else if (digitalRead(startbutton) == LOW) {
    Serial.println(0);
    delay(500);
  }
}

Description of p5.js code: The images and sounds are preloaded before the setup runs. The game has three states: start with instructions given, play with the game playing, and end with the score obtained as shown in Fig. 2. In the instructions, the image with the instructions is shown and the serial communication is set by pressing the space key button and choosing the serial port. By pressing the start button on Arduino, the number ‘0’ is sent to the p5.js, which is interpreted as the transition between the states. So, the game transitions to the game-playing state. In this state, the background image of the racing lines is displayed with the moving obstacles, bonus stars as well and the horse of the user. In the beginning, the horse is in the initial position given unless the number is sent to the p5.js by pressing one of the four buttons on Arduino. When the button is pressed, the number is sent to p5.js and the horse moves by x coordinate accordingly. Moreover, there are the moving obstacles, which appear randomly in the x coordinates of windowWidth * (3 / 12), windowWidth * (5 / 12), windowWidth * (7 / 12), windowWidth * (9 / 12) and move by y coordinate and increase the speed. Initially, these obstacles were identified as the circles as well as the horse, so the minimum distance between them was calculated as the obstacle.radius + (circleRadius / 2). The collision was identified when the distance between the obstacle radius and the circle (horse) radius was smaller than the minimum distance. When they collide, the game ends by freezing the image or sleeping for a second and then transiting to the end state. A similar approach was taken with the bonus stars, but the collision with them added extra points to the score. The score was calculated by every obstacle passing through the window, so the y coordinate of the obstacle circle is greater than the window height. In the end state, the end image is displayed with the score.

Fig. 2. Game states

The code I am proud with:  I am particularly proud with the code of calculating the distance between the obstacles and the circle (horse) to determine the collision.

// for loop accessing each obstacle in the array and calcilating the distance between the horse and obstacle as well as minimum distance for collision to occur
 for (let i = 0; i < obstacles.length; i++) {
  let obstacle = obstacles[i];
  let distance = dist(circleX, windowHeight * (2 / 3), obstacle.x, obstacle.y);
    let minDistance = obstacle.radius + (circleRadius / 2);

// if the horse collides with the obstacles, then the game is over, transitting to end game state and playing the winning sound
  if (distance < minDistance) {
    wonGame = false;
    // transitting to end game state
    gameState = 'end';
    level_win.play();
// if the horse doesn't collide with the obstacles, for each of the obstacles passed through the window height, the score is incremented. 
  } else if (obstacle.y > windowHeight && !obstacle.passed) {
      obstacle.passed = true; 
      score++; 
    }
}

Embedded p5.js code: 

Description of communication between Arduino and p5.js: there are five buttons on Arduino. Every time the button is pressed, the corresponding number is written down, which is sent to p5.js. For every one of these numbers, there are the x coordinates of the horse. Hence, the horse moves by x coordinate depending on the number taken. For instance, when the red button is pressed, the number 4 is sent to p5.js, meaning that the horse should move to the windowWidth*(3/12). 

The aspects of the project I am proud of: Overall, I am very proud of the final product because I see the development of creative coding skills from absolute zero to something that can be presented in the IM Show. Specifically, I am proud of being able to correctly create the serial communication between Arduino and p5.js, so the Arduino sends the number every time the button is pressed and that number is rightly interpreted by p5.js and moves the horse by x coordinate. Moreover, as this is my very first dynamic game project, I am proud of being able to code the moving obstacles and awards as well as the collision between them. When the horse collides with an obstacle, the game is over, while when it collides with the award, bonus points are given to the score. Last but not least, I am very proud of the physical decoration I made shown in Fig. 3. because overall the project looks aesthetically pleasing and engaging. 

Fig. 3. Decor

The areas of improvement: There are some improvements to the project that I would like to add in the future. For instance, it would be great if the score of the users could be saved and created the list of users with the highest score in the game. This would create a sense of competition with others, making the game more interesting. Furthermore, there are minor details that would enhance the experience of playing the game such as whenever the collision with the snake happens, the image of the horse and snake changes as if the snake was biting the horse or something similar. I had that idea but I couldn’t find the image which would satisfy that. Hence, I think in the future, I might draw the images of the snake and horse and import them. Additionally, the moving trees and houses on the sides would be a great addition, enhancing the experience that the horse is moving. Moreover, as an improvement for the game, it would be great if two or more people could play the game together and compete with each other. In this case, the game would look like a horse race. 

Resources used