Author: Kamila Dautkhan

Concept

Harmonic Market is a fun interactive project I built in p5.js. It takes real-time crypto prices (Bitcoin, Ethereum, or Solana) and turns them into sound and light. I chose exactly this idea because I am a finance major and very passionate about crypto. The main idea is quiet simple: the market has its own voice and you get to decide how to listen to it. Instead of just staring at numbers on a chart, this project lets you actually feel what the market is doing through sound. What does a calm market sound like? What about a super volatile or crashing day?

When you use it you become like a performer. You move your mouse around the screen to tune in to the market. Moving left and right changes how nice or tense the sound feels, it can sound smooth and beautiful when you’re in the right spot, or rough and weird when you’re not. Moving the mouse up and down changes how loud and intense everything gets. The live market data controls the base pitch and how wild the sound is, while you control the harmony on top of it. Every session sounds different and every market condition feels completely unique.

Pictures

How does the implementation work?

I mapped the mouse controls to musical ideas so it feels natural. Moving the mouse left and right changes how “in tune” the sound is. When you go left, the sound gets very tense: noise gets louder and the waveform on screen starts shaking. When you move right, everything snaps into nice harmony: the wave smooths out and green particles float up from the center. It just makes sense to me: left = tension, right = resolution. Moving the mouse up and down controls how loud and intense everything feels. Up = louder and bigger visuals. Down = quieter. If you hold SPACE, it gives a temporary 50% boost and makes particles orbit.

I made the feedback to be strong and responsive so you always know what’s happening. There’s a harmony meter on the left and an intensity meter on the right that show live numbers and colored bars. The labels “DISSONANT” and “CONSONANT” fade in and out depending on where your mouse is. I also replaced the normal cursor with a growing circle that gets bigger when intensity is high because it feels more like playing an instrument than just pointing.

How This Was Made

The whole idea and main structure came from me. I wanted to make financial data feel emotional and musical instead of just showing numbers on a chart. The concept of using mouse X to control consonance/dissonance was my original idea since I got very inspired by how people talk about markets in musical terms.

I used separate functions:

• drawOrb() — draws the glowing center orb that pulses and reacts to market volatility
• drawWaveform() — draws a three-layer wave across the screen that shows the harmony
• drawConnectorLines() — draws lines from your mouse to the orb (more lines = more harmony)
• updateSound() — controls the three oscillators and makes the sound change smoothly
• spawnAndUpdateParticles() — creates green sparks when it sounds good, red when it sounds tense
• fetchMarketData() — gets live prices from CoinGecko every 9 seconds

So, I wanted to smooth the mouse movement so the sound and visuals don’t jump and I had a very hard time with it, so I used ClaudeAI in lines 97-102. Also, I used ClaudeAI to make the harmony sound much more musical and intentional. It picks a musical interval from the scale based on mouse position, then multiplies the base pitch by that ratio (lines 107-114).

What I’m Proud Of

I’m most proud of the left/right harmony mapping. It’s not just turning volume up and down, it actually changes the musical relationship between the sounds, and you can really feel and see the difference. When you’re far left it feels chaotic and clashing. When you’re far right it feels calm and beautiful.

I’m also happy that it’s pretty easy to understand without reading instructions. The intro screen explains the controls clearly, the meters give constant feedback, and most people figure out the mapping within about 30 seconds.

Areas for Future Improvement

There are a few things I’d like to add later:

• Use more market data (like trading volume or market cap) to control reverb, rhythm
• Add touch support so it works nicely on tablets with two fingers
• Maybe let the user sing into the microphone and have the market data affect their voice instead of just synthesized sounds

Live Demo:

Click inside the frame to start • Move mouse to harmonize with the market

During the user testing, some people felt confused at the beginning. They didn’t understand that the big number in the center was the real, live Bitcoin  price updating from the internet. Some thought the sound was just random background music and didn’t realize their mouse movement was controlling the pitch and volume. They also didn’t know that the sound changing from smooth to rough was connected to market volatility. Overall, the project felt fun but needed clearer explanations about what the price, mouse controls, and sound changes actually represented. I think this experience clearly showed that without knowing its real market data, people enjoy it as a generative toy but miss the deeper meaning. Here is the video:

IMG_4224

What caught my eye in this reading was the idea that disability design should not always be about hiding disability or making products invisible. I found the discussion about glasses especially interesting. I never thought of glasses as something that used to be seen mainly as a medical device and later became a fashion item. It made me think about how design can change the way people feel about using assistive products. Instead of making people feel embarrassed, good design can make people feel confident. That idea stood out to me because it shows design is not only about function, but also about identity.

Also I found it really interesting that the reading’s challenge to the belief that discreet is always the best approach for disability-related products. I used to assume the best design would make these products as hidden as possible, but the reading made me question that. Sometimes trying to hide disability can actually reinforce stigma. I liked how the author connects fashion and disability, because at first those seem like unrelated topics, but the reading shows they can influence each other. Overall, the reading made me think of design as something more social and cultural, not just practical. It made me realize inclusive design can help change attitudes, not just solve problems.

Honestly, the Victor reading was kind of a trip because he’s basically saying our iPhones are kind of a step backward. It’s wild to think that we have all these nerve endings in our fingers but we’re stuck just swiping on flat glass all day. He calls it “Pictures Under Glass,” and it made me realize how much better it feels to use actual physical tools where you can feel the weight and the edges of things. It definitely makes me want to build something that isn’t just another touchscreen.

Connecting that to the BlinkWithoutDelay thing actually makes a lot of sense now. If you’re trying to build a cool, responsive tool like Victor is talking about, you can’t have your code stuck on a delay() command. It’s like trying to have a conversation with someone who randomly freezes for two seconds every time they blink. Using millis() is basically the only way to make sure the hardware is actually “awake” enough to feel what the user is doing in real-time.

One thing I’m still stuck on is how to actually build the 3D stuff he’s talking about. Like, it’s easy to code a button, but how do you code something that feels like “opening a jar” or sensing weight? I also definitely need to practice the if (currentMillis - previousMillis >= interval) logic more because it’s way less intuitive than just typing delay(1000). It feels like a lot of extra math just to keep the light blinking while doing other stuff.

Reading these back-to-back was honestly a bit of a reality check. I went into the “Greatest Hits” list thinking I might find some cool, niche ideas to borrow, but instead I realized that almost every “original” thought I’ve had as a beginner like: digital mirrors, the MIDI gloves is already something established. It’s a bit humbling but I really liked the author’s idea that it’s not about being the first person to use a sensor, it’s about what you actually do with it after the “wow” effect of the tech wears off.

It made me rethink my own process. Because just because I can make someone wave their hand to trigger a sound doesn’t mean it’s meaningful. If the physical action doesn’t match the emotion of the piece it just feels like a tech demo than art.

In the second reading, I loved the actor/director analogy. You don’t tell an actor exactly how to feel, you give them the props, the lighting, and the space, and let them find the emotion themselves. Our job in physical computing is basically to be the stage manager. Also, what thing that really stood out to me was that I usually see someone using the data wrong as a failure on my part, but now I’m trying to see it as a conversation. If they’re confused, that’s not really a bug, it’s actually a reflection of the design which is extremely helpful.

Concept

This project demonstrates how an arduino system uses both analog and digital inputs to control outputs in different ways. A digital sensor like a button provides simple on or off input to control an LED, while an analog sensor like a potentiometer provides a range of values that are mapped to adjust another LED’s brightness using PWM. So in simple words, the project shows how real world data can be read, processed and translated into responsive visual feedback.

Sketch

Code

const int ANALOG_SENSOR_PIN = A0;  // Potentiometer or photoresistor
const int DIGITAL_SENSOR_PIN = 2;  // Button/switch
const int DIGITAL_LED_PIN = 13;    // LED controlled digitally
const int ANALOG_LED_PIN = 9;      // LED controlled with PWM 

int analogValue = 0;
int digitalValue = 0;
int ledBrightness = 0;

void setup() {
  pinMode(DIGITAL_SENSOR_PIN, INPUT_PULLUP); 
  pinMode(DIGITAL_LED_PIN, OUTPUT);
  pinMode(ANALOG_LED_PIN, OUTPUT);
  
  Serial.begin(9600);
}

void loop() {
  analogValue = analogRead(ANALOG_SENSOR_PIN);
  
  digitalValue = digitalRead(DIGITAL_SENSOR_PIN);
  
  if (digitalValue == LOW) {  
    digitalWrite(DIGITAL_LED_PIN, HIGH);
  } else {
    digitalWrite(DIGITAL_LED_PIN, LOW);
  }

  ledBrightness = map(analogValue, 0, 1023, 0, 255);
  analogWrite(ANALOG_LED_PIN, ledBrightness);
  
  // Debug output
  Serial.print("Analog: ");
  Serial.print(analogValue);
  Serial.print(" | Digital: ");
  Serial.print(digitalValue);
  Serial.print(" | Brightness: ");
  Serial.println(ledBrightness);
  
  delay(10); // Small delay for stability
}

How it was made 

I built this project using an arduino, a button, a potentiometer, and two LEDs on a breadboard. The potentiometer was connected as an analog sensor to control the brightness of one LED, while the button was used as a digital sensor to turn the second LED on and off. I wrote the code to read both inputs, then used analog(Write) to adjust the LED brightness and digital(Write) to control the on/off LED. I also used the serial monitor to display the values for testing and debugging.

Reflection

Honestly, doing this project was very fun. Through this project, I learned how to use both analog and digital inputs with an arduino. I understood that analog inputs give a range of values which can be used to control things like brightness, while digital inputs only have two states. If I did this project again, I would try to make it more creative by adding more sensors or different types of outputs.

Sketch

The Concept

I wanted to make something that felt exciting and had that arcade game tension where you’re always on edge. The idea came from thinking about those old falling-object games, but I wanted to add my own twist, for example, what if the game itself reacted to how much time you have left? What if it got harder as you played? And what if the music changed when things got intense? So Nebula Chase became this space game where you’re flying through a nebula collecting stars while dodging bombs. Simple concept, but I put a lot of work into making it feel engaging.

The Game in Action

Start Screen: 

When you first load the game, you see the title with this “CLICK TO START” button. I made the title bounce a little using a sine wave. And in order to make it easier for the user to understand the instructions of the game, they can press ‘I’ to see them. 

Gameplay:

Once you’re playing, the screen gets busy very fast. Yellow stars fall down, those are the good ones, and red bombs come at you too. Your ship follows your mouse, and you will have only 60 seconds to grab as many stars as possible without hitting bombs. The UI at the top shows your score, timer, and lives. I made the timer turn red and the whole screen flash when you’re under 10 seconds. 

Object-Oriented Design

I used three classes to organize everything:

1. Star Class 

2. class Star {
     constructor() {
       this.x = random(width);
       this.y = random(-600, -50);
       this.speed = random(2, 4) * difficulty;
       this.wobble = random(TWO_PI);
     }
     
     move() {
       this.y += this.speed;
       this.wobble += 0.05;
       if (this.y > height + 50) {
         this.y = -50;
         this.x = random(width);
       }
     }
     
     display() {
       push();
       translate(this.x + sin(this.wobble) * 10, this.y);
       tint(255, tintAmount, tintAmount);
       image(starImg, -20, -20);
       pop();
     }
   }
   

   Each star wobbles side to side as it falls which makes them really hard to catch. The speed multiplies by the difficulty variable, so as the game goes on, everything gets faster and it makes everything harder for the user. 

   2. Bomb Class – The obstacles

   class Bomb {
     constructor() {
       this.x = random(width);
       this.y = random(-600, -50);
       this.speed = random(3, 5) * difficulty;
       this.rotation = 0;
       this.pulsePhase = random(TWO_PI);
     }
     
     move() {
       this.y += this.speed;
       this.rotation += 0.05;
       this.pulsePhase += 0.1;
       if (this.y > height + 50) {
         this.y = -50;
         this.x = random(width);
       }
     }
     
     display() {
       push();
       translate(this.x, this.y);
       rotate(this.rotation);
       let pulseSize = 1 + sin(this.pulsePhase) * 0.15;
       scale(pulseSize);
       image(bombImg, -22, -22);
       pop();
     }
   }

   I made the bombs rotate and pulse to make them feel dangerous. They also move slightly faster than stars on average, which created the pressure for the player. 

3. Particle Class

class Particle {
  constructor(x, y, col) {
    this.x = x;
    this.y = y;
    this.vx = random(-3, 3);
    this.vy = random(-3, 3);
    this.life = 255;
    this.col = col;
  }
  
  update() {
    this.x += this.vx;
    this.y += this.vy;
    this.vy += 0.1;  // Gravity
    this.life -= 5;
  }
  
  display() {
    fill(red(this.col), green(this.col), blue(this.col), this.life);
    ellipse(this.x, this.y, this.size);
  }
}

Whenever you collect a star or hit a bomb, it creates approx 15 particles that spray out in random directions. They fade out over time and fall slightly from gravity. I made this just to make the interactions feel way more satisfying. 

The Spaceship:

I didn’t want to use basic shapes for everything, so I made custom graphics using p5’s createGraphics():

playerImg = createGraphics(60, 60);
// Outer glow
playerImg.fill(0, 255, 200, 100);
playerImg.triangle(30, 5, 10, 50, 50, 50);
// Main body
playerImg.fill(0, 255, 150);
playerImg.triangle(30, 10, 15, 45, 45, 45);
// Cockpit detail
playerImg.fill(100, 200, 255);
playerImg.ellipse(30, 25, 8, 12);

Since to come up with this code was very challenging for me, I used these resources to help me navigate:

https://www.deconbatch.com/2022/01/blendmode-add.html

https://www.youtube.com/watch?v=pNDc8KXWp9E

As you can see the stars and bombs have this glowing effects  because I drew multiple overlapping circles with decreasing opacity to create that glow. The stars are yellow or white and the bombs are red with a darker center that kind of looks like a skull.

The code I’m proud of:

It is definitely the sound system since I didn’t want to just upload the mp3 because I didn’t find the suitable one. So, I decided to generate it myself , that’s the reason why I spent a lot of time on it.

function updateBackgroundMusic() {
  if (timer > 10) {
    // Calm ambient music
    bgMusic.amp(0.15, 0.5);
    bgMusic.freq(220 + sin(frameCount * 0.02) * 20);
    bassLine.amp(0.1, 0.5);
    urgentMusic.amp(0, 0.5);
  } else {
    // DRAMATIC URGENT MUSIC FOR FINAL 10 SECONDS
    bgMusic.amp(0.05, 0.3);
    urgentMusic.amp(0.25, 0.3);
    urgentMusic.freq(110 + sin(frameCount * 0.1) * 30);
    bassLine.amp(0.2, 0.3);
  }
}

For most of the game you hear this calm wave that wavers slightly (that’s the sin(frameCount * 0.02) * 20 part, it creates a slow sound in pitch). There’s also a quiet bass line. But when you hit 10 seconds left the music completely changes.  The bass gets louder and the calm music fades. I just wanted to make feel the pressure for the user.

Reflection

I’m really happy with how this game turned out. The music transition at 10 seconds is probably my favorite part because it genuinely makes the game feel more intense and interesting. The particle effects were also surprisingly easy to implement but it added so much to the feel of the game. The biggest thing I learned was about game balance. It’s one thing to make mechanics work, but making them feel good for the user is way harder. I probably spent as much time tweaking numbers like how fast things fall and etc. as I did writing the actual code.

 

I think some of the ways that computer vision differs from human vision is that while we see “meaning”, computer sees only the math behind it. Because human vision is very semantic, for example when we look at a video we instantly see a person, a mood or a story. But for a computer that same video is just a massive dump of pixel data and color coordinates. Because the computer has zero clue if it’s looking at a human or something else until we write an algorithm to prove it. To help the computer see what we’re interested in we can use cheats like  background subtraction, where the computer compares a live shot to a blank photo of the room to spot what’s new or frame differencing to track motion by subtracting one frame from the next.

The fact that computer vision is rooted in military surveillance really colors how it’s used in interactive art. I think because computer vision was born in military and law enforcement labs, they are designed to track and monitor. That’s why they bring a sense of control to interactive arts. For example, in works like Myron Krueger’s Videoplace the tracking is used for play. It turns your body into a paintbrush, giving you a role in the digital world. Also, projects like Suicide Box show that surveillance can be used to track the tragic social phenomena that the government might ignore.