Concept

I was very interested in using non-conventional inputs for RPG based games, and had initially planned to make an audio-based movement game. However, I came to the conclusion that the lag between the player voicing out the command and it executing in game would cause a loss of interest. This is when I decided to go into using a hybrid strategy where the user still controls the player using keys, but unlocks special abilities when humming.

I incorporated the idea of short but tricky levels from Level Devil, a simple 2D level based RPG game, so a player would typically try multiple times per level,  making the player feel smarter at the end of each level. Here are a few key decisions I took to give my project direction:

1. 1. Level 3 should be unusually harder and the newfound ability doesn’t help,  the player will get stuck, and they must learn to master the key controls to maneuver the player between the ladders. This accomplishment will make the next levels much easier, and make players more likely to finish the game.
   2. The game starts after enter is pressed from the start scene, the player will use either WASD or up,down,left,right arrows to move. The movement must be smooth and the player walks should feel natural.
   3. Sprites increase computing and reduce FPS, making the game slow, so I chose to create the character using code.
   4. I kept the death count and level count as indicators of progress, and decided to name each level.
   5. Sound effects, and background music must be handpicked.
   6. Player must get the stop-time and reverse-time abilities after 2 levels and abilities must grow as levels progress.
   7. Giving player and theme customisability (a suggestion from a classmate)

Sketch

Implementation details – Game States

The global state variable (managed in js/state.js) determines which update cycle and rendering pipeline draw() executes each frame:

title → tutorialControls → play ↔ dead / won (level 1,2)→ tutorialAudio → play ↔ dead / won (level 3-12) → … → ending

Every frame, draw() in js/sketch.js performs the following steps:

1. 1. Calculates a clamped delta/marginal time (dt).
   2. Executes the update logic corresponding to the active state.
   3. Applies uniform aspect ratio scaling and letterboxing transforms (translate(0, view.oy); scale(view.s);).
   4. Calls the appropriate drawing functions for world entities, traps, and UI overlays.

Script Load Order

Load order is critical for initialization-time dependencies (for instance, state.js relies on constants defined in levels.js). sketch.js must always be loaded last as the main entry point.

Assets Used

(All audio used was found from Pixabay and trimmed to fit the game sound needed)

• • Background Music (background-music.mp3): Ambient soundtrack looping during gameplay.
  • Good / Success (good.mp3): Level clear chime played upon reaching the exit gate.
  • Glass Breaking (glass-breaking.mp3): Death and impact sound effect when hitting traps.
  • Ladder (ladder.mp3): Climbing audio triggered when ascending/descending ladders.
  • New Message (new-msg.mp3): Dialogue and narrative notification pop sound.

• • Hell Theme (hell.png): Background artwork for the hell/underworld theme.
  • Nature Theme (nature.png): Unlockable background artwork for the nature theme.
  • Settings Gear (gear.svg): Vector icon for the in-game settings panel button.

Challenges Faced 

1. 1. Key inputs were a major challenge on the web editor. Often the preview canvas doesn’t initially receive keyboard focus. In the canvas, clicking the preview didn’t focus it, so keyboard events went to the editor instead.
   2. Refactoring sketch.js wasn’t easy either, maintaining SOLID principles got harder as more features were added.
   3. Humming conflicting with background noise, to combat this, I chose to simply lower the volume while recording the hum and keeping it lower in the next levels.

Code that I am Proud Of

Detecting pitch in real-time in the browser can be noisy. Rather than relying on heavy third-party speech libraries, I wrote a lightweight FFT peak-detection algorithm that isolates the dominant fundamental frequency around the player’s calibrated humming range while filtering out background noise:

// Detects the dominant pitch frequency from FFT spectrum data
function detectPitch(spectrum) {
    let maxEnergy = 0;
    let peakBin = -1;
    let nyquist = sampleRate() / 2;
    let binWidth = nyquist / spectrum.length;

    // Scan only musical vocal range (80 Hz to 600 Hz)
    let minBin = Math.floor(80 / binWidth);
    let maxBin = Math.min(Math.floor(600 / binWidth), spectrum.length - 1);

    for (let i = minBin; i <= maxBin; i++) { if (spectrum[i] > maxEnergy) {
            maxEnergy = spectrum[i];
            peakBin = i;
        }
    }

    // Return the detected frequency if it passes our confidence threshold
    return (maxEnergy > 140 && peakBin !== -1) ? peakBin * binWidth : 0;
}

Why I’m proud of this: It runs at 60 FPS directly within the p5 audio loop with zero noticeable latency, allowing the voice abilities to feel instant and responsive.

When rewinding time, standard mechanics often rewind the player along with everything else, trapping them in the same fatal mistake. I designed an asymmetric rewind system where the game world rolls back, but the player retains full agency to escape incoming hazards:

// Rewinds traps and level obstacles while leaving the player freely movable
function applyTimeRewind(targetFrame) {
    let snapshot = timeHistory[targetFrame];
    if (!snapshot) return;

    // Restore environmental state (traps, moving platforms, crumbling floors)
    for (let i = 0; i < snapshot.traps.length; i++) {
        Object.assign(level.traps[i], snapshot.traps[i]);
    }
    gameTime = snapshot.gameTime;

    // Noticeably absent: player.x and player.y are NOT restored.
    // The player stays completely active and can dodge out of the trap's path!
}

Why I’m proud of this: It turns a frustrating trial-and-error moment into a dynamic, clutch escape mechanic that feels empowering to the player.

Reflection

I am pretty happy with the final game, I learnt a lot of new things in p5 and realized how the organization of the code into classes representing various types of objects made it much easier to understand and develop. There is some scope for improvement that could be done in the future:

1. 1. Making it multiplayer. (Conflicting hums would be interesting to tackle)
   2. Adding more sound effects and tricky situations in levels.
   3. Adding a level map, so players can revisit old levels

The Concept

For my midterm project, I want to make a walking game controlled by sounds the player records. Instead of assigning fixed voice commands, the game will let each player choose a short sound for Up, Down, Left, and Right. A player could use spoken words, clicks, whistles, or other distinct sounds. I like the idea that the controls become personal to the player, and that learning to move is part of the experience.

User Interaction

The player will first enable their microphone and record one sound for each direction. Once all four sounds are saved, they can press “Start listening” and play using the microphone. Each recognized sound moves the character for a chosen number of seconds. The player can adjust that duration with a slider and re-record a sound if it is difficult to recognize.

Movement on the two axes works independently. For example, if the player makes the Up sound and then makes the Left sound while Up is still active, the character moves diagonally. The arrow keys also work, which gives me a way to test movement separately from the microphone.

Code and Design Progress

I started with a p5.js prototype of a walking character. It loads a sprite sheet, separates it into animation frames, and displays the correct row for each direction. The character animates while moving and stays within the canvas.

I then added the microphone setup, recording buttons, sound recognition, and timed movement. The code samples a sound’s frequency profile, divides it into three parts over time, and compares it with the four recorded signatures. It rejects sounds when the match is too uncertain. I also changed the movement code to keep separate horizontal and vertical timers, which allows two directions to remain active together.

Most Frightening Part

The most uncertain part is sound recognition. A person will not make a sound exactly the same way twice, and background noise could cause a false command. Rapid commands are another challenge: the program needs to notice where one sound ends and the next begins.

To reduce this risk, I built the recognition system before adding more game features. The prototype normalizes the frequency profile so a louder or quieter version of a sound can still match. It waits for a short silence to finish recording a command and rejects ambiguous matches. This gives me a working system to test with real sounds, although I still need to find out how reliably it performs in different rooms and with different players.

Next Steps

My next step is to test several sets of four sounds and record which commands are missed or confused. I can use those results to adjust the detection thresholds and the pause between commands. After the controls feel reliable, I want to design a space for the character to navigate so that the audio movement has a clear goal.

The artwork I kept thinking about after reading Levin’s essay was Cheese. The performers had to hold a smile while a computer measured it, and an alarm went off when their smiles dropped below a certain level. I can imagine laughing at that at first, then becoming very aware of my own face. It made me wonder when an artwork that responds to you starts to feel like it is judging you.

That example also made the difference between human and computer vision clearer to me. I might look at someone’s smile and wonder whether they are tired or joking. The computer needs a way to turn what its camera captures into something it can measure. Even finding a person in the image takes work. Frame differencing looks for changes between video frames, so someone standing still could disappear from its view. Background subtraction finds what differs from an empty scene, while brightness thresholding works when a person or object is clearly lighter or darker than its surroundings. I was surprised by how much the artist can help simply by changing the lighting or background.

I like Krueger’s idea that “the entire human body ought to have a role in our interactions with computers.” It opens up more playful possibilities than clicking a mouse. I would be curious to make something that reacts when a visitor turns toward it or moves away, without trying to decide what their expression means. Tracking can make a person feel involved in an artwork, but Cheese reminded me how easily that same attention can feel like surveillance. I think visitors should be able to tell when an artwork is watching them and choose whether to take part.

Concept

For this assignment, I created an interactive artwork called Don’t Notify Me. It shows a self-portrait of me trying to work on my laptop while a stream of notifications keeps breaking my concentration. Participants can write their own notification, send a blank one, reset the scene, or tap the lamp to switch on focus mode.

A normal notification takes about two seconds to pull my attention away from the laptop. Strangely, a blank notification gets an even faster reaction. Since it gives me no information, I immediately want to know what it means. One or two notifications do not bother me much, but my mood changes as they pile up. After three pings in a row, I start to look annoyed. At five, I lose my patience completely. My expression changes, the scene turns red, my replies become more direct, and the sounds get harsher.

I wanted the experience to be playful, but also familiar to anyone who has struggled to focus while notifications keep appearing. The participant is not just watching me become frustrated. They are the one causing it. Their timing and messages affect my expressions, responses, and surroundings, creating a small back-and-forth between the participant and the artwork.

Code highlight

One part I am especially proud of is the way the artwork uses CSV files to manage its timing and dialogue:

settingsTable = await loadTable("settings.csv", ",", "header");
responsesTable = await loadTable("responses.csv", ",", "header");

currentPing = {
  message,
  createdAt: now,
  reactionAt: now + (
    message ? config.normal_delay : config.empty_delay
  )
};

settings.csv contains values such as the reaction delays, streak window, and mood thresholds. responses.csv contains the character’s possible replies to calm, annoyed, furious, urgent, and empty notifications. Keeping this information outside the main sketch makes the project much easier to adjust. I can change the timing or rewrite the character’s responses without digging through all the drawing code.

The character’s mood depends on the current notification streak:

const score = Math.max(
  0,
  streak - (theme === "dark" ? config.focus_relief : 0)
);

if (score >= config.furious_at) return "furious";
if (score >= config.annoyed_at) return "annoyed";
return "calm";

Focus mode lowers the effective streak instead of erasing it. Tapping the lamp can help the character calm down, but it does not act as if the earlier interruptions never happened.

How this was made

I adapted an earlier p5.js self-portrait and turned it into a scene of me working at a table. The laptop screen faces the character, so the audience sees the back of it. While I am working, my eyes point down toward the screen. Once the reaction delay ends, my eyes move toward the notification and a speech bubble appears.

The controls beneath the canvas use a real HTML text field and buttons. Participants can type a message and press Enter, click Send Ping, send a deliberately blank notification, or reset the scene. Each notification appears as a card inside the artwork, along with a timer showing how long it is taking to catch my attention.

The program also tracks how quickly the notifications arrive. Pings sent close together build a streak, while leaving me alone for long enough resets it to zero. This makes the interaction feel less like pressing a button and more like testing someone’s patience.

One part of the reading that stood out to me was the distinction between interaction and reaction. It says, “Interaction and reaction are apples and oranges, horses of different colors, tigers of different stripes.” This made me think about how easy it is to mistake a system reacting to the user for a system actually being interactive.

A strongly interactive system, to me, is something that is open to interpretation and interaction in a way where the user is truly connected to the system through an action-reaction ecosystem. The user’s actions should have some meaning within the system, rather than simply triggering a response that could have happened regardless of what they did.

In my current sketches, I only let the user control the local state of the sketch using mouse clicks and drags. The sketch reacts to what the user does, but after reading this, I think there is more I could do to make that interaction meaningful. I could make a sketch that expects a more explicit answer or question from the user and responds back to them in a meaningful way.

It should also involve me, as the developer, communicating a message or feeling that is open to interpretation. The user should be able to affect the sketch while also forming their own interpretation of what the sketch is trying to communicate. That feels closer to the kind of interaction described in the reading, rather than simply making a sketch that reacts to inputs.

Concept

I wanted to take our in-class example of making a car class/element and use the car as a pen to scribble over the canvas. The point of this type of generative art was to create an unending and random pattern, while giving the viewer an essence of cyclicalness as the cars seemed to rush towards the right end of the canvas, and then reemerge from the other side only to repaint over their own scribbles.

Moreover, I wanted to add meaningful interactivity in the form of terrain creation, where the user can drag their pointer on the canvas to create a muddy region. This muddy region slows down the cars horizontally while maintaining their constant aggressive back-n-forth vertically. This jitter zone eventually populates with ink much faster than the rest of the canvas, and, although the mud is eventually enveloped by the scribbles, the texture of that region remains visibly distinct and dense.

Adding the telemetry HUD showing real-time average speed was a fun experiment too, giving immediate visual feedback as soon as you trap cars in your mud pits.

Code that I’m particularly proud of

 /**
   * Steers agent away from nearby cars to maintain safe personal space.
   */
  applySeparation(nearbyCars) {
    let desiredSeparation = 14;
    let steerX = 0;
    let steerY = 0;
    let neighborCount = 0;

    for (let other of nearbyCars) {
      if (other === this) continue;

      let dx = this.posX - other.posX;
      let dy = this.posY - other.posY;
      let distance = sqrt(dx * dx + dy * dy);

      // Apply repulsive force inversely proportional to distance
      if (distance > 0 && distance < desiredSeparation) {
        steerX += (dx / distance) * (desiredSeparation - distance);
        steerY += (dy / distance) * (desiredSeparation - distance);
        neighborCount++;
      }
    }

    if (neighborCount > 0) {
      this.posX += (steerX / neighborCount) * 0.04;
      this.posY += (steerY / neighborCount) * 0.04;
    }
  }
}

Why: When spawning the initial cars in a grid, I noticed that often times overlaps would happen. I started with 900 cars (30×30) initially and would use per-car seperation logic, that would reduce FPS by a lot. To reduce this calculation, I decided to implement a solution called spatial grid updating. It immidiately boosted the FPS and it future proof.

Embedded sketch

How Was This Made

This project is built around an autonomous agent model combined with an off-screen persistent canvas:

1. 1. Persistent Trails via createGraphics: Normally, calling background() in draw() clears every prior frame. To allow cars to act as drawing pens, I created an isolated rendering layer named trailLayer using p5’s createGraphics(). The car chassis, tires, mud, and HUD are redrawn continuously on the main canvas, while each car leaves permanent line strokes on trailLayer beneath them.
   2. Organic Movement & Drawing: Instead of moving in rigid straight lines, each car is given a small randomized horizontal step and an erratic vertical offset (random(-4, 4)) on every frame. This converts pure vehicle physics into an energetic, hand-drawn scribble texture.
   3. Interactive Mud Physics: The terrainPatches array records coordinates clicked or dragged by the user. On each update tick, each car performs a radius check against active terrain circles. When an intersection is detected, currentSpeed is cut to one-third of its maximum speed, concentrating the scribbles into dense clusters.
   4. Color Scheme: The color palette was curated based on vintage Bugatti racing liveries (French Racing Blue, Rouge Garance, Deep Navy, Atalante Yellow, and Obsidian Black Carbon) rendered against an off-white, parchment-toned canvas (#E4D5B7) to evoke technical drafting or blueprint paper.

References & Inspiration:

• • Craig Reynolds’ Boids Algorithm: The separation logic in applySeparation() is directly adapted from Reynolds’ classic steering behavior rules for autonomous flocking agents.
  • Spatial Hash Grids: The SpatialGrid bucket implementation was referenced from standard 2D broadphase collision detection patterns used in real-time game engines.
  • In-Class Vehicle Class: The starting chassis and wheel coordinate offsets were built out from our foundational class OOP exercise.

Problems Encountered & Solutions:

• • The Screen-Wrap Laser Bug: When cars reached the right edge and wrapped around to x = -10, line(prevX, prevY, this.posX, this.posY) drew a jarring, full-width horizontal stroke across the screen. I fixed this by adding a conditional gate (if (this.posX >= prevX)) so trails are strictly drawn while vehicles travel forward.
  • Input Flooding on Mouse Drag: Dragging the cursor across the canvas pushed hundreds of identical terrain coordinates into memory every second. I added an addTerrainPatch() helper with a minSpacing threshold of 8 pixels, ensuring clean, evenly spaced patches without choking the update loop.

Future Improvements:

Currently, there’s only one type of terrain (mud), I would like to introduce a few more, such as a zero friction zone (ice patch) that boosts horizontal velocity over the zone. Additionally, the canvas currently gets too full, and the user has no way of canvas clearing. I could add a small corner button that clears the trails, or a long press that slowly clears out trails from bottom up. Another experiment would be adding sounds to each car based on spatial circumstances.

Concept

For this assignment, I wanted to create something that felt like rain, but also resembled microorganisms under a microscope. After watching Casey Reas’ talk, I became more interested in creating a system rather than manually designing every part of the image. I wanted to start with a clear structure, rain falling downward, and then introduce randomness and interactions between the particles.

The particles are loosely inspired by microorganisms because they have translucent shapes around them and move in groups. However, I did not want to directly draw bacteria or cells. Instead, I wanted the biological feeling to come from the way the particles behave. Each particle reacts to nearby particles by attracting, repelling, and slightly aligning with them. This means that even though I set the rules, I cannot completely predict the patterns they will create.

I also added mouse interaction so that the viewer becomes part of the system. Moving the mouse pushes nearby particles away and creates a disturbance. I liked this because it makes the artwork feel less like an animation that is simply being watched and more like an environment that the viewer can affect.

Code I’m Proud Of

The part of my code I am most proud of is the interaction between particles. Instead of giving every particle a specific path, I made them respond to their surroundings.

if (distance < 70 && distance > 0) {
  nearbyCount++;
  attractionX += distanceX / distance;
  attractionY += distanceY / distance;
  alignmentX += otherParticle.velocityX;
  alignmentY += otherParticle.velocityY;

  if (distance < 18) {
    repulsionX -= distanceX / distance;
    repulsionY -= distanceY / distance;
  }
}

This checks for particles that are nearby and gives them different behaviors depending on how close they are. They are slightly attracted to particles that are farther away, while particles that get too close repel each other. They also look at the movement of nearby particles and slightly align with them. I liked this part because a fairly small amount of code can create much more complicated movement when it is repeated across hundreds of particles.

How Was This Made?

I started by creating an array of 250 particles. Each particle has an x and y position, a velocity, and a size. I initially gave all of them a downward velocity so that the overall system would resemble rain.

After that, I added the particle interactions. For every particle, the code checks the other particles within a certain distance. Based on those distances, the particles attract, repel, and align with one another. I also added random() to their movement so that they do not behave exactly the same way every time.

The background is dark purple and slightly transparent, which allows previous frames to remain visible. This creates trails behind the particles and makes the canvas itself become part of the artwork.

Finally, I added mouse interaction. When the mouse gets close to a particle, the particle is pushed away. Moving the mouse quickly creates a stronger disturbance, so the viewer can influence the behavior of the system.

Reflection and Ideas for Future Work

I think this assignment helped me understand Casey Reas’ idea of balancing structure and randomness more clearly. I started with a very controlled idea, which was simply particles falling like rain, but then allowed the system to become less predictable through randomness and particle interactions. I liked that I was defining the rules without directly defining the final image.

One thing I found interesting was that the particles sometimes begin to move together and form patterns even though some of their movement is random. This relates to Reas’ discussion of swarm dynamics and made me think more about how systems can create order on their own.

I am fairly happy with the final result, especially because the viewer can disturb the system with the mouse. In the future, I would like to experiment with changing the rules while the artwork is running, so the system itself could gradually become more unpredictable.

In the video, Casey Reas talks about moving from total structure toward slowly introducing randomness within a controlled environment, and eventually giving up even more control to the canvas itself. I particularly liked his discussion of structured order, including the structure within his own work, as well as the way he critiques the philosophy behind each artist he presents.

When he says, “I wanna make my own reality,” and begins talking about imitating biology, I understood exactly where he was coming from. As a visual artist, he seems to move from thinking about objects to thinking about behaviors, and then to thinking about the individual elements that make up those behaviors. Once those elements begin interacting according to certain rules, they can produce outcomes that are difficult to predict, what he describes as a form of “chaos.”

I also found his discussion of swarm dynamics interesting. Even when each individual element behaves somewhat randomly, the system can eventually synchronize and settle into a pattern. That raised an interesting question for me: how do you design a system that continues to feel random and alive rather than eventually becoming predictable?

My biggest takeaway from the video was the idea of applying these concepts to multisensory artwork, especially when Reas brings up John Cage’s use of chance in music. A computer can take a simple set of rules or sounds and multiply them millions of times, creating something far more complex and extraordinary than what was originally defined. At first, I might see something like that as just noise or background. But as Reas demonstrates, that “background” can be given its own behavior and brought to life in unexpected ways.

Concept:

I’ve tried to show my personality as a night person through p5.js. At first, you can see how unsatisfied I am with the lamp being turned on, when clicked, I get happy! I also tried to make my glasses as a prominent feature of my face, as they’ve been with me for a very long time. It also showcases my love for dark mode, and dark themes, specially on digital displays.

I also implemented lower and higher opacity on the lens of my glasses as if they’re Photochromatic, which is something I’m particularly a fan of.

The unique check background is also symbolic of my interest in board games, specially chess.

How this was made:

This self-portrait was made multiple elements of p5.js such as arcs, rectangles, lines, ellipses, fills, strokes, and also a combination of push(), translate(), and scale().

I started with a simple circle using ellipse, the top and side hair using more ellipses and rectangles, then the specs (using multiple lines and rectangles), and eventually the smile (arc) and the chin (also arc). I reused some code from my draw_dude() function I worked on in last class for the body, while creating an original background using checks. The lamp was a bonus as I learned corner rounding in the process of making it look sleek, now it’s an integral part of what I’m trying to show as my personality.

My toughest challenge was actually colour theory, as I couldn’t get the background to match my skin and other elements. Also the face looks rounder than I preferred. This is where the push(), translate(), and scale() came in handy, I used it to slim down my face without redoing all the hardcoded values. Next time, I would be more inclined to declare variables early.

Code I am Proud of:

To be honest, I’m proud of the entire codebase for this portrait, but if I had to pick, I’d choose draw_specs:

function draw_specs(pos_x, pos_y, size, theme = "light"){
  if (theme == "light"){
    fill("#00000080");
  } 
  if (theme == "dark") {
    fill("#00000030");
  }
  stroke("#90D5FF");
  rect(pos_x + size/10, pos_y - size/25 + 2, size/2);
  rect(pos_x - size/1.7, pos_y - size/25 + 2, size/2);
  
  stroke("#404040");
  strokeWeight(2);
  line(pos_x - size/3, pos_y - size/25, pos_x + size/3, pos_y - size/25);
  line(pos_x - size/10, pos_y + size/8, pos_x + size/10, pos_y + size/8);
  line(pos_x + size/10 + size/2, pos_y + size/8, pos_x + size, pos_y + size/5);
  line(pos_x - size/1.7 - size/2.4, pos_y + size/5, pos_x - size/1.7, pos_y + size/8);
}

The feeling of drawing my favourite accessory on a digital screen using code was very rewarding in itself. Each line and shape was simple yet intentional.

Reflection:

As a Computer Science major, I’m often induced to work on delivering an output or solving a problem. I’ve never had the opportunity to pay undivided attention to the visual elements, instead of just the functional ones. So while doing this, I realised the very essence of what I like to see as a software developer, and being able to convey my personality through my projects. It’s one of the few assignments I’ve genuinely had fun finishing.