Category: F2025 – Aya

Q1: What are some of the ways that computer vision differs from human vision?

Humans see in a flexible and intuitive way. We can recognize a friend even if they’re in the shadows, wearing different clothes, or drawn as a cartoon. Computers, on the other hand, are much more rigid. They need clear cues, like the right lighting, steady backgrounds, and often lots of training data, just to recognize something we would see instantly. Computers don’t bring context or common sense either. If I see someone running, I might guess they’re late or playing a sport; a computer just sees moving shapes. This difference means that in art, computer vision often works best when the artist designs the environment to make it easier for the machine to see, but not to interpret it like humans. 

Q2: What are some techniques we can use to help the computer see / track what we’re interested in? 

On the technical side, artists use things like motion detection (spotting what changes between frames), background subtraction (separating a moving person from a still background), or color filters (tracking a red ball). More advanced tools can follow body joints or estimate a skeleton, which is great for dance or performance. But beyond algorithms, the environment is just as important. If you give the system good lighting, a solid backdrop, or make participants wear bright colors, the system would be able to focus on one thing and spot them more easily. It’s less about forcing the computer to be “smart” and more about designing the whole setup so the vision works smoothly.

Q3: How do you think computer vision’s capacity for tracking and surveillance affects its use in interactive art?

Computer vision’s ability to track people is both a strength and a challenge for interactive art. On the positive side, tracking makes the art responsive, installations can change based on where you move, how you gesture, or even how many people are in the space. This creates a playful, engaging experience that feels alive. But because the same technology is often used for surveillance, it can also make people feel watched. That changes how the audience experiences the artwork, sometimes it’s fun, but sometimes it raises concerns about privacy. Many artists lean into this tension: some use tracking purely for interaction, while others use it to make us think critically about how surveillance works in our daily lives.

Concept 

I’m making a small interactive vertical Chicken Invaders style shooter game. The player controls a spaceship at the bottom of the screen and can move up, down, left, or right while shooting bullets upward. Chickens (enemies) continuously descend from the top of the screen, and some randomly drop eggs that can damage the player.

The game starts with a start screen that tells the player how to play: “Arrow keys to move, SPACE to shoot. Press SPACE to start.” Once the player starts, waves of chickens begin descending. The player earns points by shooting chickens and can lose lives if hit by enemy eggs. After losing all lives, the game transitions to a Game Over screen, showing the final score and instructions to restart by pressing ENTER.

Player interaction is simple and intuitive:

• Arrow keys: move the spaceship in all directions

• SPACE: shoot bullets upward

• Collision detection: bullets destroy enemies, eggs damage player

• Score and lives display updates in real time

• Continuous spawning keeps the game dynamic and engaging

Code Design

The game is designed in an object-oriented structure:

1. Player class:
   • Controls the spaceship’s position, movement, and display
   • Constrains the player within canvas boundaries
2. Bullet class:
   • Manages bullet position, movement, and collision with enemies
3. Enemy class:
   • Controls chicken position, downward movement, and spawning
   • Randomly generates enemy bullets (eggs)
4. EnemyBullet class:
   • Manages eggs dropped by enemies, movement, and collision with the player
5. Main sketch:
   • Holds a gameState variable: "start" | "playing" | "gameover"
   • Delegates drawing and updates depending on the current game state
   • Spawns new enemies continuously and updates UI elements (score, lives)

Current Game:

Risk & Challenges

The scariest part was managing continuous enemy spawning and collision detection while ensuring smooth movement and responsive player controls. If either bullets or enemy collisions failed, the gameplay would feel broken, and the game could become frustrating.

How I reduced the risk:

• I wrote small test sketches for bullets and enemy collisions first, confirming that collisions were detected reliably

• Spawn timers were implemented to gradually introduce enemies without overloading the canvas

• Player boundaries were constrained using constrain() to avoid moving off-screen

• The gameState system ensures that start, playing, and game over screens never interfere with gameplay logic

Result: collisions, enemy spawning, and player movement are smooth and reliable, so the core gameplay is stable and enjoyable.

Next Steps

• Add background music that loops during gameplay and optional sound effects for shooting and enemy hits

• Add visual enhancements: use emojis, icons, or sprites for player and chickens for more arcade-like feel

• Add multiple enemy types: faster chickens, stronger enemies, or bonus targets

• Implement power-ups: shields, rapid fire, or score multipliers

Concept:

function mouseClicked() {
  if (gameState === "start" && playButton.clicked(mouseX, mouseY)) {
    playButton.action();
  } 
  else if (gameState === "playing") {
    // Change clothes when clicking on body parts
    if (mouseX > width / 2 - 50 && mouseX < width / 2 + 50 && mouseY > 300 && mouseY < 400) {
      currentPants = (currentPants + 1) % pantsColors.length; // Change pants
    }
    else if (mouseX > width / 2 - 50 && mouseX < width / 2 + 50 && mouseY > 150 && mouseY < 300) {
      currentShirt = (currentShirt + 1) % shirtColors.length; // Change shirt
    }
    else if (mouseX > width / 2 - 50 && mouseX < width / 2 + 50 && mouseY > 400 && mouseY < 500) {
      currentShoes = (currentShoes + 1) % shoesColors.length; // Change shoes
    }

In this reading, Levin argues that the tools, techniques, and physical setups required for basic computer vision are no longer exclusive to only specialists, they can now be used by students and artists. Reading this article really got me thinking about how different human vision is from computer vision.

One way computer vision differs from human vision is that computers don’t have a built-in understanding; they just see pixels and values. They need algorithms to interpret color, brightness, motion, etc. Humans, on the other hand, have an innate understanding.

Some techniques we can use to help the computer see and track what we’re interested in include frame differencing (detecting motion by looking at changes between frames), background subtraction (using a static background and removing it to isolate objects), and brightness thresholding (using contrast to separate what you want from what you don’t).

Computer vision’s capacity for tracking and surveillance has its pros and cons. On the positive side, tracking allows for more responsive, immersive, and personal interactive art. On the downside, issues like surveillance, privacy and consent arise. If art is tracking people, there’s always a risk of misuse and discomfort.

Concept

For my midterm project, I’ve decided to make a Star Wars-themed game, where the user plays as Luke Skywalker. The aim of the game is to defeat the Stormtroopers on the spaceship. The aesthetic of my game is going to look similar to the image below. I want to go for the pixel-style simple game.

I got the inspiration for my game’s mechanics from Street Fighter. But instead of two players fighting against each other, I’ll make it a one-player game fighting against NPC enemies.

I’m thinking of making my game look something like this:

Design and User Interaction

• The player (Luke) will be able to move and attack.
• The Stormtroopers will spawn on the opposite side of the screen and run towards Luke.
• The game will include a time-limit (which goes on until all his lives/HP is gone, or to maximize points within the time-limit).

Challenges and Risks

The most complex part is probably figuring out how Stormtroopers attack. I’m probably going to program them in a single class, which will spawn all of the enemies and their attack patterns. I’m also uncertain about the point system. I’m not sure whether to go for an HP/lives system, or a points system where he gains points for killing each enemy. But either way I’ll include a time limit.

To minimize the risk, I’m planning to divide the player and NPCs into two separate classes. I’ll also experiment with a simple collision detection test to confirm that Luke’s attacks properly register against enemies.

This is what my sketch looks like so far:

<iframe

Concept

For my midterm, I wanted to make a project that isn’t just fun, but also means something to me, feels useful to create, and reflects my interests, which for me is combining women’s empowerment, sports, and music. Because this project allows so much freedom, my mind kept jumping from idea to idea. I finally decided on a concept that felt meaningful and achievable: an interactive story experience where users guide a female athlete through multiple sports — football, basketball, and martial arts — while overcoming challenges in the form of negative comments from people, culminating in a celebratory dance that reflects the user’s gameplay.

I also wanted to incorporate music, one of my biggest passions, into the gameplay. Each action the user takes (scoring a goal, making a basket, or landing a punch) will add a layer of sound to a final anthem, letting players create a rhythm-based celebration that reflects their journey and success.

To make the experience authentic and inspiring, I decided to use real female athletes as the characters for each stage: Sam Kerr for football, Breanna Stewart for basketball, and Ronda Rousey, the first female UFC champion, for martial arts. Each athlete has a quick introduction before their stage to give context and highlight their achievements and the obstacles they’ve overcome.

My project aims to:

• Educate users about the basics of each sport.
• Empower women by breaking stereotypes and highlighting achievements.
• Engage users through rhythm-based gameplay that integrates music and beats.
• Inspire users to create a unique anthem reflecting the character’s journey.

Design

I want the design to be interactive and bold. The journey starts with a fullscreen instruction screen, and each stage is preceded by a quick introduction of the athlete, with a short text and a character.

The gameplay is divided into stages:

1. Football: Users dribble and shoot as Sam Kerr. Negative comments appear as barriers, which shatter when the user scores, adding beats to the anthem.
2. Basketball: Users control Breanna Stewart, shooting and dribbling while shattering barriers and adding beats to the music.
3. Martial Arts: Users perform punches and kicks with Ronda Rousey, breaking barriers that add more beats to the anthem.
4. Celebration / Dance: The beats from previous stages combine into a unique anthem, and character performs dance moves that sync with the rhythm, celebrating the journey.

Visually, I’m keeping the design stylized and clean, using shapes for interactive objects. Sounds, like cheering, percussion, and piano, respond dynamically to user actions, making the experience engaging.

Frightening / Challenging Aspects

There are a few parts of this project that I’m worried might be tricky:

• Making the rhythm-based gameplay work: I’m unsure about getting the sounds from scoring, punching, or breaking barriers to line up and feel smooth.
• Creating each sport stage: Each sport has different moves and controls, and it might be hard to make them all work well.
• Drawing the characters and barriers: Making all the athletes and objects using shapes in p5.js could take a lot of time and get complicated.
• Combining all the beats into a final anthem: I’m unsure if the sounds from football, basketball, and martial arts will come together nicely to make a fun and satisfying rhythm at the end.

Risk Prevention

To manage the different sports stages, I will use separate layers so only the relevant athlete, objects, and barriers show up at the right time. For example, when the user moves from football to basketball, the football elements disappear and the basketball stage appears. Designing all the characters and barriers from scratch in p5.js could take a long time, so I started making them on separate canvases to work faster and keep things organized before combining them into the full project. As well as I started on the intro page. Making interactions feel natural is also tricky because I need to detect clicks, key presses, or movements on objects made of multiple shapes. I’ve started experimenting with this early to make sure the gameplay will feel smooth.

Initial Concept:

The core concept of my project is to create a game where a panda chef stacks pancakes on its tray while pancakes fall from random positions in the sky . The gameplay is intentionally simple—pancakes fall from the sky, and the player moves the panda left and right to catch them before they hit the ground. I wanted the concept to be both whimsical and approachable, something that could appeal to all ages while still having the potential for engaging mechanics like timers, scoring, and fun visuals.

Design

The panda sprite sheet is animated using a frame-based system that cycles through images depending on movement, while background elements like clouds and grass are generated with simple loops and p5 shapes for efficiency. Pancakes are handled as objects using a dedicated class, which keeps the code modular and easy to expand. I also separated core functions—like drawing the welcome screen, updating the game state, and spawning pancakes—so the program remains organized and readable. This approach makes the design not only playful on-screen but also manageable under the hood.

Potential Struggles:

The most frightening part of the project was integrating sounds into the game. I know that audio is essential to making the experience immersive, but I was unsure about the technical steps required to implement it smoothly, especially since I had issues in the past when it comes to overlapping sounds and starting and stopping it accurately. Questions like when to trigger sounds, how to loop background music, or how to balance audio levels without them being distracting added to the challenge.

How I Plan to Tackle:

To reduce this risk, I turned to existing resources and examples, especially by watching tutorials and breakdowns on YouTube. Seeing other creators demonstrate how to load and trigger sounds in p5.js gave me both practical code snippets and creative inspiration. By learning step by step through videos, I am hoping to be able to gradually integrate audio without it feeling overwhelming.

Embedded Skecth:

Reading Response:

Computer vision i would say is different from how humans see. We naturally understand depth, context, and meaning, but computers just see a grid of pixels with no built-in knowledge or ability to infer and make connections like humans do. They need strict rules, so even small changes in lighting or background can throw a computer off.

To help computers “see” better, we often set up the environment in their favor, meaning that we cater to their capabilities. Simple techniques like frame differencing (spotting motion), background subtraction (comparing to an empty scene), and brightness thresholding (using contrast) go a long way. Artists and designers also use tricks like infrared light, reflective markers, or special camera lenses to make tracking more reliable.

What’s interesting in art is how this power of tracking plays out. Some projects use computer vision to make playful, interactive experiences where people’s bodies become the controller. Others use it to critique surveillance, showing how uncomfortable or invasive constant tracking can be. So, in interactive art, computer vision can both entertain and provoke — it depends on how it’s used.

When I think about how computer vision differs from human vision, what stands out most is how little meaning computers can extract on their own. To a human, even a blurry image is packed with context, emotion, and symbolism. To a computer, it’s just a grid of pixel values that need to be processed before anything useful can be recognized. This is why the article highlights techniques like frame differencing, background subtraction, and brightness thresholding, which are methods that help a computer separate what is important from what is in the background.

To make these systems work, we often need to modify the image so the computer can interpret it. That might mean fixing the lighting, using high-contrast markers, or limiting the scope of what the computer is supposed to track. In a way, we design the world to fit the algorithm rather than expecting the algorithm to fully match the complexity of the world.

In interactive media, I think this capacity to interpret and track movement opens up exciting new directions for creativity. Computer vision gives artists a way to design works that respond directly to a person’s gestures, which is actually something I used in a project for another IM course. It transforms the audience from passive viewers into active participants, making the artwork something dynamic and alive. By combining human imagination with the computer’s ability to detect patterns, interactive art can become more immersive and responsive than ever before.

Concept + Interaction: 
I’m making a small interactive Barbie Dream House experience with mini-games in each room. You start outside the house and it tells you to “click the door”. When you do a doorbell sound plays and you go inside. The interior is a divided view with hotspots for rooms: closet, kitchen, bedroom, and living room.
Each room has a tiny activity:
Closet: wardrobe selector (browse outfits with ← / → and hit ✓ to confirm).
Kitchen: cupcake maker (choose base, pick frosting color, “bake” in an oven).
Bedroom: open a book to see a short quote or page.
Living room: click the TV to change pictures (channel changer). 

Code design:
Each interactive room is its own class/file.
Main sketch holds a simple state machine: state = “outside” | “inside” | “closet” | “kitchen” | …. When state === “closet” the sketch delegates to wardrobe.draw() and forwards clicks to wardrobe.handleClick().

I made a Wardrobe class that:

•  loads outfit sprite-pages and slices them into frames,
• centers and scales the current outfit into a fixed box,
• draws left/right arrow buttons and a confirm (✓) button,
• exposes handleClick() for arrows, page switches, and confirm.

Other rooms (Kitchen, Bedroom, LivingRoom) follow the same class pattern so swapping and testing is easy.

Here is my current progress:

https://editor.p5js.org/rma9603/full/5s96PtOnM

Biggest risk & what I did about it:
The scariest part was making outfit switching reliable: slicing sprite-sheets, handling transparency, and aligning click hitboxes. If that failed the closet would be unusable.

How I reduced the risk:
I wrote a tiny test that loads one outfit sprite-page and uses page.get() to draw each frame side-by-side so I could verify cropping and transparency immediately. I exported/uploaded transparent PNGs (no white bg) and re-tested them.

centralized coordinates: arrows, confirm button, and hitboxes compute from the same this.box values so visuals and clicks always line up.

Result: the sprite-sheet method works, i can slice pages reliably and the outfit switching is stable, so i’m confident i can implement the cupcake and other rooms the same modular way.

Next steps:

•  finish the kitchen cupcake game (choose base, frosting, bake timer).
•  Create the bedroom interaction
•  Create living room interaction
• add start/instructions screen and a “new session” reset button so people can restart without refreshing.
•  hover feedback + polish (shadows, small tweaks).
• Make it full screen.

Main Concept and User Interaction:

Super Mario has been the greatest game I’ve ever played, and I have so many good memories associated with it. I used to play it with my whole family after my dad came back from work. Since I was a child, I didn’t know how tired he was after finishing his job, but I kept begging him to play with me. Even though he was apparently exhausted, he always said yes to me, and I was absolutely happy and excited every night. Super Mario is thus a fundamental part of my childhood. For this midterm project, I wanted to bring back that childhood memory by making a simple version of Super Mario Bros 2, a Wii game I used to play with my family.

Most Uncertain Part of My Midterm Project:

The most uncertain part of my midterm project is how to actually make Super Mario run and jump using the sprite concepts that we learned in class. Since I had no experience making animations where a character responds to key presses, I started the project by completing this feature. I first went to a website called “The Spriters Resource” to get sprite images of Super Mario and then imported them into my project. But since the sprite images had a green background, I had to remove it using another website “Remove Background” and make the background transparent so that Mario matched the canvas background.

Code Snippet:

As you can see, this is the code I wrote to animate Mario running and jumping. I used the same logic that we discussed in class. If a user presses the right arrow, Mario runs to the right. If a user presses the up arrow, Mario jumps vertically. I used modulo to make sure the running animation loops back to the first frame and doesn’t go over the limit.

let isRunning = false;
  let isJumping = !onGround;

  //Make mario run towards right 
  if (keyIsDown(RIGHT_ARROW)) {
    x += speed;
    isRight = true;
    isRunning = true;
  }
  //Make mario run towards left
  if (keyIsDown(LEFT_ARROW)) {
    x -= speed;
    isRight = false;
    isRunning = true;
  }
  
  //make mario always stay inside the canvas 
  x = constrain(x, 0, width); 

  //animation for running
  if (isRunning && onGround) {
    //every 6 frame, move to the next sprite 
    if (frameCount % 6 === 0) {
      index = (index + 1) % sprites[1].length; //use modulo to loop through the same animation
    }
    drawMario(sprites[1][index]); //draw running Mario
  }
  
  //Animation for jumping 
  else if (isJumping) {
    drawMario(sprites[3][0]); //sprite for jumping
  }
  
  //sprite for idle 
  else {
    drawMario(sprites[0][0]);
  }

Sketch:

Reflections and Future Improvements:

It was relatively easier than I expected to implement the animation of Mario running and jumping because I only had to apply the concepts we learned in class, such as using nested arrays to store each sprite image and if-else statements to trigger certain actions when a certain key is pressed. However, I still need to fix one critical issue, which is when Mario runs too fast, the background image glitches. I am still not sure how to solve this issue, so I would like to ask my professor during the next class. To make it more like Super Mario Bros 2, I need to add some obstacles such as blocks, pitfalls, Koopa Paratroopas, Bullet Bills, and Goombas. I would also like to add a score tracking system where the longer you play, the higher your score gets. This is to make the game more interesting and fun.