Month: September 2025

Week 5 – Midterm Progress

After three days of painstaking brainstorming for my midterm, I came up with two directions: one was a game-like networking tool to help people start conversations, and the other was a version of Flappy Bird controlled by the pitch of your voice.

I was undoubtedly fascinated by both, but as I thought more about the project, it was clear that I wanted to experiment with generative AI. Therefore, I combined the personal, identity-driven aspect of the networking tool with a novel technical element.

The Concept

“Synthcestry” is a short, narrative experience that explores the idea of heritage. The user starts by inputting a few key details about themselves: a region of origin, their gender, and their age. Then, they take a photo of themselves with their webcam.

From there, through a series of text prompts, the user is guided through a visual transformation. Their own face slowly and smoothly transitions into a composite, AI-generated face that represents the “archetype” of their chosen heritage.

Designing the Interaction and Code

The user’s journey is the core of the interaction design, as I already came across game state design in class. I broke the game down into distinct states, which becomes the foundation of my code structure:

  1. Start: A simple, clean title screen to set the mood.
  2. Input: The user provides their details. I decided against complex UI elements and opted for simple, custom-drawn text boxes and buttons for a more cohesive aesthetic. The user can type their region and gender, and select an age from a few options.
  3. Capture: The webcam feed is activated, allowing the user to frame their face and capture a still image with a click.
  4. Journey: This is the main event. The user presses the spacebar to advance through 5 steps. The first step shows their own photo, and each subsequent press transitions the image further towards the final archetype, accompanied by a line of narrative text.
  5. End: The final archetype image is displayed, offering a moment of finality before the user can choose to start again.

My code is built around a gameState variable, which controls which drawing function is called in the main draw() loop. This keeps everything clean and organized. I have separate functions like drawInputScreen() and drawJourneyScreen(), and event handlers like mousePressed() and keyPressed() that behave differently depending on the current gameState. This state-machine approach is crucial for managing the flow of the experience.

The Most Frightening Part

The biggest uncertainty in this project was the visual transition itself. How could I create a smooth, believable transformation from any user’s face to a generic archetype?

To minimize the risk, I engineered a detailed prompt that instructs the AI to create a 4-frame “sprite sheet.” This sheet shows a single face transitioning from a neutral, mixed-ethnicity starting point to a final, distinct archetype representing a specific region, gender, and age.

To test this critical algorithm, I wrote the startGeneration() and cropFrames() functions in my sketch. startGeneration() builds the asset key and uses loadImage() to fetch the correct file. The callback function then triggers cropFrames(), which uses p5.Image.get() to slice the sprite sheet into an array of individual frame images. The program isn’t fully functional yet, but you can see the functions in the code base.

As for the use of image assets, I had two choices. One is to use a live AI API generation call; the other is to have a pre-built asset library. The latter would be easier and less prone to errors, I agree; but given the abundance of nationalities on campus, I would have no choice but to use a live API call. It is to be figured out next week.

 

Week 5: Reading Reflection

Our vision as people comes with a few prerequisites for creating a clear image in our mind, for example an object needs to be close enough with lighting that is sufficient for us to identify what it is. Computer vision is differentiated to our vision with the existence of a larger set of these prerequisites, the conditions need to be tuned to the abilities of the algorithm in place for accurate identification. With each system built for different purposes needing different unique conditions and pre-existing information to complete its purpose, while our vision is able to cover all ends such as motion detection and object detection without the need for different conditions. Which is why it might be a difficult task to integrate computer vision into interactive media, it is not only about picking the right system and mechanism, you need to set up the right circumstances for the system to successfully bring your vision to life.

However, even with the complication that come with it, the use of computer vision opens up a new realm of possibilities for interactive art. It takes the ability to interact with a user to another level with the possibility of now tracking tracking the movement of the user adding another layer to the ways the art can ‘listen, think and speak’ to the participant. This is especially true due to the increased accessibility to open source code and the technologies needed to navigate the world of computer vision, it is now easier than ever to utilize such technologies to explore themes more in depth. Though, this does raise the question that many of the works mentioned in the article tackle, which is surveillance. These technologies provide interactive art pieces with eyes, they are now partaking in the observing and tracking process of the work. This is more of a concern when it comes to those works installed in public spaces where the technology is used to detect the movement of people who did not choose to be part of the work. Which is why it is important to approach the use of such technologies with care to make the most of them in stepping up the level of interactivity in art works.

Week 5 – Reading Response

After reading Golan Levin’s “Computer Vision for Artists and Designers,” I’m left with a deep appreciation for the creativity that arose from confronting technical limitations. The article pulls back the curtain on interactive art, revealing that its magic often lies in a clever and resourceful dialogue between the physical and digital worlds, not in lines of complex code. Apparently, the most effective way to help a computer “see” is often to change the environment, not just the algorithm.

Levin shows that simple, elegant techniques like frame differencing or brightness thresholding can be the building blocks for powerful experiences, in contrast to my preexisting thought for a powerful CV system. The LimboTime game, conceived and built in a single afternoon by novice programmers who found a large white sheet of Foamcore, pushed the change in my perspective. They didn’t need a sophisticated algorithm; they just needed a high-contrast background. It suggests that creativity in this field is as much about physical problem-solving as it is about writing code. It’s a reminder that we don’t live in a purely digital world, and that the most compelling art often emerges from the messy, inventive bridge between the two.

The article also forced me to reflect on the dual nature of this technology. On one hand, computer vision allows for the kind of playful, unencumbered interaction that Myron Krueger pioneered with Videoplace back in the 1970s. His work was a call to use our entire bodies to interact with machines, breaking free from the keyboard and mouse. In the past or now, it is always joyful that our physical presence can draw, play, and connect with a digital space in an intuitive way.

On the other hand, the article doesn’t shy away from the darker implications of a machine that watches. The very act of “tracking” is a form of surveillance. Artists like David Rokeby and Rafael Lozano-Hemmer confront this directly. Lozano-Hemmer’s Standards and Double Standards, in particular, creates an “absent crowd” of robotic belts that watch the viewer, leaving a potent impression that I would not have expected from visual technology in the early 2000s.

Ultimately, this reading has shifted my perspective. I see now that computer vision in art isn’t just a technical tool for creating interactive effects. It is a medium for exploring what it means to see, to be seen, and to be categorized. The most profound works discussed don’t just use the technology; they actively raise questions about the technology. They leverage its ability to create connection while simultaneously critiquing its capacity for control. I further believe that true innovation often comes from embracing constraints, and that the most important conversations about technology could best be articulated through art.

Week 5: Pong² (Midterm Progress Report)

Concept:

Finding the concept for this project was rather difficult– I initially thought about creating a rhythm game that would use the WASD keys and the arrow keys similar to a 2D Beat Saber where the direction you cut mattered as much as your timing. I was inspired by Mandy’s project from Fall 2024 where she made a two player dance game where you have to input a specific pattern of keys in combination with your partner. I thought her project had a lot of charm to it; however, I imagine syncing the rhythm to the onscreen inputs would prove challenging so I scrapped that idea early on.

Then I revisited the project I made for week 3 where we had to use loops. Back then I just wanted to follow Professor Aya’s advice (when she visited on Monday of week 3) and use object-oriented programming to add manually controlled collision physics to the project I made for week 2. That’s how I accidentally made Pong but this week I considered seriously turning it into a fully-functioning project.

I realized that my code was going to be very messy and a long scroll from top to bottom if I kept it all in one .js file, so I looked up how to split up the files. After I created new js files I realized that setup() and draw() could only be run in the main sketch.js file so I had to find a way to work around that for the gameLogic js file which is just an imported week 3 js file. I basically made new functions and named them initiateGame() and initiateMenu() and called them in the main sketch.js setup() function; I also had to add two lines of code into the html file to have the project access these new js files.

<script src="menu.js"></script>
<script src="gameLogic.js"></script>

Updating the “PING” Game

Besides the obvious requirements to add a menu screen and a restart button, there were plenty of personal touches I wanted to add to what I made two weeks ago.

The first of which was to implement a more visually appealing score tracking system. Last time I had the game continuously repeat so I made a very minimalistic but number-based scoring system to fit with the rest of the minimalist aesthetic. Since I was adding a set number of rounds now, I wanted to use a more interesting way of representing each point. There would be a rectangle in the middle that would either flick upwards or downwards slightly based on which side took the point (kind of like a lightswitch in a way).

The next idea was to have the ability for an options menu to change the in-game volume mixer and maybe even keybinds.

Since there was also the need to reset the game without restarting the whole sketch now, I made sure to add an ESC function where you can back out to the menu from the game and also subsequently reset the scores and start a new game the next time the user clicks the button to enter the game.

//RESET SCORE
function resetGame(){ //resets the score
  roundNumber = 0;
  topScore = 0;
  topCounter = [];
  bottomScore = 0;
  bottomCounter = [];
}
...
function menuMousePressed(){
  twoPlayerButton.handleClick(mouseX, mouseY);
  resetGame();
}

I also made each button with its own class while passing functions as parameters to call when clicked. I have only test one button so far– the twoPlayerButton that leads you into the normal mode – but it works great and I’ve “scaffolded” an area for more buttons to be added the same way.

allMenuButtons.push(
  twoPlayerButton = new button(
    tempButton, //image 
    width/2 + 120, //xPos
    height/2, //yPos
    tempButton.width, //sets to uploaded file dimensions
    tempButton.height, 
    switchToGame //which function to call if clicked?
  )
);
User Interface Planning

To plan out the UI, I quickly made a Canva file that matched the dimensions of my project and made a quick sketch of what I wanted the menu to look like. I’m going for a rather

This is also how I came up with a better name for the game: Pong2  

It’s not quite Pong 2, but since the paddles are no longer restricted to one dimension along each edge and now have true 2D movement, I wanted to call it something representative of that.

For the font choice I chose Gotham for its geometric feel. Since p5js didn’t have access to Gotham, I downloaded an otf file online and placed it in my sketch folder.

For the final version of the sketch,  I want to add a faint bouncing ball in the background to make the menu screen feel more dynamic and alive too.

A detail I was quite proud about was setting up a hover checker that would turn my cursor into a hand if I was hovering over something that was clickable.

for (let btn of allMenuButtons) { 
  btn.display(); //displays the button
  
  if (btn.isHovered(mouseX, mouseY)) {
    hovering = true; //check if hovering any buttons
  }
}

if (hovering) {
  cursor(HAND);
} else {
  cursor(ARROW);
}
Identify the Most Frightening Aspect: 

The most frightening part of this project is most certainly the one player mode I want to make where you can play against an AI. Theoretically it’s just some simple math to have it sort of predict the ball’s trajectory but I imagine it would consume a considerable amount of effort to figure out how that would work.

I might drop this aspect altogether if it’s too challenging to make before the deadline but I really would like to make it work.

 

 

Week 5 Reading Reflection

When I compare computer vision to human vision, I realize how differently they work. For me, vision is immediate and layered with meaning, but for a computer, an image is nothing more than an array of numbers, vectors, and pixels. This reading reminded me of the summer I worked on an image understanding project with our faculty in the cybersecurity domain. I saw firsthand how computers don’t “see” an image the way I do. They only interpret numerical values and computations. Because of this, it’s even possible to generate two different sets of numbers that look the same to a computer, but when drawn out, they are hardly identical. That gap between what humans and computers perceive really fascinated me.

To help computers track what we want, techniques like frame differencing, background subtraction, and brightness thresholding come into play. Through this reading, I learned how important the setup is: lighting, camera placement, and reflective markers can make or break the system’s accuracy. It’s almost like computers need us to simplify reality so they can process it.

Where I feel conflicted is in the area of surveillance. In my cybersecurity project, I came to appreciate how powerful these systems can be, but also how much risk comes with that power. Since computers only “understand” numbers, they can be tricked, but they can also be used to monitor, record, and categorise people in ways that feel invasive. In interactive art, I see these tools as playful, engaging, and thought-provoking. But outside of art, they can become a threat. Surveillance powered by computer vision has the capacity to invade privacy, consolidate control, and even manipulate what we think of as reality. That tension makes me think harder about not just what computers can see, but what we allow them to see.

Week 5- Midterm Progress

Concept

For my midterm project, I wanted to build something interactive, competitive, and continuously evolving in difficulty. Inspired by the mechanics of hole.io, I decided to create a Blackhole game in p5 where the player controls a growing blackhole that consumes objects scattered across the canvas. The more the player consumes, the larger and stronger the blackhole becomes.

The concept behind this project is to give the user an engaging challenge: survive, grow, and adapt. I wanted to capture the thrill of growth mechanics where every interaction (consuming objects, avoiding hazards, gaining powerups) feels rewarding. Unlike my previous projects, this one leans toward a fast-paced arcade experience rather than something calm or exploratory.

To keep the game fun yet playable, I plan to design mechanics that gradually increase difficulty by reducing screen size. The goal is for the user to feel both challenged and accomplished as they make progress.

Design

The overall design will remain simple but dynamic, with a clear focus on gameplay elements. The blackhole will start small, almost unnoticeable, and will grow bigger as it absorbs objects.

The background will replicate the feeling of outer space, dark and starry, while asteroids and floating debris will act as obstacles and targets. Power-ups will be distinguishable by their shapes and colors, with each type offering unique abilities: faster movement, temporary freezing of asteroids, shrinking/growing effects, or invincibility.

The controls will be intuitive: the blackhole follows the player’s movement across the canvas, keeping the learning curve minimal so the focus remains on reaction and survival.

Frightening / Challenging Aspects

  • Considering the extreme conditions: what if the blackhole becomes larger than the screen? At that point, gameplay might break or lose its intended challenge.
  • Object Spawn Locations: How do I make sure that newly spawned objects do not appear inside the blackhole space, where they would be instantly consumed without interaction?
  • Minimum Distance Between Objects: Ensuring that two spawned objects are not too close to each other, which would make the game feel cluttered and unrealistic.
  • Shrinking Screen Space: The problem arises when objects fall outside the shrinking boundaries, how should those be handled?

 

Risk Prevention

  • Extreme Growth Prevention: To prevent the blackhole from growing uncontrollably, I will implement a mechanic where the hole begins to shrink if it hasn’t consumed anything in the past few seconds (e.g., 5 seconds). This ensures balance and keeps the game playable even in longer sessions.
  • Object Spawn Locations: While spawning new objects, I will check their coordinates against the blackhole’s current position and radius. If an object is detected within the hole’s space, it will be re-spawned at another random location.
  • Minimum Distance Between Objects: I plan to use a distance check between all newly spawned objects, ensuring that each object maintains a minimum distance from others. This can be achieved with a simple loop comparing coordinates and re-spawning objects if they are too close.
  • Shrinking Screen Space: I am exploring the use of constraint and filter functions to dynamically remove or reposition any objects that fall outside the shrinking boundaries. This way, the playable area remains fair and uncluttered as it reduces in size.

 

Week 5 – Midterm Project Progress

Project Concept & Design

For my midterm project, I wanted to make something that was a little different. Looking at past projects, I noticed that all the sketches used a ton of colors and soft shapes that created an aesthetically pleasing and welcoming environment for the user. So I thought, why not this time, I make the user uncomfortable? Of course, I can’t know the different things that make various people uncomfortable, so this interactive artwork + game is based on things that make me uncomfortable.

Basically, it’s a series of challenges that the user must get through in order to win. But each challenge is supposed to evoke some sort of uncomfortable emotion/feeling (hopefully) within the user. The design is such that there are 8 doors, behind each of which is a challenge that the user must complete. Once the user has clicked on a door, they are trapped in that room until they successfully complete the challenge. Only then are they given the option to return to the hallway and go through another door. The only colors I aim to use across this game is black, white, and red, as I think it throws people off and adds to the overall mood I’m going for.

Production & The most frightening part

There were two things I was most worried about. First was making the hallway. This is because I couldn’t find the right kind of doors online, and had to design my own using Canva, and then go through the complicated process of cutting out backgrounds, resizing, and figuring out the coordinates on the canvas for each door so that they align in a perfect one-point perspective. That said, I’m pretty happy with how the end result turned out.

Second, I had imagined that it would take a lot of work to implement the general algorithm for the game. That means the structure of dividing it into different rooms, coding a timed mini game for each room (while keeping everything in theme), and keeping the user trapped in a room until they solved it. To minimize this risk, I decided to divide the code into different game states, so that whenever I need to add the code for a new door, for example door 2, I can just add the required code and set the game state to room 2 and test it out. Also for elements like the popups and the timer, which get used for every door, I made them into reusable functions so that I can just call them for every door, only with different arguments (popup messages) based on what I want. Hopefully, for every door I can continue to have the format I have implemented now, which is an initRoom__ and drawRoom__ function for each door. Then it’s pretty much just following the same pattern 8 times.

Risks/Issues

My biggest problem right now is that I may have overestimated my abilities by choosing to do 8 rooms. I’m going to have to think of 8 different mini-game ideas that make people uncomfortable and implement it in the given time frame.

Next Steps

Currently I’ve only implemented 2 rooms. I need to implement the other 6, as well as decide on what to do for the end page of the game when the user finishes all the challenges. I also need to decide on a background soundtrack that will run for the entire game. I also haven’t been able to think of a name for this project yet, and that’s pretty important.

It’s much better to view the sketch in full-screen: https://editor.p5js.org/siyonagoel/full/eEPhmUoLw

Week 5 – Midterm Progress

Concept
I’m a huge football fan and also a huge supporter of Barcelona FC. I’ve always been fascinated by goalkeepers who do their best to save their team points. Those incredible moments of jumping their highest and catching the ball are something I would love to implement. I especially remember the incredible journey of Dibu Martínez (Argentina’s main goalkeeper) against France in the 2022 Qatar World Cup, where he saved two out of five crucial penalties, ultimately winning them the World Cup. So, I thought, why not step away a bit from my Dexter-inspired assignment and move into my other passions.

Highlight of the code

shoot(targetX, targetY) {
    this.totalSteps = random(20, 30); 

    let distX = targetX - this.x;
    let distY = targetY - this.y;

    this.dx = distX / this.totalSteps;
    this.dy = distY / this.totalSteps;

    this.isFlying = true;
    this.currentStep = 0;
}

The Ball class uses a simple, yet highly effective, trick to achieve smooth motion without needing complex physics calculations. For example, the shoot(targetX, targetY) method instead of calculating a constant velocity that needs to be constantly adjusted, the game calculates total movement distance and divides it by a random number of steps (this.totalSteps). Thus, giving the ball its speed.

Sketch

Uncertainties & Complexities
I didn’t use sounds for now and any kind of images or icons. So, I’m a bit concerned about that. Apart from that, I quite like the progress I’ve made, and I think I’m able to finish the project on time. But, if I do get to add those images and icons, and sound properly soon, I want to incorporate other essential stuff that would make the game a lot better. It took a while how do I ensure that the ball always hits the intended target, and I figured it could be easily done when the currentStep equals the totalSteps, so that’s how I keep track of whether ball should stop or not. Collision detection was a bit of struggle too, but I already did Intro to CS project, so I used as Professor suggested once to use circles to detect the collision by dist() between their radiuses.

Future improvements
I asked couple of my friends while I was in the library what do they think of the project, and some said that I could add different ball types like if you save a golden ball you get +3 points instead of +1, and if you don’t you lose -2 lives instead of -1. Also, I’m not quite sure how it works on p5js, but when we worked on Processing during my Intro to CS class, we were able to add CSV file to have a database of users, so we could keep track of best score across all users. Maybe, I will look into that and try to figure out how to add a database.

 

Week 5 – Reading Reflection

Reading Levin’s essay really changed how I think about what it means for a computer to “see.” One of the most striking points was that, unlike humans, a computer has no built-in way to understand what it is looking at. Levin writes that video is “just a big, dumb array of numbers”,  which really stood out to me, people automatically recognize faces, objects, and events, but a computer has to be taught step by step what to look for. This helped me appreciate how much of computer vision is really about extracting meaning from raw data rather than simply displaying images.

I also found it interesting that helping the computer “see” is as much about the physical environment as it is about the code. Levin mentions that careful control of lighting and contrast, even using infrared light or retroreflective materials, can make the difference between a system that works and one that fails. That made me realize that when we design with computer vision, we are not just writing software, but we are shaping the entire stage so the camera has the best chance to succeed.

Finally, the discussion about surveillance made me think about the double-edged nature of computer vision in art. Works like Standards and Double Standards or Sorting Daemon use tracking not just for interaction but to make us feel observed, even judged.  I then reflected on how computer vision can be playful but also unsettling  and how artists can deliberately use that feeling of being watched to make a political or emotional point. It reminded me that computer vision is not neutral, it carries cultural and ethical weight, which is part of what makes it powerful in interactive art.

Week 5 – Reading discussion

When I think about computer vision, what interests me most is how strange it feels to give a machine the ability to “see.” Human vision is so automatic and seamless that we don’t really think about it, but when you translate it into algorithms, you realize how fragile and mechanical that process is. I find it fascinating that a computer can pick up tiny details that our eyes might not notice, yet at the same time, it can completely miss the “big picture.” That makes me wonder whether computer vision is really about replicating human vision at all, or if it’s creating an entirely different way of perceiving the world.

What I find both exciting and unsettling is how computer vision plays with control. On one hand, it can feel magical when an artwork follows your movements, responds to your gestures, or acknowledges your presence (like in TeamLab). There’s an intimacy there, like the piece is aware of you in a way that a static painting could never be. On the other hand, I can’t help but think about surveillance every time I see a camera in an installation. Am I part of the artwork, or am I being monitored? That ambiguity is powerful, but it also puts a lot of responsibility on the artist to think about how they’re using the technology.

For me, the most interesting potential of computer vision in interactive art isn’t just the novelty of tracking people, but the chance to reflect on our relationship with being watched. In a world where surveillance cameras are everywhere, an artwork that uses computer vision almost automatically becomes a commentary on power and visibility, whether or not the artist intends it. I think that’s what makes the medium so rich: it’s not just about making art “see,” it’s about making us more aware of how we are seen.