Computer vision and human vision differ in interesting ways. While human vision is natural and intuitive, allowing us to recognize patterns and emotions effortlessly, computers need specific algorithms to make sense of images. For instance, what we instantly understand as motion or objects, computers detect through methods like frame differencing or background subtraction. I honestly find it intresting how rigid and task-specific computer vision is compared to our flexibility. Furthermore, to help computers “see” what we want, it uses techniques like brightness thresholding or background subtraction, and sometimes adjusts the physical environment by using better lighting or reflective markers.
Moreover, in interactive art, computer vision creates exciting new opportunities but also brings up ethical questions. For instance, Videoplace used computer vision to create playful, full-body interactions, while Standards and Double Standards used it to explore themes of authority and surveillance. However, a question that popped into my mind is that, when you consider the ability of these systems to track movements and gestures, do you feel like the line between creative interaction and surveillance can sometimes blur? This reminded me of the movie M3GAN, where AI uses computer vision to care for a child, but the surveillance becomes invasive. What if we might see something similar with interactive art or technology, where the systems that are meant to engage us could start to feel more like surveillance. Hence, it’s an interesting balance between enhancing the experience but also respecting privacy.
Category: F2024 – Aya
Assignment 5 – Midterm Progress
Concept
The concept for my midterm is simply my interest in arcade racing games and love for cars. I was inspired by car games such as Asphalt and Forza which I played for most of my childhood and with the knowledge from the class I felt compelled to try and implement some of these games in the best way I can. My objective is to come up with a visually engaging and simple game where a player navigates a car in a street road avoiding obstacles(oncoming traffic).
Other than that, I was eager to come up with something that will be 100% user engaging unlike my other projects and creating a game seemed to cater for this. I wanted to enhance user experience in my projects while also maintaining the good visual style which is appealing like the other projects.
Design
My design is centred around user interaction. It is a user engaging game where the player has to manouver the car across three lanes in all directions with the use of arrow keys and avoid obstacle cars. The background is a road, while the cars on canvas are specifically chosen to be sleek and easily distinguishable. As the player progresses, the game becomes increasingly difficult since the game speeds up gradually. I thought that this design will enhance the game clarity and ease of control, ensuring that the player quickly gets the mechanics but still feel challenged.
Frightening/Challenging Aspects
The most frightening parts while drafting and planning the project was how I will handle the collisions between the cars. This is the most crucial part for the game to function thus I can’t afford any errors in it.
Another challenge might be how I will balance the increasing speed of the game.
Risk Management
To counter this challenges, I did some research on how I can handle collisions and it turned out to be a really simple mathematical logic to compute. I was able to come up with an algorithm that calculates the distance between the player car and the obstacles and since the cars are rectangular it might be easy to handle.
Midterm Progress
Idea:
For my midterm, I originally wanted to create an interactive artwork because I assumed it was easier. After looking at some students’ previous pieces, I concluded that neither choice was easier than the other, so I settled for an interactive game that was a combination of some of my favorite childhood games, with a theme that I’ve noticed a lot in recent years. The games I thought of were: Jetpack Joyride, Subway Surfers, and Temple Run.
These endless runner games consist of a character that keeps running (or flying) while avoiding obstacles to reach the furthest distance possible. I don’t think I have the time or knowledge to create a completely endless one, so I’ll go for one that goes “You Win!” if the user can last around 1 minute 30. I decided to make the theme revolve around social media influencers that impulsively move to New York “just because it’s New York.” I’m thinking of naming it Escape from New York as word play on Marketplace’s Escape from New York Pizzeria.
Concept:
The concept is the user plays as an influencer who severely romanticizes New York and imagines it to be the way it’s portrayed in movies, only to move there and be faced with obstacles such as rats, trash, and crackheads. Their mission is to use the arrow keys to avoid driving her into the obstacles to get her out of New York.
Design:
Above is a brief idea of what I want the actual game to look like. The user will be able to move the arrow keys up and down to move her to the road without an obstacle.
So far, I’ve only worked on setting some functions and coming up with the look of the main menu page and the tutorial. I’m facing some formatting issues that need to be worked on such as the game background image’s scroll and the tutorial going off of the canvas.
Uncertainty:
Honestly, I’m uncertain about being able to properly finish the entire game with just a little over a week left because I feel like it is way too complex and out of my capabilities, so I’ll be needing a lot of help. I snooped around other students’s project codes and got intimidated by how long their codes are. I think I’ll get the hang of it the more I work on it, as my progress and understanding from today was better than that of when I worked on it 3 days ago.
I’m kind of unsure of how I’ll make the game look as if she’s driving and the obstacles are still, rather than the obstacles are moving and she’s still. I’m also uncertain of how to make sure there’s a road without an obstacle, so that she wouldn’t be trapped like in the example design. I also found myself spending a lot of time trying to find the right images to use in the game, and even the ones I found still aren’t the right ones for the final.
I think there’s a lot of planning out and learning to do beyond what I’ve already done, so I’m kind of worried on how fast I’ll have to pace myself, which is way out of my coding comfort zone. To reduce this risk, I’ll have to dedicate a good chunk of my time to it everyday, rather than only working on it the days after I have Intro to IM class. I’ve already spent a good chunk of time finding the images for the game, so now I get to focus on the code and implementation of these images. I’ll also look at and refer to p5.js tutorials, YouTube tutorials, other students’ previous works, and of course the professor’s help to learn more on how to bring this game to life.
Assignment 5 – Midterm Progress
Concept:
For my midterm I got the idea to recreate some of the arcade games, but the only difference would be that in this version we can easily access our favorite games on our devices. For my initial idea I decided to create 6 games and it would look somewhat like the image below.
This would be the starting screen. The way I want this to work is that if you press on one of the screens of the arcade machine, or if you press on the air hockey table the game corresponding with what you pressed would appear.
For the first game I decided to use a spritesheet of a skeleton and create a game from it. For my second game I decided to create a jukebox, where I will add different music and the player could change the song by pressing the arrows. For the third game I decided to make the machine not functional and that is because whenever I go to the arcade most of the machines are not working. For my fourth game I decided to add the moving squares a previous assignment of mine. And for my fifth game I decided to add a Jackpot slot machine. For my air hockey table I was thinking of not creating a two player game, but instead a one player, where the player just places the air hockey puck on the table and the puck moves around.
Code Progress:
The progress I have so far is the layout of my arcade room. I also started working on some of the games.
This is the skeleton game that I started working with. You have to press the spacebar to start the game, use the up, down, right, and left keys to move the skeleton and collect the bones, and when the points reach to 10 you have to press the “R” key to restart the game. What I want to add to this game is when the skeleton touches the sides of the screen you lose the game.
This is the error screen that I created for the broken machine. I used pixels to create this effect. What I want to add to this game is a text.
Challenges:
The challenges that I am facing with this project is how I will add all these games into one code and one game. And how will I create the interactivity if you press the screen of a game the selected game pops up.
Reading Response 4
Midterm Progress: Save Bran Stark!
Concept:
HBO’s Game of Thrones has countless iconic scenes that have been forever ingrained in the memory of 21st century pop culture, but arguably one of the most iconic and significant is the fall that crippled Bran Stark and set off the ripple of scandals that made the series’s plot as complicated as it is. While I obviously do not condone acts of violence against minors, I found the scene quite funny because of how absurd everything was: a 10 year old child who somehow managed to climb up the exterior of a tower accidentally witnesses an incestuous affair and then gets thrown out of a window. I figured that I would eternalize this scene in a more lighthearted way by creating a simple game in which multiple Brans will fall and the player, with their mouse, has to catch the boy by moving a pile of hay as a cushion for him to fall on. Each game will last 60 seconds, and players must save as many Brans as they can — and prevent the chaos that is about to spread throughout Westeros.
A rough sketch of what the game will look like.
Project Components:
- OOP: Bran Stark will be a class. Bran’s sprite and the parameters for his falling movement will be set in this class.
- Image: Using Procreate, I will illustrate a simple background image that depicts the tower from which Bran fell + grassy ground + pebbles. The sprite for Bran will also be illustrated by yours truly (in a desperate attempt to make up for amateur coding with good artistic direction).
- Sound: I plan to have the show’s opening theme, composed by Ramin Djawadi, play as soon as the game loads; it will stop when the timer runs out and restart along with the game.
- Text: The starting screen will show the game’s title in the same font as the show’s title card. The countdown and score will also be displayed.
- Shape: I will draw simple shapes to serve as buttons for starting/restarting the game.
Code Progress:
- So far, I have a rough sketch of red ellipses (placeholder for Bran) that fall at varying speeds and a yellow rectangle (placeholder for hay cushion) with a fixed y position that moves horizontally with the player’s mouse. Both of these are classes of their own.
- The Game of Thrones main title theme starts playing when the game loads.
- The checkCollision() function checks if a Bran has came into contact with the hay cushion; if so, the score increases by 1 and that specific Bran is removed while a new Bran spawns at the top.
-
let brans = []; let hay; let numberOfBran = 3; //number of Brans to be on screen at the same time let score = 0; let music; function preload() { music = loadSound("GOT Main Title.mp3"); } function setup() { createCanvas(600, 400); for (let i = 0; i < numberOfBran; i++) { brans[i] = new Bran(); } hay = new Hay(); music.play(); } function draw() { background("rgb(180,226,218)"); //display Bran and hay cushion hay.display(); for (let i = 0; i < brans.length; i++) { brans[i].display(); brans[i].move(); //increase score and reset Bran after collision if (checkCollision(brans[i], hay)) { score += 1; brans[i].reset(); } } fill(0); text("Score:" + score, 500, 80); } //check for collision between Bran and hay cushion function checkCollision(bran, hay) { let d = dist(bran.x, bran.y, mouseX + 35, hay.y + 15); return d < bran.diameter / 2; } class Bran { constructor() { this.x = random(50, 550); this.y = 0; this.speed = 3 + random(0, 7); this.diameter = 50; } display() { fill("rgb(216,78,78)"); ellipse(this.x, this.y, this.diameter); } move() { this.y += this.speed; //re-generate Brans at top of screen when Brans fall off screen at the bottom if (this.y - this.diameter / 2 > canvas.height) { this.reset(); } } reset() { this.x = random(50, 550); this.y = 0; } } class Hay { constructor() { this.x = mouseX; this.y = 330; } display() { fill("#FCD179"); rect(mouseX, this.y, 70, 30); } }
- Challenges:
- Setting a countdown for the game would be a challenge for me, as I have never tried incorporating one into my projects prior to this one.
- I will also need to figure out an algorithm that prevents the randomly generated positions of the falling Brans from overlapping.
MIDTERM PROGRESS
CONCEPT:
I couldn’t decide between a game or an artwork for my midterm project. However, I was playing on my phone, and there’s this one game that I still play to this day “Geometry Dash”. My siblings and I grew up around this game and we still love it to this day, and as the game design is basically shapes, I thought it would be the perfect game to try and re-create.
The main idea or goal is to control a character that moves through several/ repeating obstacles. The difficulty increases over time as the player’s score rises, with obstacles speeding up and appearing more frequently. There’s also a feature where the game randomly switches to an upside-down mode, adding unpredictability and complexity to keep the player more engaged.
Design:
So far, I haven’t really worked on the design but the layout of my game. I want to have a similar concept to the actual game, where the background is gradient, and changes colour that aligns to the beat of the background music and the obstacles. As for the obstacles, for now, I left them as simple shapes, rectangles, and spikes just to test everything out and see how it flows in the game. For the use of sound, I found online the original music used in Geometry Dash and implemented it in my game as well as adding a sound effect when the player dies. However, I still need to fix the background music so that when the player dies the song stops until he starts playing again, since I used the loop function it’s just playing over and over non-stop.
This is the inspiration for my design and how i would like it to turn out in the end.
User Interaction:
My user interactions are basically the player’s input in the game. The player must press the spacebar to jump. If the spacebar is held down, the player continues jumping until the key is released. As for my instructions and text, I’ve applied it in the beginning, so the game begins when the player presses the spacebar at the start screen. After a game is over, pressing “1” will restart the game. Moreover, I still need to work on the visual design of the Start page, as of now, I just left it as text. I’ve also added a score count which is displayed at the top of the screen, which increases by one as the player successfully passes an obstacle. In the game, the obstacles appear from the right side of the screen, and the player must jump to avoid them. Then the game randomly switches to an upside-down mode at higher scores, adding an extra challenge, but I still think I need to make it more complex and play around with the obstacles, as I fear the game might be too simple and boring the way it is now.
The Most Difficult Part of the Project:
The hardest part of making this game has been figuring out how to make the difficulty increase smoothly as I want the game to stay engaging throughout. I want the game to get harder as you play, but I also need to make sure it doesn’t become too hard too soon, to the point it just gets frustrating.
Collision Detection (When the Player Hits an Obstacle):
The other tricky part is making sure the game knows when the player hits an obstacle, especially the spikes. For the spike obstacles, the spikes are drawn as triangles, but I treated them as if they were inside an invisible rectangle (called a bounding box) that surrounds the spike. This makes it easier for the game to detect if the player hits the spike. Even though the spike is a triangle, the game checks if the player touches the rectangle around the spike. I used the collideRectRect() function in p5.js. This function checks if two rectangles touch each other. Even though the spike is a triangle, the game uses a rectangle around it for simpler collision detection. If the player’s rectangle overlaps with the spike’s rectangle, the game registers a hit. The same goes for the rectangle obstacles.
How I Made It Less Risky:
To make sure the game doesn’t get too hard too fast, I tested how quickly the obstacles speed up and how often they appear. By setting limits on both, I made sure that the game gradually gets harder, but not too difficult right away.
Code so far:
Reading Reflection #4: Computer Vision for Artists
As much as we may try, computer vision has yet to be able to emulate the human vision; what a computer “sees” are “critically dependent on certain unique assumptions about the real-world video scene it is expected to analyze” (Levin). And as we work towards optimizing computer vision — especially in terms of enhancing facial recognition and such — I do believe that regulations do need to be put in place so that these technologies are not appropriated by the wrong hands, with issues such as surveillance and the potential invasion of privacy being so widely debated. The Suicide Box referenced in the text feels especially macabre and ethically questionable, since the project relies on tracking the movement of unsuspecting, suicidal people who are at their most vulnerable moment. Conversations about the ethics of computer vision being implemented in such manners are difficult but necessary in this rapidly evolving technological landscape.
In terms of interactive media arts, however, I do believe that computer vision and tracking can greatly enhance immersive experiences. Once again speaking in terms of video games (because I really do love games), the eye-tracking mechanism of Skybound Games’s Before Your Eyes is an example of great computer vision implementation that came to my mind. The game is centered around the story of a boy who revisits memories of his life while preparing for the afterlife (this is a very rough synopsis, as to not spoil the game). The gameplay tracks the player’s eyes, and players blink to interact with in-game elements and advance the plot. The game’s lengthy but crucial cutscenes prove to be challenging for players to watch without blinking and accidentally skipping to the next scene. While this may seem like an unwise choice of gameplay on the developer’s part, I eventually realized how creative it was for a game dealing with themes of grief to utilize eye-tracking to capture the bittersweet feelings of wanting to hold on to fleeting moments in life but feeling yourself unintentionally letting them slip away. Before Your Eyes is proof that computer vision, when implemented creatively, also has the potential to enhance the emotional value of a project.
Reading Reflection – Week #5
Computer Vision for Artists and Designers: Pedagogic Tools and Techniques for Novice Programmers
It was interesting to find out that technological applications stem not only from their technical strength and abilities, but also from suitability of the physical space to which you try to apply computer vision. Therefore, now I can see how in interactive art there are two sided contributions between the artist and computer vision’s capacity that creates a final image that the audience sees.
As it was noted by Golan Levin in the article, computer vision creates broad opportunities for application and interaction with the real world. While it may seem that computer vision almost mimics human vision because of its surveillance practices, at its core there are very different methodologies applied to human and computer vision. While, human vision operates “generally”, analyzing and accepting any image that it sees in the surrounding environment, further generating the same level of processing, computer vision is highly dependent on what kind of image it receives and compatible it is to the vision’s capacity. Therefore, computer vision may generate unexpected processing outcomes with no consistency in the quality. Simply, the way we can help the computer to track what we’re interested in is contrasting the intended object from the background image and other unnecessary surroundings. Just like in Suicide Box project, where vertical motion was characterized as a contrasting event on the image, it was easier for a computer to identify what was tracking target.
In the socially meaningful projects, like Suicide Box, it is confusing to observe how computer vision gets just manually involved in the process of identifying vertical motion without directly understanding what kind of tragic event it gets to record. While, if we would task a specific person to perform the same job as the computer, and see how people commit suicide on the bridge, the person definitely would be left with the trauma, and it would be much harder to transform his knowledge into other forms. While, in this case, using computer vision, we are separating some subjective experiences from the process, and we are left with the intended outcome, which is easier to work on.
Reading response #4: computer vision
Computer vision has strong analytical capabilities, but only with proper programming. For example, in Limbotime (the example given by the web), the computer interprets the human body as black pixels, where the head is identified as the vertex of the pixel group. This expands the range of input compared to simpler systems like p5, where input is limited to the keyboard. It also highlights the key difference between computer vision and human vision. While computer vision relies on data and algorithms, human vision rely on context and experience, which is similar to computer vision’s logic but is much more subjective.
Optimizing physical conditions, like using reflective materials or controlled environments, can improve computer vision’s accuracy. But there are broader implications, especially regarding surveillance and privacy. I’m always surprised by how quickly we’ve adapted to technologies like Instagram filters. It’s now expected for the screen to show something when you interact with them. This also makes me wonder how computer vision might shape human vision, not just physically but in how we perceive ourselves. In terms of servailence, the privacy intrusion may not be as extreme as sci-fi films suggest, but it does subtly alter how we see and present ourselves.