Category: S2025 – Shiloh

Concept/Design

For my midterm, I wanted to create a escape room type of game, where the players have to like find objects around the room to be able to escape. In terms of design, I want to structure it so that the players must find five objects for example within a room and answer a series of questions to unlock the next clue. So, the game might begin with an introductory hint that guides the player to find the first object. Once they find it, they are either provided with another hint or asked a question. Answering the question correctly would then lead them to the next clue or object, however anwering it wrong may either let them retry or offer an extra hint. As the player progresses, they will uncover more objects, until they have all the objects needed to escape. I will probably have the game feature different rooms for the player to choose from, and each room will present a unique set of clues, objects, and questions to solve.

Challenging Code/Uncertainty

For this project I think the most complex part of the it is like an ordered clicking type of mechanism. For this game specifically, I think this is needed because I want the players to interact with objects in a specific order,  whether it’s pressing buttons or flipping swithces , I want them to do it in a specific order. By adding this system in my code, where objects must be clicked in the correct order, it will make sure that players engage with the puzzle thoughtfully rather than just clicking randomly and finding the key the first try.

To minimize this uncertainty, I tried to write the code to keep track of which shapes the player has clicked by using variables like rectClicked, triClicked, and circClicked. These start as false, meaning the player hasn’t clicked them yet. Then, in the mousePressed() function, I set rules so that the shapes can only be clicked in the correct order. For example, the player has to click the rectangle first, then the triangle, and finally the circle. If the player tries to click them out of order, the game won’t move forward. This helps make sure the game flow stays smooth and clear for the player. below is this code:

function mousePressed() {
  if (!rectClicked && mouseX > 50 && mouseX < 130 && mouseY > 100 && mouseY < 150) { //first rectangle is clicked
    rectClicked = true;
  } else if (rectClicked && !triClicked && mouseX > 170 && mouseX < 230 && mouseY > 100 && mouseY < 150) { //triangle clicked true  if rectangle clicked first
    triClicked = true;
  } else if (rectClicked && triClicked && !circClicked && dist(mouseX, mouseY, 320, 125) < 25) {//circle clicked true if rectangle and triangle clicked before
    circClicked = true;
    escape = true; //clicking circle = players escapes
  }
}

Human vision is intuitive whereas computer vision relies on algorithms, pattern recognition, and controlled environments for said algorithms to work. Computer vision processes images pixel by pixel whereas human vision takes in an image as a whole. For example, to detect movement, some computer vision algorithms detect the value different between individual pixels next to each other. However, human pattern recognition looks at the overall shape of an object. We can also better handle seeing things in different contexts. However, computers need to specifically be told to watch out for slight variations. With the rise of machine learning and neural networks that help computer vision with pattern recognition, however, it’ll get easier for computers to detect objects or movement.

Some techniques for tracking include detecting motion, detecting presence, and detection through brightness thresholding. Detecting motion means comparing pixel brightness per frame. If the brightness of a pixel changes a lot from one from to another, that can indicate movement. The total differences experienced by different pixels can give a broad picture of total movement. Detecting presence means comparing an with a preset background to see what’s different (background subtraction). Detection through brightness threshold uses a general brightness level and compares pixels to it to determine whether or not the pixel is in the foreground or background.

Tracking through computer vision has a lot of potential in interactive art. Being able to detect a user’s movement means it can influence the direction of the art. Even with musical performances for example, an artist can do a specific movement to trigger a specific sound or effect. Raves or concerts which are both audio and visual based could have a lot of potential using dance movements to aid musical production. More advanced tracking has also allowed for the detection of subtle expressions and glances, which gives artists even more to work with as they can use a user’s emotion to guide an art piece.

Project Concept

My midterm project is based on a concept I wrote in The Gazelle a few years ago: Drowning At NYUAD: Finding Space In A Ceaseless Work Culture. The article discusses the at times unhealthy work and hustle culture of NYUAD students, where a lack of sleep, overloading classes, and extracurricular commitments can be used as a metric of how “successful,” you are.

The midterm game aims to exaggerate the absurdity of it–underscoring the need to find a personal work-life balanace that enables you to enjoy (and not just survive) your undergraduate career.

Design

The game is designed to parallel mobile games, where a character is dragged along the bottom of the screen–launching projectiles towards oncoming enemies. Except, instead of enemies, the character will be defeating NYUAD-specific monsters, like SIG Training, homework all-nighter, readings, etc.

Prior to playing the game, the user will be presented with several screens that allow for customization. Specifically, the user will need to select a character and an academic weapon from a list of three options each. Below includes screenshots of the game storyboarding, designed in Figma.

Key Risks

One of the aspects that I am most concerned about is the number of screens involved in the game. To alleviate the potential issues and minimize complexity, I am going to work to encapsulate each screen as a class object, with a .hide() and .show() method. This way, the code may stay maintainable and navigable. I may also opt to include a configuration dataclass object, to pass to each scene as a reference, in order to avoid defining many global configuration variables.

Concept

I was thinking for a while of a good midterm project that would combine everything we have been taught so far and would be also fun to make and even more fun to play/experience. I came up with an arcade style game that combines the fast paced racing games I used to play as a kid to the fun and popular endless runners like “Subway Surfers”. I am making a game where the player controls car movement left and right to avoid oncoming traffic and collect coins. The more the game progresses and the more coins the user collects, the game becomes faster and therefor harder making it a fun challenge for everyone.

User interactions

• Arrow keys to move the car left and right, allowing for quick and responsive controls
• Randomly positioned traffic that appears on the screen, requiring strategic movement to avoid collision
• Randomly appearing coins that the player collects to increase their score, encouraging risk-taking and precise movement
• A progressively challenging difficulty curve where traffic increases in speed and density over time
• Game over state when the player collides with traffic, prompting a restart option to try again and improve their score

Code design

I have structured my code using object oriented programming with the following classes:

Car – Represents the player’s car and handles movement

Traffic – Represents the incoming traffic and resets to random position when it moves off screen

Coins – Represents collectable coins that appear at random positions when collected

Additionally the game includes:

Score system

Collision detection system

Car class:

class Car {
    constructor(x, y, w, h) {
        this.x = x;
        this.y = y;
        this.w = w;
        this.h = h;
    }
    display() {
        fill(0, 0, 255);
        rect(this.x, this.y, this.w, this.h);
    }
    move() {
        if (keyCode === LEFT_ARROW && this.x > 100) {
            this.x -= 50;
        } else if (keyCode === RIGHT_ARROW && this.x < 400) {
            this.x += 50;
        }
    }
}

Collision detection:

if (car.x < traffic.x + traffic.w &&
    car.x + car.w > traffic.x &&
    car.y < traffic.y + traffic.h &&
    car.y + car.h > traffic.y) {
    console.log("Collision with traffic!");
    noLoop();
}

Currently I have the car and the traffic as simple p5 box objects while I set everything up, but will change them to images as I work on the project.

Challenges and Risks

The most complex part of the project so far was implementing collision detection and ensuring objects reset properly. This was essential for making the game playable and preventing unfair conditions where the player could not avoid obstacles. I have dealt with this challenge already, but I am thinking of implementing a power up system into the game which might bring a completely new set of challenges with it.

Next steps

While the core mechanics have already been implemented there is still work to be done such as:

Add game sound

Improve the visuals

Add high score tracking

Possible implementation of power up system

 

So far this project has been a great learning experience, I am looking forward to work on it even more and refine it further!

I decided to make an experience based on a flower shop. Users can go into the shop, select flowers they like, make a bouquet, and buy flowers. I also want to implement a money system such that you need to purchase everything within the budget, and if you go over the budget, you can’t buy the bouquet. Right now, I’ve implemented being able to walk in and out of the store using a door (enter) and rug (exit), being able to select the flower stand and select flowers you like from a menu into your cart, and going to the checkout. I want to draw all the images, backgrounds, and items in this experience, but right now, I just simple stand-in pictures I drew. I’ll replace these shabby ones in the future, but I just wanted to get the mechanisms of my system working before beautifying it.

Jotting  down  ideas  before  starting:Lots of things ended up changing. I originally wanted to make everything pixel art too, but I think I’m gonna stick with more a freeform style.

Fear:

I had a lot of difficulty displaying and un-displaying different elements in the project. For example, I have a hover effect I’m proud of as it helps indicate what’s clickable to users. However, that means turning things on and off in terms of displaying them, which is a lot more complicated to implement than it intuitively feels to use. I knew I was gonna spend a lot of time figuring out how these mechanisms were gonna work, so I wanted to start early on these instead of the visual sophistication of the project. That’s why I created really simple sketches and blobs to fill in the final drawings.

I’m also not sure if I want to implement a budget; the minimum viable product should be being able to make a bouquet with flowers you chose without worrying about the money, but it’d be nice to have. Having these milestones is important because then you feel less overwhelmed. Each tiny detail can take a long time to implement, so it can feel very daunting. However, I can steadily make progress through these milestones.

Computer Vision

The articles juxstaposing of computer vision with human vision was very interesting. While some aspects are similar, such as a common use of types of lenses for both recording video and through retinas, the image processing appears to be where differences begin to proliferate.

An interesting point I found the article made was that videos are inherently recorded to store pixel information, but not necessarily scene information. For instance, a night sky is recorded as lots of black pixels–rather than some encoding of a night sky parameter enabled. This fundamental concept means that complex algorithms must be constructed to reconstruct and interpolate the scene information from pixel values. Furthermore, there are still many video encoding formats (e.g., H.264, H.265), so standardization is further lacking in this regard–introducing additional complexity to the process.

One of the techniques I found intriguing is the background subtraction technique, where an initial reference image of the set is first captured. Then, the reference is used to systematically distinguish which objects belong to the scene, and which do not.

The surveillance art, which monitored the Golden Gate Bridge, sparked considerable reflection. I found the author’s point particuarly pointed, when it was revealed that the art had captured a considerably higher number of suicides than what was noticed through traditional systems. However, I can also see how recording these events is also uniquely invasive to the subjects, who are likely unaware that they have become part of an art piece–and did not sign up to be so. This work was only made possible through computer vision.

Computer vision algorithms are not general-purpose and rely on specific assumptions about the video input, while human vision is adaptable and can interpret a wide range of visual stimuli. Unlike humans, computers cannot inherently understand or extract semantic information from visual data without additional programming. Additionally, computer vision systems may struggle with ambiguous or poorly defined scenes, whereas humans can often infer meaning from context.

Techniques to enhance computer vision include controlled illumination to improve contrast, using brightness thresholding to distinguish objects based on their brightness, and employing background subtraction to isolate moving objects. Additionally, surface treatments like high-contrast paints can make objects more detectable. Simple object tracking algorithms can also be implemented to follow specific features, such as the brightest pixel in a video frame.

Computer vision’s tracking and surveillance capabilities enable interactive art to engage audiences by responding to their movements and expressions, creating immersive experiences. However, this capacity raises ethical concerns regarding privacy and consent, as participants may be unaware of being monitored. Artists must balance the innovative use of tracking technology with responsible practices to ensure a respectful and enjoyable interaction.

How Computer Vision is Different from Human Vision

Computer vision and human vision work in very different ways. People can see and understand images quickly because our brains recognize patterns, shapes, and objects naturally. We also understand depth, colors, and movement easily. Computers, however, need special instructions to “see” things. They analyze images by looking at tiny dots called pixels and using math to find patterns. Unlike humans, computers have trouble with things like bad lighting, unclear backgrounds, or objects that look similar. This means computer vision needs special tricks to work well in different situations.

How We Help Computers See and Track Objects

Since computers do not see like humans, we use special techniques to help them track objects. One method is frame differencing, where the computer looks at two pictures and spots the differences to detect movement. Background subtraction helps the computer ignore things that do not change and focus only on moving objects. Another technique is brightness thresholding, where the computer finds objects based on how bright or dark they are. More advanced methods use color tracking or infrared cameras to improve accuracy. These tools help computers find and follow objects better, even in complex scenes.

How Computer Vision Affects Interactive Art and Surveillance

Computer vision is used in art to make interactive experiences where people can control things with their movements. For example, in Myron Krueger’s Videoplace, people’s shadows on a screen could interact with digital images. Artists also use computer vision to explore serious topics like surveillance. In Sorting Daemon, David Rokeby used it to study how cameras track people in public places. Another example, Suicide Box, recorded people jumping off a bridge to show how some events are ignored by society. While computer vision makes art more exciting, it also raises questions about privacy and how technology watches people without their permission.

My Concept:

For my midterm project, im attempting to bring Bikini Bottom to life through an engaging and nostalgic digital experience. The project features the three iconic homes from SpongeBob SquarePants, SpongeBob’s pineapple house, Squidward’s stone home, and Patrick’s simple rock. The project is a tribute to childhood memories, capturing the essence of the beloved cartoon that many grew up watching. I plan to make it more interactive by making users solve different problems using hints, enabling them to interact with different objects.

Design:

For my design I hope to closely mirror the original aesthetic of SpongeBob SquarePants, using vibrant colors and playful elements to capture the cartoon’s distinct underwater atmosphere. The houses are placed in a scene that resembles Bikini Bottom’s sandy ocean floor, with blue gradients representing the ocean background as shown below: 

The interactive element plays a crucial role in this project. As the user moves their mouse over each house, the door lights up to indicate interactivity. When a user clicks on the door, an passcode input box pops up, where the user needs to type in the correct code to unlock it (by using hints). When a user interacts with the door, they enter the house where, depending on the house chosen, will have a different theme and interactive objects that match the vibes/themes of the character. This will also be done by changing the genre of music inside each house.

Challenges:

One of the main challenges was making the opening door action simple and easy to use. Instead of just clicking the door, I had to add an input field that only appears when needed. Managing this input box so it doesn’t always stay on the screen was tricky. Another challenge was handling incorrect passcodes. I made the program display “Incorrect” on the screen. This required making sure the message only appears when needed and resets when the user tries again.

Finally, I had to ensure that when the correct passcode is entered, the screen updates immediately. This meant making sure the input box and button disappeared at the right time while smoothly transitioning to the next screen.

let doorX, doorY, doorWidth, doorHeight;
let isUnlocked = false;
let correctPasscode = "1234"; //the passcode
let userInput = "";
let message = "";

function setup() {
  createCanvas(300, 200);
  
//door position
  doorX = width / 2 - 40;
  doorY = height / 2;
  doorWidth = 80;
  doorHeight = 120;

//input box
  inputBox = createInput('');
  inputBox.position(width / 2 - 50, height / 2 + 50);
  inputBox.hide(); //hidden at the start

//create submit button
  submitButton = createButton('Enter');
  submitButton.position(width / 2 + 60, height / 2 + 50);
  submitButton.mousePressed(checkPasscode);
  submitButton.hide(); //initially hidden
}

At the moment the dimensions for clicking the door is not perfect at all, but it’s just to give an idea. (click at the bottom of the door)

passcode is “1234”

Concept

For my midterm project, I wanted to create something that is 1) funny and 2) inspired by our campus. I thought hard and long, and eventually came up with an idea to create a game based on an “incident” that happened to me at the Palms. I love sitting under the palm trees outside C2 when the weather is nice, and one day I was joyfully sitting on one of the tables doing work on my laptop when suddenly I heard a quick popping sound and felt something splatter in front of me – a bird had pooped on my laptop.

At a loss for words as to why, out of all the open space around me, it chose to land right on me, I questioned my luck and intensely cleaned around. Ever since then, whenever I go to the palms, I am reminded of that day, laugh at how “lucky” I was, and glance up from time to time to check for birds above me.

Design

The idea of the game is to dodge bird 💩 falling from the sky while at the same time trying to finish your homework. The user needs to protect the computer using the left and right arrow keys without getting hit from what is falling from above. To complete the homework, the user needs to repeatedly press the space bar until the progress bar fills up. If the user stops pressing the space, the space bar will decrease. The goal is to finish the homework as fast as possible without getting hit. The shortest time will be recorded for the user to see.

Challenging Parts

• Creating the progress bar and incorporating decay so that the progress decreases upon inactivity
• Checking for collision between the computer and bird poop
• Making bird poop fall randomly from the sky (from random columns at random times with perhaps random speeds)
• Showing time elapsed and keeping track of minimum time
• Creating a visually appealing game interface

I tried creating a simple progress bar that fills up on the pressing of the space bar. It works by drawing a rectangle where the width is based on the progress variable, and this variable is incremented when the space bar is pressed and decremented slightly every time to add to the challenge.