Computer vision differs from human vision because computers do not naturally understand images and videos in the same flexible way that humans do. Instead, computers rely on specific algorithms and assumptions to identify useful information from pixels. For example, an algorithm may need movement, differences in brightness, or differences between an object and its background in order to detect something. Because of this, computer vision can be sensitive to changes in lighting, background, or whether an object is moving.

There are several techniques we can use to help computers see and track what we are interested in. Frame differencing can be used to detect movement by comparing two video frames, while background subtraction can separate people or objects from a known background. Brightness thresholding can also identify objects based on how light or dark they are, and brightness tracking can be used to follow a bright object such as a flashlight. We can also improve computer vision by designing the physical environment, such as using strong lighting, high contrast.

I think computer vision’s ability to track and surveil people can make interactive art more interesting and responsive, but it also raises concerns about privacy, consent, and how people’s information is used. For example, people may feel uncomfortable if a camera is tracking their movements or recording their images without them knowing. I think artists can address these ethical issues by clearly informing participants about what is being collected, how the information will be used, and whether it will be stored or shared. They should also give people the choice to participate or not whenever possible. At the same time, artists can use surveillance itself as a theme to make people think about how technology is used to monitor and understand people. Therefore, I think computer vision can be very powerful in interactive art, but it should be used in a way that respects people’s privacy and gives them a clear understanding of what is happening.

Computer vision and human vision both involve looking at and analyzing objects, but how they execute that task is different. Humans mainly use their eyes and brain to perceive the image that is coming in while computers tend to have a more complicated process. Usually images are fed into a software which analyzes each numerical pixel and groups them by their attributes to recognize the image.

Some of the ways we can help a computer ‘see’ better is by using methods such as brightness detection. This helps a computer understand the boundaries of an object by measuring the brightness of each pixel and determining the shape of the object.

In my opinion, I believe that computer vision adds a whole new level of creativity and immersion to interactive art by opening up new possiblities for artists to immerse their audiences in an experience. I think that this is a positive outcome as nowadays artists can run wild with their ideas and not be limited by the laws of our reality.

Levin talks less about teaching computers to see our world and more about teaching humans to make the world easy for computers to read. I believe he adopted this approach because he wanted to demystify computer vision for novice programmers. And, to be honest, it worked! When I first thought of computer vision, it intimidated me. By acknowledging that “the discipline continues to confront numerous unsolved (and perhaps unsolvable) challenges” (1), while still breaking down the algorithms, he made an intimidating field feel slightly more approachable. Levin made me understand that a computer cannot tell whether a scene contains a person or whether it is day or night. Frame differencing, background subtraction, and brightness thresholding (the methods he describes) do not understand, per se. Instead, they compare pixels against a previous frame, a stored background, or a threshold. By being honest about how little these algorithms actually ‘know’, he makes them feel within reach, at least for me. For instance, in his Limbotime story, the students’ breakthrough came from finding a sheet of white foam core in a closet, rather than from them being programming-savvy. That detail stuck with me because it suggests you do not need to be an expert to build something that works in computer vision.

However, this also complicates this technology’s ethical use. Levin showed us how much the same simple tools can mean depending on who uses them. While he admits that computer vision was long limited to military and law-enforcement purposes, he emphasizes artworks that use that same technology in completely different ways. Rokeby’s ‘Sorting Daemon’, for example, turns automated profiling into a critique of profiling. Jeremijenko’s ‘Suicide Box’, on the other hand, uses motion detection to count something that, in her words, “was not being counted – that is, doesn’t count” (6). Levin demonstrates that these works range from the highly formal and abstract to the humorous and sociopolitical. I believe that this range exists because the techniques are somewhat ‘simple’, allowing almost anyone to use them. This is where my feeling of it being approachable got a little complicated. If tracking a person is mostly counting pixels, then the difference between an art installation and a surveillance system is largely the intention behind it. So the same accessibility that makes computer vision welcoming to beginners is also what makes it easy to misuse.

 

It is really interesting to think about how computers see completely differenly than how humans do. Like, I habe found it always so interesting that what seems so easy to say or see for a human the computer just work differently and I have to basically think as if I was a computer too. As I was reading, I realised how much more complex a human mind is by default, such as having spatial awareness, memory or even reasoning, while a computer is only able to possess some of these qualities if it was programmed to do so. As the author highlights, it is so interesting to think about that computers need to have context, need to have something that is clearly distinguishable between for example objects and humans, because fundamentally they see both as the same if it is not specified, while humans just know basically, that a sofa and a person are not the same.

The author mentioned a few ways that it could be made easier for a computer to distinguish, track or see what we are interested in, such as using infrared, brightness or frame differencing. I am a film major, and a lot of what he said about setting the scene, making sure that the thing we want to track stands out from the background and all that just felt so relatable in a way. When we think about set design in films, we also look at the camera, how it see it because everything looks SO different on a camera than in real life, so we have to make sure that when we set up lighting it highlights the actress/actor or whatever we want to film, or that they are wearing clearly outstanding clothes. I know that this is not the same as trying to get  a computer program to differenciate between things, but I felt like it was not too different either. And if we think about it, it makes so much sense, the computer doesn’t have conscience, doesn’t have a context about the world, and it needs guiding, maybe like a little child. I find the solution with the usage of the infrared the most exciting. I am not entirely sure how it works, but I feel like that technology would give us the most accurate results.

This usage of computers is further interesting, because it is able to transform compley human life into quantifiable data and thus interactive installations. I feel like for each different use of computer tracking it has a different level of capacitz, but findamentally it is definitely an important part. For example with Standards and Double Standards it is really important, in order to translate the meaning of the artwork correctly; if the computer is not able to track the movements correctly I feel like the whole message gets lost. I feel the same with Suicide Box although that is a much deeper sociological installation. With this latter example, the capacity to track is also important because it is able to expose basically “invisible” realities, that would not be made public otherwise. Just like the true number of suicides happening on the Golden Gate bridge.

For this week’s project, I used arrays and the random() function to create a Resident Evil-inspired quote generator. The program pulls quotes from different characters stored in the array and randomly displays them as text on the canvas.

A part that I am proud of in this program is my array, which stores the quotes. I used objects as well to connect each character to their quote so the program wouldn’t randomly display the wrong character and quote.

// Array with random quotes
let quotes = [
{
character: “Leon”,
quote: “Where’s everyone going? Bingo?”,
},
{
character: “Leon”,
quote: “No thanks, bro!”,
},
{
character: “Jill”,
quote: “You want S.T.A.R.S.? I’ll give you S.T.A.R.S.”,
},
{
character: “Claire”,
quote: “Hey, it’s up to us to take out Umbrella.”,
},
{
character: “Wesker”,
quote: “7 minutes. 7 minutes is all I can spare to play with you.”,
},
{
character: “Jake Baker”,
quote: “Welcome to the family, son.”,
},
];

I created this by using an array to store my quotes and using objects to link each quote to its respective character. I then used the random() function to select an object when the user clicks anywhere on the canvas. The selected quote and character are stored in a variable and displayed using the text() function. I also used different text styles and colors to make it more visually appealing.

One thing that I would like to improve on in my program is using different fonts than the ones already stored in p5js. I feel that would make it more recognizable. I would also like to experiment more with the background and maybe add some images too.

Concept

For this assignment, I created an interactive artwork called Don’t Notify Me. It shows a self-portrait of me trying to work on my laptop while a stream of notifications keeps breaking my concentration. Participants can write their own notification, send a blank one, reset the scene, or tap the lamp to switch on focus mode.

A normal notification takes about two seconds to pull my attention away from the laptop. Strangely, a blank notification gets an even faster reaction. Since it gives me no information, I immediately want to know what it means. One or two notifications do not bother me much, but my mood changes as they pile up. After three pings in a row, I start to look annoyed. At five, I lose my patience completely. My expression changes, the scene turns red, my replies become more direct, and the sounds get harsher.

I wanted the experience to be playful, but also familiar to anyone who has struggled to focus while notifications keep appearing. The participant is not just watching me become frustrated. They are the one causing it. Their timing and messages affect my expressions, responses, and surroundings, creating a small back-and-forth between the participant and the artwork.

Code highlight

One part I am especially proud of is the way the artwork uses CSV files to manage its timing and dialogue:

settingsTable = await loadTable("settings.csv", ",", "header");
responsesTable = await loadTable("responses.csv", ",", "header");

currentPing = {
  message,
  createdAt: now,
  reactionAt: now + (
    message ? config.normal_delay : config.empty_delay
  )
};

settings.csv contains values such as the reaction delays, streak window, and mood thresholds. responses.csv contains the character’s possible replies to calm, annoyed, furious, urgent, and empty notifications. Keeping this information outside the main sketch makes the project much easier to adjust. I can change the timing or rewrite the character’s responses without digging through all the drawing code.

The character’s mood depends on the current notification streak:

const score = Math.max(
  0,
  streak - (theme === "dark" ? config.focus_relief : 0)
);

if (score >= config.furious_at) return "furious";
if (score >= config.annoyed_at) return "annoyed";
return "calm";

Focus mode lowers the effective streak instead of erasing it. Tapping the lamp can help the character calm down, but it does not act as if the earlier interruptions never happened.

How this was made

I adapted an earlier p5.js self-portrait and turned it into a scene of me working at a table. The laptop screen faces the character, so the audience sees the back of it. While I am working, my eyes point down toward the screen. Once the reaction delay ends, my eyes move toward the notification and a speech bubble appears.

The controls beneath the canvas use a real HTML text field and buttons. Participants can type a message and press Enter, click Send Ping, send a deliberately blank notification, or reset the scene. Each notification appears as a card inside the artwork, along with a timer showing how long it is taking to catch my attention.

The program also tracks how quickly the notifications arrive. Pings sent close together build a streak, while leaving me alone for long enough resets it to zero. This makes the interaction feel less like pressing a button and more like testing someone’s patience.

1. What’s something (not mentioned in the reading) that drives you crazy and how could it be improved?

Something that drives me crazy is using sinks in public toilets because I can never tell if they are motion activated or if I’m supposed to press something down to turn the water on. Sometimes I put my hands under the sink waiting for the water to come out and nothing happens, so then I realize I have to press the top down. I think they could be improved by making it more obvious how the sink works, like having a small symbol or indicator showing whether you need to use the sensor or press the button.

It is such a simple thing, but I always get confused by it but i think its important to note that now after covid-19 almost everything is motion activated which i find super interesting. But even then there are issues with where to put your hands and if the sensor is broken i have to just keep wiggling my hands under it making it an awkward moment, then the person next to you says “That one doesn’t work”.

2. How can you apply some of the author’s principles of design to interactive media?

I can apply Norman’s principles of design to interactive media by making things easier for people to understand and use without having to guess what they are supposed to do. For example, if I make a button on a website, it should actually look like something that can be clicked instead of just looking like normal text. I can also use feedback, like changing the color of a button or showing something on the screen after it is clicked, so the user knows that their action worked.

I think this is similar to the public sink example because when I cannot tell if I need to use a sensor or press something, the design is not making the interaction clear enough. In interactive media, I want to make sure that users can understand what they can interact with, what will happen when they interact with it, and whether their action actually worked. This would make my projects feel less confusing and more natural to use.

Concept:

For this assignment, I had absolutely no idea what to do. I tried looking around for inspiration when the idea of incorporating song lyrics came to mind because I listened to music all the time. Once I had the idea of using song lyrics, I decided I could make the project more personal and specifically make use of song lyrics that I really enjoy. When figuring out how to design the project, a few references came to mind: the first being the lyric videos I sometimes see on TikTok (Image 1), the second being the Instagram reels that just have text on a color background (Image 2), and the third being the shared lyrics from Spotify (Image 3).

 

 

How this was made:

For this assignment, I wanted to try and use a csv file to store the lyrics so I began by reviewing a video I found titled “p5 js Working with Basic Data + CSV Files” by Jacob Rivkin and the p5 references on csv, in order to get a better understanding of it works. Originally, I planned to only add the lyrics, but once I watched the video and followed along, I decided to add additional information to the CSV file, such as the song title and artists. Once I was able to get the connection to the CSV file working, I added the mouseClicked function so that users can go between the song lyrics. 

After the functionality of the assignment was done, I began working on the visualization of the text. I started with the position of the lyrics, song name, and artist, trying to find a font size and position that would keep each section of text from overlapping because at first when I added alternative fonts some of the longer lyric segments would go a bit off screen due to the font size. In terms of other visual aspects, I tried to keep it a bit simplistic, in a manner similar to the references I had while still having some kind of unexectedness to it, which is where the changing fonts, and text and background color come in. 

Code I’m Proud Of:

async function setup() {
  createCanvas(500, 500);
  
  data= await loadTable('lyrics.csv',',','header' );

  numRows = data.getRowCount();
  songLyrics = data.getColumn('Lyrics');
  songName = data.getColumn('Song');
  songArtists = data.getColumn('Artists');

When I started working on the assignment, I was unsure about how to navigate the CSV files that I almost ended up just using an array to store the lyrics, and at that point I wasn’t even planning to add the other song information. When I found the YouTube video explaining the concept, I found it much easier than I initially thought and was able to incorporate it into my assignment.

Reflection:

Even though I think this is one of the more simple assignments, I think the fact I got to incorporate something I love into it, that being music I personally enjoy, I had more fun throughout the entire process. For future improvements, I would want to add a feature for users to add their own lyrics to be displayed, which I could use to add a more personal touch for the person experiencing the work. Additionally, one thing I wanted to add but wasn’t able to was a gradient-style background, much like the Instagram video reference. As we continue going through the semester, I’d like to come back to this assignment and hopefully add those improvements. 

• What’s something (not mentioned in the reading) that drives you crazy and how could it be improved?

One thing not mentioned in the reading that could be improved is the car dashboard. Nowadays, many car manufacturers are leaning towards a fully digital car dashboard, moving away from physical control switches, nobs and buttons. I’d like to call this the ‘iPadification’ of cars, a term that may or may not have been used somewhere before, as the dashboards of many modern cars have just become a giant iPad. Many people may think it’s just easier to have everything digitized, but I think when it comes to driving that may not always be a good thing. Focus is very important when you’re on the road, and having the physical buttons and nobs made it easier to tell if you were able to do what you wanted, such as changing the temperature. Additionally, the static nature of physical buttons made it easier to remember where certain features were, allowing drivers to build muscle memory, meaning less focus would need to be given to those tasks. Meanwhile, with a touchscreen, it may be more distracting for drivers, as there are many menus that need to be navigated for different controls, and the overall process may be longer, leading to the driver having to multitask for longer, increasing the chance of them losing focus on the road.

• How can you apply some of the author’s principles of design to interactive media?

When applying these principles to Interactive Media, since the focus is interaction, I think keeping human-centered design at the forefront is really important. Putting a priority on clearly indicating to users the intention, the means of interaction a project could have, and some kind of indication that something happened represents the affordances, signifiers, and feedback principles, which, for interactive media, may be some of the most important. If a project fails to communicate how it works, what it does, or that something happened, people would likely lose interest in interacting with said project. User attention is also very limited, so for an interactive media project, whether it’s a website, installation, or game, if users are unable to figure out how to use it or if it’s actually working, they will naturally move on, especially if there is another project a few steps away capturing their attention because it has some kind of audio or visual response to the user’s interaction.