Author: Izza Tirmizi

The main difference between computer and human vision is that while humans can instantly recognize faces, objects, and contexts, a computer sees only raw pixel data unless explicitly programmed to interpret it. We have the ability to recognize people, expressions, colors, animals, etc instantly that computers have to be taught how to do and even that to only some extent. To help computers “see” or track objects of interest, techniques such as frame differencing, background subtraction, and brightness thresholding are used to extract meaningful features from video input. These methods allow interactive systems to detect movement, isolate subjects, and respond dynamically to user actions, forming the foundation of many interactive media works. However, each of these techniques has limitations, such as sensitivity to lighting changes or reliance on high-contrast visuals, highlighting the need for carefully designed environments to optimize detection accuracy.

Still, computer vision has improved vastly over the past few decades and has especially become much more accessible for artists, designers, and even beginner programmers to incorporate vision-based interactivity into their work. Software environments like Processing, Max/MSP/Jitter, and Macromedia Director provide frameworks for integrating computer vision, either through direct coding or via plug-ins and toolkits. Additionally, stand-alone applications like BigEye and EyesWeb expand these capabilities, enabling real-time motion tracking and expressive gesture analysis that is much easier to get your hands on.

In interactive art, computer vision’s capacity for tracking and surveillance introduces both creative potential and ethical concerns. Artists can leverage motion detection and object tracking to create immersive, responsive installations that engage audiences much more than just standing there staring. However, the same tools that enable interactivity can also invoke concerns about surveillance, privacy, and data collection. Since interactive art often explores the relationship between technology and human experience, the ability of computer vision to monitor and track movement, especially without explicit user consent, raises questions about autonomy and control. What information is the computer storing? Can it be used against me? These are questions people who are being recorded don’t even have the chance to ask. Ultimately, while computer vision expands the possibilities for dynamic and participatory art, its application must be carefully considered to balance both innovation and creativity with ethical responsibility.

For this project, I was inspired by the windows at the Louvre Abu Dhabi that have quotes on them. I’ve always thought that poetry is incredibly beautiful powerful, and something that could have a section in a museum to inspire those that want to read it. Therefore, for my project, I decided to create what I call the “The Museum of Poems”. The concept involves a museum scene with a frame in the very center that displays a random quote about love or life for the person to see. My hope is that whatever they read, although short, inspires them or motivates them in some way. I gathered 2-line poems from various reddit posts, online blogs, quora posts, and even ChatGPT, and put them all into a giant list for the program to pick and display at random. My museum ended up looking like this:

The hardest part of this project was that after I initially finished, I accidentally clicked refresh and lost the entire project. I thought I had saved it, but turns out I hadn’t, and there was no way to recover it. Luckily, there was a point where I asked ChatGPT to help me solve a problem I was running into and gave it all my code, so I could copy that back. However, this was still in the earlier stage of my project, and I had to redo a lot of it from scratch including importing all my fonts and images again, putting in all the poems in the list, the gradient background, the wood flooring, etc. Speaking of the gradient background, that part along with the wood flooring is the part of the code I am most proud of. I had to learn how to use the lerpColor function, mapping properly, as well as using an image in repetition to create a pattern. The result can be seen here:

//Creates the entire background and floor of the museum
function drawMuseum() {

  //Draws the gradient background
  for (let i = 0; i <= height - 100; i++) {
    let inter = map(i, 0, height - 100, 0, 1);
    let wallColor = lerpColor(color('#ECC2EC'), color('#DCB5A5'), inter);
    stroke(wallColor);
    line(0, i, width, i); 
  }
  
 // Creates the wood texture on the floor
  for (let i = 0; i < width + 50; i += 100) {
    // Draw the first set of wood planks
    image(wood, i, height - 100, 100, 50); 
    image(wood, i - 50, height - 50, 100, 50);
    
  // Adds borders around each plank
    stroke('#BC7C64');
    noFill();
    rect(i, height - 100, 100, 50);
    rect(i - 50, height - 50, 100, 50);
  }
}

Overall, I am really proud of how the “The Museum of Poems” turned out. I wish though, that I could make it somehow look more realistic. I feel like the wall and the flooring has a pretty big contrast in terms of aesthetic which is something to consider for the future.

One thing that drives me absolutely insane is paper straws. Nothing disappoints me more than getting my coffee or any other drink really and seeing a paper straw. And no, of course I’m not against saving the environment or the turtles, but those flimsy things are horrible for mixing drinks and get all soggy so fast. Any milkshake or even whipped cream and  the straw will give up and disintegrate before my eyes, refusing to let me finish my drink without crumpling up into a useless, unpleasant, undrinkable, thing in my cup. Sometimes I even have to get my hands dirty while trying to get a firmer grip on the straw just to mix it. With sustainability being a key issue for many, there are many other biodegradable alternatives up and coming on the market like wood straws or even sugarcane straws that have much better, 24 hour durability compared to like 30 seconds with a paper straw. While places in the UAE are slowly starting to implement these, it will take time before I never have to see a paper straw again.

This highlights why, as Norman explains, human-centered design is so important. While paper straws match the societal push for sustainability and look great for companies, they don’t account for the humans actually using them that suffer through the sogginess and disintegration of paper straws themselves. One of the other things Norman talks about is signifiers and affordances. Since perceived affordance can take the form of many things, it is important to have signifiers to make clear what the instructions are. I think this is especially important in interactive media because we often assume people will know to click, move their mouse around, press the arrow keys, etc in order to make something happen on the screen. However, as a designer, things that seem self-intuitive to us may not always be self-intuitive to others and can have many affordances. Therefore, having signifiers is important to make sure our projects are understandable and usable to a broader audience.

For this assignment, my inspiration came from an image I saw while scrolling on Instagram:

The image was coupled with some headline, but I can’t remember what exactly it was. The main thing that struck me about this photo was how it didn’t seem like the orbs and their nodes had any noticeable pattern, and yet somehow, this sphere was created. Did someone create the sphere and then click around? Was there a program? What are the chances this was done randomly? Thus, I was inspired to create a program that has users create orbs that bounce around the screen when they drag their mouse around, and if they are close, create a line connecting them. The output would be constantly evolving and up to the user as they can choose to add more orbs if they want. Who knows, maybe a sphere like shape would be seen on the screen temporarily. The results of my efforts are as follows:

(Drag your mouse!)

This assignment ended up being much harder than I expected because I had to make the orbs all look and act different, but not too different that they look bad on screen. I learned a lot about the p5.vector() and the types of attributes I can apply to it when creating my orb class which can be seen below. Figuring out how to create the connection line only if the orbs are close to each other also took a little bit.

class Orb {
  constructor(x, y) {
    this.pos = createVector(x, y); // Position vector
    this.vel = p5.Vector.random2D(); // Assign velocity
    this.vel.mult(random(1, 3)); // Assign magnitude
    this.size = random(8, 20); // Random size for each orb
    this.color = color(random(100, 255), random(100, 255), random(255), 150); // Random semi-transparent color
  }
  
  move() {
    this.pos.add(this.vel); // Update position
    // Bounce off edges
    if (this.pos.x <= 0 || this.pos.x >= width) {
      this.vel.x *= -1;
    }
    if (this.pos.y <= 0 || this.pos.y >= height) {
      this.vel.y *= -1;
    }
  }
  
  display() {
    noStroke();
    fill(this.color);
    ellipse(this.pos.x, this.pos.y, this.size); // Draws orb
  }

// Determines if orbs are within 100 pixels of each other
  isCloseTo(other) {
    let distance = this.pos.dist(other.pos);
    if (distance < 100) {
      return true;
    } else {
    return false;
  }
 }
}

To improve, I think I could play around more with the colors and upping the interactivity. It could be that the user can determine the colors of the orbs or have the colors all be the same but change each time the mouse is dragged. Ideally, it would also be cool to have the speed or density with which the user clicks affect the amount of orbs created. Overall though, I had fun working on this assignment and learned a lot.

The first thing reading this chapter did is confuse me. Things I thought were interactive like buttons on websites or sketches on an iPad, were being challenged, and I wondered if that was something I could accept. I always assumed that if something responded to a user’s input, it was interactive, but this reading made me rethink that. It introduced the idea that real interactivity isn’t just about reaction but about creating an experience where the user’s choices actually shape the outcome itself. That made me reconsider how much interaction we actually do have in our digital lives. How much of what we “interact” with is pre-scripted?

After this chapter, a strongly interactive system to me is one where the user’s choices don’t just trigger these pre-scripted responses but actually change the experience in an organic way and ever-evolving way. It should feel like a conversation as Crawford describes interactivity to be rather than a simple input-output relationship. The thinking is necessary. In my own sketches, I tried to incorporate a little of this by having the art start where the user clicks it and have them be able to add on to the art as it goes on. Plus, with randomness, the art is never the same for each person. However, I would want to go even further by implementing elements that can evolve based on patterns of interaction, so users feel like they’re shaping the art as it goes rather than just clicking around. For example, the sketch could adapt its colors, speed, or density based on whether the user clicks quickly, holds down, or moves in a certain rhythm.

One of my favorite parts of Casey Reas’ Eyeo Talk was his emphasis on how randomness should be embraced and has been embraced when creating art. He gives examples like John Cage’s experimental music compositions, where random elements influenced the outcome, and Jackson Pollock’s drip paintings, which embraced chance as a key element in the artistic process. But, he also provides examples from his own work in graphic art using generative systems and computer algorithms. I feel as if many people stray away from randomness out of fear that the chance exists of a random element ruining what they hope to achieve, but Reas welcomes that chance in his art, showcasing to others that randomness can create unexpected but fascinating art that is worth appreciating. He even states that with computer generative systems, you can create bounds or limits such as how big or small to make something in order to control some of the randomness in your art, creating endless possibilities for future computer generated art that have a guide on what to do but still have unexpected elements to them.

I was inspired personally by this talk and really tried to incorporate randomness into the assignment for this week, but with some bounds to get the desired effect I wanted for my project. I created lists with a set of elements that would be acceptable to use such as a set of colors or a set of numbers and then used the random function to have the program pick a random element from that list. This way, all my lines looked different due to the random line sizes and they were varied in color due to the random colors picked. I doubt I’ll ever be able to fully give in to total randomness as I do want some sense of control over what is created, but I am open to trying it to see what could be created since the possibilities are endless.

For this assignment, I was inspired by an artwork in the Computer Graphics and Art May 1976 magazine titled “The Cubic Limit Series” by Manfred Mohr shown below:

I wanted to create a grid like it along with similar looking shapes, but instead have the shapes/doodles be randomized and made from sharp lines that rotate in different colors within each square. The result of that looked like this:

Overall, I think the assignment did end up aligning with the vision I had in my head. When I initially made the grid, I was having trouble getting the background to be black as it seemed like the background was somehow being overwritten. It took me a bit, but I eventually I figured out that the squares in the grid had an automatically applied color fill which was displaying over the background. I also had trouble with getting each doodle to be in the center of each square before I realized I had to use translate to change its origin. I did have to create a separate function to create the doodle in each square, and while it seems complicated, it just involved picking a random number from a list I had defined in an earlier variable. I am proud of getting the result to be similar to the inspiration from the image I had in mind, and the snippet of that code can be seen below:

// Function to draw a random doodle in each square
function drawSquiggle() {
  strokeWeight(1.25)
  stroke(random(colors))
  line(random(neg_numbers), random(neg_numbers), random(numbers), random(neg_numbers)); 
  line(random(numbers), random(neg_numbers), random(numbers), random(numbers));  
  line(random(numbers), random(numbers), random(neg_numbers), random(numbers));
}

In the future, I want to see if I can make the rotation be smooth as well. Even though this kind of choppy look was intentional to seem more robotic, I feel a different shape rotating smoothly inside of the grid would also look cool. I also want to get better at navigating transformations of shapes for the best placement.

For this assignment, we had to create a self-portrait in p5js. I began by deciding which elements of myself were important to incorporate. Of course, this includes basic features like eyes, nose, mouth but also things like my wavy hair or silver earrings. Once I figured out what to incorporate, I began making shapes in p5js, making sure the layers were right to get the results I wanted. The final result of my self-portrait is as follows:

As I worked, I realized making shapes in this program is quiteee annoying, especially because you need to play guess and check when figuring out coordinates. This led to it taking more time than expected, especially when it came to places where there are many shapes combined. For example, the eyes are something I am proud of because I went a step further to make them a tad bit more realistic by adding a flash spot. You can see the code for this below:

//eyes
  push()
  stroke('black')
  fill('white');
  ellipse(220,190,30,19)
  fill('black');
  circle(220,190,12)
  fill('white')
  circle(224,186,4)
  pop()
  push()
  stroke('black')
  fill('white');
  ellipse(280,190,30,19)
  fill('black')
  circle(280,190,12)
  fill('white')
  circle(284,186,4)
  pop()

Overall though, I think this assignment was a great introduction to creating objects in Javascript and required a lot of self-teaching from the references on the p5js website when it came to things like arcs, rounding out rectangles, push and pop, ellipses, etc. Breaking down myself into simple shapes really pushed my creativity when it came to combining shapes like ellipses and rectangles and circles for my hair and bangs. I was able to think outside the box on how to create what I wanted because I wasn’t even sure what the best way to get what I wanted to create was. My hope for the future is the be able to understand curves better to be able to make more custom shapes from scratch. Perhaps one day I can revisit this self-portrait and make it more realistic.