Category: Spring 2024 – Aaron (Section 001)

Golan Levin’s Computer Vision for Artists and Designers,  explains the concept computer vision for beginner programmers like myself, placing a emphasizing on utilizing vision-based detection and tracking techniques within interactive media. The initial section introduces the artistic applications of computer vision beyond industrial and military realms.

It was a very interesting read that almost shows history in interactive media through computer vision, at least in some way or the other. It is through computer vision the first computer interactive art performance was developed.

For example, ‘Standards and Double Standards’ by Rafael Lozano-Hemmer is something that intrigued me. Belts spinning around based on who’s passing by – visually cool, but there’s this metaphorical layer that adds depth. It’s the kind of stuff that makes you ponder. Which is what I would like to implement in my work.

 

For my midterm, I had to create a game primarily focused on cats, driven by my love for them. The game’s objective is to reach a score of 20 by catching as many cats as possible within thirty seconds. Additionally, there is a power-up cat that boosts your score. I encountered two unsuccessful attempts in executing my ideas, leading me to start over, which I believe turned out quite well.

INITIAL IDEAS:

My initial idea was to create a game inspired by “My Talking Tom,” a game I enjoyed playing while growing up. I aimed to introduce a twist by replacing the cat with a camel. However, when it came time to execute the idea, I was unsure of how to begin and could not find any examples to follow.

 

My second idea involved creating an interactive game using PoseNet, modeled after the classic “I’ve got your nose” game. The core code functioned well; however, implementing the menu and other deliverables proved difficult, as they interfered with the rest of the code’s functionality. If you’re interested in trying it out, I recommend visiting the actual sketch, as you’ll need your webcam.

Moving on to the actual game, I aimed to create something simple and easy to play. However, I do wish it offered more of a challenge to players. Perhaps increasing the speed or adding another character that reduces the score instead of offering power-ups could enhance the difficulty. Additionally, feedback from one of my classmates highlighted the importance of clear instructions. Given the potential for confusion, providing explicit instructions regarding the timeframe, the scoring system, and the power-up cat would make it easier for users to understand and enjoy the game.

In terms of challenges, I faced issues similar to those of my classmates, particularly with the sketch not accepting images. I was advised to either work on the code locally or clear my cache to resolve this issue. However, my main challenges were related to the stylistic aspects of the game and deciphering the keyPressed(); function. Additionally, linking all parts of the game—from the menu to the game itself, and then to the win or lose screen—before looping it all back to the menu was probably the most time-consuming aspect of this entire project.

The most enjoyable aspect of this project for me was creating the graphics. I utilized Canva and Vecteezy to source all the elements needed to achieve the pixelated Mario aesthetic I desired.

Code + Notes:

// Midterm Project by Amal Almazrouei
// Credit:
// Sound: https://www.voicy.network/
// Font: Google Fonts
// Images: https://www.vecteezy.com/ ; pinterest.com
// Graphics: canva.com
// Example Code: https://editor.p5js.org/simontiger/sketches/Dj7GS2035


// Flags to control game state
let started = false;
let gameEnded = true; // Treat game as ended initially to present start menu
let showingEndScreen = false; // Indicates if end screen is currently displayed

// Game timing and positioning variables
let startTime;
let x = 200;
let y = 200;
let powerX = 100;
let powerY = 200;

// Score and game dynamics variables
let score = 0;
let r = 48; // Radius for interactive elements
let interval = 60; // Controls frequency of game events
let frames = interval; // Countdown to trigger game events
let powerIsThere = false; // Flag for power-up appearance
let increase = 1; // Score increment
let increased = 0;

// Assets and sound variables
let img, powerImg, startScreenBg; // Image variables for game elements
let myFont; // Custom font variable
let backgroundMusic, catSound, powerUpSound, winSound, loseSound; // Sound variables

function preload() {
  // Preload all necessary media assets
  img = loadImage('Media/cat1.png'); // Main character image
  powerImg = loadImage('Media/cat2.png'); // Power-up image
  startScreenBg = loadImage('Media/startScreenBg.png'); // Start screen background image
  myFont = loadFont('assets/MyFont.ttf'); // Custom font
  // Loading sound assets
  backgroundMusic = loadSound('sounds/backgroundMusic.mp3');
  catSound = loadSound('sounds/catSound.mp3');
  powerUpSound = loadSound('sounds/powerUpSound.mp3');
  winSound = loadSound('sounds/winSound.mp3');
  loseSound = loadSound('sounds/loseSound.mp3');
}

function setup() {
  createCanvas(400, 400);
  textFont(myFont); // Set the custom font
  backgroundMusic.loop(); // Play background music on loop
}

function keyPressed() {
  // Handles key presses for restarting or transitioning from end screen
  if (gameEnded && showingEndScreen) {
    showingEndScreen = false; // Exit end screen to show start menu
  } else if (!started && !showingEndScreen && gameEnded) {
    resetGame(); // Reset and start game from menu
  }
}

function resetGame() {
  // Reinitialize game variables for a new session
  started = true;
  gameEnded = false;
  showingEndScreen = false;
  startTime = millis(); // Reset game timing and positioning
  x = 200;
  y = 200;
  score = 0;
  r = 48;
  interval = 60;
  frames = interval;
  powerIsThere = false;
  increase = 1;
  increased = 0;
  clear(); // Clear the screen before starting
}

function mousePressed() {
  // Handle player interactions with game elements
  if (started && !gameEnded && !showingEndScreen) {
    if (dist(mouseX, mouseY, x, y) < r) {
      catSound.play();
      // Random chance for score increase
      const rand = random(1);
      if (rand < 0.02) {
        increase = 50;
      } else if (rand < 0.1) {
        increase = 10;
      } else {
        increase = 1;
      }
      score += increase;
      increased = 20;
      interval--;
      r--;
    }
    if (powerIsThere && dist(mouseX, mouseY, powerX, powerY) < r) {
      powerUpSound.play();
      score++;
      interval = 60;
      r = 48;
      powerIsThere = false; // Remove power-up after clicking
    }
  }
}

function draw() {
  // Main animation loop: Update game screen based on state
  if (!started && !showingEndScreen) {
    gameStartScreen(); // Show start screen if game hasn't started
  } else if (started && !gameEnded) {
    // Game play updates
    let currentMillis = millis();
    background(135, 206, 235); // Set background
    // Display power-up image if present
    if (powerIsThere) {
      image(powerImg, powerX - r, powerY - r, r * 2, r * 2);
    }
    // Display main character image
    image(img, x - r, y - r, r * 2, r * 2);
    // Display score and time left
    textAlign(LEFT, TOP);
    textSize(30);
    fill(255, 0, 255); // Set text color
    text(score, 15, 15);
    let timeLeft = 30 - Math.floor((currentMillis - startTime) / 1000);
    text("Time: " + timeLeft, 15, 50);
    // Update positions and check for game end conditions
    frames--;
    if (frames <= 0) {
      frames = interval;
      x = random(width);
      y = random(height);
      if (random(1) < 0.07) {
        powerIsThere = true;
        powerX = random(width);
        powerY = random(height);
      } else {
        powerIsThere = false;
      }
    }
    if (score >= 20) {
      endGame(true); // Player won
    } else if (currentMillis - startTime > 30000) {
      endGame(false); // Time's up, player lost
    }
  }
}

function endGame(win) {
  // Handle game end: Show win or lose screen
  started = false;
  gameEnded = true;
  showingEndScreen = true;
  background(135, 206, 235); // Optionally reset background
  textAlign(CENTER, CENTER);
  textSize(32);
  // Display win or lose message
  if (win) {
    winSound.play();
    fill(0, 0, 128); // Blue color for win
    text("You Win!", width / 2, height / 2);
  } else {
    loseSound.play();
    fill(0, 0, 128); // Blue color for lose
    text("You Lose!", width / 2, height / 2);
  }
  fill(255, 0, 255); // Magenta color for subtitle
  textSize(24); // Smaller font size for subtitle
  text("Press any key to continue", width / 2, height / 2 + 20);
}

function gameStartScreen() {
  // Display start screen at game launch
  image(startScreenBg, 0, 0, 400, 400);
}

Credit + Resources:

Sound: https://www.voicy.network/
Font: Google Fonts
Images: https://www.vecteezy.com/ ; pinterest.com
Graphics: canva.com
Example Code: https://editor.p5js.org/simontiger/sketches/Dj7GS2035

 

 

 

The article provides a comprehensive examination of how computer vision technologies have evolved from highly specialized tools to accessible instruments for artistic expression and design. The inclusion of historical and contemporary examples of interactive art, such as Myron Krueger’s Videoplace and Golan Levin’s own Messa di Voce, serves not only to illustrate the potential applications of computer vision in art but also to inspire readers to think creatively about how they might employ these tools in their own work. The article does not shy away from addressing the challenges and ethical considerations inherent in surveillance and data collection, using projects like the Suicide Box by the Bureau of Inverse Technology to provoke critical thought about the implications of computer vision technology. The workshop example, LimboTime, serves as a tangible outcome of Levin’s pedagogic approach, demonstrating how novices can rapidly prototype and implement an interactive game using basic computer vision techniques. This example encapsulates the article’s core message: that with the right guidance and tools, computer vision can become a powerful medium for artistic exploration and expression.
By demystifying computer vision and providing practical tools and techniques for novices, Levin opens up new avenues for creative expression and interaction, reinforcing the idea that art and technology are not disparate fields but complementary facets of human creativity.

When I heard how the midterm would be to create a game, my mind immediately went to my favorite type of game: Rhythm Games. I wanted to try and recreate one of my favorite arcade games called Dance Dance Revolution where the player would have to stand on the arrows on the ground that would correspond to the rhythm of the background music.

This is the DDR Arcade Machine:

I successfully created three lives that would get lost when the player would miss a ball. I also managed to create a “Game Over” state which I have to modify. However, I need to work on the interactive part of it. The mechanics that I have currently are Mouse click based, meaning that a player would have to click on the target area in order to gain points. When I tried to play it using a mouse, it felt very awkward so I will probably change it to the arrow keys which would be way more convenient.

Update: Okay, turns out that It does not even work properly with the mouse no matter how hard I try. I will be switching the strategy to the arrow keys.  I found the exact arrows that I would want to use which are the following. Now I have to figure out how I could code the arrow keys and assign them to each of the targets.

This is what I would like it to look like:

Concept:

The core idea of my game reimagines the classic mechanics of the retro game Snake, introducing a playful twist where the protagonist is a dog on a mission to eat as much homework as possible. In this game, players navigate a dog around the screen, aiming to collect and consume homework pieces scattered across the play area. Each piece of homework the dog eats not only adds points to the player’s score but also visually enlarges the dog’s face, making the game increasingly challenging and humorous as the dog’s appearance grows comically large.

To add a layer of strategy and urgency, the game is set against a ticking clock. Players must race against time, strategizing to eat as much homework as they can before the timer expires. The score is a combination of two elements: the physical size of the dog’s face, which grows with each piece of homework eaten, and a numerical value that increases with the homework consumed. This dual scoring system provides immediate visual feedback and a quantifiable measure of success, engaging players in a quest to beat their own high scores or compete with others.

A highlight of some code that you’re particularly proud of:

function drawDog(x, y) {
  push(); // Start a new drawing state
  translate(x - width / 2, y - height / 2); // Center the drawing on the dog's position
  scale(0.25)
  
  var colorW = color(255,255,255);
  var colorBL = color(0,0,0);
  var colorBR = color(160,82,45);
  var colorP = color(255,182,193);
  
  // Ears
  push();
  noStroke();
  fill(colorBR); 
    // Right ear
    rotate(-PI/2.2);
    translate(-400,30);
    ellipse(width/2, height/4, 150, 50);  

    // Left ear
    rotate(PI/-12);
    translate(-56,245);
    ellipse(width/2, height/4, 150, 50);
  pop();
  
  // Base
  push();
  noStroke();
  fill(colorW);
  ellipse(width/2, height/2, 200, 200)
  pop();
  
  // Mouth
  push();
  noStroke();
  fill(colorBL);
  translate(150,210);
  arc(50, 50, 80, 80, 0, HALF_PI + HALF_PI);
  pop();
  
  // Tongue
  push();
  noStroke();
  fill(colorP);
  translate(-25,65);
  rect(width/2, height/2, 50, 35, 20)
  pop();
  
  // Tongue detail
  push();
  fill(219,112,147);
  ellipse(width/2, 277.5, 5, 25);
  stroke(125);
  pop();
  
  // Nose
  push();
  noStroke();
  fill(colorBL);
  translate(142,150);
  triangle(30, 75, 58, 100, 86, 75);
  pop();
  
  // Nose shine
  push();
  noFill();
  stroke(colorW);
  strokeWeight(2);
  noFill();
  strokeJoin(MITER);
  beginShape();
  scale(0.5, 0.5);
  translate(380,437);
  vertex(10, 20);
  vertex(35, 20);
  endShape();
  pop();
  
  // Eyes
  push();
  noStroke();
  fill(colorBR);
  rect(220, 160, 60, 60, 20, 30, 20, 40);
  fill(colorBL);
  ellipse(250, 190, 35, 35);
  ellipse(150, 190, 35, 35);
  fill(colorW);
  ellipse(150,180,10,10);
  ellipse(250,180,10,10);
  pop();
}

 

Embedded sketch:

Reflection and ideas for future work or improvements:

Progress on the game development is quite encouraging at this point. I’ve successfully tackled the challenge of scaling the dog’s head in a way that ensures all facial features remain proportionate, regardless of its size. This was a crucial step to maintain the visual consistency and appeal of the game, ensuring that as the dog “eats” more homework and its face grows, it still looks natural and retains the humorous charm intended for the game’s design.

The next significant hurdle I’m facing involves developing a robust logic system for the game. Specifically, I need to implement a mechanism where the growth of the dog’s head is directly tied to the action of consuming homework pieces (HW) within the game. This means creating a set of rules and conditions in the game’s code that accurately tracks each piece of homework the dog eats and translates that into proportional increases in the size of the dog’s head.

Going into the midterm project I originally had the idea to make a two player game of Russian roulette, but eventually I changed my mind. I have now decided to make a two player, Mexican standoff game. Where there is a countdown and whoever presses the shoot button first wins. Player 1 will have a button on the left hand side of the keyboard like the letter ‘S’ and player 2 will have a button on the right hand side of the keyboard like the letter ‘L’ for example.

I spent most of my time working on the scene management. By that I mean the functionality around being able to switch to and from scenes. For example, going from the main menu to the tutorial screen or to the credits or game screen and back. I also decided for this project I would use visual studio code to program the code. It has worked perfectly up to this point where I have to now upload the code onto the website so that I can share it here. For some reason, as of writing this I am having the problem where whenever I run the code in the website editor the website decides to turn white. I thought this was a problem with chrome and maybe one of my extensions so I tried switching to Safari but had the same problem. Here is the sketch below. I managed to fix the issue and I explain the

I was having the issue of the website editor going blank, because one of my functions returns a console.error(). At the time when I had made the function return such a value I didn’t think it would be an issue in the first place because it would never happen, and it never did, but because a function could return such a value it crashed the editor. Note to self: don’t try to be fancy with the values you return

At the moment the program is very bare bones. In the future I plan to have the main title an image, and maybe the main menu background will be an animated one. Then for the game each player will be a pixelated version of a cowboy or something along those lines, and there will be some kind of background. I may also make the mexican standoff game a best of 5.

Lastly, thank you Pi for telling me about scene management, and thank you professor for helping with uploading the code from Visual Studio Code to the website editor.

I found the article to be very interesting, before reading I had a very primitive idea as to how a computer can recognize objects in a video or anything else to do with a computer processing a video. But, as I was reading I was reminded of examples of “computer vision” as the author put it. On the way to the Intro to IM class there is a screen and camera, and displayed on the screen is whatever is in front of the camera but the brightness of (nearly) every pixel is represented as a character. The end result is a black and white image but with characters to represent the brightness of most of the pixels. I was also reminded of a time where during the thunderstorm which had happened recently I met someone as I was walking to D2 who was having trouble photographing a lightning strike. In my head I thought that he could have benefited from some kind of program which could do all the work for him. I thought about it, and the idea I came up with was if the camera was recording all the time but deleting footage older than 30 seconds (to save space) then if the camera detects a spike in brightness it saves the last 30 seconds plus some footage after the lightning strike (this could be done by delaying the time between detecting the spike and saving the footage). Of course I don’t really know how to implement it but in theory it could work… I think.

I also learned a lot from the article. There were techniques mentioned, most of which which I would never be able to come up with myself. “Frame differencing,” “background subtraction,” and “brightness thresholding.” While I do not have a great idea of how to implement these techniques I think the most valuable thing I took away from the article were the names as I could always search them up and learn more afterwards. Fortunately they also linked some processing programs at the bottom that I could use to learn more.

Lastly, I noticed that the article was somewhat outdated. It was released in 2006. I felt it was worth mentioning because at the time machine learning was no where near as advanced as it is today so I would have liked to have learned more about how machine learning could be used to improve “computer vision.”

 

For my midterm project I will be creating an interactive artwork entitled Wanderer which will take the form of a visual loop featuring a figure that walks through various landscapes. The interaction itself will be fairly simple as the user will only be able to initiate the figure’s movement and change the landscape in which the figure is walking.  I have chosen to simplify the interaction as a means to draw attention towards the work’s visual qualities which will (hopefully) be satisfying to look at. To fulfil the brief, I plan to incorporate an “energy” meter which will cause the session to restart if it reaches zero. As of right now, I have started working on a sprite sheet and will make progress on the backgrounds in the coming days.

let spritesheet;
let sprites = [];
let direction = 1;
let step = 0;
let x;
let y;
let speed = 7;
let stepSpeed = 60;
let animationTimer;
let automaticMovement = false; 

function preload() {
  spritesheet = loadImage("walk-sprite.png");
}

function setup() {
  createCanvas(600, 600);

  let w = spritesheet.width / 17;
  let h = spritesheet.height / 2;

  for (let y = 0; y < 2; y++) {
    for (let x = 0; x < 17; x++) {
      sprites.push(spritesheet.get(x * w, y * h, w, h));
    }
  }

  x = width / 2;
  y = height / 2;

  imageMode(CENTER);
}

function draw() {
  background(255);

  if (automaticMovement) {
    direction = 1; 
    x += speed;
    step = (step + 1) % 12;

  translate(x, y);
  scale(direction, 1);
  image(sprites[step], 0, 0);
}
}

function keyPressed() {
  clearInterval(animationTimer);

 
  automaticMovement = true;

  // then set the interval
  animationTimer = setInterval(() => {
    step = (step + 1) % 12;
  }, stepSpeed);
}

function keyReleased() {
  clearInterval(animationTimer);


  automaticMovement = false;
}

function mouseClicked() {

  automaticMovement = true;
}

 

From what I understand, “computer vision” refers to a broad range of algorithms that let computers make sophisticated analyses of digital images and movies. The reading begins with defining computer vision and then explores its use in interactive arts, emphasizing Marvin Minsky’s early miscalculation of the field’s complexity. The reading was educational, especially the portion that highlighted how many different kinds of artworks use computer vision. These works of art range widely in style, from abstract and formal pieces to ones with comedic or political overtones, and this is important in regard to understanding the different ways in which we can communicate art.

Moreover, other artists mentioned explore a wide spectrum of human movement, from close studies of facial expressions to broader gestures and even entire body movements. Overall, the reading underscores the rich potential of computer vision in the realm of interactive art, showcasing its versatility and impact on artistic expression.

“Computer Vision for Artists and Designers” by Golan Levin introduces computer vision as a set of algorithms enabling computers to intelligently interpret digital images and videos, emphasizing its recent accessibility to novice programmers and interactive-media artists. It demystifies computer vision for beginners, focusing on its applications in interactive art, elementary computer vision techniques, physical optimization for computer vision, multimedia authoring tools for computer vision.

The author also explores the evolution of computer vision as a medium for artistic expression, outlining its recent democratization and tracing its historical origins. It highlights how computer vision methods are being applied to a wide range of artistic mediums, video games, and home automation systems. In addition, he deconstructs basic computer vision methods including frame differencing, background subtraction, brightness thresholding, and basic object tracking and provides an understanding of how to use and use them in interactive media. It highlights how crucial physical optimization is for computer vision and offers suggestions for setting up conditions that support reliable algorithmic performance.

Furthermore, he also examines multimedia authoring tools for computer vision, including major software development environments and their corresponding computer vision toolkits or plug-ins, such as Processing, Macromedia Director, and Max/MSP/Jitter. Additionally, it provides an example of a workshop project named LimboTime that shows how computer vision techniques can be used to create an interactive game. LimboTime highlights the wider application of computer vision in art and design by demonstrating the accessibility and possibilities for non-programmers to create vision-based interactive systems.

Overall, this reading offers us a comprehensive overview of computer vision, catering to novice programmers and artists, and highlights its growing significance in interactive art and design, offering practical insights and resources for implementing computer vision techniques in various artistic and design contexts.