Month: February 2024

Reading through Golan Levin’s musings on computer vision feels like sitting down with a mad scientist who’s decided to use his powers for art instead of world domination. It’s like he’s taken this super complex tech thing and turned it into a playground. Imagine computers squinting at us, trying to make sense of what they’re seeing, and then artists stepping in saying, “Let’s make it fun.” Levin essentially tells us that with a bit of creativity, even the geekiest tech can be as expressive and unpredictable as a paint-splattered canvas. Who knew algorithms could be party tricks for the digital age? It’s a reminder that innovation isn’t just about serious business applications; it’s also about making cool stuff just because we can.

Computer Vision is full of bits and bytes. There is no way perfect algorithm to obtain the computer vision of the background. The computer vision seems to only see the whole back as a series of pixels. This means that without human to define what these pixels are, the computer does not know what to do with the image in their vision. In other words, the image data is a stream of nonsense to the computer.

The way the computer analyze the changes in its vision is similar to how the sensors work, I think. For example, in one of the techniques mentioned in the reading, “detecting motion” is done by comparing the pixels of the first frame (F1) and the subsequent frame (F2). Any changes in pixels detected by the processor can be concluded that was a movement in the vision. However, it is clear that this technique is unable to recognize what type of shape is it as well as the color of the moving objects. Similarly, since this algorithm is created to only detect motion, it does not understand the general image that it is capturing.

Therefore, computer vision can only do good at the jobs that t is assigned.

Concept

For this project, I’ve drawn inspiration from a childhood game my brother used to play, called the “Fishing game.” It involved catching fish with a fishing hook within a time limit. I’ve adapted this concept into my own version of the game, adding some twists and modifications. Instead of a traditional fishing game with a box of fish, my game features fish scattered randomly across the canvas. The objective remains the same: capture each fish individually using a fishing hook (the mouse) and deposit them into a fish bowl before time runs out. This adaptation allows for a unique gameplay experience while still capturing the essence of the original game.

Design

1. The game begins with an instructions page featuring a button labeled “Click here to continue” at the bottom.

2. Upon clicking, the player is directed to a page where they can select the number of fish to be displayed on the canvas, adjusting the game’s difficulty.

3. After choosing the level of difficulty, the player clicks a button labeled “Click here to start the game.”

4. Upon starting the game, a countdown timer begins, indicating the time limit for completing the game.

5. Background music plays throughout the game, creating an immersive experience.

6. If the player successfully captures all the fish before the timer runs out, they win the game.

7. Upon winning, a new page displays congratulating the player on their victory and showing their completion time.

8. A victory music plays to celebrate the player’s success.

9. If the player fails to capture all the fish before the timer expires, they lose the game.

10. Upon losing, a new page shows that the player has lost the game.

11. An end game music plays to signal the conclusion of the game.

Challenges with the project

1. Implementing the functionality to drag and drop each fish individually to the fish bowl has posed a significant challenge.

2. Creating an on-screen countdown timer that accurately tracks the remaining time and triggers the end game page when it reaches zero has been another obstacle.

3. Providing users with the ability to select their preferred difficulty level by choosing the number of fishes in the game initially proved to be problematic, requires for code refining.

4. Design challenge: Positioning the fish bowl on a table above water and making the users to drag the fish from the water surface into the bowl, which would make it aesthetically pleasing.

Code

let fishes = [];
let fishBowl;
let fishingHook;
let timer;
let gameStarted = false;
let gameFinished = false;
let numFishes = 5; //  default number of fishes


function preload() {
  
  fishBowl = loadImage('bowl.png'); // Load image for fish bowl
  fishingHook = loadImage('hook.png'); // Load image for fishing hook
  gameMusic = loadSound('gamemusic.mp3'); // Load background music
  victoryMusic = loadSound('victorymusic.mp3'); // Load victory music
  losingMusic = loadSound('losingmusic.mp3'); // Load losing music
  
}

function setup() {
  
  createCanvas(800, 600);
  timer = new Timer(numFishes); // timer object with the number of fishes
  
}

function draw() {
  background("#2196F3");

  if (!gameStarted) {
    displayInstructions(); // Display instructions if game hasn't started
  } 
  
  else {
    timer.update(); // Update timer

    if (!gameFinished) {
      // Draw fish bowl
      image(fishBowl, width / 2 - 50, height / 2 - 50, 150, 150);

      // Draw fishing hook
      image(fishingHook, mouseX - 25, mouseY - 25, 50, 50);

      // Display and update fishes
      for (let fish of fishes) { //checks each elements of the "fishes" array
        fish.display();
        
        //hooking the fishes to the hook
        if (dist(mouseX, mouseY, fish.x, fish.y) < 25) {
          fish.hooked = true;
        
        }
        
        if (fish.hooked) {
          fish.move(mouseX, mouseY); //the fish is hooked to the hook
        }
      }

      // Check if all fishes are inside fish bowl
      let allFishesInside = true;
      for (let fish of fishes) {
        if (!fish.insideBowl) {
          allFishesInside = false;
          break;
        }
      }
      if (allFishesInside) {
        gameFinished = true;
        timer.stop();
        
        //play music
        if (timer.timeLeft > 0) {
          
          victoryMusic.play(); // Play victory music if game finished before countdown ends
          gameMusic.stop(); // Stop background music

        } 
        
        else {
          
          losingMusic.play(); // Play losing music if countdown ends before game finished
          gameMusic.stop(); // Stop background music

        }
        
      }
    } 
    
    else {
      fill(255)
      textSize(40);
      textAlign(CENTER, CENTER);
      text("Game Over!", width / 2, height / 2);
      text("Time left: " + timer.getTime() + " seconds", width / 2, height / 2 + 40);
      
    }
  }
}

function displayInstructions() {
  
  // Display instructions
  fill('rgb(0,0,134)'); //color of ellipse
  ellipse(width/2, height/2,650,150); //ellispe in the center behind the circle
  fill(255); //color of the texts
  textSize(40); //size of texts
  textAlign(CENTER, CENTER); //texts in the center of the canvas
  text("Click to start the game", width / 2, height / 2);
  
}


function mousePressed() {
  if (!gameStarted) { //if the game is not started 
    gameStarted = true; //the variable is set to true and it resets the fishes array
    fishes = []; // Reset fishes array
    
    //adding fishes to the canvas
    for (let i = 0; i < numFishes; i++) {
      fishes.push(new Fish(random(width), random(height)));
    }
    
    timer.start(); //countdown starts
    gameMusic.loop(); // Play background music on loop

    
  }
}

class Fish {
  constructor(x, y) {
    this.x = x;
    this.y = y;
    this.hooked = false;
    this.insideBowl = false;
    this.fishImage = loadImage('fish.png'); // Load fish image
  }

  display() {
    image(this.fishImage, this.x - 25, this.y - 25, 50, 50); // Draw fish image
  }

  move(x, y) {
    this.x = x;
    this.y = y;
    
    // Checking if the fish is inside the fish bowl
    if (dist(x, y, width / 2, height / 2) < 50) {

      this.insideBowl = true; // If the distance is less than 50, then the insideBowl property is set to true


    }
  }
}


class Timer {
  constructor(numFishes) {
    this.totalTime = numFishes * 1; // Adjust the total time based on the number of fishes
    this.timeLeft = this.totalTime;
    this.running = false;
  }

  start() {
    this.startTime = millis();
    this.running = true;
  }

  stop() {
    this.endTime = millis();
    this.running = false;
  }

  update() {
    if (this.running) {  //if the timer is running
      
      let timePassed = millis() - this.startTime; //calculates the time passed since the timer started
      
      this.timeLeft = max(0, this.totalTime - floor(timePassed / 1000)); // Calculate remaining time
      
      if (this.timeLeft === 0) {
        this.stop(); // Stop the timer when time runs out
      }
    }
  }

  getTime() {
    return this.timeLeft;
  }
}

Future Improvements and plans

1. Add more colors and deigns to the overall design of the game to make it more attractive, I want to make the design more aesthetically eye pleasing.

2. Add an instructions page to guide users on how to play the game.

3. Enable users to drag the fishes individually to the fish bowl for a more interactive experience.

4. I want to add more music to the game like subtle sounds and musics when the user interacts with the game.

5. Refine all aspects of the code to align with the initial plans and ensure smooth functionality.

Diving into the article about computer vision for artists and designers felt like opening a door to a room where art meets science in the most fascinating way. I’ve always thought of computer vision as something you’d find in sci-fi movies or high-tech security systems, not something that could be part of creating art. The idea that artists are using this technology to make interactive pieces where the artwork changes based on how people move around it is pretty cool. It’s like the art is alive, reacting to us just as much as we react to it.

Reading about projects like Videoplace and Sorting Daemon really got me thinking. It’s one thing to look at a painting on a wall, but it’s something entirely different to be part of the art itself. The thought that my movements could influence an art piece, creating something unique each time, is both exciting and a bit mind-blowing. It’s not just about observing; it’s about participating, and that changes the whole experience of what art can be.

The technical side of things, like how computer vision can track movements and interpret them into visual changes in an art piece, was a bit complex but also intriguing. I didn’t realize so much went into making these installations work. It’s not just about having a good idea for an art piece; it’s also about solving problems, like making sure the lighting is right so the cameras can see what they need to. This made me appreciate the art even more, knowing the blend of creativity and technical skill that goes into it.

However, what stuck with me the most was thinking about the bigger picture—like what it means for a machine to watch us and then create art based on that. It’s a bit like the machine is an artist too, interpreting our actions in its own digital way. And while it’s amazing, it also raises questions about privacy and how much we’re comfortable with machines “seeing.”

Overall, the article was a peek into a future where art and technology blend in ways that make the audience a part of the artwork. It’s a reminder that creativity knows no bounds, especially when artists start playing around with tech to see what new forms of expression they can discover. It makes me wonder what other surprises the art world has in store as technology keeps advancing.

Inspiration

I grew up in a village without a park, a place where families, friends, and pets gather to enjoy nature and each other’s company. To experience the joy of a park, I had to travel to nearby villages or cities. This sparked my love for parks and inspired my midterm project: creating a virtual park. My goal is to recreate the sense of community and fun found in a park, from children playing on swings to people walking their dogs and old friends catching up. This project is my way of bringing the park experience to those who, like me, have always admired it from afar.

Concept

Building on the foundational inspiration, the core idea of my midterm project is to craft an immersive user experience that mimics the tranquility and community feel of a real park. To achieve this, I’ve begun by setting up a serene background image that serves as the canvas for the park’s life. Central to the scene are two swings, each with a figure that moves in harmony with the swing’s motion. To ensure these movements are as natural as possible, I’ve implemented the lerp() function, which allows for smooth transitions and adds a lifelike quality to the animations.

Adding another layer of interaction to the park, I’ve introduced a jukebox feature. This allows users to engage with the environment actively by selecting and controlling the music to suit their mood, enhancing the personal experience of the park. While the music feature is currently in development, with plans to enable song changes and stops, it promises to be a significant aspect of user engagement.

Looking ahead, I plan to enrich the park’s atmosphere with spritesheets for various people movements, creating a dynamic and bustling park scene. This will include groups of people sitting on the grass or on carpets, adding to the communal vibe. Additionally, to further the immersion, I intend to incorporate ambient sounds, such as birds chirping, which will be a constant backdrop to the park experience, not subject to user control. This sound layer aims to deepen the user’s connection with the virtual environment, making it feel alive and vibrant. Moreover, diversifying the playground equipment, especially with different swings, will offer a variety of interactions for the user, simulating the choice and freedom one finds in a real park. This expansion not only adds visual appeal but also invites users to explore and find their favorite corner of the virtual park.

Challenging Aspects

Drawing Complexities: The first significant challenge is the detailed drawing of park structures and objects using p5.js. Every element, from swings to human-structure, requires time-consuming manual drawing. This process demands a high level of precision and artistic skill to ensure the park’s visual appeal and thematic consistency.

Interactive Features: Introducing interactivity, such as clickable buttons and objects that respond to mouse hover, adds another layer of complexity. For example, making the jukebox highlight when hovered over involves sophisticated input detection and dynamic response coding, enhancing user engagement but also increasing development complexity.

Animating Characters with Spritesheet: Utilizing spritesheets for character movement presents a formidable challenge. Ensuring these animated figures navigate the park without colliding with objects or wandering off the grass involves intricate collision detection and boundary management.

References:

https://www.youtube.com/watch?v=8uLVnM36XUc

The article provides a comprehensive overview of the history, techniques, and applications of computer vision in the interactive media arts. It is interesting to see how artists have used computer vision to create novel and expressive experiences, such as Myron Krueger’s Videoplace, Rafael Lozano-Hemmer’s Standards and Double Standards, and David Rokeby’s Sorting Daemon. The article also explains some of the basic algorithms and tools for implementing computer vision, such as frame differencing, background subtraction, brightness thresholding, and object tracking. I find these algorithms useful and inspiring for my own projects, as they enable me to detect and analyze the movements and gestures of people and objects in real time.

Messa di Voce by Golan Levin and Zachary Lieberman was very interesting to me. This is an audiovisual performance that uses computer vision to track the locations of the performers’ heads and analyze the sounds they produce. The system then generates graphics that are projected from the performers’ mouths, creating a dynamic and expressive dialogue between sound and image. I think this artwork is a creative and engaging example of how computer vision can augment and transform human communication and expression.

I found Cheese by Christian Möller pretty funny. This is an installation that uses computer vision to measure the sincerity of the smiles of six actresses. The system uses an emotion recognition algorithm to detect the changes in the facial expressions of the actresses, and alerts them when their smiles fall below a certain threshold. I think this artwork is a humorous and critical commentary on the social and cultural expectations of happiness and beauty, as well as the limitations and biases of computer vision.