Computer Vision is Hard

So cameras, computer vision and cool stuffs! Yay! For this week, I messed around with the Video and OpenCV libraries. this resulted in two programs: 1) a program that taces the webcam capture and runs an edge-identifying algorithm to re-draw the capture in cool, computer-y ways, and 2) an attempt at a function that is able to identify the inside of a shape drawn with a black marker. Fun times.

Continue reading “Computer Vision is Hard”

Practical Processing

Jackal Radio uses a piece of software called SAM broadcaster to create the radio stream. It has a bunch of built in features which don’t really work. You can only hook up one external input, which means that you have to use an external application to mix input from a variety of sources (mic, line in, turntable) which then gets piped into SAM. One thing we’ve been trying to implement lately is soundbites. You know how radio stations always have their characteristic little soundbite? That’s what we want. Stuff like “Jackal Radio: The galaxy’s only college radio station” or “Jackal Radio: Jackal Radio’s Jackal Radio.” I built a little processing sketch that can play these little bits at the press of a button. I made a GIF to demonstrate this. Check it out! As always, why work on your capstone when you can work on radio stuff?

soundbite_example

We love Scott and his Cat :)

In this assignment, I made a picture by using processing. The basic idea is to create two layers of images after importing two images.  One image is full of words while the other is Scott’s portrait, the gradients of words sketches outline of the portrait. Then I decided to show all of our names on the portrait of Scott, it would be awesome! Here is the work:)

We love Scott & his cat

Continue reading “We love Scott and his Cat :)”

Convolutions

Convolving two functions and amounts to dragging one function over another. Computing the convolution of two functions involves computing the integral of the product of the two functions as one is shifted while the other is stationary. There is a cool theorem (the convolution theorem) that states that the convolution of two functions is equal to their product in Fourier space. I leverage this theorem in my processing code to compute the convolution of the webcam feed with a gaussian. The position of the gaussian can be shifted by moving the cursor across the frame. By switching modes, one can increase the standard deviation of the gaussian, which results in less blurring. I made a GIF that demonstrates this functionality.

convolutions

Tinted Frame

For this week’s assignment, we were asked to do things either video or image on processing. First, I chose four world-famous paintings as building blocks to the effect I wanna create. Then I first tried to make one of them shrink in size. If the mouse is pressed, the whole screen would be tinted with random color. I did not set the background so that as the image moves it leaves colored frames behind. Then I assigned different keys( m for Mona Lisa, g for the American Gothic, c for Scream, and s for Sunflowers) to the corresponding images. And that’s basically what I did.

Here are a couple screenshots I made while running the processing code:

Screen Shot 2015-11-23 at 2.06.57 PM

Continue reading “Tinted Frame”

Video manipulation – What is happening?

In my project for this week, the setup function shows the image captured from the camera and asks for a color to search for (the color can be selected by clicking on some point of the image), then the program displays pixels of a pre-recorded video instead of all the pixels of the image captured from the camera that are of that color (and within a threshold), while the rest of the image turns to random noise (colors). I feel like the computer is having a hard time finding the color I want it to look for (and similar ones to that color), and is looking for colors of similar darkness instead. I’ll try working on this but I like what is happening anyway.

 

Here are the code and the video files I used for my project:

https://drive.google.com/a/nyu.edu/folderview?id=0BwGOKPbccFXuVGV5OWxmR05TODA&usp=sharing

(For testing: I find it best to click on my mouth or some sort of middle-dark, possibly pinkish color, that gives the best results.)

Distortion Machine

I wrote a program that has the user choose a certain theme, and then the output is a distortion using tints and image overlays for some interesting effects. I liked the idea of having a choice in switching between themes and creating distortions; it’s something I want to include in my final project, perhaps as a heavily distorted background to the interactive designs.

import processing.video.*;
PImage gradient;
PImage madonna;
PImage marley;
PImage merica;
PImage new_media;
int home = 0;

Capture video;
void setup(){
 size(640,480);
 gradient = loadImage("GRADIENT.jpg");
 madonna = loadImage("MADONNA.jpg");
 marley = loadImage("MARLEY.jpg");
 new_media = loadImage("NEW_MEDIA.jpg");
 video = new Capture(this, width,height);
 video.start();
}
void draw(){
 if (video.available()){
 video.read();
 }
if(home == 0){
main_screen();
}
hover();
}

void main_screen(){
background(20);
textSize(20);
PFont myFont = createFont("AnonymousPro",50);
textFont(myFont);
text("CHOOSE YOUR MOOD",50,100);
image(gradient,30,150,100,100);
image(madonna,150,150,100,100);
image(marley,270,150,100,100);
image(new_media,390,150,100,100);
}

void hover(){
 if((mouseX >= 30 )&&(mouseX <= 130) && (mouseY >= 150 )&&(mouseY <= 250 )){
 home = 1;
 gradient();
 }
 
 if((mouseX >= 150 )&&(mouseX <= 250) && (mouseY >= 150)&&(mouseY <= 250 )){
 home = 1;
 madonna();
 }
 if((mouseX >= 270)&&(mouseX <= 370) &&(mouseY >= 150 )&&(mouseY <= 250 )){
 home = 1;
 bob_marley();
 }
 if((mouseX >= 390)&& (mouseX <= 490) &&(mouseY >= 150 )&&(mouseY <= 250)){
 home = 1;
 new_media_pink();
 }
 
 if(mousePressed == true){
 home = 0;
 }
}

 void gradient(){
 video.loadPixels();
 tint(255,127, 0, 127);
 for(int i = -150;i<width;i+=100){
 tint(255+i,127+i,0,127);
 image(video,i,0);
 }
 }
 
 void new_media_pink(){
 video.loadPixels();
 tint(255,150, 0, 225);
 image(video,-100,0);
 tint(255,127, 0, 225);
 image(video,-50,0);
 tint(255,100, 50, 175);
 image(video,100,0);
 tint(255,50, 100, 125);
 image(video,50,0);
 tint(255,25, 150, 75);
 image(video,0,0);
 }
 
 void bob_marley(){
 video.loadPixels();
 for(int i = 0;i<width;i+=30){
 for(int j=0;j<height;j+=40){
 tint(i,j, 0, 127);
 image(video,i,j,30,40);
 }
 }
 }
 
 void madonna(){
 video.loadPixels();
 for(int i = 0;i<width;i+=30){
 tint(255,127, 50, 127);
 image(video,i-30,0,i,60);
 tint(210,127, 100, 127);
 image(video,i-30,60,i,90);
 tint(170,127, 150, 127);
 image(video,i-60,150,i-30,130);
 tint(130,127, 200, 127);
 image(video,i-60,280,i-30,200);
 }
 }

Album_Cover 2015-11-23_0106

Wannabe Mateo

I made a snake game.

You push the buttons to move the snake and try not to die.

 

Also there’s a cool space background just to make things more interesting.

Snake_Space

 

import processing.serial.*;
Serial myPort;

ArrayList<Sphere> spheres;

int xPos = 400;
int yPos = 300;
int Xspeed = 1;
int Yspeed;
int diam = 10;
float []  coordinatesX = {};
float [] coordinatesY = {};
int game = 0;

void setup(){
  size(800,600);
  background(30);
  fill(255);
  String portName = "/dev/tty.usbserial-DA01LJI0";
  myPort = new Serial(this,portName, 9600);
  myPort.bufferUntil('\n');

spheres = new ArrayList<Sphere>();

}
void draw(){
  
  background(30);
  spheres.add(new Sphere());
   for (int i = spheres.size()-1; i>=0; i--) { // manipulate the objects
    Sphere s = spheres.get(i);
    s.update();
    s.render();
    //s.bounds();
    if (s.isDead()) {
      spheres.remove(i);
    }
  }
  
 game = 1;
  if(game == 1){ 
    fill(255); 
    stroke(255);
    xPos = constrain(xPos,0,800);
    yPos = constrain(yPos,0,600);
    xPos += Xspeed;
    yPos += Yspeed;
  

    for (int i = 0; i < coordinatesX.length; i++){
      
      rect(coordinatesX[i],coordinatesY[i],diam,diam);
      
      if ((xPos == coordinatesX[i] && yPos == coordinatesY[i]) || xPos == 0 || xPos == 800 || yPos == 0 || yPos == 600 ){
        game = 2;
  }
  }
  rect(xPos,yPos,diam,diam);
  //'UP' command
 
  coordinatesX = append(coordinatesX,int(xPos));
  coordinatesY = append(coordinatesY,int(yPos));
 
  if(game == 2){
    background(30);
    textSize(20);
    text("Bruh you dead.",200,height/2);
    yPos = 0;
    xPos = 0;
    fill(30);
  }
}
}
void serialEvent(Serial myPort) {
  String input = myPort.readString();
  String[] numbers = split(input, ',');
  float[] values = float(numbers);
  
//'UP' command
   if(values[2] == 1){
    Yspeed= -3;
    Xspeed=0;
  }
//'DOWN' command
    if(values[1] == 1){
      Yspeed = 3;
      Xspeed = 0;
    }
//'LEFT' command
    if(values[3] == 1){
      Yspeed = 0;
      Xspeed = -3;    
    }
    if(values[0] == 1){
      Yspeed = 0;
      Xspeed = 3; 
    }
    myPort.write('x');
  }


class Sphere {
  // variables for the object
  float xPos;
  float yPos;
  float xSpeed;
  float ySpeed;
  float diam;
  float alpha;

  Sphere() {
    alpha = random(250,  255);
    xPos = 400;
    yPos = 300;
    xSpeed = random(-7.,7.);
    ySpeed = random(-7.,7.);
    diam = 100;
    
  }

  boolean isDead() {
    if (alpha <= 0) {
      return true;
    } else {
      return false;
    }
  }
  // declare all my functions
  void render() {
    alpha -=3;
    diam--;
    fill(210, alpha);
    //for(int i = 0; i<100;i = i+5){
    ellipse(xPos, yPos, xSpeed, ySpeed );
   // ellipse(xPos-i,yPos+i,xSpeed,ySpeed);
   // ellipse(xPos+i,yPos-i,xSpeed,ySpeed);
   // ellipse(xPos-i,yPos-i,xSpeed,ySpeed);
   // }
   
   if (mousePressed == true){
    for(int i = 0; i<100;i = i+50){
      ellipse(xPos+i, yPos+i, xSpeed, ySpeed );
       ellipse(xPos-i,yPos+i,xSpeed,ySpeed);
      ellipse(xPos+i,yPos-i,xSpeed,ySpeed);
       ellipse(xPos-i,yPos-i,xSpeed,ySpeed);
    }
   }
     
  }

  void update() {
    xPos-=xSpeed;
    yPos-=ySpeed;  
    
      
    
      
      
    
  }

  void bounds() {
    if (xPos >= width || xPos <= 0) {
      xSpeed = xSpeed * -1;
    }
    if (yPos >= height || yPos <= 0) {
      ySpeed = ySpeed * -1;
    }
  }

}

This is the code for Arduino:

void setup() {
  // put your setup code here, to run once:
pinMode(2,INPUT); //GREEN
pinMode(3,INPUT); //YELLOW
pinMode(5,INPUT); //RED
pinMode(7,INPUT); //BLUE


Serial.begin(9600);
Serial.println("0,0,0,0");
}

void loop() {
  // put your main code here, to run repeatedly:
  if (Serial.available() > 0) {
 int processing = Serial.read();

 Serial.print(digitalRead(2));
 Serial.print(",");
 delay(1);

 Serial.print(digitalRead(3));
 Serial.print(",");
 delay(1);

 Serial.print(digitalRead(5));
 Serial.print(",");
 delay(1);

 Serial.println(digitalRead(7));
 delay(1);

}
}