Month: April 2019

I have always been fascinated with using nature as a medium for interaction and a source for new creation, especially in the area of music. And, for the final project, I want to focus on using water as a medium to create music and visualize the flow and beauty created with manipulating the dynamic features of water. Ripples and waves are often easily created by an external force, but I wanted to insert something more into those movements that can be appealing both visually and audibly.

In terms of technical side, the project entails the use of a projector, a kinect, a container to contain shallow level of water, and a stand that will hold the project and kinect in place above the container.

The ideal instance of a user interaction would be where users would play around with the water on the sides of the container, creating different waves and ripples on the surface. The kinect will capture the depth of the water, and color the points that are at higher level than the base level of the surface, which will be projected on the water through the projector.

Meanwhile, depending on the location of the ripples/waves, either a recorded or a set of notes will be played in a loop. The more waves created will lead the music either playing faster or at a higher pitch.

The area of projection and detection for the projector and kinect will be smaller than the area of the container (as seen in the figure below) in order to reduce the instances of where hands are detected as waves or ripples.

The expected size of the board would be 120cm x 120cm x 20cm. I would still need to calculate and see the level of the support for the projector and the kinect, but the area that the kinect and the projector will cover the area for 100cm x 100cm. I will mainly be using Processing for the program, in attention to the sound library and the library for connecting kinect with processing.

Four most difficult, complicated or frightening aspects:

1. Detecting water waves/ripples using Kinect
2. Playing sound depending on the location of particles/points on a certain dimension
3. Changing the pitch of the sound depending on the number of points in a certain area
4. Aligning the Kinect with the projector in terms of visuals

As a conceptual focus for our Final Project, Jana and I are considering the complex social fabric of our university. Namely, the heavy focus on our perceptions of perfection and our ideas surrounding productivity. Vox Media created a video essay on the nature of our perceptions of productivity, describing how we often consider forms of self-care as precursors and necessary acts for the sake of our own professional, academic and all-around utilitarian productivity. This notion is frequently reproduced at NYUAD as we witness many slouched students scattered around our campus stressing about the next assignment due, or whether they have a worthwhile internship lined up for summer.

As a conceptual attendance sheet, our project would be placed one of the entrances to the exhibit, asking visitors to input data pertaining to their relationship to leisure, self-care, and perceptions and indicators of productivity.

As a way of measuring these relationships, we collect the following data from visitors: level of stress (pressure sensor), hours of sleep (typed input). We also want to understand the current state of mind an individual is experiencing upon entry to the show. To determine this, we will use Paul Ekman’s five foundational emotions (joy, sadness, disgust, fear, anger). Here we will use a keypad or coloured buttons.

 

Libraries; sound, minim.

This data, after testing, will be visualized into a live animation. The three biggest fears with anticipating are

1) coding simple data into a visually compelling form;

2) creating reliable and durable forms of input;

3) collecting cumulative data that relies on user involvement.

 

Classwork:

PImage source;
int increment=10;
float diam;
float z;
void setup() {
  size(512, 512, P3D);
  source = loadImage("baboon.png");
  noStroke();
}

void draw() {
  background(0);
  rotateY (0.5);
  source.loadPixels();
  for (int y=0; y<height; y+=increment) {
    for (int x=0; x<width; x+=increment) {
      int loc = x+(y*width);
      color pix=source.pixels[loc];
      diam = map(brightness(pix), 0, 255, 1, 5)*map(mouseX, 1, width, 10, 30);
      pushMatrix();
      translate (x, y, diam);
      fill(pix);
      box(diam/5);
      popMatrix();
    }
  }
  source.updatePixels();
  
}

 

 

Notes on Final Project:

A toggle button to see the path of movement of the objects in space, as opposed to the objects themselves. Also between showing a transparent fill, or just stroke.

Add noise to the movement path.

With this weekly assignment, I attempt to create a Digital version of the analogue crossbow that I designed for my midterm, using the same 2 potentiometer, 1 button remote controller.

I spent quite a while fiddling with the code in the original handshake example, trying to get it to input data from multiple sensors connected to my Arduino. That being said, when I found the sketch that gave me the syntax to do it properly, I had most of the inputs properly formatted.

I ended up having to Add a third input to the code (for my button). Basically it would register pressed as a 1 and unpressed as a 0, then send it into the processing sketch, where I could make an if statement in order to draw a mark.

void setup() {
  Serial.begin(9600);
  Serial.println("0,0,0");
  pinMode(3, OUTPUT);
  pinMode(5, OUTPUT);
  pinMode(trigPin, INPUT);
}

void loop() {
  while (Serial.available()) {
    right = Serial.parseInt();
    left = Serial.parseInt();

  
  int trigRead = digitalRead(trigPin);
  if (trigRead == HIGH) {
    fire = 1;
  }
  else {
    fire = 0;
  }
//  Serial.print(trigRead);

  if (Serial.read() == '\n') {
    digitalWrite(3, right);
    digitalWrite(5, left);
    int sensor = analogRead(A0);
    delay(0);
    int sensor2 = analogRead(A1);
    delay(0);
    Serial.print(sensor);
    Serial.print(',');
    Serial.print(sensor2);
    Serial.print(',');
    Serial.println(fire);

  }
}

}

import processing.serial.*;
Serial myPort;
int xPos=0;
int yPos=0;
int button=0;
//boolean Button = false;
boolean onOff=false;
boolean onOff2=false;
//Array[] marks;
//Mark mark;

void setup() {
  size(960, 720);
  printArray(Serial.list());
  String portname=Serial.list()[1];
  println(portname);
  myPort = new Serial(this, portname, 9600);
  myPort.clear();
  myPort.bufferUntil('\n');
}

void draw() {
  //background(255);
  drawTarget();
  drawCross();

  if (button == 1) {
    drawMark();
   //marks.add(new Mark);
  } 
 
  
  //if (keyPressed){
  //  drawTarget();
  //}

}

void serialEvent(Serial myPort) {
  String s=myPort.readStringUntil('\n');
  //String t=myPort.readStringUntil('\n');
  s=trim(s);
  //t=trim(t);
  if (s!=null) {
    println(s);
    int values[] = int (split(s, ','));
    if (values.length ==3) {
      xPos=(int)map(values[0], 0, 1023, 0, width);
      yPos=(int)map(values[1], 0, 1023, 0, height);
      button= (int) values[2];
    }
  }



  myPort.write(int(onOff)+","+int(onOff2)+"\n");
}

void drawMark() {
  noStroke();
  fill(20, 15, 0);
  ellipse (xPos,yPos, 10, 10);
}


void drawCross() {

  fill(0, 255, 150);
  noStroke();
  rectMode(CENTER);
  rect(xPos, yPos, 3, 30);
  rect(xPos, yPos, 30, 3);
  rect(xPos-20, yPos, 3, 15);
  rect(xPos+20, yPos, 3, 15);
  rect(xPos, yPos-20, 15, 3);
  rect(xPos, yPos+20, 15, 3);
}

void drawTarget() {
  fill(200, 118, 70, 240);
  noStroke();
  rectMode(CENTER);
  rect(width/2, height/2, width, height);
  fill(240);
  rect(width/2, height/2, 400, 600);
  fill(255, 255, 0, 150);
  rect(width/2, height/2, 300, 450);
  fill(240);
  rect(width/2, height/2, 200, 300);
  fill(255, 255, 0, 150);
  rect(width/2, height/2, 100, 150);

  fill(240);
  rect(width*.17, height*.7, 150, 225);
  fill(255, 255, 0, 150);
  rect(width*.17, height*.7, 100, 150);
  fill(240);
  rect(width*.17, height*.7, 50, 75);

  fill(240);
  rect(width*.83, height*.7, 150, 225);
  fill(255, 255, 0, 150);
  rect(width*.83, height*.7, 100, 150);
  fill(240);
  rect(width*.83, height*.7, 50, 75);

  fill(240);
  rect(width*.17, height*.3, 150, 225);
  fill(255, 255, 0, 150);
  rect(width*.17, height*.3, 100, 150);
  fill(240);
  rect(width*.17, height*.3, 50, 75);

  fill(240);
  rect(width*.83, height*.3, 150, 225);
  fill(255, 255, 0, 150);
  rect(width*.83, height*.3, 100, 150);
  fill(240);
  rect(width*.83, height*.3, 50, 75);
}

At the end, I attempted to make an array list in order to save points on the target that I had shot, but could not figure out how to make it work with my code.

Thanks for reading!

As an initial idea for my final project, I wanted to create an interactive art installation that would act as a parallel universe to the audience’s body. Conceptually speaking, I am intrigued by the prospect of combining technology and computing with visual arts and performance. The project also acts as an exploration regarding certain dynamics – relationship between colors, space, and the audience to each other. This could also be done through a change in graphics and patterns when to people experience the installation together. The main dynamic of the project would be the camera detecting the movement of the user and then generating colorful geometric patterns to follow the movement of the body. Each section of the projection should display a different color if the user stands in front of that specific section.

I’ll need:

• 1 or 2 Kinects
• 1 projector
• 5 silk screens

I keep finding myself wanting to press “command /” to comment out a part of an essay, or some thought I had, to save it for later. I have spent so much of my life actively avoiding learning how to program, it has been an interesting experience, finally getting into it. When working on a project, it feels like there is always some part of my mind trying to process what is happening in the code I am working, even when I am doing something else. In this way it has really become an interesting part of my life, that I never thought it would.

I feel Like I have improved my coding skills significantly, but with every weekly project, I feel like I end up kicking my own ass. Just one little conceptual misunderstanding can throw me off and send me into a downward spiral of confusion for hours. It is like trying to fly a plane without knowing what all the buttons do.

In that way, I really hate how limited i feel. It was much easier for me to conceptually understand physical things at the beginning of the semester, like circuitry and mechanical systems. Sometimes when I rely on just programming, I feel like I am sliding backwards down a slide, and end up feeling stupider than the day before.

I really want to keep improving my skills, but it is hard on my conscience and confidence to always be working just above the range of my ability.

Still, I am fascinated by computing. I wouldn’t keep working so hard at leaning it if I wasn’t. It helps me to see the world a little differently. I am really grateful that I stepped out of my comfort zone.

For this week, I thought it would be really cool to make my snake game more advanced by using a joystick to control the movement using both Processing and Arduino. For this I used 4 male and 4 female cables and a joystick. I faced different challenges such as the arduino not transferring to processing or only two sides of the joystick working and it was more challenging than I initially thought, but it I got it working in the end!

Arduino:

void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);
  Serial.println("0,0");
}

void loop() {
  if(Serial.available()>0){
    char inByte=Serial.read();
    int sensor = analogRead(A0);
    delay(0);
    int sensor2 = analogRead(A1);
    delay(0);
    Serial.print(sensor);
    Serial.print(',');
    Serial.println(sensor2);
  }
}

Processing:

int grid = 20; //How big each grid square will be
import processing.serial.*;
PVector food;
int speed = 10;
boolean dead = true;
int highscore = 0;
Snake snake;
Serial myPort;
int xPos=0;
int yPos=0;

void setup() {
  size(500, 500);
  snake = new Snake();
  food = new PVector();
  newFood();
    String portname=Serial.list()[2];
  println(portname);
  myPort = new Serial(this,portname,9600);
  //myPort.clear();
  //myPort.bufferUntil('\n');
}

void draw() {
  background(0,0,50);
  fill(200,200,200);
  if (!dead) {
    
    if (frameCount % speed == 0) {
      snake.update(xPos,yPos);
    }
    snake.show();
    snake.eat();
    fill(200,50,100);
    rect(food.x, food.y, grid, grid);
    textAlign(LEFT);
    textSize(15);
    fill(255);
    text("Score: " + snake.len, 10, 20);
  } else {
    textSize(25);
    textAlign(CENTER, CENTER);
    text("Alien snake Game\nAre you up for the challenge?\nClick to start" + "\nHighscore: " + highscore, width/2, height/2);
  }
}

void newFood() {
  food.x = floor(random(width));
  food.y = floor(random(height));
  fill( random(255), random(255), random(255), random(255));
  food.x = floor(food.x/grid) * grid;
  food.y = floor(food.y/grid) * grid;
}

void mousePressed() {
  if (dead) {
    snake = new Snake();
    newFood();
    speed = 10;
    dead = false;
  }
}

void serialEvent(Serial myPort){
  String s=myPort.readStringUntil('\n');
  s=trim(s);
  if (s!=null){
    int values[]=int(split(s,','));
    if (values.length==2){
      xPos=(int)map(values[0],0,1023,0, width);
      yPos=(int)map(values[1],0,1023,0, height);
    }
  }
  println(xPos+ " "+yPos);
  myPort.write('0');
}

class Snake {
  PVector pos;
  PVector vel;
  ArrayList<PVector> hist;
  int len;
  int moveX = 0;
  int moveY = 0;

  Snake() {
    pos = new PVector(0, 0);
    vel = new PVector();
    hist = new ArrayList<PVector>();
    len = 0;
  }

  void update(int x, int y) {

    if (x>300 && y<300 && y>200 && snake.moveX != 2) {
      snake.vel.x = -1;
      snake.vel.y = 0;
    } else if (x<200 && y<300 && y>200 && snake.moveX != -1) {
      snake.vel.x = 1;
      snake.vel.y = 0;
    } else if (x<300 && x>200 && y<200 && snake.moveY != 2) {
      snake.vel.y = -1;
      snake.vel.x = 0;
    } else if (x<300 && x>200 && y>300 && snake.moveY != -1) {
      snake.vel.y = 1;
      snake.vel.x = 0;
    }

    hist.add(pos.copy());
    pos.x += vel.x*grid;
    pos.y += vel.y*grid;
    moveX = int(vel.x);
    moveY = int(vel.y);

    pos.x = (pos.x + width) % width;
    pos.y = (pos.y + height) % height;

    if (hist.size() > len) {
      hist.remove(0);
    }

    for (PVector p : hist) {
      if (p.x == pos.x && p.y == pos.y) {
        dead = true;
        if (len > highscore) highscore = len;
      }
    }
  }

  void eat() {
    if (pos.x == food.x && pos.y == food.y) {
      len++;
      if (speed > 5) speed--;
      newFood();
    }
  }

  void show() {
    noStroke();
    fill( random(255), random(255), random(255), random(255));
    ;
    rect(pos.x, pos.y, grid, grid);
    for (PVector p : hist) {
      rect(p.x, p.y, grid, grid);
    }
  }
}
void keyPressed() {
  if (keyCode == LEFT && snake.moveX != 1) {
    snake.vel.x = -1;
    snake.vel.y = 0;
  } else if (keyCode == RIGHT && snake.moveX != -1) {
    snake.vel.x = 1;
    snake.vel.y = 0;
  } else if (keyCode == UP && snake.moveY != 1) {
    snake.vel.y = -1;
    snake.vel.x = 0;
  } else if (keyCode == DOWN && snake.moveY != -1) {
    snake.vel.y = 1;
    snake.vel.x = 0;
  }
}

 

Physical computing has been one of the primary technique that I has changed the way I see and my responses to the world. As an interactive media and computer science majors, I have realized that “software” is not the only method to present ideas to tackle rising issues, and physical computing allowed me to bring out these ideas to the real world. It may not necessarily make me a good person with physical computing at this point, but it is definitely providing me the path to become one as I develop and dive further into physical computing world. It is opening up possibilities and now I just have to create them!

It’s a “Hello World” in Physical Computing Now!

After reading Tega Brain’s “The Environment is Not a System”, I wanted to focus on the idea of portraying the nature and the environment, where human intervention can be a trigger to the destruction of nature.

The project entails a camera and processing to provide an interactive experience that shows the following idea. The general outline of an audience’s body will be captured through the camera, and the processing code will create growing plants on the outline. However, depending on the proximity of the audience, the stages of the plants will change (which is shown on the drawing above). As the audience gets closer to the camera, the plants will wither away – showing how some of our nature environment is good for us to stay away from it, especially if we want to protect it.

Concept of the Game:

3D videogame controls an iron man glove and a button to shoot, obstacles will be shown flying at the user and the user would have to aim and destroy the obstacle.

Materials Needed:

• glove with Ultrasonic or Infrared sensors
• button controller
•  Environment made in processing
• Box Surrounding the user to detect the distance.

Procedure:

1. We will create a glove similar to the iron man glove with infrared sensors and ultrasonic light that will detect the distance from the hand to the box that will surround the user.
2.  The values gathered by the sensors will cause movement in an aiming box in the processing environment
3. If the aiming box is at the same location as the obstacle and the user shoot,  a laser beam will be shown in the screen and the obstacle will be destroyed
4. The player will have one minute to destroy as many obstacles as possible. At the end of the game, we will show a scoreboard with the highest scores.

 

Creditst to the following videos and avengers Endgame for giving us the inspiration to this videogame.

 

 

 

I decided to create a physical controller for my simple version of Flappy Bird, that I coded for the OOP assignment. I tried to code a button which will control the movement of the object in the game, and with some help from Rick, I was able FINALLY figure it out! I set up a button state and used it in the draw function, since I have three states for the game – pressing the button will change the button state from 0 to 1 then change the state of the game also from 0 to 1.

Arduino Code:

int buttonPin = 2;
int button;

void setup() {
  Serial.begin(9600);
  Serial.println('0');
  pinMode(buttonPin, INPUT);
}
 
void loop() {
    char inByte=Serial.read();
    button = digitalRead(buttonPin);
    delay(0);
    Serial.println(button);
}

Processing Code:

import processing.serial.*;
Serial myPort;
int state = 0; 
int score=0;
boolean currentState = false;
boolean prevState = false;
int arduinoButton = 0;
bird b = new bird();
pillar[] p = new pillar[3];


void setup(){
 size(500,700);
 
 printArray(Serial.list());
  String portname=Serial.list()[7];
  println(portname);
  myPort = new Serial(this,portname,9600);
  myPort.clear();
  myPort.bufferUntil('\n');
  
 int i; 
 for (i = 0; i < 3; i++){
   p[i]=new pillar(i);
 };
}

void draw(){
  background(0);
  if (arduinoButton == 1 && prevState == false) {
    currentState = true;
    prevState = true;
    state = 1;
  }
  
  if (arduinoButton == 1) {
    b.jump();
  }
  
  if (state==0){
    // intro state
    text("Click to Play",155,240);
    b.move(p);
  }
  
  else if (state==1){
    // start state 
    b.move(p);
    b.drag();
    b.checkCollisions();
    rect(20,20,100,50);
    fill(255);
    text(score,30,58);
  }
  
  else {
    // end state 
    rect(150,100,200,50);
    rect(150,200,200,50);
    fill(255);
    text("game over",170,140);
    text("score",180,240);
    text(score,280,240);
  }
  
  b.drawBird();
  
  for(int i = 0;i<3;i++){
    p[i].drawPillar();
    p[i].checkPosition();
  }
  
  stroke(255);
  textSize(32);
}
void mousePressed(){
  state = 1;
} 

void serialEvent(Serial myPort){
  String s=myPort.readStringUntil('\n');
  s=trim(s);
  if (s!=null){
    arduinoButton = int(s);
  }
  println(arduinoButton);
}