Author: Suman Ansari

Final Project Review

My final project was meant to be a very loosely based game on collecting water and watching a plant grow and having a real life replica of the plant mimicking this movement.

Initial screen:
The first stage:

The second stage:

Final screen:

The layout at the showcase:

After the user-testing I did in class and outside of it, I realized some fundamental problems with the game. Firstly, the motive of the game was not really clear to the people interacting with it until after they started the game; however, by that point it was pretty intuitive that they had to move the plant in real life to see the change on screen. Also, I felt as if labelling the project a ‘game’ implied that there was a way to win (getting a certain number of drops within a certain number of time); therefore, when there was no real indication if people won or loss, they kind of didn’t know what to do. This was an error on my part as I should’ve made a very clear winning/losing situation or just a win-win situation but with a more obvious ending. For this, I wanted a flower to grow to portray the final stage of growth but I didn’t know how I could implement this in the short amount of time I had considering I would have to change a lot of the code.

I think what did work was the implementation of handles on the pot, this was a clear indicator that people had to hold it from both sides. The arrows on the board it rested on, also helped. Moreover, once people did start, a lot of them appreciated the calmness they felt in doing the act and a lot go gasps of excitement were heard when the plant did grow on screen. A few people commented on the good color palette/’clean’ look of the whole experiment and I felt very proud of that because neatness is pretty important to me personally. Another viewer also commented how the movement of this game closely resembled activities that are used with patients that are exercising mobility of body parts and how this could extend to that application which I found very, very cool.

Personally, although a few aspects of the project did not end up looking like I had hoped they would, I have never felt more proud being present around people looking at my work and smiling/laughing because of it — it felt amazing. It especially felt great when Craig and Sarah told me I should be proud of myself and I was! — I worked on this code for 2 weeks straight (with help) but also did most of it by myself which I never thought I would be able to say about something that looked like this. I think that’s something that this course taught me, there’s a lot of stuff I CAN do, I just need to stop telling myself that I can’t. So overall, I know there’s a lot to be improved upon but I think I’m still pretty happy with the result. With special thanks to Aaron, Jack, Ali, Yousra, the UNIX lab and Daniel Shiffman.

Code:

Game

import processing.video.*;
import jp.nyatla.nyar4psg.*;
Capture cam;
MultiMarker nya;

Catcher catcher;    
Timer timer;  
Drop[] drops;    
//End[] ends = new End[10];;
int totalDrops = 0;
int numCaught = 0; 
PImage flowers;
PImage bFlowers;
PImage background;
PImage black;
//PImage end;
int level;
// 1: First plant stage
// 2: Second plant stage

int time;
String timeString = "00";
//String counter = "00";
int initialTime;
int interval = 1;
int totalTime = 60000;
float x=0;
int value = 32;
int intTimeString;

int gameScreen;
// 0: Initial Screen
// 1: Game Screen
// 2: Game-over Screen

void setup() { 
  fullScreen(P3D);
  //size(640,480,P3D);
  fill(0);
  background = loadImage("background.jpg");
  catcher = new Catcher(60); // Create the catcher with a radius of __
  drops = new Drop[2500]; 
  timer = new Timer(300); 
  flowers = loadImage("flowers.png");
  bFlowers = loadImage("bFlowers.png");
  //end = loadImage("end.png");
  initialTime = millis();

  //for (int = i; i < ends.length; i++) {
  //  ends[i] = new End(100+i*100, 300, random(32,72));
  //}
    
  cam=new Capture(this, 1440, 900);
  nya=new MultiMarker(this, 1440, 900, "camera_para.dat", NyAR4PsgConfig.CONFIG_PSG);

  nya.addNyIdMarker(0, 80);
  cam.start();
}

void draw() {  
  background(0);
  if (gameScreen == 0) {
    initScreen();
  } else if (gameScreen == 1) {
    if (cam.available() !=true) {
    }
    gameScreen();
    cam.read();
    nya.detect(cam);
    pushStyle();
    imageMode(CORNER);
    //nya.drawBackground(cam);
    popStyle();
    if ((!nya.isExist(0))) {
    } else {
      x += (nya.object2ScreenCoordSystem(0, 0, 0, 0).x-x)*.1;
      //println(x);
    }
    catcher.setLocation(width-x);
    catcher.display();
  } else if (gameScreen == 2) {
    gameOverScreen();
  }
}
void initScreen() {
  background(255);
  image(bFlowers, width/2, height/3);
  imageMode(CENTER);
  fill(50, 100, 150);
  textSize(18);
  textAlign(CENTER);
  text("MOVE THE PLANT SLOWLY TO COLLECT AS MANY WATER DROPS AS YOU CAN", width/2, height/2);
  text("PRESS SPACEBAR TO START", width/2, height/1.7);
}

void keyPressed() {
  if (value == 32) {
    startGame();
    time = 0;
  }
}
void startGame() {
  gameScreen=1;
}

void gameScreen() {
  imageMode(CORNER);
  image(background, 0, 0);
  catcher.display();

  if (timer.isFinished()) {
    drops[totalDrops] = new Drop();
    totalDrops ++ ;
    if (totalDrops >= drops.length) {
      totalDrops = 0;
    }
    timer.start();
  }

  for (int i = 0; i < totalDrops; i++ ) {
    drops[i].move();
    drops[i].display();
    if (catcher.intersect(drops[i])) {
      drops[i].caught();
      numCaught++;
    }
  }
  
  if (numCaught==15) {
    level = 1;
  } else if (numCaught==25) {
    level = 2;
  }

  if (millis() - initialTime > interval)
  {
    time += 1;
    timeString = nf(time, 2);
    initialTime = millis();
  }

  intTimeString = parseInt(timeString)/60;
  timeString = Integer.toString(intTimeString);
  text("TIME: " + timeString, width/9, height/8);
  textSize(22);
  if ((time>=totalTime) || (numCaught>=30)) { 
    gameOverScreen();
  }
}

void gameOverScreen() {
  cam.stop();
  background(255);
  fill(50, 100, 150);
  textAlign(CENTER);
  textSize(22);
  text("GAME OVER", width/2, height/2.3);
  textSize(20);
  text("PRESS THE SCREEN TO REPLAY", width/2, height/2);
  imageMode(CORNER);
   //or (int = i; i < ends.length; i++) 
  //ends[i].ascend();
  //ends[i].display();
  //ends[i].top();
}

void mousePressed() {
    gameScreen = 1;
    //loop();

Catcher class

class Catcher {
  float r;    // radius 
  color col;  // color
  float x, y; // location

  int w = 200;
  int h = 580 ;
  Catcher(float tempR) {
    r = tempR;
    col = color(50, 10, 10, 150);
    x = 0;
    y =  height - 200;
  }

  void setLocation(float tempX) {
    x = tempX;
  }

  void display() {
    stroke(0);
    fill(col);

    int startPoint = 200*level;

    PImage flower = flowers.get(startPoint, - height/3, w, h);
    image(flower, x, y);
    imageMode(CENTER);
  }
  
  

  // A function that returns true or false based on if the catcher intersects a raindrop
  boolean intersect(Drop d) {
    // Calculate distance
    float distance = dist(x, y + 120, d.x, d.y); 

    if (distance < r + d.r) { 
      return true;
    } else {
      return false;
    }
  }
}

Drops class

class Drop {
  float x, y;   // Variables for location of raindrop
  float speed;  // Speed of raindrop
  color c;
  float r;      // Radius of raindrop

  Drop() {
    r = 8;                   // All raindrops are the same size
    x = random(width);       // Start with a random x location
    y = -r*4;                // Start a little above the window
    speed = random(3, 6);    // Pick a random speed
    c = color(50, 100, 150); // Color
  }

  // Move the raindrop down
  void move() {
    // Increment by speed
    y += speed;
  }

  // Check if it hits the bottom
  boolean reachedBottom() {
    // If we go a little beyond the bottom
    if (y > height + r*4) { 
      return true;
    } else {
      return false;
    }
  }

  // Display the raindrop
  void display() {
    // Display the drop
    fill(c);
    noStroke();
    for (int i = 2; i < r; i++ ) {
      ellipse(x, y + i*4, i*2, i*2);
    }
  }

  // If the drop is caught
  void caught() {
    // Stop it from moving by setting speed equal to zero
    speed = 0; 
    // Set the location to somewhere way off-screen
    y = -1000;
  }
}

Timer class

class Timer {

  int savedTime; // When Timer started
  int totalTime; // How long Timer should last

  Timer(int tempTotalTime) {
    totalTime = tempTotalTime;
  }

  // Starting the timer
  void start() {
    // When the timer starts it stores the current time in milliseconds.
    savedTime = millis();
  }

  boolean isFinished() { 
    // Check how much time has passed
    int passedTime = millis()- savedTime;
    if (passedTime > totalTime) {
      return true;
    } else {
      return false;
    }
  }
  
  int getTime() {
    return millis()/600;
  }
}

A class of flowers I was considering ending at the end screen:

//class End {
  
//  float x, y; // location
//  float diameter;
  
//  Bubble (float tempX, float tempY, float tempD) {
    
//    x = tempX;
//    y = tempY;
//    diameter = tempD;
    
//  }
  
//  void ascend() {
    
//    y--;
//    x = x+random(-2, 2);
//  }
  
//  void display() {
//    imageMode(CENTER);
//    image(end, x, y, diameter, diameter);
    
//  }
  
//  void top() {
//    if (y < diameter/2) {
//      y = diameter/2;
      
//    }
//  }
  
//}

Fin.

So… I’m definitely not close to finishing my game and I know it’s missing a lot of things but I decided to test it out anyway so far.  All of my users said the same things which I kind of already knew but:

1). Although the collecting droplets is pretty intuitive, there should be an initial start screen with instructions/info and “press start”.

2). The timer should count down  instead of up (right now is confused as a score). So, could also include a score.

3). There needs to be a “try again” or “you win” kind of ending because people can’t actually tell how they’ve done.

So far for my final project, I worked on the software for the game. I made 3 distinct classes: one for the raindrops, one for a timer and one for the ‘catcher’. I also added a timer onto the screen; however, it is currently counting up and needs to do the opposite. I also worked out how to print out the values for each time a raindrop intersects with the ‘catcher’ which I can further use to build the levels of my game. I also found the images I would like to use in an array to portray the growing flower and need to code this next as opposed to the ‘catcher’.

Catcher catcher;    // One catcher object
Timer timer;        // One timer object
Drop[] drops;       // An array of drop objects
int totalDrops = 0;
int numCaught = 0; // Starting a count for the number of caught raindrops

PImage background;

void setup() {
  size(480, 270);
  background = loadImage("background.jpg"); // Putting in a background
  catcher = new Catcher(32); // Create the catcher with a radius of 32
  drops = new Drop[1000];    // Create 1000 spots in the array
  timer = new Timer(300);    // Create a timer that goes off every 300 milliseconds
  timer.start();             // Starting the timer
}

void draw() {
  background(255);
  // Set catcher location
  catcher.setLocation(mouseX, height); 
  // Display the catcher
  catcher.display(); 

  // Check the timer
  if (timer.isFinished()) {
    // Deal with raindrops
    // Initialize one drop
    drops[totalDrops] = new Drop();
    // Increment totalDrops
    totalDrops ++ ;
    // If we hit the end of the array
    if (totalDrops >= drops.length) {
      totalDrops = 0; // Start over
    }
    timer.start();
  }
  
{
  text(timer.getTime(), 420, 50);
  fill(30);
  textSize(30);

  // Move and display all drops
  for (int i = 0; i < totalDrops; i++ ) {
    drops[i].move();
    drops[i].display();
    if (catcher.intersect(drops[i])) {
      drops[i].caught();
      numCaught++;
      print(numCaught);
    }
  }
}
}

class Catcher {
  float r;    // radius
  color col;  // color
  float x, y; // location

  Catcher(float tempR) {
    r = tempR;
    col = color(50, 10, 10, 150);
    x = 0;
    y = 0;
  }

  void setLocation(float tempX, float tempY) {
    x = tempX;
    y = tempY;
  }

  void display() {
    stroke(0);
    fill(col);
    ellipse(x, y, r*2, r*2);
  }

  // A function that returns true or false based on
  // if the catcher intersects a raindrop
  boolean intersect(Drop d) {
    // Calculate distance
    float distance = dist(x, y, d.x, d.y); 

    // Compare distance to sum of radii
    if (distance < r + d.r) { 
      return true;
    } else {
      return false;
    }
  }
}

class Drop {
  float x, y;   // Variables for location of raindrop
  float speed;  // Speed of raindrop
  color c;
  float r;      // Radius of raindrop

  Drop() {
    r = 8;                   // All raindrops are the same size
    x = random(width);       // Start with a random x location
    y = -r*4;                // Start a little above the window
    speed = random(1, 5);    // Pick a random speed
    c = color(50, 100, 150); // Color
  }

  // Move the raindrop down
  void move() {
    // Increment by speed
    y += speed;
  }

  // Check if it hits the bottom
  boolean reachedBottom() {
    // If we go a little beyond the bottom
    if (y > height + r*4) { 
      return true;
    } else {
      return false;
    }
  }

  // Display the raindrop
  void display() {
    // Display the drop
    fill(c);
    noStroke();
    for (int i = 2; i < r; i++ ) {
      ellipse(x, y + i*4, i*2, i*2);
    }
  }

  // If the drop is caught
  void caught() {
    // Stop it from moving by setting speed equal to zero
    speed = 0; 
    // Set the location to somewhere way off-screen
    y = -1000;
  }
}

class Drop {
  float x, y;   // Variables for location of raindrop
  float speed;  // Speed of raindrop
  color c;
  float r;      // Radius of raindrop

  Drop() {
    r = 8;                   // All raindrops are the same size
    x = random(width);       // Start with a random x location
    y = -r*4;                // Start a little above the window
    speed = random(1, 5);    // Pick a random speed
    c = color(50, 100, 150); // Color
  }

  // Move the raindrop down
  void move() {
    // Increment by speed
    y += speed;
  }

  // Check if it hits the bottom
  boolean reachedBottom() {
    // If we go a little beyond the bottom
    if (y > height + r*4) { 
      return true;
    } else {
      return false;
    }
  }

  // Display the raindrop
  void display() {
    // Display the drop
    fill(c);
    noStroke();
    for (int i = 2; i < r; i++ ) {
      ellipse(x, y + i*4, i*2, i*2);
    }
  }

  // If the drop is caught
  void caught() {
    // Stop it from moving by setting speed equal to zero
    speed = 0; 
    // Set the location to somewhere way off-screen
    y = -1000;
  }
}

class Timer {

  int savedTime; // When Timer started
  int totalTime; // How long Timer should last

  Timer(int tempTotalTime) {
    totalTime = tempTotalTime;
  }

  // Starting the timer
  void start() {
    // When the timer starts it stores the current time in milliseconds.
    savedTime = millis();
  }

  // The function isFinished() returns true if 5,000 ms have passed. 
  // The work of the timer is farmed out to this method.
  boolean isFinished() { 
    // Check how much time has passed
    int passedTime = millis()- savedTime;
    if (passedTime > totalTime) {
      return true;
    } else {
      return false;
    }
  }
  
  int getTime() {
    return millis()/600;
  }
}

Next, I’m going to be working on using the IR camera and learning how to program it with the interaction I would like.

Computer Vision for Artists and Designers: Pedagogic Tools and Techniques for Novice Programmers

This week’s reading introduced us to “computer vision” which is a method that allows computers to make use of digital input e.g. videos and images and make inferences. Since the new wave of digital art has come forth, this method of work allows a lot of room for creativity for artists to work with. The 4 ways mentioned in the article about how computer vision can work is: 1). Detecting motion: movements of people  within the video frame can be detected and quantified using frame differencing. 2). Detecting presence: background subtraction. 3). Detection through brightness thresholding: objects of interest can be distinguished based on their brightness in a threshold value. 4). Simple object tracking: finds the location of the single brightest pixel in every fresh frame of video.

For my final project, this text really encouraged me to not forget about the physical components of my work as I may get too involved in software and forget about the other constraints. The physical aspects of the project are just as important as the software; therefore, it reminded me to work out which software techniques are going to be best compatible with the available physical conditions. As I am hoping to use IR camera and simple object tracking, I learned that “using IR significantly improves the signal-to-noise ratio of video captured in low-light circumstances”, since light is critical, I need to make sure I test my project out beforehand in the location of the exhibition.

In class, I played around with the 3D effect on processing. I also worked on a sketch that changed the brightness of the image as it zoomed into the pixels; however, I didn’t save it correctly so can’t find the record :(.

After more research into my project and from the feedback I got in class, I would like to create a game where through Processing, on the screen, there appears rain drops, an empty pot and a timer. Outside of the screen, the player will also be holding a similar looking pot that has an IR LED attached to it and below the screen will be an IR camera. I would like the IR to detect the persons movement on an x-axis/horizontal movement so that the pot on screen appears to move with the person. The goal of the game is to collect as many water droplets in a specified amount of time to grow your plant – this will happen in approximately 4 stages, e.g. 15 droplets will produce a sprout, 25 will produce a flower bud and so on.

Equipment and construction needs:

• IR camera and LED
• Screen
• Plant pot with IR LED

Programming:

• Program a timer that starts and stops
• Raindrops that fall across the screen but stop once they ‘enter’ the pot
• The pot itself that will move across the x-axis and also its changing states of growth

3 most difficult, complicated, frightening aspects:

• Being a novice at coding, my first and biggest fear is the software coding aspect of the project (and that basically sounds like the whole thing but) I have not yet worked at all with IR sensors and processing so I know I will find this very difficult. I’m not sure how aligning the visuals with the IR will work yet.
• Making the transitions between each phase of the plant growth seem natural.
• Making the droplets fall into the ‘pot’ for a desired number and restarting the game once at full capacity etc.

For my final project, I wanted to do a sort of continuation of my midterm project (the Sunflower); however, this time I would have used a real plant with a moisture sensor and other data sensors that I could then analyze on a screen using processing (AKA to know when to water the plant etc.). However, I didn’t think this involved much interaction (or the interaction was just too long term).

So then, I thought about creating an interactive game, a form of Dance Dance Revolution but for the hand. Again, I liked this concept a lot because the interaction is more clear but it lacked creativity.

So I’m considering combining the two and creating a game where the player has to move the plant on screen (using tangible buttons) and collect water droplets. Hopefully, as they collect more water, the more the plant grows. I’m not yet sure I want to do this 100% though. For this, I would need my Arduino, buttons (arrows), wiring and a box of some sort. The rest would be coded into Processing/Arduino directly.

Personally, this class has opened my eyes to a whole new world. Before this, computing (and I think mostly programming) was a very distant concept to me. It was not something that I thought could add anything to my life.  However, after doing all the work and especially the readings that we have done this semester, I truly think that computing is one of the most important things in the world and affects many aspects of my and other people’s lives that I had never considered.

I think computing has brought the world a lot closer together by improving communication – be it, between individuals through machines, between individuals and machines or machines on machines – the capacities of these have greatly increased innovations and have revolutionized in how the world responds to issues. I’m not yet sure how computing could make me a better person specifically but it has definitely taught me patience and perseverance – to not give up when something goes wrong but instead, to keep at it until it works (or ask for help!). It has also taught me to have a lot more respect for those that are in the computational field as I understand how things that appear simple on the surface are actually more complicated than you think. I know that this is not necessarily my field of interest and is actually very far from my comfort zone, but I hope I continue to learn more and enhance my skills in this area because I really have enjoyed pushing myself.

I combined Processing and Arduino in this example by getting an RGB LED to change colors according to the pixels on the screen (using processing).

The codes:

int redPin   = 9;   
int greenPin = 10;
int bluePin  = 11;  

long int inByte;

void setup()
{
  pinMode(redPin,   OUTPUT);  
  pinMode(greenPin, OUTPUT);
  pinMode(bluePin,  OUTPUT);

  Serial.begin(9600);
}


void outputColour(int red, int green, int blue) {
  analogWrite(redPin, red);
  analogWrite(bluePin, blue);
  analogWrite(greenPin, green);
}


int* getColour() {
  int* colour;
  int i;

  i = 0;

  while (i < 4)
  {
    if (Serial.available() > 0) {
      colour[i] = Serial.read();
      i++;
    }
  }

  return colour;
}

void loop()
{
  if (Serial.available() > 0) {

    inByte = Serial.read();

    if (inByte == 'C') {
      int* one;
      one =  getColour();

      outputColour(one[1], one[2], one[3]);

    }
  }

  delay(10);
}

import processing.serial.*;

PImage img;  
Serial myPort;

void setup() {
  //set these to the size of the image

  size(512, 512);
  // printArray(Serial.list());
  // String portname=Serial.list()[1];
  // println(portname);

  img = loadImage("pixels.jpg");
  myPort = new Serial(this, Serial.list()[1], 9600);
}

void draw() {
  background(0);
  image(img, 0, 0);
  img.loadPixels();
}

void mousePressed() 
{
  myPort.write("CL");
  myPort.write(int(red(img.pixels[mouseX+mouseY*img.width])));
  myPort.write(int(green(img.pixels[mouseX+mouseY*img.width]))); 
  myPort.write(int(blue(img.pixels[mouseX+mouseY*img.width])));
}

I got help using the following tutorials:

https://www.hackster.io/hardikrathod/control-arduino-using-gui-arduino-processing-2c9c6c

https://www.instructables.com/id/RGBs-with-Arduino-and-Processing/

For this project, I kept it pretty simple as I tried to get my head around how to split string arrays. I basically wrote down a bunch of words that were in my head throughout this week and the text kinda follows the pattern around the screen like the thoughts do in my brain? If that makes sense. I think there’s a lot of ways I can develop this further and with better aims e.g. I feel like similar typography/graphics is used in a lot of advertisements when actors’ thoughts crowd the screen without them actually saying anything etc. It would be cool to develop this more aesthetically as well.

int i = 0;
String quote = "food mom dubai midterms exams stress game of thrones chocolate stress endgame cinema work friends laugh chicken deadlines ramadan eid grandma weather statistics snapchat wedding";
String [] myArray = split (quote, " ");
PFont f;

void setup() {
  size(800, 600);
  background(0);
  f = createFont("FreeSansBold.ttf", 32);
  textFont(f);
  println();
}

void draw() {

  i++; 
  if (i > (myArray.length-1)) {
  i = 0;
 
}

  fill(187, 178, 255, 55); 
  translate(mouseX, mouseY);
  text(myArray[i], 0, 0);
  delay(100); 
}