Final project Progress: Fatema Nassar, Ingy El Sheikh

Idea:

Basically, our idea was to imitate a sketchbook.The user gets to draw with their hands. There are 5 different generative art pieces, hence five different ‘brushes’ that the user can draw with.The user can choose which brush to use, by pushing  the button corresponding to each brush connect to. arduino.We also decided to incorporate an element of allowing the user to choose the color they want to draw with using a color gradient on processing, that we connected to Arduino through a slide potentiometer. The user can reset the drawing, showing them a blank sketch so they. start the drawing again. We also, wanted to incorporate allowing the user to save the drawing they made to make it more personal to each individual, but we haven’t done that yet.

Process:

This is the order we chose to complete the project in:

  1. First of all, we finalized the idea since we were not working together initially. We decided on everything we needed and the process we would complete the project in.
  2. The brushes were complete (the 5 generative art pieces that play as the brushes)
  3.  started working on the fiducial QR code (which was a pain !) in order to allow the user to use their hands to draw on the sketch, we decided we will have a glove with the QR stuck to it and the user will wear the glove to draw, luckily we were able to get it to work. Simultaneously we were working on the arduino side of things, completing first the color gradient. The color gradient on processing is actually an image and the get(); function was used to get the color at the position of the rectangle that was also mapped and connected to arduino through the slide potentiometer.
  4. Completed the the creation of the box using the laser cutter ! as well as finishing connecting the arcade buttons on arduino and creating code to connect it to processing, so that each button corresponds to a different brush.
  5. We then finished assembling the box and creating the reset button (to return the sketch back to being blank again). We decided to create a box, so that the arduino doesn’t show as it is not very visually pleasing, and in order to ensure that nothing moves around as we are using a lot of wires that may move around.
  6. We conducted user testing. 

Difficulties:

First of all, the QR code was very difficult to get to work, but after so many days of trying to figure it out we were able to get it work. On the arduino side of things, there were so many connection on the arduino, and so much code that sometimes, they were glitches and loose wires which was very frustrating.  But after several tests and removing the wires and re attaching them , we were able to get everything working well.

User Testing:

User #1:

User #2:

Feedback& next steps:

The feedback for both users was. generally the same:

  1. The camera is flipped, hence when the user moves their hand to the left the drawing draws to the right. We are aware of that and it is our next step.
  2. The QR code sometimes does not get picked up, so the drawing stops, this is because the user is holding the QR code as just a paper now,  so. when they hold it wrong on bend it backwards the camera doesn’t pick it up. In order to fix that we will stick it on a glove that the user will wear, so the qr is always the right side up, and straight, hence it will be fixed to the camera will always pick it up.
  3. Lastly, the users wanted us to incorporate adding the names of the brushes to the box so they know which brush corresponds to which button. However, we are a bit skeptical about doing so because we feel like we would be giving to much instruction on what the buttons do. We will take more opinions on this matter.

processing code:

import processing.video.*;
import jp.nyatla.nyar4psg.*;
import processing.serial.*;
import java.util.*;

int n=1, mode=1;
PVector mouseloc = new PVector(0, 0);
PVector p;

Capture cam;
MultiMarker nya;

Brush myBrush = new Brush(1, 2, mouseloc);


// serial stuff
Serial myPort;
int xPos=0;
int yPos;
boolean onOff=false;
boolean onOff2=false;
PImage myImage;
color pix;

int [] val = {0, 0, 0, 0, 0, 0, 0, 0};

void setup() {
  size(1280, 720, P3D);
  colorMode(RGB, 100);
  //fullScreen(P3D);

  printArray(Serial.list());
  String portname=Serial.list()[0];
  println(portname);
  myPort = new Serial(this, portname, 9600);
  myPort.clear();
  myPort.bufferUntil('\n');
  myImage  = loadImage("color.jpeg");

  printArray(Capture.list());
  cam=new Capture(this, 1280, 720);
  nya=new MultiMarker(this, width, height, "data/camera_para.dat", NyAR4PsgConfig.CONFIG_PSG);
  nya.addARMarker("data/patt.hiro", 40);//id=0
  cam.start();

  background(0);
  loadPixels();
}

int c=0;

void draw() {

  c++;
  if (cam.available() !=true) {
    return;
  }

  pushMatrix();
  cam.read();
  scale(-1, 1);
  //image(cam, -cam.width, 0);
  translate(width, 0);
  nya.detect(cam);
  nya.drawBackground(cam);
  popMatrix();




  if ((!nya.isExist(0))) {
    return;
  }
  p=nya.screen2ObjectCoordSystem(0, width/2, height/2);
  updatePixels();

  if (mode==1) {
    drawScreen();
  }
  // nya.endTransform();

  if (mode==2) {
    // save("pic"+String.valueOf(n++)+".tif");
    // delay(10);
    // takeScreenShot();
    // delay(10);
  }
}


void drawScreen() {
  //brush color picker
  imageMode(CENTER);
  fill(0);
  rectMode(CENTER);
  noStroke();
  rect(myImage.width/2, myImage.height/2, myImage.width, myImage.height*1.5);
  image(myImage, myImage.width/2, myImage.height/2, myImage.width, myImage.height);
  color pix = get(xPos, myImage.height/2);
  fill(pix);
  stroke(0);
  rect(xPos, myImage.height/2, 10, myImage.height*1.5);

  buttonPressed();
  myBrush.setCol(pix);

  mouseloc.x = p.x;
  mouseloc.y = p.x;
  myBrush.draww(mouseloc);
  loadPixels();
}

void buttonPressed() {
  if (val[0] == 1 ) {
    myBrush.setbrush(1);
  } else if (val[1] == 1 ) {
    myBrush.setbrush(2);
  } else if (val[2]==1 ) {
    myBrush.setbrush(3);
  } else if (val[3]==1 ) {
    myBrush.setbrush(4);
  } else if (val[4]==1 ) {
    myBrush.setbrush(5);
  }
  // reset
  else if (val[5]==1 ) {
    println("bla");
    background(0);
    loadPixels();
  }
}

void serialEvent(Serial myPort) {
  String s=myPort.readStringUntil('\n');
  s=trim(s);
  if (s!=null) {
    println(s);
    int values[]=int(split(s, ','));
    if (values.length==9) {
      xPos=(int)map(values[0], 89, 912, 0, myImage.width);

      //yPos=(int)map(values[1],0,1023,0, height);
      //yPos=height/2-230;
      //myBrush.setbrush(2);

      val[0] = (int)values[1];  // read it and store it in val[]
      val[1] = (int)values[2];
      val[2] = (int)values[3];
      val[3] = (int)values[4];
      val[4] = (int)values[5];
      val[5] = (int)values[6];//reset
      val[6] = (int)values[7];//draw
      val[7] = (int)values[8];//saveScreen
    }
    myPort.write(int(onOff)+","+int(onOff2)+"\n");
  }
}
class Brush {

  int brushN;

  float stepSize, linelength, diam;
  float angle;
  float x, y, tempX, tempY;
  color col;
  //
  int counter;
  PFont font;
  float fontSize;
  String letters;
  float  fontSizeMin;
  //

  //linemoduelw
  PShape lineModule;

  float [] numX={ -100, -70, 0, 70, 100, 70, 0, -70};
  float [] numY={ 0, 70, 100, 70, 0, -70, -100, -70};

  Brush(int defaultt, float step_size, PVector mouse) {
    // 1 blur
    // 2 random cylinder
    // 3 dynamic shapes
    // 4 text lines
    // 5 rotating lines
    brushN = defaultt;
    stepSize = step_size;
    angle = 270;
    x = mouseX;
    y = mouse.y;
    col = color(255);
    linelength = 100;
    diam = 2;
    fontSizeMin=200;

    fontSize=50;
    letters = "It isn't true that my mattress is made of cotton candy. It was the first time he had ever seen someone cook dinner on an elephant. Homesickness became contagious in the young campers' cabin. Every manager should be able to recite at least ten nursery rhymes backward.";
  }
  void setbrush(int brushn) {
    brushN=brushn;
  }
  void setSize(float size) {
    stepSize=size;
  }
  void setCol(int colorr) {
    col=colorr;
  }
  void setLineLength(float len) {
    linelength=len;
  }
  void setDiam(float diamm) {
    diam= diamm;
  }

  void setAngle(float ang) {
    angle=ang;
  }

  void draww(PVector mouse) {

    switch (brushN) {
    case 2:
      setLineLength(100);
      setSize(5);
      setDiam(2);
      blur(mouse);
      break;

    case 3:
      setLineLength(100);
      setSize(3);
      randomprisms(mouse);
      break;

    case 4:

      dynamicShapes(mouse);
      break;

    case 5:
      textlines(mouse);
      break;

    default :
      setLineLength(100);
      strokeWeight(1);
      rotatingLines(mouse);
      break;
    }
  }

  void helper_randomPrisms() {
    float angle = TWO_PI/8;
    float rad = 100 * random(0.5, 1);
    for (int i=0; i<8; i++) {
      numX[i]=cos(angle * i) * rad;
      numY[i]=sin(angle * i) * rad;
    }
  }

  void blur(PVector mouse) {
    float [] blurX ={ -stepSize, -stepSize, 0, stepSize, stepSize, stepSize, 0, -stepSize};
    float [] blurY ={         0, stepSize, stepSize, stepSize, 0, -stepSize, -stepSize, -stepSize};
    float d = dist(x, y, mouse.x, mouse.y);

    if (d > stepSize) {
      // gets the angle between the mouse and the location of the brush so that next point is drawn in the right intended location
      angle = (float)Math.atan2(mouse.y-y, mouse.x-x);

      tempX = x;
      tempY = y;

      nya.beginTransform(0);
      //
      //translate(x, y);
      rotate(angle);
      noStroke();
      fill(col, 40);

      //
      for (int i=0; i<500; i++) {
        int dir =  int(random(0, 7));
        rotate(random(5));
        println(blurX[dir]);
        tempX=blurX[dir]*random(1, 10);
        tempY=blurY[dir]*random(1, 10);
        circle(tempX, tempY, diam);
      }
      //
      nya.endTransform();

      x = mouse.x;
      y = mouse.y;
    }
  }

  void randomprisms(PVector mouse) {
    float d = dist(x, y, mouse.x, mouse.y);

    if (d>stepSize) {
      // gets the angle between the mouse and the location of the brush so that next point is drawn in the right intended location
      angle = (float)Math.atan2(mouse.y-y, mouse.x-x);
      for ( int i=0; i<8; i++) {
        numX[i]+=random(0-stepSize/2, stepSize/2);
        numY[i]+=random(0-stepSize/2, stepSize/2);
      }

      nya.beginTransform(0);
      //
      //translate(x, y);
      rotate(angle+random(-0.1, 0.1));
      stroke(col);
      strokeWeight(stepSize/4);
      rectMode(CENTER);
      noFill();
      rect(0, 0, linelength*random(0.95, 1)*d/100, linelength*random(0.95, 1)*d/100);

      nya.endTransform();

      x=x+(float)Math.cos((angle))*stepSize;
      y=y+(float)Math.sin((angle))*stepSize;
    }


    helper_randomPrisms();
  }

  void dynamicShapes(PVector mouse) {
    float d = dist(x, y, mouse.x, mouse.y);
    println("x: ",x,"y", y," mouse.x: ", mouse.x,"mouse.y: " ,mouse.y);
    if (d>stepSize) {
      //
      lineModule = createShape();
      stroke(col);
      strokeWeight(1);
      noFill();
      lineModule.beginShape();

      lineModule.vertex(100, 0); // first point

      lineModule.vertex(0, 100);// /
      lineModule.vertex(-100, 0);// \
      lineModule.vertex(0, -100);// /
      lineModule.vertex(100, 0);// \

      lineModule.endShape();
      // gets the angle between the mouse and the location of the brush so that next point is drawn in the right intended location
      angle = (float)Math.atan2(mouse.y-y, mouse.x-x);


      nya.beginTransform(0);
      //
      //translate(mouse.x, mouse.y);
      rotate((angle-PI));
      shape(lineModule, 0, 0, d, noise(1+frameCount*0.0001)*10);
      //
      nya.endTransform();


      x=x+(float)Math.cos((angle))*stepSize;
      y=y+(float)Math.sin((angle))*stepSize;
    }
  }

  void textlines(PVector mouse) {
    float d = dist(x, y, mouse.x, mouse.y);
    d= constrain(d, 60, 7000);
    if (d>stepSize) {
      // gets the angle between the mouse and the location of the brush so that next point is drawn in the right intended location
      angle = (float)Math.atan2(mouse.y-y, mouse.x-x);
      fontSize= fontSizeMin*60/d;
      font = createFont("Calibri", fontSize);

      textFont(font, fontSize);
      char newLetter = letters.charAt(counter);
      stepSize = textWidth(newLetter);
      fill(col);
      //stroke(col);


      nya.beginTransform(0);
      //
      //translate(x, y);
      
      rotate(angle);
      text(newLetter, 0, 0);
      //
      nya.endTransform();

      counter++;

      if (counter>letters.length()-1) counter=0;

      x = x + (float)Math.cos(angle)*stepSize;
      y = y + (float)Math.sin(angle)*stepSize;
    }
  }

  void rotatingLines(PVector mouse) {

    nya.beginTransform(0);
    strokeWeight(1);
    stroke(col);
    
    rotate(radians(angle));
    line(0, 0, linelength, 0);
    //
    nya.endTransform();

    angle +=5;

    linelength= int(noise(50+0.01*frameCount)*300);
  }
}

Arduino code:

int left = 0;
int right = 0;
int buttonOne = 9;
int buttonTwo = 10;
int buttonThree = 11;
int buttonFour = 12;
int buttonFive = 13;
int buttonSix = 3;//reset
int buttonSeven = 5;//Draw
int buttonEight = 4;//saveScreen


int val1 = 0;
int val2 = 0;
int val3 = 0;
int val4 = 0;
int val5 = 0;
int val6 = 0;
int val7 = 0;
int val8 = 0;

void setup() {
  pinMode(buttonOne, INPUT);
  pinMode(buttonTwo, INPUT);
  pinMode(buttonThree, INPUT);
  pinMode(buttonFour, INPUT);
  //   pinMode(buttonFive, INPUT);
  Serial.begin(9600);
  Serial.println("0,0");

}
void loop() {
  while (Serial.available()) {
    right = Serial.parseInt();
    left = Serial.parseInt();
    if (Serial.read() == '\n') {

      int sensor = analogRead(A0);
      delay(1);
      val1 = digitalRead(buttonOne);
      delay(1);

      val2 = digitalRead(buttonTwo);
      delay(1);

      val3 = digitalRead(buttonThree);
      delay(1);
      //
      val4 = digitalRead(buttonFour);
      delay(1);

      val5 = digitalRead(buttonFive);
      delay(1);

      val6 = digitalRead(buttonSix);
      delay(1);

      val7 = digitalRead(buttonSeven);
      delay(1);
      
      val8 = digitalRead(buttonEight);
      delay(1);
  


      Serial.print(sensor);
      Serial.print(',');
      Serial.print(val1);
      Serial.print(',');
      Serial.print(val2);
      Serial.print(',');
      Serial.print(val3);
      Serial.print(',');
      Serial.print(val4);
      Serial.print(',');
      Serial.print(val5);
      Serial.print(',');
      Serial.print(val6);//
      Serial.print(',');
      Serial.print(val7);//
      Serial.print(',');
      Serial.println(val8);
    }
  }



}

 

Leave a Reply