Idea
As you might remember, we jumped between a few ideas, and after brainstorming with Aaron and long conversations among each other, we decided on an idea. When we started, it was a bit vague, but the more progress we made the more everything made sense and became clearer.
So, our project is an experience in which two users answer questions that are meant to explore how similar these users are to each other by mapping the data from their answers onto the screen and then visualizing it gradually as users interact with the set-up. This visualization happens when users’ traces intersect in the area of a question where they answered similarly to reflect connection. The “bloom” we used to resemble connection differs based on how close the answers were, making it the most dramatic when users have the exact same answer.
This final idea ended up reminding us of a lot of things that we find fascinating and enjoy:
- The concept of an etch-a-sketch
- A card game called “We are Not Really Strangers”
- Colors!!
- Human connection
The Experience
To take you through our game, we will demonstrate our Arduino set-up and the three different screens that make up our experience:
For Arduino, we wanted to have two different “stations” for each user. Each user on their breadboard has 4 buttons (reflecting Strongly disagree, disagree, agree, strongly agree for answering questions), and two potentiometer knobs for the etch- a – sketch.
Although finding pieces was an initial struggle, people on campus were generous enough to lend us some pieces — especially Amina! Shout out to you :-).
As you can see we had to work with the larger potentiometers, which initially caused a glitch due to loose wires, but then when taped down they worked just fine.
As for the code, we used an ascii handshake to send one button reading and the 4 values from the potentiometers in a comma-separated string, which we trimmed, divided and casted in processing. For the buttons, we gave each button a unique value so we can indicate the answer chosen in processing (rather than just depending on 0 and 1).
int button1Pin = 2;
bool prevButton1 = LOW;
int button2Pin = 4;
bool prevButton2 = LOW;
int button3Pin = 8;
bool prevButton3 = LOW;
int button4Pin = 10;
bool prevButton4 = LOW;
int button5Pin = 13;
bool prevButton5 = LOW;
int button6Pin = 12;
bool prevButton6 = LOW;
int button7Pin = 7;
bool prevButton7 = LOW;
int button8Pin = 5;
bool prevButton8 = LOW;
void setup() {
pinMode(button1Pin, INPUT);
pinMode(button2Pin, INPUT);
pinMode(button3Pin, INPUT);
pinMode(button4Pin, INPUT);
pinMode(button5Pin, INPUT);
pinMode(button6Pin, INPUT);
pinMode(button7Pin, INPUT);
pinMode(button8Pin, INPUT);
Serial.begin(9600);
Serial.println("0,0");
}
void loop() {
if (Serial.available() > 0) {
char inByte = Serial.read();
int u1Op1 = digitalRead(button1Pin);
int u1Op2 = digitalRead(button2Pin) + 2;
int u1Op3 = digitalRead(button3Pin) + 4;
int u1Op4 = digitalRead(button4Pin) + 6;
int u2Op1 = digitalRead(button8Pin) + 10;
int u2Op2 = digitalRead(button7Pin) + 12;
int u2Op3 = digitalRead(button6Pin) + 14;
int u2Op4 = digitalRead(button5Pin) + 16;
int knob1U1 = analogRead(A0);
delay(1);
int knob2U1 = analogRead(A1);
delay(1);
int knob1U2 = analogRead(A4);
delay(1);
int knob2U2 = analogRead(A5);
if (u1Op1 == 1 && prevButton1 == LOW) {
Serial.print(u1Op1);
Serial.print(',');
} else if (u1Op2 == 3 && prevButton2 == LOW) {
Serial.print(u1Op2);
Serial.print(',');
} else if (u1Op3 == 5 && prevButton3 == LOW) {
Serial.print(u1Op3);
Serial.print(',');
} else if (u1Op4 == 7 && prevButton4 == LOW) {
Serial.print(u1Op4);
Serial.print(',');
} else if (u2Op1 == 11 && prevButton5 == LOW) {
Serial.print(u2Op1);
Serial.print(',');
} else if (u2Op2 == 13 && prevButton6 == LOW) {
Serial.print(u2Op2);
Serial.print(',');
} else if (u2Op3 == 15 && prevButton7 == LOW) {
Serial.print(u2Op3);
Serial.print(',');
} else if (u2Op4 == 17 && prevButton8 == LOW) {
Serial.print(u2Op4);
Serial.print(',');
}
else {
Serial.print(-1);
Serial.print(',');
}
Serial.print(knob1U1);
Serial.print(',');
Serial.print(knob2U1);
Serial.print(',');
Serial.print(knob1U2);
Serial.print(',');
Serial.println(knob2U2);
prevButton1 = u1Op1;
prevButton2 = u1Op2 - 2;
prevButton3 = u1Op3 - 4;
prevButton4 = u1Op4 - 6;
prevButton5 = u2Op1 - 10;
prevButton6 = u2Op2 - 12;
prevButton7 = u2Op3 - 14;
prevButton8 = u2Op4 - 16;
}
}
- Start Screen: we wanted to offer the users some quick pointers before they start. Here, we tried our best not to spoil what’s going to happen afterward but communicate all the needed information.
2. Question Screen: Here, all the questions display in a grid, one after the other. When both users answer, the next question appears, and all answers are recorded and the answer difference between them calculated. This answer difference becomes the measure of their similarity that affects the blooming.
3. Drawing screen: Here, users use the potentiometer knobs to move around the screen, if they intersect in an area where answer difference = 0, they will see a dramatic explosion of color happen in front of them. This explosion will be smaller for answer difference 1 and 2, and non-existent for answer difference 3.
4.End Screen:
This appears after the users “save” their sketch and it explains the concept to them.
Some Main Features in the Code + Key Challenges during Implementation
Questions:
We had a question class, which creates class objects in a grid that we display one by one as users answer. This class includes the text display, variables to record user answers, the variable to calculate answer difference, and booleans to check if the question has been answered.
The Traces:
In the beginning, we were just trying the code using the keyboard, and we were drawing points. We were moving the point one pixel at a time. We, later, realized that this wouldn’t work with the potentiometers. It was also causing way too much discontinuity in the traces. To slightly fix this, and get a better effect overall, we started recording the lastX and lastY read by the potentiometers, and drawing lines from (lastX, lastY) to (currentX,currentY) instead of points.
Collisions
One of the main issues we ran into was detecting collisions between the traces.
There was a lot to think about in order to only detect valid collisions.
We needed to compare the color of pixels in two different PGraphics layers, each one containing the trace of one user. Whenever a pixel location is colored in both of the layers, a collision is detected.
However, we also needed to keep track of collision locations so that a collision wouldn’t be detected twice in the same location.
Since the strokeWeight of the traces is larger than just 1 pixel, and the lines are not just at 90-degree angles (they can be as squiggly as your heart desires) we also needed to check if a collision has already happened within a radius close to the current collision detected.
The bloom
For the bloom, it took us a while to figure out what we wanted it to look like. We ended up using code we found on OpenProcessing (credits go to the post https://www.openprocessing.org/sketch/486216 ), as our base. We struggled a bit to incorporate it into our code, but we tweaked parts of it and changed some parameters to get it to work with our idea.
User Test:
Testing this experience with people actually tied our concept together, it suddenly made much more sense when we saw how the users went about it and their reactions. For example, in the case of the video above, after the users read through the end screen their reactions were something like: “oh! This is a painting about our friendship”, then discussing their answers and how similar and different they are, and then asking if they could print it. Although we tested it with friends, we think this could be a great conversation starter for two strangers!
Future Improvements
Perhaps adding more questions or delving deeper with what we ask could be an interesting improvement to this experience, adding more meaning to it. Furthermore, in terms of the Arduino setup, we really wanted to make it more appealing and better divided, however, the limitations made it a bit difficult to make it look neater.
Finally, if we had more chances to user test we would’ve definitely been able to reveal details in the user experience that could be enhanced.
Code
Main Code
import processing.serial.*;
Serial myPort;
PGraphics user1, user2, blooms;
ArrayList<Integer[]> collisions;
int c_index = -1;
IntList collisionLocations;
Question[] questions;
int screen = 0;
int questionNumber = 0;
int Answer = 0;
int currentU1Answer = 0;
int currentU2Answer = 0;
String timestamp;
int qCols = 3;
int qRows = 3;
//int goingThruQs = 0;
int x;
int y;
int lastX;
int lastY;
int x2;
int y2;
int lastX2;
int lastY2;
float a, b, c, d, e, f ;
PImage traceBg;
PImage questionBg;
PImage startScreen;
PImage endScreen;
void setup() {
background(0);
fullScreen();
println(width + " " + height);
String portname = Serial.list()[2];
myPort = new Serial (this, portname, 9600);
myPort.clear();
myPort.bufferUntil('\n');
collisionLocations = new IntList();
x = 0;
y = height/2;
lastX = 0;
lastY = height/2;
x2 = width;
y2 = height/2;
lastX2 = width;
lastY2 = height/2;
questions = new Question[qCols*qRows];
int questionIndex = 0;
for (int r2 = 0; r2 < qRows; r2++) {
for (int c2 = 0; c2 < qCols; c2++) {
questions[questionIndex] = new Question( questionIndex, c2*640, r2*360);
questionIndex +=1;
}
}
collisions = new ArrayList();
user1 = createGraphics(width, height);
user2 = createGraphics(width, height);
blooms = createGraphics(width, height);
pts = new ArrayList<Particle>();
traceBg = loadImage("bg2darker.jpg");
startScreen = loadImage("startScreen.png");
questionBg = loadImage("bg.jpg");
questionBg.resize(width, height);
endScreen = loadImage("endScreen.png");
}
void draw() {
if (screen == 0) {
image(questionBg, 0, 0, width, height);
image(startScreen, 50, 50 );
}
if (screen == 1) {
questions[questionNumber].display();
goingThroughQuestions(questionNumber);
if (questions[questionNumber].user1Answered && questions[questionNumber].user2Answered && questionNumber < 8 ) {
questions[questionNumber].answerDifference = abs(questions[questionNumber].user1Answer - questions[questionNumber].user2Answer);
println(questions[questionNumber].answerDifference);
questionNumber += 1;
}
if (questionNumber == 8 && questions[questionNumber].user1Answered && questions[questionNumber].user2Answered) {
questions[questionNumber].answerDifference = abs(questions[questionNumber].user1Answer - questions[questionNumber].user2Answer);
image(traceBg, 0, 0, width, height);
screen = 2;
}
}
if (screen == 2) {
stroke(183,150,172,50);
strokeWeight(5);
line(x, y, lastX, lastY);
user1.beginDraw();
user1.stroke(255);
user1.strokeWeight(5);
user1.line(x, y, lastX, lastY);
user1.endDraw();
lastX = x;
lastY = y;
stroke(120,150,220, 50);
strokeWeight(5);
line(x2, y2, lastX2, lastY2);
user2.beginDraw();
user2.stroke(255, 0, 0);
user2.strokeWeight(5);
user2. line(x2, y2, lastX2, lastY2);
user2.endDraw();
lastX2 = x2;
lastY2 = y2;
//image(user1, 0, 0);
//image(user2,0,0);
user1.loadPixels();
user2.loadPixels();
for (int i=0; i<width; i++)
{
for (int j=0; j<height; j++)
{
if ( user1.get(i, j) == color(255) && user2.get(i, j) == color(255, 0, 0) &&!exists(i, j) && !existsInRadius(i, j))
{
Integer[] c= {i, j};
collisions.add(c);
collisionLocations.append(location(c[0], c[1]));
c_index += 1;
yes = true;
}
}
}
blooms.beginDraw();
blooms.smooth();
blooms.colorMode(HSB);
blooms.rectMode(CENTER);
if (c_index > -1)
{
bloomValues(questions[collisionLocations.get(c_index)].answerDifference);
//blooms.ellipse(collisions.get(i)[0], collisions.get(i)[1], diameter, diameter);
if (a != -1 && b != -1 && c != -1 && d != -1 && e != -1 && f != -1) {
drawcool(blooms, collisions.get(c_index)[0], collisions.get(c_index)[1], a, b, c, d, e, f);
}
}
blooms.endDraw();
image(blooms, 0, 0);
}
if (screen == 3) {
image(questionBg,0,0);
image(endScreen,50,50);
}
}
void goingThroughQuestions(int i) {
//answer is not zero only when a button has been pressed
if ( Answer == 1 || Answer == 3 || Answer == 5 || Answer == 7) { //if one of these buttons is pressed then it's the first user
if (!questions[i].user1Answered) { // if the question hadn't been answered already
questions[i].user1Answer = currentU1Answer; //assign the user 1 arduino variable to the user1 question variable
questions[i].user1Answered = true;
//println("user 1 " + questions[i].user1Answer);
//not sure if the following 2 lines make sense, to me it's like restarting and waiting for a new button press
currentU1Answer = 0;
Answer = 0;
}
}
if ( Answer == 11 || Answer == 13 || Answer == 15 || Answer == 17) {
if (!questions[i].user2Answered) {
questions[i].user2Answer = currentU2Answer;
questions[i].user2Answered = true;
currentU2Answer = 0;
Answer = 0;
}
}
}
void serialEvent(Serial myPort) {
String s=myPort.readStringUntil('\n');
s=trim(s);
if (s!=null) {
int values[]=int(split(s, ','));
if (values.length==5) {
Answer = (int)values[0];
x=(int)map(values[1], 0, 1023, 0, width);
y=(int)map(values[2], 0, 1023, 0, height);
x2=(int)map(values[3], 0, 1023, 0, width);
y2=(int)map(values[4], 0, 1023, 0, height);
}
}
if (Answer != -1) {
if (Answer == 1 || Answer == 3 || Answer == 5 || Answer == 7) {
currentU1Answer = Answer;
}
//second set of buttons for user 2
if (Answer == 11 || Answer == 13 || Answer == 15 || Answer == 17) {
currentU2Answer = Answer - 10 ;
}
}
myPort.write('0');
}
//function to check if collision already exists in array
boolean exists(int x, int y)
{
for (int i=0; i<collisions.size(); i++)
{
if (collisions.get(i)[0]==x && collisions.get(i)[1]==y)
{
return true;
}
}
return false;
}
//function to check if collisions exists in radius
boolean existsInRadius (int x, int y)
{
for (int i=0; i<collisions.size(); i++)
{
if (sq(collisions.get(i)[0] - x) + sq(collisions.get(i)[1] - y) <= 20*20)
{
return true;
}
}
return false;
}
//function to check in which question's "area" the collision happened in
int location(int x, int y) {
int qIndex = 0;
for (int r2 = 0; r2 < qRows; r2++) {
for (int c2 = 0; c2 < qCols; c2++) {
if (x >= c2*640 && x <= (c2+1)*640 && y >= r2*360 && y <= (r2+1)*360) {
return qIndex;
}
qIndex +=1;
}
}
return -1;
}
//function to take parameters for the bloom based on the answer difference
void bloomValues(int diff) {
if (diff == 0) {
//lifespan
a = 70; //80;
b = 150; //120;
//decay
c = 0.7;
d = 0.99;
//weightRange
e = 60;
f = 180; //120;
} else if (diff == 2) {
//lifespan
a = 55;
b = 80;
//decay
c = 0.63;
d = 0.9;
//weightRange
e = 20;
f = 100;
} else if (diff == 4) {
//lifespan
a = 30;
b = 65;
//decay
c = 0.63;
d = 0.83;
//weightRange
e = 8;
f = 35;
} else if (diff == 6) {
a = b = c = d = e = f = -1;
}
}
void keyPressed() {
if (key == 'S' || key == 's') {
timestamp = year() + nf(month(), 2) + nf(day(), 2) + "-" + nf(hour(), 2) + nf(minute(), 2) + nf(second(), 2);
saveFrame(timestamp+".png");
screen = 3;
}
}
void mouseClicked(){
screen = 1;
}
Bloom Class
ArrayList<Particle> pts;
boolean yes;
int coolingTime;
void drawcool(PGraphics pg, Integer x, Integer y, float a, float b, float c, float d, float e, float f) {
pg.beginDraw();
if (yes)
{
for (int i=0;i<10;i++) {
Particle newP = new Particle(x, y, i+pts.size(), i+pts.size(), a,b,c,d,e,f);
pts.add(newP);
}
}
for (int i=0; i<pts.size(); i++) {
Particle p = pts.get(i);
p.update();
p.display(pg);
}
for (int i=pts.size()-1; i>-1; i--) {
Particle p = pts.get(i);
if (p.dead)
{
pts.remove(i);
}
}
coolingTime++;
stop();
//yes = false;
pg.endDraw();
}
void stop(){
if (coolingTime>1)
{
yes = false;
coolingTime = 0;
}
}
//CLASS
class Particle{
PVector loc, vel, acc;
int lifeSpan, passedLife;
boolean dead;
float alpha, weight, weightRange, decay, xOffset, yOffset;
color c;
float a, b,cc,d,e,f;
Particle(float x, float y, float xOffset, float yOffset, float thisa , float thisb , float thisc, float thisd, float thise ,float thisf){
loc = new PVector(x,y);
a = thisa;
b = thisb;
cc = thisc;
d = thisd;
e = thise;
f = thisf;
float randDegrees = random(360);
vel = new PVector(cos(radians(randDegrees)), sin(radians(randDegrees)));
vel.mult(random(1));
acc = new PVector(0,0);
lifeSpan = int(random(a, b)); //HERE IS HOW LONG THE ANIMATION IS (timewise)
decay = random(cc, d); //HERE IS HOW FAR IT GOES
c = color (random(255), random(255), 255, 75);
weightRange = random(e,f);//random(5,90); //HERE IS THE SIZE
this.xOffset = xOffset;
this.yOffset = yOffset;
}
void update(){
if(passedLife>=lifeSpan){
dead = true;
}else{
passedLife++;
}
alpha = float(lifeSpan-passedLife)/lifeSpan * 70+50;
weight = float(lifeSpan-passedLife)/lifeSpan * weightRange;
acc.set(0,0);
float rn = (noise((loc.x+frameCount+xOffset)*0.01, (loc.y+frameCount+yOffset)*0.01)-0.5)*4*PI;
float mag = noise((loc.y+frameCount)*0.01, (loc.x+frameCount)*0.01);
PVector dir = new PVector(cos(rn),sin(rn));
acc.add(dir);
acc.mult(mag);
float randDegrees = random(360);
PVector randV = new PVector(cos(radians(randDegrees)), sin(radians(randDegrees)));
randV.mult(0.5);
acc.add(randV);
vel.add(acc);
vel.mult(decay);
vel.limit(3);
loc.add(vel);
}
void display(PGraphics pg){
pg.beginDraw();
pg.strokeWeight(0);
pg.stroke(0, alpha);
pg.point(loc.x, loc.y);
pg.strokeWeight(weight);
pg.stroke(c);
pg.point(loc.x, loc.y);
pg.tint (255,125);
pg.endDraw();
}
}
Question Class
class Question {
//variables to store input answers
PImage image;
int user1Answer;
int user2Answer;
int answerDifference;
//question location (corresponding to area on screen)
int x;
int y;
int coverX = 0;
//question box dimensions (it could be something that we design and upload to processing(?))
int w = 640;
int h = 360;
int c = 255; //alpha
int questionNumber;
String questionText;
boolean user1Answered = false;
boolean user2Answered = false;
boolean sameAnswer;
Question(int tempQNumber, int tempX, int tempY) {
questionNumber = tempQNumber;
x = tempX;
y = tempY;
image = loadImage("question"+(questionNumber+1)+".png");
}
void display() {
//background(0);
image(questionBg,0,0);
fill(0);
//noStroke();
stroke(255);
image(image,x, y);
fill(255);
//textSize(40);
//text(questionText, x+w/4, y+h/4);
}
}
