I’m interested in the idea of using cameras and live video with Processing, and so for my final project, I want to create some form of a “self-portrait generator”, where I’ll use concepts from generative art along with video. I’m not completely confident that I have enough knowledge in this premise, but I’ve found enough relevant examples online that I can refer to as a start.
For the interaction, I will either use a screen where the user can re-assemble the portrayed image by touching the screen, or by using a regular screen, an infrared sensor and allowing the user to manipulate the self-portrait using hand gestures. I will have to think more about the affordances and clarity of interaction for each approach, perhaps through user testing.
For this project, I’ll need:
-one, or perhaps two cameras.
-a touch screen OR a regular screen
-(possibly) an infrared sensor
As for visuals, I’m interested in imitating a style of art (ex: pointillism, cubism, post-impressionism, suprematism) using Processing, and merging that with video to create an interactive self-portrait where the user can then intervene in their own way. For example, if I were to use a piece by Malevich as inspiration, the camera footage would interact in a certain way with the shapes, and the user would be able to rearrange and reassemble them.
As I mentioned already, I am not sure of how I would want the visuals to look like yet, but this is a quick sketch of how I would want the setup to look like.
Soooo, I have recreated my table tennis game but using a physical controller a.k.a. Potentiometer. It turned out to be better than the previous version since more than one key cannot be pressed at a time and the sensor transfer their values once a frame. I have encountered a problem though during coding. The thing is that when you output data from arduino to processing we use serial monitor. It transfers one line at a time, when i needed two. What Jack advised me to do is to put all the data in one line and separate using a comma (CSV file? really? how didnt I come up with that? Am I really a CS major?) Here is a short video:
And here is the code:
1)Code for arduino:
void setup() {
// put your setup code here, to run once:
Serial.begin(9600);
Serial.println("0,0");
}
void loop() {
// put your main code here, to run repeatedly:
if (Serial.available()>0){
char inByte = Serial.read();
int sensor = analogRead(A0);
int sensor1 = analogRead(A1);
delay(0);
Serial.print(sensor);
Serial.print(",");
Serial.println(sensor1);
}
}
class padle {
float x;
float y;
float w;
float h;
color fillVal = color(0, 255, 0);
padle(float _x, float _y, float _w, float _h) {
x = _x;
y = _y;
w = _w;
h = _h;
}
void update() {
}
void display(float x, float y, float w, float h) {
fill(fillVal);
rect(x, y, w, h, 5);
}
void run() {
update();
display(x,y,w,h);
}
}
Final project idea:
My idea is to make a claw game where you pull out toys. But all of that will be on the screen except the controller. After that an actual toy that you have pulled out in the game falls from behind the screen. The idea is to make an unexpected result. The user won’t have an idea that the toy from the computer will fall out from behind the computer. Other than that I have some ideas like making a game in processing
and using a physical controller (sensors, buttons), but Aaron didn’t really liked this idea 🙁
This week I struggled with coming up with an idea on which of my processing designs I could combine with arduino. I ended up going with my game, popping balloons. But which aspect would I change? I played around with a button to pop them, a potentiometer to change the speed in which they moved, but then settled on using a potentiometer to change how many circles would appear at the start of the game.
My biggest challenge with this was rearranging certain aspects of my code in order to fit how I had defined the sensor. Then, once I had done that, I struggled with the sensor reading being on a loop. Jack helped me add in a boolean to have it only read once at the start of the game.
My initial impulse was to focus on a processing heavy project as me and physical computing did not turn out to be very good friends – but primarily because I would like to work on a bigger, more challenging project in Processing which we did not get to during the second half of the semester.
I would like to create an art-piece using computer vision. This would form a silhouette of the person standing in front of the camera that would mimic the person’s movement – and making them pretty much one person.
What would be happening on the screen, however, would be the most important. I would like the silhouette to be filled with a lot of tiny little objects – flowers, hearts, ice-cream, a lot of happy lively colorful objects. But in between those I would insert some very ugly objects (snakes, scary clowns, spiders etc.) which would not be visible through the beauty. Next to the person, there would be a stand with three buttons that would filter what the silhouette is made of – just pretty things, just ugly things, or both (probably marked as “The Pretty, The Ugly and The Reality”).
In order to make the experience unique for each person, I would like to have it separated by walls/curtains so only one person at a time can enter and interact. Also, I would like them to first interact with the piece without pressing the filters. I was thinking of doing this through putting instructions on the floor- which would indicate that stepping into a marked circle and interacting is the first step, and pressing the buttons the second.
For my final project, I want to create morse code machine that is controlled by Arduino and is displayed in Processing. I don’t know any Morse code so I would have to spend some time learning the language too. My main purpose of creating this project would be to teach other people how morse code works. Since Morse code is made of dots and dashes, I want to represent those dots and dashes in a more interesting way in Processing. For the Arduino part of the project, I would have buttons that triggers some sound and then have that signal send to Processing. That Processing program would take in the input from the Arduino and draw out the letter the person tapped out.
I might also add a game in the Processing part with letters that pop up. For example, Processing would show the letter A and the player would have to tap out A. This would continue until the game is over or whenever the player chooses to finish.
What I would need:
Momentary Switch Buttons
Piezo Buzzer
Acrylic panel for holding the buttons — laser cutter
LED lights — for emphasize on whether it’s a dot or dash
I’m trying to keep the Arduino part simple so that I could focus more on how I’m going to represent Morse Code in a creative manner. There would be two parts to choose from
Learning Morse code
Playing a Morse code game to test how good you are
Write on this blog a paragraph or two about what computing means to you at this point. Is it adding something to your life? Is it helping you become a better person? What are you getting out of it, what do others get from it?
I’ve never really worked with circuits before and have only played with the basics of it from my electronics class back in high school. I learned about parallel/series circuits and all the semantics/calculations but never put it into practice until this class. From everything we’ve learned from Arduino and Processing, I think I’ve grasped enough basics to create (almost) anything I want. I loved that we’ve learned how to connect software and hardware into one and I’m excited to make my own electronics to better my life. It’s a nice change from just making software/websites to making physical devices. Now that I’ve taken this class, I feel more confident with building actual products. Mark Zuckerberg recently posted about how he made his wife a device that emits a faint night between certain hours for a few minutes at night. If I hadn’t taken this class, I might think this project is very complicated, but I no longer think so.
For the project, I created a controller for my game from two weeks ago. My controller moves the bar in the game left to right. For this, I used a potentiometer because it seemed intuitive. If the player loses, the red light goes off and if they win, the green light goes off.
void loop() {
xPos = analogRead(A0);
delay(0);
Serial.write(xPos/4);
int winInt = Serial.parseInt();
int loseInt = Serial.parseInt();
if (winInt) {
digitalWrite(won, 1);
}
if (loseInt) {
digitalWrite(lost, 1);
}
}
For this week’s exercise, I have decided to connect arduino to one of my previous exercise – the recreation on Computer Graphics & Art. The structure of the code is similar to the previous exercise, but I have added conditional statements on each if-statements that draws the strokes based on the location. For the physical part on the Arduino, I have mapped the value of the potentiometer into the integer range from 0 to 8, where each integer serves as an opening indicator for each layer. When the mapped value is at 0, none of the strokes are initiated but as you increment the value using the potentiometer, the layers show the strokes – and the general flow of the layers starts from the center as the initial layer and expands to the outer layers.
The video below is a short demo of the exercise:
<Source Code – Arduino>
void setup() {
// put your setup code here, to run once:
Serial.begin(9600);
Serial.println('0');
}
void loop() {
if(Serial.available()>0){
char inByte=Serial.read();
int sensor = analogRead(A0);
delay(0);
Serial.println(sensor);
}
}
Physical computing means building interactive physical systems by the use of software and hardware that can sense and respond to the analog world. – wikipedia
This is the Wikipedia definition of physical computing, but it is a very general, very vague one. Starting this course, I had a general idea on what we might be doing, but I never really expected to connect so much with “emotion” and with “being”. My previous notion of interactive art consisted of the following: “Make something move with code and wire”. The layers were not quite clear and it was a sort of “black box”. Now that we are almost at the end of the semester, with so much basic knowledge about physical computing, I can finally see all the layers that might be there in, say, for example, an interactive installation.
Is it helping me become a better person? I don’t know if it’s helping me become a better person, but it’s helping me evolve into a different type of artist. When I think about what type of projects I want to do in the future, I no longer think exclusively of paintings and “archivable” art. I think of interactive installations I want to build and causes I want to help with these installations.
Is it adding something to my life? Yes, it is adding skills to my list of “things I can pull off under extreme stress, pressure, and approaching deadlines”. It is giving me an extension, or rather a whole other creative outlet. While coding is never going to make me as emotional as painting does, it is a big part of interactive art and helps bring things to life.
What are am I getting out of it? Currently, I would say the satisfaction of getting this course over with and (hopefully) passing. In the future perhaps a job, and maybe a medium through with I can make a change in the world, regardless of how small it may be. All big change starts with small attempts towards a larger goal.
What do others get from it? The experience. Everyone has their own interpretations of a piece. Which is what makes physical computing so special. It takes people beyond just the visuality of the project. You can touch it (not in all cases of course). In other words, you get to be “one with the piece”.
For this week’s assignment I built upon the assignment I created two weeks ago for the Object Oriented Programming game. I added a joystick to control the square’s movement, as well as a potentiometer so the user can choose a color for the square throughout the game.
It was challenging to figure out how to send the information from the Arduino to Processing, but I am a lot more confident now in understanding how the Arduino sends information to Processing and how Processing uses this information.
The user uses the potentiometer attached to a breadboard, while they can use the joystick to control the customized square’s movement.
Here is a video of the joystick and potentiometer in action:
Below is the code for the Processing sketch:
import processing.serial.*;
Serial myPort;
int xPos, yPos;
int xMove;
int bgValue;
float monsterSpotY = random(300, 560);
float monsterSpotX = random(200, 560);
float monsterRadius = 40;
int gravity = 1;
int coloramt = 0;
class Player {
float x;
float y;
float up;
float down;
float left;
float right;
float speed;
float maxJump;
float side;
float prevY;
Player() {
x = 0;
y = 580;
up = 0;
down = 0;
left = 0;
right = 0;
speed = 4;
maxJump = 400;
side = 20;
prevY = 1;
}
void display() {
fill(255, 100, finalbgValue);
rect(x, y, side, side);
}
void update() {
x = x + (xPos) * speed;
if (prevY == 1) {
gravity = 1;
y = y + (yPos + gravity) * speed;
}
//prevY = y;
println(prevY);
println("ypos is " + yPos);
if (x <= 10) {
if (y >= height - 10) {
x = 10;
y = height - 10;
}
else if (y < height - 10) {
x = 10;
}
} else if (x >= 590) {
if (y >= height - 10) {
x = 590;
y = height - 10;
} else if (y < height - 10) {
x = 590;
}
}
else if (y <= 0) {
y = 0;
} else if (y >= 590) {
y = 590;
prevY = 1;
}
if (y <= 480) {
prevY = 2;
}
if (prevY == 2) {
gravity = 3;
y = y + (yPos + gravity) * speed;
}
}
}
class Monster {
float x;
float y;
float radius;
float ySpeed;
float monsterSpotY;
float monsterSpotX;
color monsterColor;
Monster() {
radius = monsterRadius;
ySpeed = random(2, 6);
monsterSpotX = random(0, width);
monsterSpotY = random(0, 100);
}
void show() {
//monsterColor = color(map(monsterSpotY, 0, height, 0, 255), map(monsterSpotY, 0, height, 100, 150), map(monsterSpotY, 0, height, 150, 255));
fill(coloramt);
noStroke();
ellipse(monsterSpotX, monsterSpotY, monsterRadius, monsterRadius);
}
void fall() {
monsterSpotY = monsterSpotY + ySpeed;
if (monsterSpotY > height) {
monsterSpotY = random(-200, 300);
monsterSpotX = random(0, width);
}
}
void run() {
fall();
show();
}
}
int counter = 0;
int gameState = 0;
int finalbgValue;
Player myPlayer;
Monster myMonsters[];
void setup() {
size(600, 600);
myPlayer = new Player();
int monsterCount = 100;
myMonsters = new Monster[monsterCount];
for (int i = 0; i < monsterCount; i++) {
myMonsters[i] = new Monster();
}
printArray(Serial.list());
String portname=Serial.list()[69];
println(portname);
myPort = new Serial(this, portname, 9600);
//myPort.bufferUntil('\n');
}
void draw() {
background(255);
// configuration state; user can select color.
if (gameState == 0) {
textAlign(CENTER);
fill(0);
text("Color your hero!", width / 2, 200);
fill(255, 100, bgValue);
rectMode(CENTER);
stroke(0);
rect(width / 2, 290, 120, 120);
fill(100);
rect(width / 2, 395, 140, 30);
fill(0);
text("Click here to play!", width / 2, 400);
println(mouseX);
println(mouseY);
if (mousePressed == true && mouseX > 230 && mouseX < 370 && mouseY > 383 && mouseY < 414) {
gameState = 1;
finalbgValue = bgValue;
}
} else if (gameState == 1) {
myPlayer.display();
myPlayer.update();
gameOver();
fill(0);
text(counter, width - 40, 30);
int counterIncrement = counter /7;
for (int i = 0; i < counterIncrement / 20; i+= 1) {
myMonsters[i].run();
if (i == counterIncrement / 20) {
i = 0;
println(i);
}
}
counter++;
} else if (gameState == 2) {
textAlign(CENTER);
background(255);
text("You Died! Your score is: " + counter, width/2, height / 2);
text("Click here to play again", width / 2, height / 2 + 30);
if (mousePressed == true && mouseX > 230 && mouseX < 360 && mouseY > 320 && mouseY < 340) {
gameState = 1;
counter = 0;
myPort.stop();
// ^ myport clears serial before running setup again
setup();
}
}
}
void serialEvent(Serial myPort) {
String temp=myPort.readStringUntil('\n');
temp=trim(temp);
if (temp != null) {
int values[]=int(split(temp, ','));
//split string of inputs by commas and end at line break "/n"
if (values.length == 3) {
xPos=(int)(values[0]);
yPos=(int)(values[1]);
bgValue = (int)(values[2]);
xPos =(int) map(xPos, 0, 1023, -3, 3);
yPos =(int) map(yPos, 0, 765, 3, -3);
bgValue = (int) map(bgValue, 0, 1023, 0, 255);
}
}
}
void gameOver() {
for (int i = 0; i < 30; i++) {
if (myPlayer.x + 20 > myMonsters[i].monsterSpotX - 20 && myPlayer.x - 20 < myMonsters[i].monsterSpotX && myPlayer.y - 20 < myMonsters[i].monsterSpotY && myPlayer.y + 20 > myMonsters[i].monsterSpotY - 10) {
gameState = 2;
} else {
}
}
}
I think that computing has become a very important part of my life because it has opened up many ways for me to express myself and create. I find it fascinating how computers are able to do pretty much anything we want as long as we are able to code it and provide it all the information necessary. This is still one of the most frustrating yet rewarding parts of computing for me: learning how to code. It can be so tantalizing yet exciting when I want to create something I conceived and I really have to understand why the program is doing this and not doing that. I will always feel like there is something that I can improve upon for all my projects, more techniques I can learn and more creative barriers to tackle using my skills in computing.
But what excites me the most is computing’s potential. Computers have clearly changed the world dramatically over the past few decades, and I often think about how this impacts the way people think and the way we function. I have mixed feelings about computers and their effects on society and this is a big reason as to why I am so drawn to interactive media and computing. I am especially excited to see how computing will change the world in the future, and am thrilled to be a part of this process and gaining the skills necessary to perhaps shape it. In summary, computing means a lot to me: it means power, it means frustration, it means being precise, it evokes feelings of dread in me when thinking about the future, apprehension, while also making me feel happy, playful and in awe of how convenient life can be in the age of computers. The usefulness of computing in my personal life and as a potential career, combined with how important it will be for humanity’s future make it one of the most interesting things that I have ever had the opportunity to explore.