For our final project we made an interactive dollhouse. From our final project proposal we made a few changes to the interaction for the house.
CONCEPT:
For each room in our house, we wanted a functionality and a type of interaction.
We initially wanted to have multiple interactions for every room as well as have virtual tasks that one could complete in processing. However, given the time constraints, this was not very feasible. As such we decided to pivot.
Currently, the user can interact with the dollhouse by placing a doll at the front of the room. When the doll is placed in front of the room the lights in the room turn on and the display [currently a computer screen] will display wallpaper and play one form of media, either a video, audio, or both depending on the room. Each room consists of three LEDs connected in series to avoid running multiple long wires to the breadboard.
PROCESS:
Our process included a lot of side-by-side physical and technical work.
The first steps were to create a physical dollhouse, which was made out of cardboard and measured to fit exactly on a laptop, to make the background the screen of it. For the coding, we worked on being able to separately trigger each part on the screen that corresponds to the room, and we did this with buttons.
how the Leds were fit into the cardboard
Next, the dollhouse was fit with LEDs and pressure sensors. To do this, we needed longer wires — from Aayush’s Circuit Breakers class– and to solder the wires together to connect them to the breadboard. We also used pressure sensors from Suzan’s Sensors Body and Motion class, which was perfect for the main room trigger. We wanted to have some additional functionalities, such as a button for the doorbell, a physical game room, and a button to switch between sceneries on the balcony, however were limited with equipment, so settled for that. At this point, there was also time spent on creating the furniture for the dollhouse, like couches, chairs, tables, and beds.
Next, we had to put it all together. Once we met more and discussed, we decided to let go of the idea of a physical game, and instead created a bathroom, where the sensor triggers a flush. At this point, we worked on the software functionalities of each room; such as playing the sounds or movies, as well as software design and illustrations, and physical refinements like the wallpaper, wall decor, and tidying up the wires.
We placed all the wires and Arduino in the attic of the house and covered the attic from the front. We also decorated the house more and cleaned it up.
Rooms:
Balcony:
balcony with a hollow window
When triggered, music is played on loop because of the piano, and the view is a moving scenery. This was created by layering the hand-drawn png illustration with an animated scenery found on youtube, creating the window look.
Bathroom:
The bathroom plays a flushing noise once when you enter.
Bedroom:
Same as the balcony room, the illustration is a png layer on top of the video found on youtube, with the dimensions of the window being empty.
Living Room:
This room plays a TV in the background, the video of choice being Happy, by pharell williams
Dining room:
animated dining room
This is playing a small gif, (which we converted to a movie), of the smoke on the pot moving, just to show that there is a movement happening.
Each room was paired with 3 LED’s that light up upon entry.
Almost complete!
OBSTACLES:
Thankfully, we didn’t encounter any big obstacles, but some things that were not the smoothest.
Running the wires was a bit difficult to manage
We had a soldering accident when trying to connect a sensor to a wire, and burnt the sensor
Some of the furniture (like the toilet or the piano) was difficult to figure out how to create
Adding the video to processing
We initially had a 15 second long flush, and couldn’t shorten it on processing, so we had to find another flush audio
REFLECTION:
We are really happy with how this finally turned out, and even joke about creating a start up. We wish we could have added more interactions and functionalities, and also a doll, but are happy with it anyway.
This is a simple to play Trivia game for children. There are Ten questions with only two choices ; A OR B.
To pick A the child must press the Green Button and to pick B he press the Red button. If the child gets the answer correct, they will automatically move on to the next question. If the child gets the answer wrong, they would be taken to a page that tells them they are wrong and that they must press r to go back and start again.
Once the kids get to the end of the game, the will be congratulated on making it to the end and given the option to play again.
Process: For the arduino we started out by testing the buttons using LED lights the LED lights when pressed were supposed to turn on there was difficulty getting them to work I connected some of them wrong but once I figured that out I moved on to finding the best serial code 4 this game. We worked with send multiple to processing and send byte to processing.
I had some difficulty trying to make that work and I ended up mixing and matching codes
After asking I found out that the code that I should be using is handshake
Handshake had some problems because it wasn’t obvious that I had to have arduino send something back from processing after further inspection that was solved.
I was finally able to use the buttons on my game for processing.
Creating the game was not very difficult are used images for the children to be more amused by and I kept the questions simple since I knew this is a kids game I did not want to make it very difficult on the children to solve this but at the same time I give them options that were very close and alike hoping it could create a little confusion for the child.
Arduino took me a very long time to work with and that was with a lot of help eventually everything came together so here’s my final product
For the final project I wanted to go with the arcade vibe of Arduino and create a game.
With simple controls in the form of the potentiometer the game relies on the player controlling a bird, which wants to fly high into the air. On its way, the bird encounters stormy clouds, which it should avoid at all cost. Also, to reward the bird on its adventure, it will encounter some delicious smaller birds, which work as the scoring mechanism for the game.
As the player will collect more points, the animation frequency will increase making the game progressively harder and harder.
To add some spice to the game I introduced a SLOW TIME button, which significantly decreases the animation frequency.
The projects seemed a bit daunting with all the vertical animations and ways of achieving top quality responsiveness from Arduino kit. I smoothed out the potentiometer values and used a single button to control the flow of the game as well as the SLOW MODE.
In my project Arduino provided the input for the bird control. The bird movement was mapped from the potentiometer, which fits the role perfectly. In addition to that, the slow mode button provided a mechanism for slowing down vertical transitions.
The processing layer was responsible for displaying the game as well as the logic. It takes the potentiometer value from Arduino and displays the bird on an according position. It also takes care of the slow mode triggers from the button.
The main challenge of the project was working with all the animations as they change throughout the game. I couldn’t just change the frameRate of the game, as that would case sudden drops in frameRate, rather than make the transition smooth. Therefore I worked around that by introducing the bgFreq which stands for background frequency. That variable controlled the speed at which all the pieces of UI were moving on the screen, depending on the gameScore.
During user testing I played a lot with different parameters of the game. The bird speed, as well as the cloud speed. As discussed in the class I decided on making the bird speed uniform and same as that of the background. That makes it easier for the user to predict the future position of the bird and plan their moves ahead more efficiently, increasing the strategical aspect of the game. The clouds on the other hand vary in speed, adding an extra layer of difficulty to an already difficult game. Arcade games never were easy, so I believe that choice of adding extra difficulty to my game was crucial.
The Arduino setup of the final work:
Footage of actual gameplay:
At last, here is a link to my project if you wanted to play with it yourself:
When I was in 10th grade, I really got into ghost-busting shows. The thrill of seeing people go to renowned places of horror was amazing. Especially when none of it had anything to do with me, as I sit comfortably in my couch.
One of the devices that caught my attention appears in the show Buzzfeed Unsolved. When the hosts Ryan Bergara and Shane goes into places that are suspected to have entities, they open a sound-making device called the “spirit box”. The spirit box allegedly goes through multiple radio channels at a rapid pace, producing a sort of white noise. In this white noise, it is believed that ghost and entities would be able to make some sort of sound to communicate with the living
Specifics
For my project, I wanted to make a device that mimics what the spirit box does, adding some additional features. The system is composed of an Arduino and my laptop (processing). On the Arduino, there are two systems.
The detection system includes a photoresistor and an ultrasonic distance sensor. When the user walks into a haunted place, the photoresistor will notice the light getting darker, giving it an indication that there is a possibility for ghost activity. Then, the distance sensor will detect whether the user is close to a corner, wall, or object. These are all areas that are likely to have paranormal activity. There is also a piezo disk on the Arduino. While detecting, the piezo disk will inconsistently beep. There’s no particular use for it, just to add tension and mimic the original spirit-box.
Once an activity is detected, the box sends the signal to the controlling centre (laptop/processing). The box is able to detect what type of ghost it is as well. A good one, a neutral one, or an evil one.
On processing, there is a “game” that is used to “release” the ghost from its half-dead state. The game difficulty varies based on the type of ghost. The game also utilizes the second system on the Arduino, the “control system”. This is composed of a knob (potentiometer) and a button.
The Game
The Game layouts look like this
Good ghost:
Neutral ghost:
Evil ghost:
The disk in the middle is a “ghostnet”. To control the ghost net, the players use the “control system” on the Arduino. The knob controls direction, and the button controls move or stop. There are 20 spectres every time a ghost is caught, and the number of spectres required to be caught depends on the “evilness” of the ghost. A certain amount of time is given to the players to finish this task, and a red countdown is displayed on the screen. A count for spectres left to catch is also displayed on the screen in white.
Success Screen (Jacob’s your boss):
Fail Screen:
Arduino Layout
Pitfalls and Future Improvements
One of the main troubles that I ran into was the control system for the game. The knob and the buttons were a bit hard to use for the game. Although I originally intended for the controls in the game to be harder, I had to lower the difficulty holistically to account for the difficult control system.
Another difficulty and potential improvement was the mobility of the system. To use the system, the user has to bring a laptop, while taking care of the wire between the Arduino and the computer. For future purposes, the connection between the platforms could be made wireless.
Game Demonstration
Detection Demonstration
Full Code
Arduino
#include "pitches.h"
int piezo = 2;
bool piezoState = false;
int trig = 3;
int echo = 4;
int button = 5;
int buttonState = 0;
int prevButton = 0;
long timer;
int timerlength = 100;
int degree;
//ghost types
bool good = false;
bool neutral = false;
bool evil = false;
int ghostType = 0;
int ghost = 0;
void setup() {
Serial.begin (9600);
Serial.write(0);
timer = millis() + timerlength;
pinMode(trig, OUTPUT);
pinMode(echo, INPUT);
pinMode(button, INPUT);
}
void loop() {
int knobValue = analogRead(A1);
buttonState = digitalRead(button);
Piezo();
if (good == false && neutral == false && evil == false){
Evaluate();
}
else{
noTone(piezo);
knobValue /= 4;
degree = map(knobValue, 0, 255, 0, 12);
Serial.print(degree);
Serial.print(',');
Serial.println(buttonState);
}
}
void Evaluate() {
// For Distance
int time;
int distance;
digitalWrite(trig, HIGH);
delay(0.01);
digitalWrite(trig, LOW);
time = pulseIn(echo, HIGH);
// speed of sound = 343 m/s = 0.0343 cm/µs
// distance = time * speed
distance = time * 0.0343 / 2;
// Serial.print(distance);
// Serial.println(" cm");
// For brightness
int brightness = analogRead(A0);
//overall evaluation
if (millis()%10 == 1 && (Serial.available()>0)){
// Serial.println("here");
if (brightness <= 40 && brightness > 30 && distance <= 30 && distance > 20){
ghost = random(1, 101);
if (ghost < 2){
good = true;
Serial.write(ghost);
}
}
if (brightness <= 20 && brightness > 10 && distance <= 20 && distance >10){
ghost = random(1,101);
if (ghost = 1){
good = true;
Serial.write(ghost);
}
if (ghost = 2){
neutral = true;
Serial.write(ghost);
}
}
if (brightness <= 10 && distance <= 10){
ghost = random(1,101);
if (ghost == 1){
good = true;
Serial.write(ghost);
}
if (ghost == 2){
neutral = true;
Serial.write(ghost);
}
if (ghost == 3){
evil = true;
Serial.write(ghost);
}
}
}
}
void Piezo() {
if (good == false && neutral == false && evil == false){
if (millis() > timer){
tone(piezo, C5);
timer = millis() + timerlength;
}
else if (millis() == timer){
noTone(piezo);
}
}
}
Processing
Main code
import processing.serial.*;
Serial myPort;
import processing.sound.*;
SoundFile die;
SoundFile music2;
PFont font;
//images
PImage bg;
PImage bg1;
PImage bg2;
PImage bg3;
PImage good;
PImage neutral;
PImage evil;
PImage net;
//values:
int ghost = 0;
boolean go = false;
int ndegree = 0;
int fire = 0;
boolean win = false;
//objects:
Net n1;
Spectre[] spectre; //initializing all the spectre
int count = 20;
int livecount;
void setup(){
size(1200,800);
printArray(Serial.list());
String portname=Serial.list()[3];
println(portname);
myPort = new Serial(this,portname,9600);
bg = loadImage("bg.jpeg");
bg1 = loadImage("bg1.jpg");
bg2 = loadImage("bg2.jpg");
bg3 = loadImage("bg3.jpg");
good = loadImage("good.png");
neutral = loadImage("neutral.png");
evil = loadImage("evil.png");
net = loadImage("net.png");
die = new SoundFile(this, "die.mp3");
music2 = new SoundFile(this, "music2.mp3");
//initializing objects
n1 = new Net(width/2, height/2);
spectre = new Spectre[count];
int index = 0;
for (int x = 0; x < count; x++) {
spectre[index++] = new Spectre(random(0, width), random(0, height), 40, 40);
}
for (int i =0; i<count;i++){ //spectre speed initializing
spectre[i].xSpeed = random(-2,2);
spectre[i].ySpeed = random(-2,2);
}
music2.loop();
}
void draw(){
background(0);
//detection phase
if (ghost == 0){
font = createFont("BreatheFire-65pg.ttf", 60);
textFont(font);
textAlign(CENTER);
text("detecting . . .", width/2, 350);
}
//detected
if (ghost != 0){
menu();
}
if (go == true){
game();
checkwin();
}
}
void game(){
if (ghost == 1){
background(bg1);
}
if (ghost == 2){
background(bg2);
}
if (ghost == 3){
background(bg3);
}
n1.display();
n1.move();
n1.checkEdges();
for (Spectre spectre : spectre) {
spectre.display();
spectre.collision();
spectre.update();
spectre.checkEdges();
}
font = createFont("BreatheFire-65pg.ttf", 60);
textFont(font);
textAlign(CENTER);
text(livecount, width/2, 185);
}
void menu(){
if (go == false){
background(bg);
fill(255);
font = createFont("BreatheFire-65pg.ttf", 60);
textFont(font);
textAlign(CENTER);
if (ghost == 1){
text("A kind ghost is in your vicinity. \n set it free", width/2, 185);
count = 10;
}
if (ghost == 2){
text("A neutral ghost is in your vicinity. \n proceed with caution", width/2, 185);
count = 15;
}
if (ghost == 3){
text("An evil ghost is in your vicinity. \n succeed or die", width/2, 185);
count = 20;
}
livecount = count;
font = createFont("Georgia", 40);
textFont(font, 30);
text("Use the spiritbox knob to control ghostnet direction \n Use spiritbox button to move ghostnet \n Catch all spectres for mission success", width /2, 550);
stroke(0);
fill(0);
rectMode(CENTER);
rect(width/2, 680, 200, 72);
fill(255);
textFont(font, 60);
font = createFont("BreatheFire-65pg.ttf", 42);
textFont(font);
textAlign(CENTER);
text("PLAY", width/2, 700);
if (mouseX>=(width/2 - 100) & mouseX<=(width/2 + 100) & mouseY>=650 & mouseY<=720){
stroke(255);
noFill();
rect(width/2, 680, 200, 72);
if (mousePressed){
go = true;
music.stop();
}
}
}
}
void checkwin(){
if (livecount < 1){
win = true;
background(bg);
fill(255);
font = createFont("BreatheFire-65pg.ttf", 60);
textFont(font);
textAlign(CENTER);
text("mission success \n report back to Jacob", width/2, 185);
myPort.write(int(win));
}
}
void serialEvent(Serial myPort){
if (ghost == 0){
ghost = myPort.read();
}
else if (ghost != 0){
String s=myPort.readStringUntil('\n');
s=trim(s);
if (s!=null){
int values[]=int(split(s,','));
if (values.length==2){
//println(values);
ndegree = values[0];
fire = values[1];
}
}
}
myPort.write(0);
}
Ghost Net Class
class Net {
float x;
float y;
float diameter;
int score;
float xSpeed;
float ySpeed;
Net(float tempX, float tempY) {
x = tempX;
y = tempY;
diameter = 45;
score = 0;
}
void display() {
imageMode(CENTER);
image(net,x,y,diameter,diameter);
}
void move() {
xSpeed = -(cos(ndegree) * 3);
ySpeed = - (sin(ndegree) * 3);
if (fire == 1){
x += xSpeed;
y += ySpeed;
}
}
void checkEdges() {
if (y>height) {
ySpeed = -ySpeed;
}
if (y<0) {
ySpeed = -ySpeed;
}
if (x>width) {
xSpeed = -xSpeed;
}
if (x<0) {
xSpeed = -xSpeed;
}
}
}
Spectre Class
class Spectre {
float x;
float y;
float w;
float h;;
float xSpeed, ySpeed;
boolean caught = false;
Spectre(float tempX, float tempY, float tempW, float tempH) {
x = tempX;
y = tempY;
w = tempW;
h = tempH;
xSpeed = ySpeed = 0;
}
void display() {
if (caught == false){
if (ghost == 1){
image(good, x, y, w, h);
}
if (ghost == 2){
image(neutral, x, y, w, h);
}
if (ghost == 3){
image(evil, x, y, w, h);
}
}
}
//collision function for when players catch the right and wrong spectre
void collision() {
if ((n1.x - n1.diameter < x && x < n1.x + n1.diameter) & (n1.y - n1.diameter < y && y < n1.y + n1.diameter)){
x = -50;
y = -50;
caught = true;
livecount --;
die.play();
}
}
void update(){
x += xSpeed;
y += ySpeed;
}
void checkEdges() {
if (y>height) {
ySpeed = -ySpeed;
}
if (y<0) {
ySpeed = -ySpeed;
}
if (x>width) {
xSpeed = -xSpeed;
}
if (x<0) {
xSpeed = -xSpeed;
}
}
}
As you might remember, we jumped between a few ideas, and after brainstorming with Aaron and long conversations among each other, we decided on an idea. When we started, it was a bit vague, but the more progress we made the more everything made sense and became clearer.
So, our project is an experience in which two users answer questions that are meant to explore how similar these users are to each other by mapping the data from their answers onto the screen and then visualizing it gradually as users interact with the set-up. This visualization happens when users’ traces intersect in the area of a question where they answered similarly to reflect connection. The “bloom” we used to resemble connection differs based on how close the answers were, making it the most dramatic when users have the exact same answer.
This final idea ended up reminding us of a lot of things that we find fascinating and enjoy:
The concept of an etch-a-sketch
A card game called “We are Not Really Strangers”
Colors!!
Human connection
The Experience
To take you through our game, we will demonstrate our Arduino set-up and the three different screens that make up our experience:
For Arduino, we wanted to have two different “stations” for each user. Each user on their breadboard has 4 buttons (reflecting Strongly disagree, disagree, agree, strongly agree for answering questions), and two potentiometer knobs for the etch- a – sketch.
Although finding pieces was an initial struggle, people on campus were generous enough to lend us some pieces — especially Amina! Shout out to you :-).
As you can see we had to work with the larger potentiometers, which initially caused a glitch due to loose wires, but then when taped down they worked just fine.
As for the code, we used an ascii handshake to send one button reading and the 4 values from the potentiometers in a comma-separated string, which we trimmed, divided and casted in processing. For the buttons, we gave each button a unique value so we can indicate the answer chosen in processing (rather than just depending on 0 and 1).
Start Screen: we wanted to offer the users some quick pointers before they start. Here, we tried our best not to spoil what’s going to happen afterward but communicate all the needed information.
2. Question Screen: Here, all the questions display in a grid, one after the other. When both users answer, the next question appears, and all answers are recorded and the answer difference between them calculated. This answer difference becomes the measure of their similarity that affects the blooming.
3. Drawing screen: Here, users use the potentiometer knobs to move around the screen, if they intersect in an area where answer difference = 0, they will see a dramatic explosion of color happen in front of them. This explosion will be smaller for answer difference 1 and 2, and non-existent for answer difference 3.
4.End Screen:
This appears after the users “save” their sketch and it explains the concept to them.
Some Main Features in the Code + Key Challenges during Implementation
Questions:
We had a question class, which creates class objects in a grid that we display one by one as users answer. This class includes the text display, variables to record user answers, the variable to calculate answer difference, and booleans to check if the question has been answered.
The Traces:
In the beginning, we were just trying the code using the keyboard, and we were drawing points. We were moving the point one pixel at a time. We, later, realized that this wouldn’t work with the potentiometers. It was also causing way too much discontinuity in the traces. To slightly fix this, and get a better effect overall, we started recording the lastX and lastY read by the potentiometers, and drawing lines from (lastX, lastY) to (currentX,currentY) instead of points.
Collisions
One of the main issues we ran into was detecting collisions between the traces.
There was a lot to think about in order to only detect valid collisions.
We needed to compare the color of pixels in two different PGraphics layers, each one containing the trace of one user. Whenever a pixel location is colored in both of the layers, a collision is detected.
However, we also needed to keep track of collision locations so that a collision wouldn’t be detected twice in the same location.
Since the strokeWeight of the traces is larger than just 1 pixel, and the lines are not just at 90-degree angles (they can be as squiggly as your heart desires) we also needed to check if a collision has already happened within a radius close to the current collision detected.
The bloom
For the bloom, it took us a while to figure out what we wanted it to look like. We ended up using code we found on OpenProcessing (credits go to the post https://www.openprocessing.org/sketch/486216 ), as our base. We struggled a bit to incorporate it into our code, but we tweaked parts of it and changed some parameters to get it to work with our idea.
User Test:
Testing this experience with people actually tied our concept together, it suddenly made much more sense when we saw how the users went about it and their reactions. For example, in the case of the video above, after the users read through the end screen their reactions were something like: “oh! This is a painting about our friendship”, then discussing their answers and how similar and different they are, and then asking if they could print it. Although we tested it with friends, we think this could be a great conversation starter for two strangers!
Future Improvements
Perhaps adding more questions or delving deeper with what we ask could be an interesting improvement to this experience, adding more meaning to it. Furthermore, in terms of the Arduino setup, we really wanted to make it more appealing and better divided, however, the limitations made it a bit difficult to make it look neater.
Finally, if we had more chances to user test we would’ve definitely been able to reveal details in the user experience that could be enhanced.
import processing.serial.*;
Serial myPort;
PGraphics user1, user2, blooms;
ArrayList<Integer[]> collisions;
int c_index = -1;
IntList collisionLocations;
Question[] questions;
int screen = 0;
int questionNumber = 0;
int Answer = 0;
int currentU1Answer = 0;
int currentU2Answer = 0;
String timestamp;
int qCols = 3;
int qRows = 3;
//int goingThruQs = 0;
int x;
int y;
int lastX;
int lastY;
int x2;
int y2;
int lastX2;
int lastY2;
float a, b, c, d, e, f ;
PImage traceBg;
PImage questionBg;
PImage startScreen;
PImage endScreen;
void setup() {
background(0);
fullScreen();
println(width + " " + height);
String portname = Serial.list()[2];
myPort = new Serial (this, portname, 9600);
myPort.clear();
myPort.bufferUntil('\n');
collisionLocations = new IntList();
x = 0;
y = height/2;
lastX = 0;
lastY = height/2;
x2 = width;
y2 = height/2;
lastX2 = width;
lastY2 = height/2;
questions = new Question[qCols*qRows];
int questionIndex = 0;
for (int r2 = 0; r2 < qRows; r2++) {
for (int c2 = 0; c2 < qCols; c2++) {
questions[questionIndex] = new Question( questionIndex, c2*640, r2*360);
questionIndex +=1;
}
}
collisions = new ArrayList();
user1 = createGraphics(width, height);
user2 = createGraphics(width, height);
blooms = createGraphics(width, height);
pts = new ArrayList<Particle>();
traceBg = loadImage("bg2darker.jpg");
startScreen = loadImage("startScreen.png");
questionBg = loadImage("bg.jpg");
questionBg.resize(width, height);
endScreen = loadImage("endScreen.png");
}
void draw() {
if (screen == 0) {
image(questionBg, 0, 0, width, height);
image(startScreen, 50, 50 );
}
if (screen == 1) {
questions[questionNumber].display();
goingThroughQuestions(questionNumber);
if (questions[questionNumber].user1Answered && questions[questionNumber].user2Answered && questionNumber < 8 ) {
questions[questionNumber].answerDifference = abs(questions[questionNumber].user1Answer - questions[questionNumber].user2Answer);
println(questions[questionNumber].answerDifference);
questionNumber += 1;
}
if (questionNumber == 8 && questions[questionNumber].user1Answered && questions[questionNumber].user2Answered) {
questions[questionNumber].answerDifference = abs(questions[questionNumber].user1Answer - questions[questionNumber].user2Answer);
image(traceBg, 0, 0, width, height);
screen = 2;
}
}
if (screen == 2) {
stroke(183,150,172,50);
strokeWeight(5);
line(x, y, lastX, lastY);
user1.beginDraw();
user1.stroke(255);
user1.strokeWeight(5);
user1.line(x, y, lastX, lastY);
user1.endDraw();
lastX = x;
lastY = y;
stroke(120,150,220, 50);
strokeWeight(5);
line(x2, y2, lastX2, lastY2);
user2.beginDraw();
user2.stroke(255, 0, 0);
user2.strokeWeight(5);
user2. line(x2, y2, lastX2, lastY2);
user2.endDraw();
lastX2 = x2;
lastY2 = y2;
//image(user1, 0, 0);
//image(user2,0,0);
user1.loadPixels();
user2.loadPixels();
for (int i=0; i<width; i++)
{
for (int j=0; j<height; j++)
{
if ( user1.get(i, j) == color(255) && user2.get(i, j) == color(255, 0, 0) &&!exists(i, j) && !existsInRadius(i, j))
{
Integer[] c= {i, j};
collisions.add(c);
collisionLocations.append(location(c[0], c[1]));
c_index += 1;
yes = true;
}
}
}
blooms.beginDraw();
blooms.smooth();
blooms.colorMode(HSB);
blooms.rectMode(CENTER);
if (c_index > -1)
{
bloomValues(questions[collisionLocations.get(c_index)].answerDifference);
//blooms.ellipse(collisions.get(i)[0], collisions.get(i)[1], diameter, diameter);
if (a != -1 && b != -1 && c != -1 && d != -1 && e != -1 && f != -1) {
drawcool(blooms, collisions.get(c_index)[0], collisions.get(c_index)[1], a, b, c, d, e, f);
}
}
blooms.endDraw();
image(blooms, 0, 0);
}
if (screen == 3) {
image(questionBg,0,0);
image(endScreen,50,50);
}
}
void goingThroughQuestions(int i) {
//answer is not zero only when a button has been pressed
if ( Answer == 1 || Answer == 3 || Answer == 5 || Answer == 7) { //if one of these buttons is pressed then it's the first user
if (!questions[i].user1Answered) { // if the question hadn't been answered already
questions[i].user1Answer = currentU1Answer; //assign the user 1 arduino variable to the user1 question variable
questions[i].user1Answered = true;
//println("user 1 " + questions[i].user1Answer);
//not sure if the following 2 lines make sense, to me it's like restarting and waiting for a new button press
currentU1Answer = 0;
Answer = 0;
}
}
if ( Answer == 11 || Answer == 13 || Answer == 15 || Answer == 17) {
if (!questions[i].user2Answered) {
questions[i].user2Answer = currentU2Answer;
questions[i].user2Answered = true;
currentU2Answer = 0;
Answer = 0;
}
}
}
void serialEvent(Serial myPort) {
String s=myPort.readStringUntil('\n');
s=trim(s);
if (s!=null) {
int values[]=int(split(s, ','));
if (values.length==5) {
Answer = (int)values[0];
x=(int)map(values[1], 0, 1023, 0, width);
y=(int)map(values[2], 0, 1023, 0, height);
x2=(int)map(values[3], 0, 1023, 0, width);
y2=(int)map(values[4], 0, 1023, 0, height);
}
}
if (Answer != -1) {
if (Answer == 1 || Answer == 3 || Answer == 5 || Answer == 7) {
currentU1Answer = Answer;
}
//second set of buttons for user 2
if (Answer == 11 || Answer == 13 || Answer == 15 || Answer == 17) {
currentU2Answer = Answer - 10 ;
}
}
myPort.write('0');
}
//function to check if collision already exists in array
boolean exists(int x, int y)
{
for (int i=0; i<collisions.size(); i++)
{
if (collisions.get(i)[0]==x && collisions.get(i)[1]==y)
{
return true;
}
}
return false;
}
//function to check if collisions exists in radius
boolean existsInRadius (int x, int y)
{
for (int i=0; i<collisions.size(); i++)
{
if (sq(collisions.get(i)[0] - x) + sq(collisions.get(i)[1] - y) <= 20*20)
{
return true;
}
}
return false;
}
//function to check in which question's "area" the collision happened in
int location(int x, int y) {
int qIndex = 0;
for (int r2 = 0; r2 < qRows; r2++) {
for (int c2 = 0; c2 < qCols; c2++) {
if (x >= c2*640 && x <= (c2+1)*640 && y >= r2*360 && y <= (r2+1)*360) {
return qIndex;
}
qIndex +=1;
}
}
return -1;
}
//function to take parameters for the bloom based on the answer difference
void bloomValues(int diff) {
if (diff == 0) {
//lifespan
a = 70; //80;
b = 150; //120;
//decay
c = 0.7;
d = 0.99;
//weightRange
e = 60;
f = 180; //120;
} else if (diff == 2) {
//lifespan
a = 55;
b = 80;
//decay
c = 0.63;
d = 0.9;
//weightRange
e = 20;
f = 100;
} else if (diff == 4) {
//lifespan
a = 30;
b = 65;
//decay
c = 0.63;
d = 0.83;
//weightRange
e = 8;
f = 35;
} else if (diff == 6) {
a = b = c = d = e = f = -1;
}
}
void keyPressed() {
if (key == 'S' || key == 's') {
timestamp = year() + nf(month(), 2) + nf(day(), 2) + "-" + nf(hour(), 2) + nf(minute(), 2) + nf(second(), 2);
saveFrame(timestamp+".png");
screen = 3;
}
}
void mouseClicked(){
screen = 1;
}
Bloom Class
ArrayList<Particle> pts;
boolean yes;
int coolingTime;
void drawcool(PGraphics pg, Integer x, Integer y, float a, float b, float c, float d, float e, float f) {
pg.beginDraw();
if (yes)
{
for (int i=0;i<10;i++) {
Particle newP = new Particle(x, y, i+pts.size(), i+pts.size(), a,b,c,d,e,f);
pts.add(newP);
}
}
for (int i=0; i<pts.size(); i++) {
Particle p = pts.get(i);
p.update();
p.display(pg);
}
for (int i=pts.size()-1; i>-1; i--) {
Particle p = pts.get(i);
if (p.dead)
{
pts.remove(i);
}
}
coolingTime++;
stop();
//yes = false;
pg.endDraw();
}
void stop(){
if (coolingTime>1)
{
yes = false;
coolingTime = 0;
}
}
//CLASS
class Particle{
PVector loc, vel, acc;
int lifeSpan, passedLife;
boolean dead;
float alpha, weight, weightRange, decay, xOffset, yOffset;
color c;
float a, b,cc,d,e,f;
Particle(float x, float y, float xOffset, float yOffset, float thisa , float thisb , float thisc, float thisd, float thise ,float thisf){
loc = new PVector(x,y);
a = thisa;
b = thisb;
cc = thisc;
d = thisd;
e = thise;
f = thisf;
float randDegrees = random(360);
vel = new PVector(cos(radians(randDegrees)), sin(radians(randDegrees)));
vel.mult(random(1));
acc = new PVector(0,0);
lifeSpan = int(random(a, b)); //HERE IS HOW LONG THE ANIMATION IS (timewise)
decay = random(cc, d); //HERE IS HOW FAR IT GOES
c = color (random(255), random(255), 255, 75);
weightRange = random(e,f);//random(5,90); //HERE IS THE SIZE
this.xOffset = xOffset;
this.yOffset = yOffset;
}
void update(){
if(passedLife>=lifeSpan){
dead = true;
}else{
passedLife++;
}
alpha = float(lifeSpan-passedLife)/lifeSpan * 70+50;
weight = float(lifeSpan-passedLife)/lifeSpan * weightRange;
acc.set(0,0);
float rn = (noise((loc.x+frameCount+xOffset)*0.01, (loc.y+frameCount+yOffset)*0.01)-0.5)*4*PI;
float mag = noise((loc.y+frameCount)*0.01, (loc.x+frameCount)*0.01);
PVector dir = new PVector(cos(rn),sin(rn));
acc.add(dir);
acc.mult(mag);
float randDegrees = random(360);
PVector randV = new PVector(cos(radians(randDegrees)), sin(radians(randDegrees)));
randV.mult(0.5);
acc.add(randV);
vel.add(acc);
vel.mult(decay);
vel.limit(3);
loc.add(vel);
}
void display(PGraphics pg){
pg.beginDraw();
pg.strokeWeight(0);
pg.stroke(0, alpha);
pg.point(loc.x, loc.y);
pg.strokeWeight(weight);
pg.stroke(c);
pg.point(loc.x, loc.y);
pg.tint (255,125);
pg.endDraw();
}
}
Question Class
class Question {
//variables to store input answers
PImage image;
int user1Answer;
int user2Answer;
int answerDifference;
//question location (corresponding to area on screen)
int x;
int y;
int coverX = 0;
//question box dimensions (it could be something that we design and upload to processing(?))
int w = 640;
int h = 360;
int c = 255; //alpha
int questionNumber;
String questionText;
boolean user1Answered = false;
boolean user2Answered = false;
boolean sameAnswer;
Question(int tempQNumber, int tempX, int tempY) {
questionNumber = tempQNumber;
x = tempX;
y = tempY;
image = loadImage("question"+(questionNumber+1)+".png");
}
void display() {
//background(0);
image(questionBg,0,0);
fill(0);
//noStroke();
stroke(255);
image(image,x, y);
fill(255);
//textSize(40);
//text(questionText, x+w/4, y+h/4);
}
}
This project is very special for me because it’s centered around the first show I have ever watched and one of my favorites ever. I have tried to encapsulate every aspect of the theme of the show into the project as possible and the result is pretty satisfying.
Main Challenge:
One of the biggest challenges for me was to be able to do all the functionalities I wanted efficiently. Passing by reference cannot be done in Java except using objects types so I had to find my way around that. I created a matrix of arrays storing answers for the questions with each row representing a character in the show. But in order for me to randomize the order of the answers every single time the game is played, I used the StringList class. I created a list for every answer, which is a copy of a row in the matrix, then I shuffle using the shuffle() function and I display. The original matrix is kept for me to be able to check which answer exactly is chosen to increment the respective character’s score accordingly. I created a function to check which character’s answer exactly was chosen, and I use the same code for every question page with the appropriate changes. But in order for me to to have all these if conditions to not change question unless a button was pressed and to be able to identify the answer properly, I had to actually write numerous lines of code. But the runtime of the code is actually not that long because it is just mainly if conditions which have a runtime of O(1).
Implementation:
I used the main theme song in the background at all times and the player can lower the volume because it can be too repetitive. The frames of the options and in the instructions and the buttons on the Arduino board all have the same pattern of colors in the same order (red, blue, yellow, red, yellow blue). I used the same shade of purple and gold as in Monica’s apartment throughout the game. I also used the same font as in the original series’ logo. The yellow bulb indicates that the game is waiting for you to choose an answer, the blue blinks while you are pressing, and the green lights up when you finish the trivia. I made it so the instructions appear only during the first time you start the game and not if you replay because by then you surely know how to play. I made it so the choices change color if you hover over them using the mouse to make it more interactive. I was worried people might think this means they can click to choose, but then if they try to click nothing will happen, and the instructions page at the start actually explains that they should click on the Arduino board, so it should be clear enough. Whenever an option is chosen, the choice color changes momentarily as well like when the mouse hovers over it to indicate what exactly was actually chosen. I used 4 questions from the websites I included as reference in the code just to have an existing reference, but I made up the remaining 7 questions on my own in order to make it more personal. The questions at first are very easy, meaning that you can guess right away which answer represents which of the six characters. But as you go further, the answers are less direct and you cannot always guess right away who is who, which is just because the player is improving in answering the questions by time and to make it more interesting and as if it’s an actual realistic test that is interactive.
This was the Arduino progress in the first stage.
Background Image of the trivia.
This was how I built the logic for the volume knob of the theme song.
The result screens I designed at the end of the trivia based on the answers.
I actually tried to choose the most general traits of the characters so that regardless of the combination of answers leading to such a final result, the player will most likely have at least most of the things which I included in the short paragraph describing their character. I also made it 11 questions (a prime number) so that there will never be a tie in answers where I will have to choose one as winner or put both which is just lame.
This is the final result from the Arduino perspective.
Instructions for the trivia. I tried to make it as visual and realistic as possible, and I even used the same colors to make it clearer for the player. The same font is used consistently in order to maintain the theme at all times.
This is the actual final result on the computer.
Following is the Arduino code for the project:
int choice1 = 0;
int choice2 = 0;
int choice3 = 0;
int choice4 = 0;
int choice5 = 0;
int choice6 = 0;
int last = 0;
int start = 0; //some variables to store the values from the serial by parseInt
int one = 7;
int two = 6;
int three = 5;
int four = 4;
int five = 3;
int six = 2; //outlets of the six buttons
int ledYellow = 8;
int ledBlue = 9;
int ledGreen = 10; //outlets of the three led bulbs
void setup() {
Serial.begin(9600);
Serial.println("0,0,0");
pinMode(ledYellow, OUTPUT);
pinMode(ledBlue, OUTPUT);
pinMode(ledGreen, OUTPUT);
pinMode(one, INPUT);
pinMode(two, INPUT);
pinMode(three, INPUT);
pinMode(four, INPUT);
pinMode(five, INPUT);
pinMode(six, INPUT);
}
void loop() {
while (Serial.available()) {
choice1 = Serial.parseInt();
choice2 = Serial.parseInt();
choice3 = Serial.parseInt();
choice4 = Serial.parseInt();
choice5 = Serial.parseInt();
choice6 = Serial.parseInt();
last = Serial.parseInt(); //stores whether or not the game is off
start = Serial.parseInt(); //stores whether or not the game has started
if (Serial.read() == '\n') {
int sensor = analogRead(A0);
delay(1);
int button1 = digitalRead(one);
delay(1);
int button2 = digitalRead(two);
delay(1);
int button3 = digitalRead(three);
delay(1);
int button4 = digitalRead(four);
delay(1);
int button5 = digitalRead(five);
delay(1);
int button6 = digitalRead(six);
delay(1);
if (start == 1) { //as soon as the game begins after instructions yellow lights
if (button1 == HIGH || button2 == HIGH || button3 == HIGH || button4 == HIGH || button5 == HIGH || button6 == HIGH) { //while any button is pressed, light blue and dim rest
digitalWrite(ledBlue, HIGH);
digitalWrite(ledYellow, LOW);
}
else { //otherwise only keep yellow lit
digitalWrite(ledBlue, LOW);
digitalWrite(ledGreen, LOW);
digitalWrite(ledYellow, HIGH);
}
}
else { //if game is not on switch yellow and blue off
digitalWrite(ledYellow, LOW);
digitalWrite(ledBlue, LOW);
if (last == 1) { //if it was specifically on the end page where a pic of character shows, light up the green bulb signifiying end
digitalWrite(ledGreen, HIGH);
}
else { digitalWrite(ledGreen, LOW);}
}
Serial.print(button1);
Serial.print(",");
Serial.print(button2);
Serial.print(",");
Serial.print(button3);
Serial.print(",");
Serial.print(button4);
Serial.print(",");
Serial.print(button5);
Serial.print(",");
Serial.print(button6);
Serial.print(",");
Serial.println(sensor); //print everything back to complete the handshake
}
}
}
void setup() {
// put your setup code here, to run once:
Serial.begin(9600);
}
void loop() {
// put your main code here, to run repeatedly:
int sensor = analogRead(A0);
delay(1); //avoid getting noisy values.
sensor /= 4; //why is it incrementally devided by 4? maybe slow it down
//also 2^8 is 256 or 1024/4 sends one byte at a time to the serial,
Serial.write(sensor);
}
It’s been quite a journey. I put tremendous time and effort trying to perfect the details and coordinations of everything in the game. Hope you like it. At first I had a bit of trouble deciding which idea to stick to, but I am glad I ended up coming through with the “SPACE DEFENSE” game. As same as other final projects for this course, I brought Arduino and Processing together to make the game. When I first designed the development of the game, I broke it down into three main milestones with the respective order of priority:
Processing game design
Arduino architecture
Serial connection
Accomplishing the first milestone, therefore, took up the most time. I initially made the game without the Arduino part and included the use of the keyboard to make the development more convenient and seamless. This is because it seemed rather easier for me to replace the keyboard use with the button use through serial communication than leaving numerous chunks blank. I still commented out not all but most of the parts necessary to run the keyboard to play the game. So find parts as shown below and simply uncomment them to use the keyboard instead! (apart from the uncommenting you need to do some tweaking into the code of course)
In terms of the code in Processing, I have in total of 4 classes (Player, Laser, Item, and Asteroid). I used polygons instead of images on asteroids for three reasons. First, I resized the images in Processing and that made the whole game slow. After talking with Aaron, I realized I could have cropped or resized images outside of Processing. However, there was another reason I preferred not to use an image for an asteroid. I thought it would be awkward to recreate the specific movement effect on an image, and the way my code generated different shapes, sizes, trajectories made the game more resourceful. Lastly, employing shapes instead of simply importing images seemed like a bigger challenge to me so I went with it. This is because these polygons have different shapes and vertices and so the collision effect on them required more thinking and effort than on uniform images.
The biggest challenge for me was controlling and coordinating the movement of different objects. This is because I had to manage a set of situations where different objects collide with one another. The next challenge was serial communication. I just had to spend some time figuring out the use of data transferred from Arduino within different classes. Also because when an error arose I had to check both Processing and Arduino, so that required extra investigation on a broader scope. After completing the general skeleton code of the game, I initially connected the maneuvering of the spaceship to the potentiometer. So depending on the degree of rotation, the spaceship would either move left or right. But after trying this for a while, I realized how inconvenient and clumsy it gets when using the potentiometer. The rotation of the button was simply too difficult to carry out that it was annoying to invest so much energy when playing the game. Below is a little sneak peak of how the potentiometer was set up in my Arduino board at first.
With this problem and user experience in mind, I decided to use push buttons. I also decided to get rid of UP & DOWN buttons because it made physiologically little sense that LEFT & RIGHT & UP & DOWN buttons are all aligned next to each other. So the spaceship was only allowed to move sideways and this was enough. I then added two push buttons – one for shooting bullets and the other for resetting the game. Below is how my Arduino looked like in the end:
In terms of the design, I tried to make it sort of retro. The song, the font, the color, the spaceship have all been designed in a way that it makes you feel nostalgic (at least for me). The sound track of the game is from a 1996 arcade game called “Out Run”, which is a car racing game but it also suits well in my game in my opinion. Here are the images and sounds I used.
starship (player)
earth (item)
ouch.mp3 (when the player collides with an asteroid)
laser.mp3 (when shooting bullets)
soundtrack.mp3 (sound track from Out Run 1996 arcade game)
life.mp3 (when the player gets an item and health goes up)
crash.mp3 (when an asteroid collides with the land)
Of course, a simple set of instructions are displayed before the game is initiated with the game title on top. I did not add any extra effects on this part because it is retro. To talk about how the game is run, the player can maneuver the starship left and right in order to shoot bullets at the falling “asteroids” that are polygons. I have limited the maneuver to left and right for the reason mentioned earlier. If a player collides with an asteroid he/she will lose 2 health points. If an asteroid collides with the earth, the health will also decrease by 10, as it is a fatal and collateral damage to the global community. If a player successfully hits an asteroid with a bullet, the asteroid will vanish and will be deducted from the total number of asteroids in the current level. There will be earth-looking items falling too and you should shoot at it to get 10+ on your health. Your level goes up when you eliminate all the asteroids in the level. Each level you go up, there will be 1 less earth item and 2 more asteroids. Without further ado, here’s a demo video.
// Main
import ddf.minim.*;
/************* Serial **************/
import processing.serial.*;
Serial myPort;
int xPos=0;
int left, right, b, reset;
/************* Serial **************/
Player player;
Item item;
PFont font, font2;
PImage starship, earth, galaxy;
ArrayList<Asteroid> asteroids;
ArrayList <Laser> laser;
ArrayList<Item> items;
boolean instruction = true;
boolean [] keys = new boolean[128];
String message;
int[] land;
int health, asteroidCount, itemCount;
int asteroid_ = 0;
int item_ = 0;
int level = 1;
int count = 1;
//sound effect
Minim minim;
AudioPlayer soundtrack, sound, sound2, sound3, sound4;
void setup() {
size(480, 640);
if (count == 1){
/************* Serial **************/
printArray(Serial.list());
String portname = Serial.list()[5];
println(portname);
myPort = new Serial(this,portname,9600);
myPort.clear();
myPort.bufferUntil('\n');
/***********************************/
}
// Sound effect processing
minim = new Minim(this);
soundtrack = minim.loadFile("soundtrack.mp3");
sound = minim.loadFile("laser.mp3");
sound2 = minim.loadFile("crash.mp3");
sound3 = minim.loadFile("ouch.mp3");
sound4 = minim.loadFile("life.mp3");
sound2.setGain(-10);
// Images
starship = loadImage("starship.png");
earth = loadImage("earth.png");
// Font
font2 = createFont("Cambria-bold",60);
font = createFont("Phosphate-Inline",60);
// asteroid_ == 0 means when you start the game
// else meanse after Level 1
health = 100;
if (asteroid_ == 0 && count == 1){
asteroidCount = 10;
itemCount = 5;
soundtrack.rewind();
soundtrack.play();
}
else {
asteroidCount = 10 + asteroid_;
itemCount = 5 + item_;}
/********** Initiating Objects ***********/
// Player
player = new Player();
// Meteor
asteroids = new ArrayList<Asteroid>();
for (int i=0; i < asteroidCount; i++) {
asteroids.add(new Asteroid());}
// Bullet
laser = new ArrayList();
// Earth item
items = new ArrayList<Item>();
for (int i=0; i < itemCount; i++) {
items.add(new Item());}
/*******************************************/
// Background
land = new int [width/10+1];
for (int i = 0; i < land.length; i++) {
land[i] = int(random(10));}
message = "Level ";
}
void draw() {
if (instruction){
galaxy = loadImage("instruction_background.jpg");
//Using the width and height of the photo for the screen size
galaxy.resize(width,height);
image(galaxy,0,0);
textAlign(CENTER);
textFont(font);
fill(#FFFF00);
text("SPACE DEFENSE",width/2,height/2-120);
textFont(font2);
textSize(20);
fill(255);
text("Protect the planet from falling asteroids!\n* * *\nPress YELLOW to move left\nPress BLUE to move right\nPress RED to shoot\nPress GREEN to restart\n* * *\nClick anywhere to proceed",width/2,height/2);
return;
}
background(0);
fill(255);
noStroke();
ellipse(random(width), random(height-50), 3, 3);
textSize(32);
fill(255);
text(message+level, width/2, 100);
drawEarth();
for (int i= asteroids.size()-1; i >= 0; i--) {
Asteroid p = asteroids.get(i);
p.update();
p.display();
if (player.checkCollision(p)) {
if (health > 0) {
health -= 2;
}
sound3.rewind();
sound3.play();
}
for (int j= laser.size()-1; j >= 0; j--) {
Laser b = laser.get(j);
// If an asteroid appears on screen and collides with a bullet
if ((p.pos.y + p.r > 0) && (p.checkCollision(b))) {
if (asteroids.size() > 0){
asteroids.remove(i);
laser.remove(j);
asteroidCount = asteroids.size();
}
sound2.rewind();
sound2.play();
}
}
}
for (int i= items.size()-1; i >= 0; i--) {
Item it = items.get(i);
it.update();
it.display();
for (int j= laser.size()-1; j >= 0; j--) {
Laser b = laser.get(j);
// If an item appears on screen and collides with a bullet
if ((it.po.y + 25) > 0 && (it.checkCollision(b))) {
if (items.size() > 0){
laser.remove(j);
items.remove(i);
if (health >= 100) {
continue;
}
health += 10;
sound4.rewind();
sound4.play();
}
}
}
}
if (asteroids.size() <= 0) {
delay(1000);
count = 0;
asteroid_ += 2;
item_ -= 1;
setup();
level += 1;
}
player.display();
player.move();
for (int i= laser.size()-1; i >= 0; i--) {
Laser b = laser.get(i);
b.move();
b.display();
}
if (b == 1) {
Laser temp = new Laser(player.pos.x+16, player.posy);
laser.add(temp);
sound.rewind();
sound.play();
}
if (reset == 0) {
count = 0;
asteroid_ = 0;
item_= 0;
level = 1;
setup();
}
textSize(32);
if (health > 0) {
text(health, width - 100, 50);
} else {
text(0, width - 100, 50);
}
if (health <= 0) {
String over = "Game Over";
textSize(60);
text(over, width/2, height/2);
}
textSize(32);
fill(255, 0, 0);
text(asteroidCount, 100, 50);
}
void asteroidShape(float a, float b, float r, int vertices) {
float x = map(r, 0, 40, 50, 255);
float degree = TWO_PI / vertices;
color col = color(x/2, x/3, 100);
beginShape();
for (float i = 0; i < TWO_PI; i += degree) {
float sx = a + cos(i) * r;
float sy = b + sin(i) * r;
fill(col);
noStroke();
//curveVertex(sx, sy);
vertex(sx, sy);
}
endShape(CLOSE);
}
void drawEarth() {
fill(250, 150, 0, 60);
noStroke();
beginShape();
vertex(0, 640);
for ( int i=0; i < land.length; i++) {
vertex( i * 11, 640 - 40 - land[i] );
}
vertex(480, 640);
endShape(CLOSE);
}
void mousePressed() {
if (instruction) {
instruction = false;
}
//level = 1;
}
void keyPressed() {
keys[keyCode] = true;
}
void keyReleased() {
keys[keyCode] = false;
}
/************* Serial **************/
void serialEvent(Serial myPort){
String s = myPort.readStringUntil('\n');
s = trim(s);
if (s != null){
int values[] = int(split(s,','));
if (values.length == 4){
left = int(values[0]);
right = int(values[1]);
b = int(values[2]);
reset = int(values[3]);
}
}
myPort.write('0');
// For key board use:
//float xPos = myPort.read();
//player.pos.x = map(xPos, 0.0, 225.0, 0.0, 390.0)
//myPort.write(0);
}
/************* Serial **************/
// Laser class
class Laser{
PVector p;
PVector a;
PVector v;
int w, h;
Laser(float tx, float ty){
p = new PVector(tx, ty);
a = new PVector(0,-0.2);
v = new PVector(0, -2.5);
}
void display(){
noStroke();
fill(random(255), random(255), random(255));
rect(p.x, p.y, 5, 12);
}
void move(){
v.add(a);
p.add(v);
}
}
// Item class
class Item{
PVector po;
PVector ac;
PVector ve;
Item(){
po = new PVector(random(width), random(-height, 0));
ac = new PVector(0, 0.001);
ve = new PVector(0, 0);
}
void update(){
ve.add(ac);
ve.limit(1);
po.add(ve);
// Readjust location if the player goes off screen on the side
if((po.x < -25) || (po.x > width + 25)){
po.x = random(width);
po.y = random(-height+100, 0);
}
if (po.y > height - 25){
po.y = random(-height+100,0);
po.x = random(width);
// meteor crash
sound2.rewind();
sound2.play();
}
}
void display(){
pushMatrix();
translate(po.x, po.y);
image(earth, 0, 0);
popMatrix();
}
boolean checkCollision(Laser b){
if((b.p.y + 6) <= po.y + 25 && b.p.x >= po.x - 10 && b.p.x <= po.x + 50){
sound.rewind();
sound.play();
return true;
}
else {
return false;
}
}
}
// Asteroid class
class Asteroid{
PVector pos;
PVector acc;
PVector vel;
int vertices;
float r;
color c;
Asteroid(){
pos = new PVector(random(width), random(-height, 0));
acc = new PVector(random(-0.1, 0.1), random(0.1, 0.7));
vel = new PVector(0, 0);
vertices = int(random(3, 15)); // what type of polygon? triangle ~ 15-gon
r = random(10, 45);
}
void update(){
vel.add(acc);
vel.limit(2);
pos.add(vel);
// Readjust location if the player goes off screen on the side
if((pos.x + r < 0) || (pos.x - r > width)){
pos.x = random(0, width);
pos.y = random(-height, 0);
}
if (pos.y + r > height){
if (health > 0){
health -= 10; // reduce health
}
pos.y = random(-height,0);
pos.x = random(0, width);
// meteor crash
sound2.rewind();
sound2.play();
}
}
void display(){
pushMatrix();
translate(pos.x, pos.y);
rotate(frameCount / 50.0);
asteroidShape(0, 0, r, vertices);
popMatrix();
}
boolean checkCollision(Laser b){
if((b.p.y + 6) <= pos.y + r && b.p.x >= pos.x - r && b.p.x <= pos.x + r){
sound.rewind();
sound.play();
return true;
}
else {
return false;
}
}
}
The game is sort of a recreation of the Flappy Bird game but I added my own features to make it different from the normal Flappy Bird game that we are all familiar with.
The main idea of the game is for the user to be able to control the Bird through two different methods of their own choosing at the start of the game: Photoresistor or Microphone, and the objective is to control the bird and avoid all of the incoming covid 19 viruses. If the user manages to stay alive for 30 seconds, then the user wins. However, I added a twist where each time the user wins, the covid viruses’ speed will be even faster, therefore increasing the level of difficulty for the user.
Walkthrough of the Game
When first started, the screen will display several things for the users first such as the Instructions, the types of controls that the user can choose to play with (Photoresistor or Microphone), a brief tutorial for the user about the two types of controls and how to use them, and with a brief note for the users about how the virus will increase in speed each time the user wins. Once finished reading, there will be a start button for the user to press to start playing.
Once the user has either won or lost, there will be a win or lose screen with instructions for the user to press the mouse to restart and to play again.
Press the start button again to play once more!
Difficulties/Challenges
Throughout the entire process of creating this, the hardest part was definitely with the coding ,whereas the hardwiring part was extremely easy as compared to the coding.
Regarding some of the major difficulties I encountered in coding, only some because I had encountered a lot of big and small ones, I’m just going to go through each one by one.
The first part where I struggled a bit would be mapping the microphone values to the up and down movement of the bird. The reason I struggled with this is because I have never done anything like this before, so I had to search up the basics through the Processing forum about amplitude and what is it and other tutorial videos online to learn about how to incorporate it into my game. Also, I sought out help with Professor to know more about how to use various things such as what is an amplitude analyzer, etc. However, in the end, I managed to learn more about this new feature that I’ve never used before thanks to this experience.
The second part that I struggled quite a lot with as well would most likely be when I wanted to add both modes (photoresistor and microphone) into a single file and to even add the option for the user to be able to choose which mode they want to play in using the input coming from the Arduino board as it will be decided with the buttons. However, I was able to get over this difficulty as well by consulting with Jack and Aaron.
There are definitely a lot more difficulties that I had encountered throughout this entire process, but mostly are minor coding mistakes as compared to the aforementioned ones. These are also easily overcame by myself by simply looking at the problem in a different perspective and to not overthink some solutions as the answer could just simply be a slight change of position for a piece of code or something.
const int blueSwitch = 3;
const int redSwitch = 4;
const int sensor = A0;
void setup() {
// put your setup code here, to run once:
Serial.begin(9600);
Serial.println("0,0");
pinMode(blueSwitch, INPUT);
pinMode(redSwitch, INPUT);
pinMode(sensor, INPUT);
}
void loop() {
if (Serial.available() > 0) {
char inByte = Serial.read();
int photoresistor = analogRead(sensor);
delay(10);
int sensor2 = digitalRead(blueSwitch);
int sensor3 = digitalRead(redSwitch);
Serial.print(photoresistor);
Serial.print(',');
Serial.print(sensor2);
Serial.print(',');
Serial.println(sensor3);
}
}
Processing Code
Code for Main Sketch
import processing.serial.*;
import processing.sound.*;
Amplitude amp;
AudioIn in;
float smoothedNum = 0;
Serial myPort;
Virus [] virus;
Clouds [] cloud;
Bird bird;
Timer timer;
PImage Birdimg;
PImage Virusimg;
PImage Cloudimg;
boolean buttonPressed;
int gameState;
int numberOfVirus = 1;
int buttonX, buttonY, buttonW, buttonH;
int numberofClouds = 5;
int interactionMode;
String S= "You Win! \n Press the mouse to restart";
String Instructions= "Instructions: \n Control the Bird Up & Down and Avoid All Viruses \n \n Choose Control Mode: \n Red Button = Photoresistor \n Blue Button = Microphone\n\nControl Tutorial:\n Photoresistor= Hover your hand over the photoresistor and move your hand up and down to control the bird \nMicrophone= Control the Bird with your voice \n\n NOTE: \n Each time you win, the Viruses will become faster next round \n \n";
SoundFile gameplayBGMusic;
SoundFile WinBGMusic;
SoundFile LoseBGMusic;
void setup() {
size (1400, 600);
printArray(Serial.list());
String portname=Serial.list()[4];
println(portname);
myPort = new Serial(this, portname, 9600);
//myPort.clear();
//myPort.bufferUntil('\n');
gameState=0;
//Assume the start button has not been pressed
buttonPressed = false;
//Parameters for the start button
buttonW=335;
buttonH=100;
textSize(buttonH);
buttonX=(width-buttonW)/2;
buttonY=(height-buttonH)/2;
Birdimg= loadImage("Bird Picture #1.png");
Virusimg= loadImage("Virus Picture #1.png");
Cloudimg = loadImage ("Clouds.png");
cloud = new Clouds [numberofClouds];
for (int i=0; i<cloud.length; i++) {
cloud[i] = new Clouds(Cloudimg);
}
virus = new Virus [numberOfVirus];
for (int i=0; i< virus.length; i++) {
virus[i] = new Virus(Virusimg);
}
bird = new Bird(Birdimg);
timer = new Timer();
amp = new Amplitude(this);
in = new AudioIn(this, 0);
in.start();
amp.input(in);
gameplayBGMusic = new SoundFile(this, "Gameplay BG Music.mp3");
WinBGMusic = new SoundFile(this, "Win BG Music.mp3");
LoseBGMusic = new SoundFile(this, "Lose BG Music.mp3");
}
void draw() {
background(173, 253, 255);
//start screen w/ instructions
println(interactionMode);
if (gameState == 0) {
//insert starting screen code here
background(255);
fill(237, 203, 29);
//String Instructions= "Instructions: \n Control the Bird Up & Down and Avoid All Viruses \n \n Choose Control Mode: \n Red Button = Photoresistor \n Blue Button = Microphone\n\nControl Tutorial:\n Photoresistor= Hover and move your hand up and down the photoresistor \nMicrophone= Control the Bird with your voice \n\n NOTE: \n Each time you win, the Viruses will become faster next round \n \n";
textAlign(CENTER);
textSize(20);
text(Instructions, width/2, height/2-250);
fill(139, 0, 139);
rect(buttonX, buttonY+200, buttonW+65, buttonH);
fill(255);
textSize(50);
text("PRESS TO START", buttonX+200, buttonY+200+buttonH-10);
} else if (gameState == 1) { //gameplay screen for Photoresistor Mode
//insert game code here
for (int i=0; i<virus.length; i++) {
virus[i].drawVirus();
virus[i].moveVirus();
}
for (int i=0; i<cloud.length; i++) {
cloud[i].drawClouds();
cloud[i].moveClouds();
}
bird.drawUserBird();
bird.checkEdge();
timer.showTimer();
collisiondetection();
if (interactionMode == 1) {
float temporaryAmplitude;
temporaryAmplitude = amp.analyze();
temporaryAmplitude = map(temporaryAmplitude, 0, 0.2, height, 0);
temporaryAmplitude = constrain(temporaryAmplitude, 0, height);
//controlling the speed at which the object is reaching its destination
//takes more frames to arrive at its destination
bird.locy += (temporaryAmplitude- bird.locy)*.1;
}
} else if (gameState == 2) { //Win Screen
//insert win screen code here
//numberOfVirus= +1;
//for (int i=0; i<virus.length; i++) {
// virus[i].xspeed -=5;
//}
gameplayBGMusic.stop();
background(0);
fill(random(255), random(255), random(255));
textAlign(CENTER);
textSize(80);
text(S, width/2, height/2);
if (mousePressed) {
for (int i=0; i<virus.length; i++) {
virus[i].xspeed -=10;
virus[i].locx = width+50; //this makes them appear from off the screen
virus[i].locy = random(height); //this makes it appear at random heights
virus[i].timer1= millis()+1000; // this is the time interval for each virus appearing
}
WinBGMusic.stop();
reset();
}
} else if (gameState == 3) { // Lose Screen
//insert lose screen code here
String G="You Lose! Game Over";
String R="Press the mouse to restart";
background(0);
fill(random(255), random(255), random(255));
textAlign(CENTER);
textSize(80);
text(G, width/2, height/2);
textSize(50);
text(R, width/2, height/2+100);
if (mousePressed) {
for (int i=0; i<virus.length; i++) {
virus[i].locx = width+50; //this makes them appear from off the screen
virus[i].locy = random(height); //this makes it appear at random heights
virus[i].timer1= millis()+1000; // this is the time interval for each virus appearing
}
LoseBGMusic.stop();
reset();
}
}
}
//if the start button is pressed
void mousePressed() {
if ( mouseX > buttonX && mouseX < buttonX+buttonW+65 && mouseY > buttonY+200 && mouseY < (buttonY+200)+buttonH) {
//buttonPressed=true;
gameState=1;
gameplayBGMusic.play();
timer= new Timer();
}
}
void reset() {
gameState=0;
//Assume the start button has not been pressed
//buttonPressed = false;
//Parameters for the start button
buttonW=335;
buttonH=100;
textSize(buttonH);
buttonX=(width-buttonW)/2;
buttonY=(height-buttonH)/2;
//Birdimg= loadImage("Bird Picture #1.png");
//Virusimg= loadImage("Virus Picture #1.jpeg");
//virus = new Virus [numberOfVirus];
//for (int i=0; i< virus.length; i++) {
// virus[i] = new Virus(Virusimg);
//}
bird = new Bird(Birdimg);
timer = new Timer();
}
void collisiondetection() {
for (int i=0; i<virus.length; i++) {
if (dist(virus[i].locx, virus[i].locy, bird.locx, bird.locy)< virus[i].viruswidth + bird.radius) {
//println(dist(balls[i].locx, balls[i].locy, myBall.locx, myBall.locy));
gameplayBGMusic.stop();
LoseBGMusic.play();
gameState=3; //if it collides, changes the boolean to true, making the lose screen appear
}
}
}
void serialEvent(Serial myPort) {
//read string of information
String s=myPort.readStringUntil('\n');
s=trim(s);
//if there's sth in s
if (s!=null) {
//split the string s based on commmas and put it into integer array "values"
int values[]=int(split(s, ','));
/*as long as there's a total of three arrays of information from Arduino
*/
if (values.length==3 ) {
if (interactionMode == 0) {
//photoresistor value
int temporaryBirdValue;
temporaryBirdValue = values[0];
//bird.locy=values[0];
temporaryBirdValue = (int(map(temporaryBirdValue, 390, 833, height, 0)));
temporaryBirdValue = constrain(temporaryBirdValue, 0, height);
//this is the smoothing algorithm for the bird movement
bird.locy += (temporaryBirdValue - bird.locy)*.05;
}
//this is values for blueswitch
if (values[1] ==1) { //if the blueswitch is pressed change to photoresistor mode
interactionMode = 0;
}
//this is values for redswitch
if (values[2] == 1) { //if the redswitch is pressed change to microphone mode
interactionMode = 1;
}
//code to map the values of the photoresistor to the width of the screen
//gotta change the values for the range of the photoresistor
}
}
//println(xPos);
myPort.write('0');
}
Code for Bird
class Bird {
PImage Birdimg;
//int locx, locy;
float locx, locy;
float Birdwidth, Birdheight;
//no x speed as the x coordinates will be determined by values of photoresistor
float yspeed=5;
float radius=30;// for now, as we'll use a ball to determine the range of collision for the bird image
Bird(PImage birdimg) { //constructor for the Bird object
Birdimg= birdimg;
locx = width/4;
locy = height/2;
Birdwidth = 4*radius;
Birdheight = 4*radius;
}
void drawUserBird(){
//fill(255,10,243);
//stroke(255);
//ellipse(locx, locy, Birdwidth, Birdheight);
image(Birdimg, locx, locy, Birdwidth, Birdheight);
}
void checkEdge(){
if (locy+10 < 0){
locy +=10;
}
if (locy+10 >height){
locy -=10;
}
if (locx+10 < 0){
locx +=10;
}
if (locx+10 >width){
locx -=10;
}
}
}
Code for Clouds
class Clouds {
float locx, locy;
float cloudwidth, cloudheight;
float speed = -3;
PImage Cloudimg;
long timer1;
Clouds(PImage _img) {
Cloudimg = _img;
locx = random(0+10, width-10);
locy = random(0+10, height-10);
cloudwidth = 120;
cloudheight = 120;
timer1= millis()+1000; // this is the time interval for each virus appearing
}
void drawClouds() {
imageMode(CENTER);
//ellipse(locx, locy, cloudwidth, cloudheight);
image(Cloudimg, locx, locy, cloudwidth, cloudheight);
}
void moveClouds() {
if (millis()>timer1) {
locx += speed; //only move the x positions
}
if (locx<-20) {
locx = width+50;
locy = int(random(20, height-20));
}
}
}
Code for Timer
class Timer {
//basic definitions for Timer
float locx, locy;
float timerwidth, timerheight;
int countdown=30;
int seconds;
float startTime;
Timer() {// constructor for Timer object
textSize(50);
startTime= millis()/1000 + countdown;
seconds = int(startTime- millis()/1000);
}
void showTimer() {
if (seconds<0) {
startTime= millis()/1000 + countdown;//resets the timer and restarts it when less than 0
//if seconds is zero, display "game over"
gameplayBGMusic.stop();
WinBGMusic.play();
gameState=2; //if the time ends, it means the person has won and avoided all figures
} else {
seconds = int(startTime- millis()/1000);
fill(255, 0, 0);
text(seconds, width/2, 100);
}
}
}
Code for Viruses
class Virus {
int locx;
float locy;
long timer1;
float viruswidth, virusheight;
//the virus balls only need xspeed as it will only move towards the left
float xspeed= -10;//it's negative 10 as it moves to the left
int radius=30;
PImage Virusimg;
//constructor for the Virus objects
Virus(PImage _img) {
Virusimg = _img;
locx = width+50; //this makes them appear from off the screen
locy = random(height); //this makes it appear at random heights
viruswidth = 4*radius;
virusheight = 4*radius;
timer1= millis()+1000; // this is the time interval for each virus appearing
}
void drawVirus() {
//stroke(0);
//ellipse(locx, locy, viruswidth, virusheight);
imageMode(CENTER);
image(Virusimg, locx, locy, viruswidth, virusheight);
}
void moveVirus() {
if (millis()>timer1) {
locx += xspeed; //only move the x positions
}
if (locx<-20){
locx = width+50;
locy = int(random(20, height-20));
}
}
}
Are you tired yet of Zoom? Maybe not the meetings and the people but just the ever repeating procedure of logging in? Whether your answer would be yes or no, let us propose a more engaging way to join a Zoom meeting, one that involves more than your sense of touch when typing on your keyboard:
Try our musically gamified way to enter your next Zoom meeting!
How it works
You start (our new) Zoom app (Processing).
You are prompted to observe and listen to a password pattern presented in sound and light which makes it more accessible as it offers the same information as different sensory data. The sound is played by Arduino (Arduino).
Repeat the password pattern with the Arduino buttons (Arduino).
If you repeated the correct password pattern, you will now join our Zoom meeting, see yourself in your own little participant window, and receive a welcome message from us (Processing). If you entered a wrong password, you can try again.
If you decide to leave the meeting, you will see a final congratulation notes and then have the possibility to play this “musically game” to join our meeting again (Processing).
Technical Design in Arduino
Arduino circuit
Visual Design in Processing
Step 1: Open our version of Zoom and click to join our meeting.Step 2: Observe and/or listen to the password pattern presented to you.
Step 3: Repeat the Password using the colored buttons on Arduino.Step 4: Welcome! You entered the correct password and made it to our meeting. You will see yourself in your own participant window.Step 5: You left the meeting, congrats for finding the password. Do you want to play again?
Challenges & Learning
In the initial development phase of this project, after agreeing on the details of the user’s journey and the functionality, we worked independently: Amina focused on Arduino while Susanne focused on P5.js. Then bringing both parts together seemed a bit more difficult at first glance as we had previously mainly worked with serial communication between Arduino and Processing but decided to use P5.js in this project as it would allow the live camera input of the user’s camera when joining our Zoom meeting. We shared our code and tried following online tutorials that involved downloading the App p5.serialcontrol for indirect serial communication. Susanne unnecessarily run into trouble because she downloaded the app for the wrong operation system. Moving on with the appropriate App for Apple OS, some individual and communal experimentation and debugging, our Arduino and P5.js sketches eventually became friends 🙂 Happy that their idea came into life, Amina and Susanne could not stop playing their own game for some time… 🙂
Considerations for future Improvements
The current version of our musically Zoom game uses only one level of difficulty as we focused less on levels but more on the whole narrative of entering Zoom for the first development. Future versions could be improved by generating new, random password patterns every time the game is played. Currently, the use can enter the password pattern again after submitting a wrong password but needs to rely on memory. For a future version, we could possibly present the password pattern again after faulty user input to assist their success.
For now we are happy to present our concept in a working game but are always happy to hear your feedback for future version!
You want to play, too?
In order to play our game follow these simple steps:
Download the P5.serialcontrol application appropriate for your operating system.
Bird Picture 
Cloud Picture
