Concept:
We originally wanted our final project to do some sort of generative artwork, but somehow we ended up doing a project completely unrelated. Our goal with this project is to raise awareness of issues happening around the world that aren’t covered enough in the media. Although there are certainly many issues that should be highlighted we chose to focus on six issues happening in six different countries. We also wanted to include ways to learn more about these issues as becoming aware, and learning more is the first step toward change.
CNN has daily news coverage on issues happening around the world, they do this in a 10-minute video that is intended to be short and easily accessible. Taking inspiration from this we titled our project MY6 (Mariam + Yesenia, 6 issues) and aimed to showcase 6 issues through videos that give insight into the issues in short 2-minute videos. Although we went through many ideas on how to execute this project, including puzzles and floor tiles, in thinking about the experience of the user we landed on this:
Implementation:
To bring this idea to life, we used a project to project a world map onto a surface that outlined the six different countries. By touching one of the six countries, a force sensor would trigger a video to play that again, summarizes the issues, after which we compiled a list of ways to learn more. With each of these videos, we wanted the people who are affected by these issues to be at the forefront of the presentation of these issues. By doing this, we hope that the lives of those affected by these issues are humanized as opposed to being numbers reported, or people affected by another issue happening in the world.
To do this we used p5js to display the videos, and Arduino to trigger the videos. For the videos we used the setup function to hide the videos unless they are triggers, and if statements in the draw function to play the videos when they are triggered, as well as statements to hide the other videos and not play them unless all other videos also aren’t playing. We also used the play function along with an else statement to stop the video after it played once to stop it from looping.
For the map that the user will interact with we used force sensors and positioned them in the same place the map would be projected, we taped them down onto a surface and then laid a sheet of white paper over it to create somewhat of a touch screen. The user can replay a video after it has been played, and also click on any video, in any order to learn about a different country.
P5JS Code
const pts = []
var size = 0.6;
let alpha = 255;
let left = 0;
let right = 0;
let rVal0 = 0;
let rVal1 = 0;
let rVal2 = 0;
let rVal3 = 0;
let rVal4 = 0;
let rVal5 = 0;
let img1;
let vid0;
let vid1;
let vid2;
let vid3;
let vid4;
let vid5;
let playVid0 = false;
let playVid1 = false;
let playVid2 = false;
let playVid3 = false;
let playVid4 = false;
let playVid5 = false;
function preload () {
imgA = loadImage("outline map.png");
vid0 = createVideo("china.mp4");
vid1 = createVideo("sri lanka.mp4");
vid2 = createVideo("ethiopia.mp4");
vid3 = createVideo("palestine.mp4");
vid4 = createVideo("france.mp4");
vid5 = createVideo("mexico.mp4");
}
function setup() {
createCanvas(innerWidth, innerHeight);
pixelDensity(5);
textSize(18);
setUpSerial();
vid0.hide();
vid0._onended = ()=> {
playVid0 = false
}
vid1.hide();
vid1._onended = ()=> {
playVid1 = false
}
vid2.hide();
vid2._onended = ()=> {
playVid2 = false
}
vid3.hide();
vid3._onended = ()=> {
playVid3 = false
}
vid4.hide();
vid4._onended = ()=> {
playVid4 = false
}
vid5.hide();
vid5._onended = ()=> {
playVid5 = false
}
}
function keyPressed() {
if (key == " ") {
// important to have in order to start the serial connection!!
setUpSerial(SELECT_PORT);
}
}
function draw() {
image(imgA, 0, 0, innerWidth, innerHeight);
fill(100);
stroke(255);
strokeWeight(1);
if (!serialActive) {
text("Press Space Bar to select Serial Port", 20, 30);
} else {
text("Connected", 20, 30);
}
if (rVal0 == 1 && playVid0 == false && playVid1 == false && playVid2 == false && playVid3 == false && playVid4 == false && playVid5 == false) {
print("video0");
vid0.play();
playVid0 = true;
}
if ( playVid0 ){
image(vid0, 300, 120, 560, 316);
} else {
playVid0 = false;
}
if (rVal1 == 1 && playVid1 == false && playVid0 == false && playVid2 == false && playVid3 == false && playVid4 == false && playVid5 == false) {
print("video1");
vid1.play();
playVid1 = true;
}
if ( playVid1 ){
image(vid1, 330, 250, 560, 316);
} else {
playVid1 = false;
}
if (rVal2 == 1 && playVid2 == false && playVid0 == false && playVid1 == false && playVid3 == false && playVid4 == false && playVid5 == false) {
print("video2");
vid2.play();
playVid2 = true;
}
if ( playVid2 ){
image(vid2, 150, 300, 560, 316);
} else {
playVid2 = false;
}
if (rVal3 == 1 && playVid3 == false && playVid0 == false && playVid1 == false && playVid2 == false && playVid4 == false && playVid5 == false) {
print("video3");
vid3.play();
playVid3 = true;
}
if ( playVid3 ){
image(vid3, 200, 330, 560, 316);
}
else{
playVid3 = false;
}
if (rVal4 == 1 && playVid4 == false && playVid0 == false && playVid1 == false && playVid2 == false && playVid3 == false && playVid5 == false) {
print("video4");
vid4.play();
playVid4 = true;
}
if ( playVid4 ){
image(vid4, 200, 250, 560, 316);
} else {
playVid4 = false;
}
if (rVal5 == 1 && playVid5 == false && playVid0 == false && playVid1 == false && playVid2 == false && playVid3 == false && playVid4 == false) {
print("video5");
vid5.play();
playVid5 = true;
}
if ( playVid5 ){
image(vid5, 220,200 , 560, 316);
} else {
playVid5 = false;
}
}
function readSerial(data) {
////////////////////////////////////
//READ FROM ARDUINO HERE
////////////////////////////////////
if (data != null) {
// make sure there is actually a message
// split the message
let fromArduino = split(trim(data), ",");
// if the right length, then proceed
if (fromArduino.length == 6) {
// only store values here
// do everything with those values in the main draw loop
rVal0 = fromArduino[0];
rVal1 = fromArduino[1];
rVal2 = fromArduino[2];
rVal3 = fromArduino[3];
rVal4 = fromArduino[4];
rVal5 = fromArduino[5];
print(fromArduino[0]+ " "+fromArduino[1]+ " "+fromArduino[2]+ " "+fromArduino[3]+ " "+fromArduino[4]+ " "+fromArduino[5]);
}
//////////////////////////////////
//SEND TO ARDUINO HERE (handshake)
//////////////////////////////////
let sendToArduino = "\n";
writeSerial(sendToArduino);
}
}
Arduino Code
//Arudino Code
int pForce0 = 0;
int pForce1 = 0;
int pForce2 = 0;
int pForce3 = 0;
int pForce4 = 0;
int pForce5 = 0;
void setup() {
Serial.begin(9600);
// start the handshake
while (Serial.available() <= 0) {
Serial.println("0,0,0,0,0,0"); // send a starting message
delay(300); // wait 1/3 second
}
}
void loop() {
// wait for data from p5 before doing something
while (Serial.available()) {
if (Serial.read() == '\n') {
int force0 = analogRead(A0);
delay(2);
int sensor0 = 0;
if (force0 > 200 && pForce0 < 100) {
sensor0 = 1;
}
pForce0 = force0;
int force1 = analogRead(A1);
delay(2);
int sensor1 = 0;
if (force1 > 200 && pForce1 < 100) {
sensor1 = 1;
}
pForce1 = force1;
int force2 = analogRead(A2);
delay(2);
int sensor2 = 0;
if (force2 > 200 && pForce2 < 100) {
sensor2 = 1;
}
pForce2 = force2;
int force3 = analogRead(A3);
delay(2);
int sensor3 = 0;
if (force3 > 200 && pForce3 < 100) {
sensor3 = 1;
}
pForce3 = force3;
int force4 = analogRead(A4);
delay(2);
int sensor4 = 0;
if (force4 > 200 && pForce4 < 100) {
sensor4 = 1;
}
pForce4 = force4;
int force5 = analogRead(A5);
delay(2);
int sensor5 = 0;
if (force5 > 200 && pForce5 < 100) {
sensor5 = 1;
}
pForce5 = force5;
Serial.print(sensor0);
Serial.print(",");
Serial.print(sensor1);
Serial.print(",");
Serial.print(sensor2);
Serial.print(",");
Serial.print(sensor3);
Serial.print(",");
Serial.print(sensor4);
Serial.print(",");
Serial.println(sensor5);
}
}
}
Challenges:
One of the first issues we encountered was getting the videos to play over the map as it was playing under the map as a separate screen. To solve this issue we had to add a position to the video as well as hide the video before and after playing. Once we fixed this issue we then made each video work individually, which was quite simple, but when trying to make all of them work, none of the videos would play. For this, we realized there was an issue within Arduino that wasn’t sending the value of the force sensor separately so that p5js could read each value separately and use it for the appropriate video. Once we fixed this issue, we then had to make it so that each video would play only one at a time, and only play once the video playing was over, for this to the if statements that required the video to not be playing to start we added that other videos had to also be not playing.
Overall, the issue we encountered the most was consistency as the videos would play and then not play, even when we changed nothing but reopened the tab. What we realized was that there was an issue with the serial communication, and only when we closed the tabs properly, would the p5js and Arduino be able to communicate properly.
Reflection:
In the end, we not only learn new concepts in implementing our idea but were able to apply a lot of what we learned this semester. Although our project was not as creative as we originally each individually wanted, it is something that we both this is important and truly care about. Moving forward we would add more countries, or even do all the countries, although that would not be as engaging for the user, however, they could pick a few countries they are interested in. We also realized how limited information for some of these issues is as we spent quite a bit of time trying to find resources and compile ways to learn more. Although this is only one small thing, it is a step towards raising awareness for issues in a way that engages the views, gives them information, and is a way to learn more. If nothing else, we hope everyone is able to learn something new and want to learn more through our project!