During this week’s assignment, we were required to use Arduino by experimenting with a way to construct a hands-free switch (one that does not require an included analog/digital switch). Approaching this assignment, I had intentions of utilizing the Arduino in a way that would be beneficial.
code:
// Define pin for buzzer
const int buzzerPin = 9;
// Define alarm time in hours and minutes
const int alarmHour = 7;
const int alarmMinute = 30;
void setup() {
// Set the buzzer pin as an output
pinMode(buzzerPin, OUTPUT);
}
void loop() {
// Get the current time
int currentHour = hour();
int currentMinute = minute();
// Check if it is time for the alarm
if (currentHour == alarmHour && currentMinute == alarmMinute) {
// Turn on the buzzer
digitalWrite(buzzerPin, HIGH);
delay(1000);
// Turn off the buzzer
digitalWrite(buzzerPin, LOW);
delay(1000);
}
}
method:
Throughout the 12th week of the semester, another Interactive Media course, Communication and Technology, required us to experiment with going through a digital detox. As a result, I faced the issue of not being able to wake myself up since I ultimately pushed my phone away from me. That being said, I found this assignment perfect as I could make my own digital clock while also including the “hands-free” concept. The hands-free aspect was to add foil to an alarm clock’s hour hand to recognize when it hits a specific hour. This is displayed in the illustration below. To further utilize this, I adapted a simple alarm clock script that exerts a sound as soon as the signal is detected, requiring the person to wake up and pull the foil until they have to place it on again before sleeping.
sketch:
The hands-free element is when the hour arm of the clock moves to 8:00AM and eventually makes contact with the other foil piece that houses the wire connected to the Arduino, further turning on the buzzer.
future improvements:
Possibly present an element that would automatically replace the foil after it is removed to prevent further redundancy.
Initially, we wanted to create a simple art project using multiple servos to control panels. That was very short-lived. Our concept slowly went from a simple interactive art piece that looked pretty to an interactive monster called a Demogorgon from the Netflix series Stranger Things. We wanted to make something interesting that could catch users’ attention longer than an art piece. Since our Demogorgon was not going to necessarily be a game with a winning or losing outcome, we still had to figure out ways to make it more interactive. We took our assigned reading: Emotion & Design: Attractive Things Work Better by Don Norman into consideration and decided that in order to convey what we wanted we needed to make the Demogorgon come to life and make it something that is interesting to look at in addition to the code and interact itive component. To make both the interactive component and the Demogorgon fit nicely together, we decided to use certain scenes of the show to inspire the movements. In the show, the protagonist Eleven defeats a Demogorgon by putting her hand up since she has the ability to manipulate living beings however she wants through her hands. This inspired us to make it so the user is able to open the Demogorgon face flaps we build as their hand gets closer to them through a distance-measuring sensor. As we went along with our project, we realized how ambitious we were with our concept and simply tried our best to make it happen.
For our project, we used the ultrasonic distance measuring sensors, LED lights, flex sensors, 3-d printing, cardboard, wood, acrylic, and paint. Though we went through a lot of hiccups including but not limited to spending hours soldering cables we ended up not using, one of us going into isolation because of COVID-19, short circuits, and re-thinking our concept a few times… we kind of did it.
Interaction Design
Physical
This was a very time-consuming aspect of project, especially since we really wanted it to make look as much as a Demogorgon’s head as possible with what we had. The wood that was available to us was pretty thick and we knew we had to cut holes in it to make openings for the 3-d printed and handmade Demogorgon flaps so it was going to take a long time to cut the wood and put everything together. Luckily, the woodshop was nice enough to let us borrow a Jigsaw so even though it was still a tedious process of drilling two holes where we wanted one hole for the flap to be and then using the Jigsaw to go in and follow the outline, it made it go a lot faster. We did this process seven times. For the flaps of the Demogorgon, at first, we 3-d printed three of them but it was taking a long time so for the last four we decided to build them out of cardboard for time’s sake. The 3-d printed ones still looked pretty cool though and definitely added an appeal to it, in our opinion. Plus, the different materials fit the Demogorgon vibe. We painted and added red and white for the skin tone. We also had to make longer cables to reach the Demogorgon flaps when our Arduino port is hidden beneath the box/head.
Arduino Code
Our Arduino code controlled the flex sensors, server motors, and ultrasonic measuring sensor. The sens0rs were mapped to values from 0 to 180 and then output at different rates depending upon the configuration. The output rates helped stabilize the servo motors which were prone to jittering.
// Clears the trigPin condition
digitalWrite(trigPin, LOW);
delayMicroseconds(100);
// Sets the trigPin HIGH (ACTIVE) for 10 microseconds
digitalWrite(trigPin, HIGH);
delayMicroseconds(100);
digitalWrite(trigPin, LOW);
// Reads the echoPin, returns the sound wave travel time in microseconds
duration = pulseIn(echoPin, HIGH);
// Calculating the distance
distance = duration * 0.034 / 2; // Speed of sound wave divided by 2 (go and back)
// Displays the distance on the Serial Monitor
distanceMapped = map(distance, 0, 70, 22, 180);
distanceMapped=constrain(distanceMapped, 30, 180); //this allows the output to stay between the constraints. without this it goes up to like a few thousand for some reason
int distanceMappedforOuter = distanceMapped + 10; // the outer petals will open wider, to give a flared look
myOuterServos.write(distanceMappedforOuter);
delay(75);
myInnerServos.write(distanceMapped);
delay (75);
Code to control servos with the Flex Sensors:
int OutsideFlexValue;
int InnerFlexValue;
int servoPositionInside;
int servoPositionOutside;
OutsideFlexValue = analogRead(OutsideFlexPin);
// delay(15); //adding slight delays to make it stable
InnerFlexValue = analogRead(InsideFlexPin);
// delay(15);
servoPositionOutside = map(OutsideFlexValue, 170, 500, 0, 180); // need it to rest at a midwards position
servoPositionOutside = constrain(servoPositionOutside, 0, 180);
// int round10(int servoPositionOutside);
servoPositionInside = map(InnerFlexValue, 100, 360, 0, 180);
servoPositionInside = constrain(servoPositionInside, 0, 180);
// servoPosition = constrain(servoPosition, 45, 180); //for the servos that go beyond 180
myOuterServos.write(servoPositionOutside);
delay(75);
myInnerServos.write(servoPositionInside);
delay(75);
To reduce jittering the following line of code was added to constrain the servo motors from going beyond set parameters.
distanceMapped = map(distance, 0, 70, 22, 180);
distanceMapped=constrain(distanceMapped, 30, 180); //this allows the output to stay between the constraints. without this it goes up to like a few thousand for some reason
p5.js Code
Our p5.js code focused on the screen that the user would see and use to interact with our Demogorgon. We used the game state and included a starting screen followed by an introduction and instruction screen. We realize not everyone may know what a Demogorgon is or have watched Stranger Things, so we made it feel an arcade game you would go up to, get an introduction and instructions for, and start playing around with it whether or not you fully grasp the inspiration behind it.
There was an issue that we couldn’t quite figure out. After adding the second sound, the audio would start glitching after a while. At first, we thought it was just the sound so we tried it out with different sounds but no luck. When we decided to search if there was a solution or if the code was wrong, we found a thread from 2020 that discussed how the newest versions of p5.js have audible distorted effects on Chrome and there is no fix yet. The audio just keeps getting distorted and eventually the window shuts down and you have to refresh the page.
The p5js code can be accessed on Abigail and I’s p5js sketches directory on the p5js website.
Circuit Diagram for the Project:
Since our project was rather complicated and needed a lot of cables, we decided to draw a schematic for it.
Problem(s) 🙁
Though we knew what needed to be communicated between the Arduino and p5.js, it worked only sometimes. The problems started when we started off by checking out the Arduino MEGA from the IM Lab and tailoring our code to it. However, since we got an extension past the last day to return items to the IM Lab due to reasons neither of us could control, we had to switch back to the Arduino we received in our kit. For some reason what we had did not work after that and we had to figure out why. We switched up the code again but only some parts of it would work and only sometimes as p5.js kept shutting down and the serial connection kept disconnecting. We tried to ask for help through friends and Facebook groups. No one was able to help us though and as we ran out of time we accepted our fate. The Demogorgon beat us (sorta).
Some testing and progress videos:
Testing the Flex sensors interfaced with the servo motors:
Testing the ultrasonic distance sensor interfaced with the servos:
Problems that we ran into and how we tried to solve them:
The first problem we ran into pretty early on was a jittering in the servo motors that could not be controlled. The jittering would make the servos move even when the code was not running. After some debugging, we realized that the problem originated with the 5v supply on the main board, and switching the servos to an external power source would reduce it. It could be made smoother by adding another earth wire connected to one of the unused GND pins
Another problem we ran into was the 3d printing of some of the parts. The models we had used were pulled off the internet and were supposed to be printed within a few hours but the first 3 iterations were all just a mess of melted filament since the model would move around. After consulting the lab assistants we realized scraping the surface of the building area of the printer solved the issue immediately.
A bug that we ran into while working on the Arduino-p5js serial communication was that the communication would halt and spout an error ‘Failed to open serial port’. We realized that this could be fixed most of the time by closing the serial monitor on the IDE. However, at one point it persisted despite restarting, and according to some forums, it was a problem with incompatible drivers that plague computers with versions of windows later than 10.
Another bug that we ran across that caused us the most trouble was that the p5js terminal would not work or crash periodically after adding sound effects that were to be played on the terminal. We could not find a viable solution for this problem.
Future Improvements:
I think we focused on Professor’s warning that the physical component was going to take a really long time too much. Our concept depended a lot on being appealing to the audience so we wanted to make sure we made our Demogorgon look really good and spooky. However, I think we spent so much time on it considering the limited time we had that we did not think about how buggy the code would be considering everything we were trying to make it do at once.
Figure out a more stable way for communication between p5js and Arduino.
Use a stable operating system that causes least amounts of problems with regards to drivers
Use a stable search engine to run p5js smoothly
Reflection:
Overall, this project was an interesting one for us. I never would have thought that I would attempt to do something like this when I started this class. I am very proud of our physical component, our creativity, and our perseverance despite the non-stop obstacles that kept coming our way whether they were from our code or life events.
I was able to build the physical component, cutting everything with the jigsaw, putting it together, initiated the 3-d printing process, and painted it. I was actually very proud of myself for getting my hands dirty with the jigsaw and being able to use it. It felt good to use my hands to build something instead of beating my brain into figuring out code. I also drew the schematic and helped build most of the p5.js code. I soldered all of the server motor cables together to make them reach our Demogorgon and made sure to take pictures of our progress throughout the time we were working on the project.
Though I never learned to like coding while taking this class, it was definitely a learning experience. When I did get things on my own and processed what was going on, it felt really rewarding! This class was very different than what I am used to but I very much enjoyed learning something other than Political Science and Law. It was a great class and it was amazing seeing what my classmates who know this stuff well build and code.
This week we learnt that serial communication is a way for devices, like an Arduino microcontroller and a computer, to exchange data. It allows the Arduino to send data to the computer or for the computer to send data to the Arduino.
One way to establish serial communication between an Arduino and a computer is to use p5.js, a JavaScript library for creating interactive graphics and visualizations. To use serial communication with p5.js, we will need to use the p5.serialport library, which is a p5.js library that provides a wrapper around the popular serialport library for Node.js.
Once we have set up the p5.serialport library and created a p5.Serial object, we can use the serial.read() function to read data from the serial port and the serial.write() function to write data to the serial port. We can also use the serial.on() function to set up event listeners for different events, like when data is received or when the serial port is opened or closed.
Exercise 1:
In class we practiced serial communication using a potentiometer example. We modified the potentiometer example demonstrated in class to create a ball that bounces horizontally across the screen at a speed that is determined by the value of the potentiometer. In our Arduino code, we used serial communication to send the value of the potentiometer to the p5.js sketch, which was then used to control the velocity of the bouncing ball:
Arduino Code:
void loop() {
// put your main code here, to run repeatedly:
int potentiomenter = analogRead(A0);
int mappedPot = map(potentiomenter, 0, 1023, 0, 255);
Serial.write(mappedPot);
delay(100);
n our p5.js sketch, we created a ball object using a Circle class with properties for the x and y positions and a move() method. The move() method only updates the x position of the ball, which is calculated using the value of the potentiometer that was read from the Arduino using serial communication. The y position of the ball remains constant along the horizontal axis at the height/2 position. The p5js code for this exercise:
// variable to hold an instance of the p5.webserial library:
const serial = new p5.WebSerial();
// HTML button object:
let portButton;
let inData; // for incoming serial data
let outByte = 0; // for outgoing data
function setup() {
createCanvas(400, 300); // make the canvas
// check to see if serial is available:
if (!navigator.serial) {
alert("WebSerial is not supported in this browser. Try Chrome or MS Edge.");
}
// if serial is available, add connect/disconnect listeners:
navigator.serial.addEventListener("connect", portConnect);
navigator.serial.addEventListener("disconnect", portDisconnect);
// check for any ports that are available:
serial.getPorts();
// if there's no port chosen, choose one:
serial.on("noport", makePortButton);
// open whatever port is available:
serial.on("portavailable", openPort);
// handle serial errors:
serial.on("requesterror", portError);
// handle any incoming serial data:
serial.on("data", serialEvent);
serial.on("close", makePortButton);
}
function draw() {
background(0);
fill(255);
text("sensor value: " + inData, 30, 50);
}
// if there's no port selected,
// make a port select button appear:
function makePortButton() {
// create and position a port chooser button:
portButton = createButton("choose port");
portButton.position(10, 10);
// give the port button a mousepressed handler:
portButton.mousePressed(choosePort);
}
// make the port selector window appear:
function choosePort() {
if (portButton) portButton.show();
serial.requestPort();
}
// open the selected port, and make the port
// button invisible:
function openPort() {
// wait for the serial.open promise to return,
// then call the initiateSerial function
serial.open().then(initiateSerial);
// once the port opens, let the user know:
function initiateSerial() {
console.log("port open");
}
// hide the port button once a port is chosen:
if (portButton) portButton.hide();
}
// pop up an alert if there's a port error:
function portError(err) {
alert("Serial port error: " + err);
}
// read any incoming data as a string
// (assumes a newline at the end of it):
function serialEvent() {
inData = Number(serial.read());
console.log(inData);
}
// try to connect if a new serial port
// gets added (i.e. plugged in via USB):
function portConnect() {
console.log("port connected");
serial.getPorts();
}
// if a port is disconnected:
function portDisconnect() {
serial.close();
console.log("port disconnected");
}
function closePort() {
serial.close();
}
Exercise 2:
In our sketch, we created an interactive visual that represents a sun rising and falling in the sky and linked it to the brightness of an LED. When the up arrow key is pressed, the sun “rises” in the sky and the background color changes to a bright magenta. When the down arrow key is pressed, the sun “falls” and the background color deepens to a dark blue. We achieved this effect by using the y position of the sun, which was created using the Circle class from the previous exercise, to control the RGB values of the background color.
Arduino Code:
void setup() {
Serial.begin(9600);
pinMode(2, OUTPUT);
pinMode(5, OUTPUT);
while (Serial.available() <= 0) {
Serial.println("0,0"); // send a starting message
delay(200);
}
}
void loop() {
while (Serial.available() > 0) {
// read the incoming byte:
int inByte = Serial.read();
analogWrite(5,inByte);
Serial.println();
}
}
p5js Code:
let serialPort; // variable to hold an instance of the serialport library
let portName = "COM3"; // fill in your serial port name here
let currentXPos=0;
let currentYPos=240;
let switchStatus=0;
let value;
function setup() {
createCanvas(640, 480);
serialPort = new p5.SerialPort(); // make a new instance of the serialport library
serialPort.on("list", printListOfPorts); // set a callback function for the serialport list event
serialPort.on("connected", serverConnected); // callback for connecting to the server
serialPort.on("open", portOpened); // callback for the port opening
serialPort.on("data", serialDataReceived); // callback for when new data arrives
serialPort.on("error", serialError); // callback for errors
serialPort.on("close", portClosed); // callback for the port closing
serialPort.list(); // list the serial ports
serialPort.open(portName); // open a serial port
}
function draw() {
background(255);
value = map(mouseX, 0, width, 0, 255);
}
// get the list of ports:
function printListOfPorts(portList) {
// portList is an array of serial port names
for (let i = 0; i < portList.length; i++) {
// Display the list the console:
print(i + " " + portList[i]);
}
}
function serverConnected() {
print("Connected to server.");
}
function portOpened() {
print("The serial port opened.");
}
function serialDataReceived() {
// read a string from the serial port
// until you get carriage return and newline:
let incomingString = serialPort.readLine();
serialPort.write(value);
}
function serialError(err) {
print("Something went wrong with the serial port. " + err);
}
function portClosed() {
print("The serial port closed.");
}
Exercise 3:
For the final assignment, we had to use bidirectional serial communication, which basically involved controlling a p5js sketch with an Arduino and changing the p5js sketch to modify the arduino. We made the ball move on the Arduino by using a distance sensor to modify the wind speed in p5js. Depending on the ball’s velocity, we programmed the p5js sketch so that an LED will turn on each time the ball bounces.
Arduino Code:
Arduino code
void setup() {
Serial.begin(9600);
pinMode(2, OUTPUT);
while (Serial.available() <= 0) {
Serial.println("0"); // send a starting message
delay(300); // wait 1/3 second
}
}
void loop() {
while (Serial.available() > 0) {
// read the incoming byte:
int inByte = Serial.read();
analogWrite(2, inByte);
int WindPower = analogRead(A0);
Serial.println(WindPower);
}
}
p5js Code:
let serialPort; // variable to hold an instance of the serialport library
let portName = "COM7"; // replace with your serial port name
let speed;
let weight;
let location;
let acceleration;
let wind;
let resistance = 0.99;
let mass = 50;
let windVal;
function setup() {
createCanvas(640, 360);
noFill();
location = createVector(width/2, 0);
speed = createVector(0,0);
acceleration = createVector(0,0);
weight = createVector(0, 0.5*mass);
wind = createVector(0,0);
serialPort = new p5.SerialPort(); // create a new instance of the serialport library
serialPort.on("list", printPortList); // set a callback function for the serialport list event
serialPort.on("connected", serverConnected); // callback for connecting to the server
serialPort.on("open", portOpen); // callback for the port opening
serialPort.on("data", serialEvent); // callback for when new data arrives
serialPort.on("error", serialError); // callback for errors
serialPort.on("close", portClose); // callback for the port closing
serialPort.list(); // list the serial ports
serialPort.open(portName); // open a serial port
}
// get the list of ports:
function printPortList(portList) {
// portList is an array of serial port names
for (let i = 0; i < portList.length; i++) {
// Display the list the console:
print(i + " " + portList[i]);
}
}
function serverConnected() {
print("Connected to server.");
}
function portOpen() {
print("The serial port opened.");
}
function serialEvent() {
// read a string from the serial port until a carriage return and newline are received
let inString = serialPort.readLine();
print(inString);
if (inString.length > 0){
windVal = map(inString, 0, 1023, -3, 3);
}
}
function serialError(err) {
print("There was an error with the serial port: " + err);
}
function portClose() {
print("The serial port closed.");
}
function draw() {
background(255);
wind.x = windVal;
applyForce(wind);
applyForce(weight);
speed.add(acceleration);
speed.mult(resistance);
location.add(speed);
acceleration.mult(0);
ellipse(location.x,location.y,mass,mass);
if (location.y > height-mass/2 - 25){
serialPort.write(255);
}
else serialPort.write(0);
if (location.y > height-mass/2) {
speed.y *= -0.9; // A little dampening when hitting the bottom
location.y = height-mass/2;
Overall, using serial communication with p5.js allows us to create interactive graphics and visualizations that can communicate with and control devices like the Arduino.
The three of us (Tarek, Aadil, and I) collectively decided to make something similar to an arcade machine as it was a very exciting part of our childhoods. Therefore, the project is a collection of three games that are designed to be played using an Arduino controller. The games include Flappy Bird, a popular mobile game in which the player controls a bird and navigates through obstacles; Racing Game, in which the player controls a race car and avoids colliding with the randomly generated cars as the race car overtakes them; and Space Invaders, a classic arcade game in which the player controls a spaceship and fights against invading aliens. The project was named 1962 in commemoration of the first arcade opened in 1962.
Arduino to P5js communication:
int button = 2;
int pot = A0;
int button1 = 4;
int button2 = 7;
int button3 = 8;
int lastPotValue;
int lastbutton;
long previousmill = 0;
long timebutton = 500;
void setup() {
// put your setup code here, to run once:
Serial.begin(9600);
pinMode(button, INPUT_PULLUP);
pinMode(button1, INPUT_PULLUP);
pinMode(button2, INPUT_PULLUP);
pinMode(button3, INPUT_PULLUP);
pinMode(pot, INPUT);
}
int getpot(){
int potValue = analogRead(pot)/255 ;
int temp;
if(potValue == 2 || potValue == 1){
temp = 1;
}else if(potValue == 3 || potValue == 4){
temp = 2;
}else{
temp = 0;
}
return temp;
}
void loop() {
int potValue = getpot();
int buttonState = !digitalRead(button);
long currentmill = millis();
int buttonstate1 = !digitalRead(button1);
int buttonstate2 = !digitalRead(button2);
int buttonstate3 = !digitalRead(button3);
int game = 0;
if(buttonstate1 == 1){
game = 1;
}
if(buttonstate2 == 1){
game = 2;
}
if(buttonstate3 == 1){
game = 3;
}
Serial.println(String(buttonState) + "," + String(potValue) + "," + String(game));
if(buttonState == 1 && currentmill - previousmill >= timebutton){
previousmill = currentmill;
lastbutton = buttonState;
}
}
We implemented this using the web serial. Here is how it briefly works:
The user connects an Arduino board to their computer using a USB cable.
The user writes and uploads a sketch (see above for the code) to the Arduino board that defines the behavior of the board and the data that it will send to the computer.
The user opens a P5.js sketch in their web browser and includes the p5.webserial.js library in their code.
The user adds event listeners to their P5.js sketch that will be called when the user connects or disconnects from the Arduino board, when the Arduino board is ready to be connected to, when there is an error communicating with the Arduino board, or when data is received from the Arduino board.
The user calls the getPorts() method of the p5.WebSerial object to check for any available Arduino boards. If an Arduino board is available, the portavailable event listener is called, which can be used to open a connection to the Arduino board.
Once the connection to the Arduino board is established, the user can send data to the Arduino board using the send() method of the p5.WebSerial object. The user can also receive data from the Arduino board using the data event listener, which is called whenever data is received from the Arduino board.
The user can use the received data from the Arduino board to control the behavior and appearance of their P5.js sketch. The user can also send data from the P5.js sketch to the Arduino board to control the behavior of the Arduino board.
When the user is finished using the Arduino board, they can close the connection to the board using the close() method of the p5.WebSerial object.
The getRndInteger() function is a helper function that returns a random integer between two given values. This function is used to randomly generate the heights of the pipes in the game. The Bird and Pipes classes define the objects that appear in the game. The Bird class has a draw() method that is used to draw the bird on the screen, an update() method that is used to update the bird’s position and velocity, a flap() method that causes the bird to jump upwards, and a checkDeath() method that checks if the bird has collided with any of the pipes and ends the game if necessary. The Pipes class has an update() method that updates the positions of the pipes and a drawPipes() method that draws the pipes on the screen. Overall, the code defines a simple game in which the player controls a bird and must avoid colliding with pipes by jumping over them. The game keeps track of the player’s score and ends if the bird hits a pipe.
Racing Game:
The generateCars() function is used to randomly generate cars that appear on the screen and the displayCars() function is used to draw the cars on the screen. The displayScore() function is used to display the player’s current score on the screen. The potentiometer returns three readings: 0,1, and 2 based on the positioning. Based on the number being returned by the potentiometer – we handle the car movement.
Space Invaders:
// both games =================================== function score(){ textSize(32); fill(250); text("Score: "+currentScore,20,50); } function increaseD(){ if(currentScore === 1 + prevScore){ difficulty += 0.5; prevScore = currentScore; // console.log(difficulty); } return random(1,5)+difficulty; }
//space invadors ======================================
function startPage(){
textSize(27);
fill(250);
text("Space invador",27,250);
textSize(15);
text("press enter to start",52,290);
}
function removeRocks(){
rocks.splice(0,rocks.length);
rocksctr = 0;
}
function displaybullets(){
for(let i = 0; i < bullets.length; i++){
bullets[i].display();
if(bullets[i].y < 0){
bullets.splice(i,1);
numBullets--;
}
}
// console.log(numBullets);
}
function generaterocks(){
let rand = int(random(0, 100));
let rand2 = int(random(0, 100));
if(rand % 7 == 0){
if(rand2 % 3 == 0){
if(rand2 % 2 == 0 && rand % 2 == 0){
rocks[rocksctr] = new boulders();
rocks[rocksctr].display();
// console.log(rocksctr);
rocksctr++;
}
}
}
}
function displayrocks(){
for(let i = 0; i < rocks.length; i++){
rocks[i].display();
// console.log(">",rocks.length);
let temp = false;
for(let j = 0; j < bullets.length; j++){
if(bullets[j].didcollide(rocks[i])){
temp = true;
bullets.splice(i,1);
numBullets--;
}
}
if(mainship.didcollide(rocks[i])){
rocks.splice(i,1);
rocksctr--;
gamestatus = "end";
bomb.play();
losing1.play();
}else if(rocks[i].y > height || temp){
rocks.splice(i,1);
rocksctr--;
}
}
}
var timechecker = 0.5;
function makebullet(x,y){
// console.log(x);
bullets[numBullets] = new bulletClass(x,y);
m0 = millis(); //time when laser created
if(timechecker>0.3){
bullets[numBullets].display();}
numBullets++;
m = millis();
timechecker = m - m0;
} //tried to disable continous shooting maybe look into it later
// end space invadors ================================
// start racing car game functions=====================
function startPage2(){
textSize(27);
fill(250);
text("Car racing",63,255);
textSize(15);
text("press enter to start",52,290);
}
function generateCars(){
let rand = int(random(0, 100));
let rand2 = int(random(0, 100));
if(rand % 7 == 0 && carrs.length < 4){
if(rand2 % 3 == 0){
if(rand2 % 2 == 0 && rand % 2 == 0){
carrs[carsctr] = new cars();
carrs[carsctr].display();
// console.log(carsctr);
carsctr++;
}
}
}
}
function displayCars(){
for(let i = 0; i < carrs.length; i++){
carrs[i].display();
// console.log(">",carrs.length);
let temp = false;
if(maincar.didcollide(carrs[i])){
checklanes(0,carrs[i]);
carrs.splice(i,1);
carsctr--;
currentScore = 0;
// bomb.play();
gamestatus = "end";
losing2.play();
// gamestatus = "end";
// bomb.play();
}else if(carrs[i].y > height || temp){
checklanes(0,carrs[i]);
carrs.splice(i,1);
carsctr--;
currentScore++;
cargoing.play();
}
}
}
function checklanes(x,other){
if(x === 1){
if(lanes2[other.temp] === 1){
other.temp = int(random(0,4));
other.x = lanes[other.temp];
checklanes(1,other);
}else{
lanes2[other.temp] = 1;
}
}else if(x === 0){
lanes2[other.temp] = 0;
}
}
function removeCars(){
carrs.splice(0,carrs.length);
carsctr = 0;
}
Initialization:
we initialized a lot of variables that would be used by the serial and the three games. the pre load function was also used to prepare the necessary pictures and sounds as well as fonts. the we set up what was necessary in the set up function.
let whichgame = 0;
let whichgameprev = 0;
// space invadors
let mainship;
let bullets = [];
let numBullets = 0;
let arrRocks;
let rocks = []
let lasersound;
let rocksctr = 0;
let difficulty = 0; // both games
let currentScore = 0; //both games
let prevScore = 0; //both games
let gamestatus = "start"; // both games
let rate = 0; // both games
let widthh = 400;
//racing game
let arrCars;
let carrs = [];
let carsctr = 0;
let lanes = [8,88,168,248,328];
let lanes2 = [0,0,0,0,0];
//flappy bird
var menu = 0;
var SCROLL_SPEED = 4;
var SCORE = 0;
let oof;
let bruh;
let music;
var bird ;
var pipes;
var lives = 5;
const GRAVITY = 8.81;
const JUMP_HEIGHT = 6.0;
const GROUND_HEIGHT = 20;
const WIDTH = 600;
const HEIGHT = 550;
//-------------arduino----------------
// let x=0;
// var c;
let values = [];
// variable to hold an instance of the p5.webserial library:
const serial = new p5.WebSerial();
// HTML button object:
let portButton;
let inData; // for incoming serial data
let outByte = 0;
function preload() {
main = loadImage('mainpicture.png');
//space invadar
img = loadImage('Hs4QN2.gif');
startScreen = loadImage('startscreen.gif');
ship = loadImage('Untitled-1.png');
bullet = loadImage('bullet.png');
soundFormats('mp3', 'ogg');
lasersound = loadSound('lasersound.mp3');
bomb = loadSound('explo.mp3');
rock1 = loadImage('rock1.png');
rock2 = loadImage('rock2.png');
rock3 = loadImage('rock3.png');
gameoverpng = loadImage('gameover.png');
mainFont = loadFont('PressStart2P-vaV7.ttf');
alternateFont = loadFont('metal lord.otf');
losing2 = loadSound('losing2.wav');
arcade = loadSound('arcade.mp3');
sp = loadSound('space.wav');
//racing car game;
imgg = loadImage('maincar.png');
car1 = loadImage('car1.png');
car2 = loadImage('car2.png');
car3 = loadImage('car3.png');
backgroundd = loadImage('background.png');
backgrounddd = loadImage('final.gif');
cargoing = loadSound('mixkit-fast-car-drive-by-1538.wav');
startscreen = loadImage('startscreen.png');
done = loadImage('gameovercar.png');
losing1 = loadSound('losing1.wav');
carpassing = loadSound('carpassing.wav');
extraedge = loadImage('extraedge.png');
//flappy bird
music = loadSound("bgmusic.mp3");
bg = loadImage('bg11.png');
home = loadImage('homescreem.png');
b = loadImage('bird.png');
jump = loadSound('flap-1.mp3');
oof = loadSound('oof.mp3');
}
function setup() {
createCanvas(600, 550);
arcade.play();
if (!navigator.serial) {
alert("WebSerial is not supported in this browser. Try Chrome or MS Edge.");
}
// if serial is available, add connect/disconnect listeners:
navigator.serial.addEventListener("connect", portConnect);
navigator.serial.addEventListener("disconnect", portDisconnect);
// check for any ports that are available:
serial.getPorts();
// if there's no port chosen, choose one:
serial.on("noport", makePortButton);
// open whatever port is available:
serial.on("portavailable", openPort);
// handle serial errors:
serial.on("requesterror", portError);
// handle any incoming serial data:
serial.on("data", serialEvent);
serial.on("close", makePortButton);
//space invadors
mainship = new spaceship();
arrRocks = [rock1,rock2,rock3] ;
textFont(mainFont);
//racing
maincar = new main_car();
arrCars = [car1,car2,car3] ;
//flappy bird
bird = new Bird(WIDTH / 2, HEIGHT / 2, 30);
pipes = new Pipes(60, 200, 130);
}
The draw function: this is where all the functions get called:
The draw() function starts by clearing the background of the canvas with background(0), then it checks the value of whichgame and renders the appropriate game. The code uses several other functions, such as controls(), score(), and startPage(), to handle game mechanics and display game elements.
In the first if statement, the code checks if whichgame is equal to 0, and if so, it displays two images: main and extraedge and this is like the default screen. In the second if statement, the code checks if whichgame is equal to 1 and, if so, it displays the game for whichgame 1 which is space invadors. This game has several possible states (running, end, start) and the code uses if statements to handle each state. The third and fourth if statements do the same thing for games 2 and 3, car race and flappy bird respectively.
The boulders class represents the falling boulders, and the bulletClass class represents bullets that the player can shoot to destroy the boulders. The spaceship class represents the player’s spaceship, and the cars class represents the cars that the player must avoid. The main_car class is a subclass of spaceship, and it appears to have the same functionality as the spaceship class.
The boulders class has a display() function that is used to draw the boulder on the screen, a move() function that is used to update the boulder’s position, and a width() function that is used to determine the width of the boulder. The bulletClass class has a display() function that is used to draw the bullet on the screen, a move() function that is used to update the bullet’s position, and a didcollide(other) function that is used to check if the bullet has collided with another object.
The spaceship and main_car classes have a display() function that is used to draw the spaceship or car on the screen, a move() function that is used to update the spaceship or car’s position, and a didcollide(other) function that is used to check if the spaceship or car has collided with another object. The cars class has the same functions as the boulders class, but it is used to represent cars rather than boulders.
Flappy Bird: Use the button on the Arduino or the UP key to jump.
Racing Game: Use the potentiometer or left and right keys to control the car’s steering.
Space Invaders: Use the potentiometers or left and right keys to control the spaceship’s movement and button or UP key to fire lasers.
User testing of Final game with physical controls:
What we are proud of:
Firstly, we are particularly proud of the artistic expression we have been able to give through the games and the console set-up. We had a lot of fun painting and wiring the console so that parts of our personalities were expressed. We are also happy that we were able to optimize the code over time to make the games smoother, this was a different experience to just debugging the code.
Here is the final sketch:
The console:
We wanted to design a simple console that allows the suer to toggle through the game and also play the game efficiently. We also had the goal of using as minimal resources as possible: hence we decided to use an old shoe box that was made out of recycled materials and apply paint over it.
The concept was to use AI to control an arduino robot. Using the machine learning module handsfree.js, I mapped different hand gestures and finger positions to control an arduino robot I created. The arduino robot had 4 DC motors for movement, 1 ultrasonic sensor to measure the distance of nearest obstacles, 1 buzzer and two LED lights.
Interaction Design:
A video webcam feed in p5js maps your fingers and palms, calculates the positions of the two hands and sends the data to arduino. Based on the palm positions, arduino decides whether to move forward, backward, left or right. Arduino calculates the distance between the robot and the nearest obstacle ahead of it and sends the data to p5js, which then displays the distance to you.
Also, by pinching your index fingers in p5js, you’re able to increase and decrease the speed of the DC motors. Anytime your start the webcam in p5js, this sends data to arduino and it plays a sound using the buzzer to signal readiness to receive the data.
Arduino Code:
The arduino code uses two libraries: “NewPing.h” and “TimerFreeTone.h”. The NewPing library makesre using the ultrasonic sensor easy and convenient. The TimerFreeTone also simplifies the use of the tone function and makes working with the buzzer seamless.
The additional header file labeled “pitches.h” contains sounds that are played when arduino is ready to receive data from p5js.
#include "pitches.h"
#include <NewPing.h>
#include <TimerFreeTone.h>
//left_wheels
const int left_ain1Pin = 3;
const int left_ain2Pin = 4;
const int left_pwmAPin = 5;
//left wheels speed
int leftspeedA=200;
//right wheels
const int right_bin1Pin = 8;
const int right_bin2Pin = 7;
const int right_pwmBPin = 6;
//right wheels speed
int rightspeedB=200;
//ditance sensor
const int trig = 9;
const int echo = 10;
const int max = 100;
//distance
NewPing dist(trig, echo, max);
int direct;
//leds
int greenlight = 2;
int redlight = 12;
//state machine for timer
unsigned long previousMillis;
long interval = 1000;
//foward motion function
void forward(){
analogWrite(left_pwmAPin, leftspeedA);
analogWrite(right_pwmBPin, rightspeedB);
digitalWrite(left_ain1Pin, LOW);
digitalWrite(left_ain2Pin, HIGH);
digitalWrite(right_bin1Pin, HIGH);
digitalWrite(right_bin2Pin, LOW);
}
//backward motion function
void backward(){
analogWrite(left_pwmAPin, leftspeedA);
analogWrite(right_pwmBPin, rightspeedB);
digitalWrite(left_ain1Pin, HIGH);
digitalWrite(left_ain2Pin, LOW);
digitalWrite(right_bin1Pin, LOW);
digitalWrite(right_bin2Pin, HIGH);
}
//left motion function
void left(){
analogWrite(left_pwmAPin, leftspeedA);
analogWrite(right_pwmBPin, rightspeedB+50);
digitalWrite(left_ain1Pin, LOW);
digitalWrite(left_ain2Pin, LOW);
digitalWrite(right_bin1Pin, HIGH);
digitalWrite(right_bin2Pin, LOW);
}
//right motion function
void right(){
analogWrite(left_pwmAPin, leftspeedA+50);
analogWrite(right_pwmBPin, rightspeedB);
digitalWrite(left_ain1Pin, LOW);
digitalWrite(left_ain2Pin, HIGH);
digitalWrite(right_bin1Pin, LOW);
digitalWrite(right_bin2Pin, LOW);
}
//stop all movements
void stop(){
analogWrite(left_pwmAPin, leftspeedA);
analogWrite(right_pwmBPin, rightspeedB);
digitalWrite(left_ain1Pin, LOW);
digitalWrite(left_ain2Pin, LOW);
digitalWrite(right_bin1Pin, LOW);
digitalWrite(right_bin2Pin, LOW);
}
//playing christmas themed songs as the car moves
void play(){
int tempo = 180;
int notes = sizeof(melody) / sizeof(melody[0]) / 2;
// this calculates the duration of a whole note in ms
int wholenote = (60000 * 4) / tempo;
int divider = 0, noteDuration = 0;
// iterate over the notes of the melody.
// Remember, the array is twice the number of notes (notes + durations)
for (int thisNote = 0; thisNote < notes * 2; thisNote = thisNote + 2) {
// calculates the duration of each note
divider = melody[thisNote + 1];
if (divider > 0) {
// regular note, just proceed
noteDuration = (wholenote) / divider;
} else if (divider < 0) {
// dotted notes are represented with negative durations!!
noteDuration = (wholenote) / abs(divider);
noteDuration *= 1.5; // increases the duration in half for dotted notes
}
// we only play the note for 90% of the duration, leaving 10% as a pause
TimerFreeTone(11, melody[thisNote], noteDuration * 0.9, 10);
}
}
void setup() {
Serial.begin(9600);
pinMode(left_ain1Pin, OUTPUT);
pinMode(left_ain2Pin, OUTPUT);
pinMode(left_pwmAPin, OUTPUT); // not needed really
pinMode(right_bin1Pin, OUTPUT);
pinMode(right_bin2Pin, OUTPUT);
pinMode(right_pwmBPin, OUTPUT);
//leds
pinMode(greenlight, OUTPUT);
pinMode(redlight, OUTPUT);
play();
}
void loop() {
direct = dist.ping_cm();
if(direct == 0){
direct = 100;
}
Serial.println(direct);
//reading values for p5js
//toggles between 1 and 0 to specify which movement
int fo = Serial.parseInt(); //controls forward movement
int ba = Serial.parseInt(); //controls back movement
int le = Serial.parseInt(); //controls left movement
int ri = Serial.parseInt(); //controls right movement
int sp = Serial.parseInt(); //controls speed
//read the speed from p5js and assign it to the speed of the wheels
leftspeedA = sp;
rightspeedB = sp;
//checking to see which direction to move as per the command from p5js
if(fo != 0 || ba != 0 || le != 0 || ri != 0){
//set a time stamp here
unsigned long currentMillis = millis();
//calculate interval between and current stamp and previous stamp
//compare to the set duration (interval)
long itr = currentMillis - previousMillis;
if(fo == 1 && itr > interval){
previousMillis = currentMillis;
//turn green light on when moving forward
digitalWrite(greenlight, HIGH);
forward();
}
else if (ba == 1 && itr > interval){
previousMillis = currentMillis;
digitalWrite(redlight, HIGH);
backward();
}
else if (le == 1 && itr > interval){
previousMillis = currentMillis;
//turn green light on when moving left
digitalWrite(greenlight, HIGH);
left();
}
else if (ri == 1 && itr > interval){
previousMillis = currentMillis;
//turn green light on when moving right
digitalWrite(greenlight, HIGH);
right();
}
}else{
stop();
digitalWrite(greenlight, LOW);
digitalWrite(redlight, LOW);
}
}
p5js uses two additional libraries: handsfree.js library and the webserial library. The handsfree.js is a machine learning module that enables detection of the hands, postures and hand movement. The webserial library enables serial communication between arduino and p5js.
let speed = 50;
let distance;
//handsfree module
let running = false;
let once = false;
//forward, back, left, right
let direction = [0, 0, 0, 0];
let direct;
//image
let robo;
let right;
let left;
let back;
//fonts
let rboto;
let header;
//states
let start_state = true;
let instruct_state = false;
let command_state = false;
// This is like pmouseX and pmouseY...but for every finger [pointer, middle, ring, pinky]
let prevPointer = [
// Left hand
[{x: 0, y: 0}, {x: 0, y: 0}, {x: 0, y: 0}, {x: 0, y: 0}],
// Right hand
[{x: 0, y: 0}, {x: 0, y: 0}, {x: 0, y: 0}, {x: 0, y: 0}]
]
// Landmark indexes for fingertips [pointer, middle, ring, pinky]...these are the same for both hands
let fingertips = [8, 12, 16, 20]
//preload function
function preload(){
robo = loadImage("robo.png");
header = loadFont("DiplomataSC-Regular.ttf");
rboto = loadFont("Roboto-Regular.ttf");
left = loadImage("left.jpg");
right = loadImage("right.jpg");
forward = loadImage("forward.jpg");
back = loadImage("back.jpg");
pinch_r = loadImage("right_pinch.jpg");
pinch_l = loadImage("left_pinch.jpg");
}
function setup() {
sketch = createCanvas(windowWidth, 200)
// Colors for each fingertip
colorMap = [
// Left fingertips
[color(0, 0, 0), color(255, 0, 255), color(0, 0, 255), color(255, 255, 255)],
// Right fingertips
[color(255, 0, 0), color(0, 255, 0), color(0, 0, 255), color(255, 255, 0)]
]
// #1 Turn on some models (hand tracking) and the show debugger
// @see https://handsfree.js.org/#quickstart-workflow
handsfree = new Handsfree({
showDebug:true,
hands: true,
})
handsfree.enablePlugins('browser')
handsfree.plugin.pinchScroll.disable()
handsfree.update({
setup:{
canvas:{
hands: {
$el: true,
width: 400,
height: 400
},
wrap: {
$el: true,
width: 400,
height: 400
},
video: {
$el: true,
width: 400,
height: 400
}
}
}
})
}
function draw() {
start();
instruct();
command();
}
function keyPressed()
{
if(keyCode === ENTER){
setUpSerial();
command_state = true;
}
}
function fingerPosition () {
// Check for pinches and create dots if something is pinched
const hands = handsfree.data?.hands
if (hands?.pinchState) {
// Loop through each hand
hands.pinchState.forEach((hand, handIndex) => {
// Loop through each finger
hand.forEach((state, finger) => {
if (hands.landmarks?.[handIndex]?.[fingertips[finger]]) {
// Landmarks are in percentage, so lets scale up
let x = 640 - hands.landmarks[handIndex][fingertips[finger]].x * 640
let y = hands.landmarks[handIndex][fingertips[finger]].y * 480
// Set the position of each finger
prevPointer[handIndex][finger] = {x, y}
}
})
})
}
//pinch right left index finger to decrease volume
if (hands?.pinchState && hands.pinchState[0][0] === 'released') {
if(speed > 50){
speed-=10;
}
}
//pinch right index finger to increase volume
if (hands?.pinchState && hands.pinchState[1][0] === 'released') {
if(speed < 200){
speed +=10;
}
}
//pinch right middle finger to play christmas songs
if (hands?.pinchState && hands.pinchState[1][1] === 'held') {
christmas = 1;
}else{
christmas = 0;
}
let posx_right= int(prevPointer[1][1].x);
let posy_right = int(prevPointer[1][1].y);
let posx_left= int(prevPointer[0][1].x);
let posy_left = int(prevPointer[0][1].y);
if( (posy_right < 150) && (posy_left < 150) && (posx_right > 320 && posx_left < 320) ){
direction[0] = 1;
direct = "foward";
}
else{
direction[0] = 0;
direct = "";
}
if((posy_left >240) && (posy_right > 240) && (posx_right > 320 && posx_left < 320)){
direction[1] = 1;
direct = "backward"
}else{
direction[1] = 0;
}
if((posx_left > 0 && posx_left < 160) && (posx_right > 0 && posx_right < 320)){
direction[2] = 1;
direct = "left"
}
else{
direction[2] = 0;
}
if(posx_left > 320 && posx_right > 480){
direction[3] = 1;
direct = "right"
}
else{
direction[3] = 0;
}
}
function readSerial(data) {
if(data != null){
distance = int(data);
}
let sendToArduino = direction[0] +"," + direction[1]+"," + direction[2]+","+direction[3]+","+ speed + "\n";
writeSerial(sendToArduino);
}
function windowResized() {
resizeCanvas(windowWidth, 200);
}
function sethandsfree(){
if(running == false){
buttonStart = createButton('START Webcam')
buttonStart.size(100, 100);
let scol = color(76, 187, 23);
buttonStart.style('background-color', scol);
buttonStart.class('handsfree-show-when-stopped')
buttonStart.class('handsfree-hide-when-loading')
buttonStart.mousePressed(() => handsfree.start())
// Create a "loading..." button
buttonLoading = createButton('...loading...')
buttonLoading.size(windowWidth);
buttonLoading.class('handsfree-show-when-loading')
running = true;
}
if(running == true){
// Create a stop button
buttonStop = createButton('STOP Webcam')
buttonStop.size(100, 100);
let stcol = color(128,0,0);
//buttonStop.position(windowWidth-100);
buttonStop.style('background-color', stcol);
buttonStop.class('handsfree-show-when-started')
buttonStop.mousePressed(() => handsfree.stop())
running = false;
}
}
function start(){
if(start_state == true){
createCanvas(600, 600);
background(255, 255, 255);
image(robo, 0, 200, 600, 400);
//header
textSize(70);
fill(0, 102, 153);
textAlign(CENTER)
textFont("header");
text("MetaDrive", 300, 60);
//click for instructions
textSize(20);
fill(0, 102, 153);
textAlign(CENTER)
textFont("rboto")
text("AI controlled robot", 300, 100);
text("Click to view Instructions!", 300, 150);
}
}
function instruct(){
if (instruct_state == true){
createCanvas(600, 600);
background(255, 255, 255);
textSize(40);
fill(0, 102, 153);
textFont("header");
text("INSTRUCTIONS", 300, 30);
textAlign(LEFT);
textSize(20);
textWrap(WORD);
textFont("rboto")
fill(0);
//movement
text("The Robot moves based on your hand gestures.Make sure to stand in the center of the video feed window: ", 20, 50, 550, 50);
text("Forward movement, raise both hands slightly above the mid section of the video window: ", 40, 120, 420, 50);
text("Right movement, move both hands to the right half of the video window: ",40, 190, 420, 50);
text("Left movement, move both hands to the left half of the video window: ", 40, 260, 420, 50);
text("Backward movement, invert both hands as shown: ", 40, 330, 420, 50);
//speed
text("Increasing and Decreasing Speed: ", 20, 370, 420, 50);
text("Increase - Pinch Right Index Finger: ", 40, 410, 420, 50);
text("Decrease - Pinch Left Index Finger: ", 40, 450, 420, 50);
fill(0, 102, 153);
text("Press Enter to Begin controlling the robot. Select 'Generic CDC usbmoderm' as Serial port ", 20, 500, 550, 50);
//instruction images
image(forward, 480, 120, 40, 40);
image(right, 480, 190, 40, 40);
image(back, 480, 330, 40, 40);
image(left, 480, 260, 40, 40);
image(pinch_r, 480, 410, 40, 40);
image(pinch_l, 480, 450, 40, 40);
instruct_state = false;
}
}
function command(){
if(command_state == true){
createCanvas(windowWidth, 200);
if(serialActive){
background(255);
textSize(30);
fill(0, 102, 153);
textAlign(CENTER)
text("MetaDrive", windowWidth/2, 30);
textSize(20);
fill(0);
textAlign(LEFT);
text("Speed: "+ speed, 30, 100);
text("Direction: "+ direct, 300, 100);
if(distance < 10){
fill(255, 0, 0);
}
text("Object Distance: "+ distance, 30, 150);
fingerPosition()
if(once == false){
sethandsfree()
once = true;
}
}
}
}
function mousePressed(){
if(start_state == true){
start_state = false;
instruct_state = true;
}
}
The p5js code has three pages: the start page, instruction page and command page. The three pages move in order based on the events that the user executes. The command page is where the interaction happens between p5js and arduino.
The start page:
The instruction page:
The control page:
Communication between p5js and arduino:
p5js sends 5 integer controls for forward, back, left, right and speed in arduino. Arduino sends the distance measured by the ultrasonic sensor to p5js.
Video Demo:
Project Pictures:
Aspects I’m proud of:
I’m particularly proud of how I drew the hand skeleton, measured the hand gestures and hand positions from the handsfree.js.
function fingerPosition () {
// Check for pinches and create dots if something is pinched
const hands = handsfree.data?.hands
if (hands?.pinchState) {
// Loop through each hand
hands.pinchState.forEach((hand, handIndex) => {
// Loop through each finger
hand.forEach((state, finger) => {
if (hands.landmarks?.[handIndex]?.[fingertips[finger]]) {
// Landmarks are in percentage, so lets scale up
let x = 640 - hands.landmarks[handIndex][fingertips[finger]].x * 640
let y = hands.landmarks[handIndex][fingertips[finger]].y * 480
// Set the position of each finger
prevPointer[handIndex][finger] = {x, y}
}
})
})
}
//pinch right left index finger to decrease volume
if (hands?.pinchState && hands.pinchState[0][0] === 'released') {
if(speed > 50){
speed-=10;
}
}
//pinch right index finger to increase volume
if (hands?.pinchState && hands.pinchState[1][0] === 'released') {
if(speed < 200){
speed +=10;
}
}
//pinch right middle finger to play christmas songs
if (hands?.pinchState && hands.pinchState[1][1] === 'held') {
christmas = 1;
}else{
christmas = 0;
}
let posx_right= int(prevPointer[1][1].x);
let posy_right = int(prevPointer[1][1].y);
let posx_left= int(prevPointer[0][1].x);
let posy_left = int(prevPointer[0][1].y);
if( (posy_right < 150) && (posy_left < 150) && (posx_right > 320 && posx_left < 320) ){
direction[0] = 1;
direct = "foward";
}
else{
direction[0] = 0;
direct = "";
}
if((posy_left >240) && (posy_right > 240) && (posx_right > 320 && posx_left < 320)){
direction[1] = 1;
direct = "backward"
}else{
direction[1] = 0;
}
if((posx_left > 0 && posx_left < 160) && (posx_right > 0 && posx_right < 320)){
direction[2] = 1;
direct = "left"
}
else{
direction[2] = 0;
}
if(posx_left > 320 && posx_right > 480){
direction[3] = 1;
direct = "right"
}
else{
direction[3] = 0;
}
}
Future Improvements:
I plan to make the communication between arduino and p5js wireless and probably attach a camera to the robot to access the robot arial vision. Another area of improvement is to remodel the robot’s body and 3D print the model.
The concept of my game is based on the Chrome running dinosaur game, which I adapted for my midterm project. I wanted to learn to work with sprites, so I decided that the pony will be the main theme. For the final, I adapted it more, and when the player jumps physically, the pony will jump on the screen, too.
Implementation
Description of interaction design
The game starts when the player clicks on the mouse and they have to connect the port by pressing the space bar. After that, the player has to jump over the little dragon and earn points by collecting a pink magic box, they have 3 lives. The score and remaining lives are displayed at the top of the screen.
For the physical part, I connected two force resistors to Arduino, by soldering really long wire to the sensor and sticking them onto the board.
Arduino code is pretty simple, I just get the sensor values and send them to p5 by also adding a coma.
Schematic and circuit
Description of p5.js code
Because I implemented a collision library, it calculates to what extent the pony collides with a Draco, which is why depending on where/when they collide, lives change non-linearly. For example, if it doesn’t jump at all it is -1 point, but if the pony touches the Draco slightly around half a point is deducted. This gives the players grace jump, as the collision area is not too precise and bigger. And the game ends when there are no lives left.
I had trouble restarting the game and realized that my lives were not updated whenever I want to restart. That is why I created a restart function that resets all the variables.
//to restart the game and scores
const restartGame = () => {
gameLost=false;
score=0;
lives=3;
lives1=3;
lastFrame=0;
frameDelay=500;
lastFrame1=0;
frameDelay1=200;
dracos = [];
stars=[];
pinkiePie= new PinkiePie(walk);
new Draco();
new Star();
loop();
}
Description of communication between Arduino and p5.js
function readSerial(data) {
////////////////////////////////////
//READ FROM ARDUINO HERE
////////////////////////////////////
//console.log("data"+data);
if (data != null) {
//for button
//if(data==0)
let fromArduino = split(trim(data), ",");
// if the right length, then proceed
if (fromArduino.length == 2) {
sensor1= fromArduino[0];
sensor2 = fromArduino[1];
console.log("s1 "+sensor1);
console.log("s2 "+sensor2);
if((sensor1 <= 1) && (sensor2<=1)){
pinkiePie.jump();
}
}
}
}
Here, I get the data and split it on comma, and store the two values in an array, assign them to two sensor variables. When a player stands on the board, there is some amount of pressure, when they jump->almost no pressure, and when they land on the board there is a force again. That is why, I made an if statement when there is no force, Pinkie Pie will jump.
What are some aspects of the project that you’re particularly proud of?
Overall, I am really proud of how the project turned out and the experience players have when the game is dependent on their physical activity. I am also happy that I learned how to work with another type of sensor, which means that I can work with more sensors and hardware features in the future thanks to the knowledge I got from the class. I love how sound also adds interactive experience, as well as lives implementation.
What are some areas for future improvement?
There is a problem with the sensitivity of the game, as players should practice and get used to the pace of the game. This could be improved by adding more sensors or maybe trying to implement other sensors, too. I also think the game could be improved by adding more levels or speeding up the pace when players play longer.
The three of us collectively decided to make something similar to an arcade machine as it was a very exciting part of our childhoods. Therefore, the project is a collection of three games that are designed to be played using an Arduino controller. The games include Flappy Bird, a popular mobile game in which the player controls a bird and navigates through obstacles; Racing Game, in which the player controls a race car and avoids colliding with the randomly generated cars as the race car overtakes them; and Space Invaders, a classic arcade game in which the player controls a spaceship and fights against invading aliens.
Arduino to P5js communication:
int button = 2;
int pot = A0;
int button1 = 4;
int button2 = 7;
int button3 = 8;
int lastPotValue;
int lastbutton;
long previousmill = 0;
long timebutton = 500;
void setup() {
// put your setup code here, to run once:
Serial.begin(9600);
pinMode(button, INPUT_PULLUP);
pinMode(button1, INPUT_PULLUP);
pinMode(button2, INPUT_PULLUP);
pinMode(button3, INPUT_PULLUP);
pinMode(pot, INPUT);
}
int getpot(){
int potValue = analogRead(pot)/255 ;
int temp;
if(potValue == 2 || potValue == 1){
temp = 1;
}else if(potValue == 3 || potValue == 4){
temp = 2;
}else{
temp = 0;
}
return temp;
}
void loop() {
int potValue = getpot();
int buttonState = !digitalRead(button);
long currentmill = millis();
int buttonstate1 = !digitalRead(button1);
int buttonstate2 = !digitalRead(button2);
int buttonstate3 = !digitalRead(button3);
int game = 0;
if(buttonstate1 == 1){
game = 1;
}
if(buttonstate2 == 1){
game = 2;
}
if(buttonstate3 == 1){
game = 3;
}
Serial.println(String(buttonState) + "," + String(potValue) + "," + String(game));
if(buttonState == 1 && currentmill - previousmill >= timebutton){
previousmill = currentmill;
lastbutton = buttonState;
}
}
We implemented this using the web serial. Here is how it briefly works:
The user connects an Arduino board to their computer using a USB cable.
The user writes and uploads a sketch (see above for the code) to the Arduino board that defines the behavior of the board and the data that it will send to the computer.
The user opens a P5.js sketch in their web browser and includes the p5.webserial.js library in their code.
The user adds event listeners to their P5.js sketch that will be called when the user connects or disconnects from the Arduino board, when the Arduino board is ready to be connected to, when there is an error communicating with the Arduino board, or when data is received from the Arduino board.
The user calls the getPorts() method of the p5.WebSerial object to check for any available Arduino boards. If an Arduino board is available, the portavailable event listener is called, which can be used to open a connection to the Arduino board.
Once the connection to the Arduino board is established, the user can send data to the Arduino board using the send() method of the p5.WebSerial object. The user can also receive data from the Arduino board using the data event listener, which is called whenever data is received from the Arduino board.
The user can use the received data from the Arduino board to control the behavior and appearance of their P5.js sketch. The user can also send data from the P5.js sketch to the Arduino board to control the behavior of the Arduino board.
When the user is finished using the Arduino board, they can close the connection to the board using the close() method of the p5.WebSerial object.
The getRndInteger() function is a helper function that returns a random integer between two given values. This function is used to randomly generate the heights of the pipes in the game. The Bird and Pipes classes define the objects that appear in the game. The Bird class has a draw() method that is used to draw the bird on the screen, an update() method that is used to update the bird’s position and velocity, a flap() method that causes the bird to jump upwards, and a checkDeath() method that checks if the bird has collided with any of the pipes and ends the game if necessary. The Pipes class has an update() method that updates the positions of the pipes and a drawPipes() method that draws the pipes on the screen. Overall, the code defines a simple game in which the player controls a bird and must avoid colliding with pipes by jumping over them. The game keeps track of the player’s score and ends if the bird hits a pipe.
Racing Game:
The generateCars() function is used to randomly generate cars that appear on the screen and the displayCars() function is used to draw the cars on the screen. The displayScore() function is used to display the player’s current score on the screen. The potentiometer returns three readings: 0,1, and 2 based on the positioning. Based on the number being returned by the potentiometer – we handle the car movement.
Space Invaders:
// both games =================================== function score(){ textSize(32); fill(250); text("Score: "+currentScore,20,50); } function increaseD(){ if(currentScore === 1 + prevScore){ difficulty += 0.5; prevScore = currentScore; // console.log(difficulty); } return random(1,5)+difficulty; }
//space invadors ======================================
function startPage(){
textSize(27);
fill(250);
text("Space invador",27,250);
textSize(15);
text("press enter to start",52,290);
}
function removeRocks(){
rocks.splice(0,rocks.length);
rocksctr = 0;
}
function displaybullets(){
for(let i = 0; i < bullets.length; i++){
bullets[i].display();
if(bullets[i].y < 0){
bullets.splice(i,1);
numBullets--;
}
}
// console.log(numBullets);
}
function generaterocks(){
let rand = int(random(0, 100));
let rand2 = int(random(0, 100));
if(rand % 7 == 0){
if(rand2 % 3 == 0){
if(rand2 % 2 == 0 && rand % 2 == 0){
rocks[rocksctr] = new boulders();
rocks[rocksctr].display();
// console.log(rocksctr);
rocksctr++;
}
}
}
}
function displayrocks(){
for(let i = 0; i < rocks.length; i++){
rocks[i].display();
// console.log(">",rocks.length);
let temp = false;
for(let j = 0; j < bullets.length; j++){
if(bullets[j].didcollide(rocks[i])){
temp = true;
bullets.splice(i,1);
numBullets--;
}
}
if(mainship.didcollide(rocks[i])){
rocks.splice(i,1);
rocksctr--;
gamestatus = "end";
bomb.play();
losing1.play();
}else if(rocks[i].y > height || temp){
rocks.splice(i,1);
rocksctr--;
}
}
}
var timechecker = 0.5;
function makebullet(x,y){
// console.log(x);
bullets[numBullets] = new bulletClass(x,y);
m0 = millis(); //time when laser created
if(timechecker>0.3){
bullets[numBullets].display();}
numBullets++;
m = millis();
timechecker = m - m0;
} //tried to disable continous shooting maybe look into it later
// end space invadors ================================
// start racing car game functions=====================
function startPage2(){
textSize(27);
fill(250);
text("Car racing",63,255);
textSize(15);
text("press enter to start",52,290);
}
function generateCars(){
let rand = int(random(0, 100));
let rand2 = int(random(0, 100));
if(rand % 7 == 0 && carrs.length < 4){
if(rand2 % 3 == 0){
if(rand2 % 2 == 0 && rand % 2 == 0){
carrs[carsctr] = new cars();
carrs[carsctr].display();
// console.log(carsctr);
carsctr++;
}
}
}
}
function displayCars(){
for(let i = 0; i < carrs.length; i++){
carrs[i].display();
// console.log(">",carrs.length);
let temp = false;
if(maincar.didcollide(carrs[i])){
checklanes(0,carrs[i]);
carrs.splice(i,1);
carsctr--;
currentScore = 0;
// bomb.play();
gamestatus = "end";
losing2.play();
// gamestatus = "end";
// bomb.play();
}else if(carrs[i].y > height || temp){
checklanes(0,carrs[i]);
carrs.splice(i,1);
carsctr--;
currentScore++;
cargoing.play();
}
}
}
function checklanes(x,other){
if(x === 1){
if(lanes2[other.temp] === 1){
other.temp = int(random(0,4));
other.x = lanes[other.temp];
checklanes(1,other);
}else{
lanes2[other.temp] = 1;
}
}else if(x === 0){
lanes2[other.temp] = 0;
}
}
function removeCars(){
carrs.splice(0,carrs.length);
carsctr = 0;
}
Initialization:
we initialized a lot of variables that would be used by the serial and the three games. the pre load function was also used to prepare the necessary pictures and sounds as well as fonts. the we set up what was necessary in the set up function.
let whichgame = 0;
let whichgameprev = 0;
// space invadors
let mainship;
let bullets = [];
let numBullets = 0;
let arrRocks;
let rocks = []
let lasersound;
let rocksctr = 0;
let difficulty = 0; // both games
let currentScore = 0; //both games
let prevScore = 0; //both games
let gamestatus = "start"; // both games
let rate = 0; // both games
let widthh = 400;
//racing game
let arrCars;
let carrs = [];
let carsctr = 0;
let lanes = [8,88,168,248,328];
let lanes2 = [0,0,0,0,0];
//flappy bird
var menu = 0;
var SCROLL_SPEED = 4;
var SCORE = 0;
let oof;
let bruh;
let music;
var bird ;
var pipes;
var lives = 5;
const GRAVITY = 8.81;
const JUMP_HEIGHT = 6.0;
const GROUND_HEIGHT = 20;
const WIDTH = 600;
const HEIGHT = 550;
//-------------arduino----------------
// let x=0;
// var c;
let values = [];
// variable to hold an instance of the p5.webserial library:
const serial = new p5.WebSerial();
// HTML button object:
let portButton;
let inData; // for incoming serial data
let outByte = 0;
function preload() {
main = loadImage('mainpicture.png');
//space invadar
img = loadImage('Hs4QN2.gif');
startScreen = loadImage('startscreen.gif');
ship = loadImage('Untitled-1.png');
bullet = loadImage('bullet.png');
soundFormats('mp3', 'ogg');
lasersound = loadSound('lasersound.mp3');
bomb = loadSound('explo.mp3');
rock1 = loadImage('rock1.png');
rock2 = loadImage('rock2.png');
rock3 = loadImage('rock3.png');
gameoverpng = loadImage('gameover.png');
mainFont = loadFont('PressStart2P-vaV7.ttf');
alternateFont = loadFont('metal lord.otf');
losing2 = loadSound('losing2.wav');
arcade = loadSound('arcade.mp3');
sp = loadSound('space.wav');
//racing car game;
imgg = loadImage('maincar.png');
car1 = loadImage('car1.png');
car2 = loadImage('car2.png');
car3 = loadImage('car3.png');
backgroundd = loadImage('background.png');
backgrounddd = loadImage('final.gif');
cargoing = loadSound('mixkit-fast-car-drive-by-1538.wav');
startscreen = loadImage('startscreen.png');
done = loadImage('gameovercar.png');
losing1 = loadSound('losing1.wav');
carpassing = loadSound('carpassing.wav');
extraedge = loadImage('extraedge.png');
//flappy bird
music = loadSound("bgmusic.mp3");
bg = loadImage('bg11.png');
home = loadImage('homescreem.png');
b = loadImage('bird.png');
jump = loadSound('flap-1.mp3');
oof = loadSound('oof.mp3');
}
function setup() {
createCanvas(600, 550);
arcade.play();
if (!navigator.serial) {
alert("WebSerial is not supported in this browser. Try Chrome or MS Edge.");
}
// if serial is available, add connect/disconnect listeners:
navigator.serial.addEventListener("connect", portConnect);
navigator.serial.addEventListener("disconnect", portDisconnect);
// check for any ports that are available:
serial.getPorts();
// if there's no port chosen, choose one:
serial.on("noport", makePortButton);
// open whatever port is available:
serial.on("portavailable", openPort);
// handle serial errors:
serial.on("requesterror", portError);
// handle any incoming serial data:
serial.on("data", serialEvent);
serial.on("close", makePortButton);
//space invadors
mainship = new spaceship();
arrRocks = [rock1,rock2,rock3] ;
textFont(mainFont);
//racing
maincar = new main_car();
arrCars = [car1,car2,car3] ;
//flappy bird
bird = new Bird(WIDTH / 2, HEIGHT / 2, 30);
pipes = new Pipes(60, 200, 130);
}
The draw function: this is where all the functions get called:
The draw() function starts by clearing the background of the canvas with background(0), then it checks the value of whichgame and renders the appropriate game. The code uses several other functions, such as controls(), score(), and startPage(), to handle game mechanics and display game elements.
In the first if statement, the code checks if whichgame is equal to 0, and if so, it displays two images: main and extraedge and this is like the default screen. In the second if statement, the code checks if whichgame is equal to 1 and, if so, it displays the game for whichgame 1 which is space invadors. This game has several possible states (running, end, start) and the code uses if statements to handle each state. The third and fourth if statements do the same thing for games 2 and 3, car race and flappy bird respectively.
The boulders class represents the falling boulders, and the bulletClass class represents bullets that the player can shoot to destroy the boulders. The spaceship class represents the player’s spaceship, and the cars class represents the cars that the player must avoid. The main_car class is a subclass of spaceship, and it appears to have the same functionality as the spaceship class.
The boulders class has a display() function that is used to draw the boulder on the screen, a move() function that is used to update the boulder’s position, and a width() function that is used to determine the width of the boulder. The bulletClass class has a display() function that is used to draw the bullet on the screen, a move() function that is used to update the bullet’s position, and a didcollide(other) function that is used to check if the bullet has collided with another object.
The spaceship and main_car classes have a display() function that is used to draw the spaceship or car on the screen, a move() function that is used to update the spaceship or car’s position, and a didcollide(other) function that is used to check if the spaceship or car has collided with another object. The cars class has the same functions as the boulders class, but it is used to represent cars rather than boulders.
Flappy Bird: Use the button on the arduino or the UP key to jump.
Racing Game: Use the potentiometer or left and right keys to control the car’s steering.
Space Invaders: Use the potentiometers or left and right keys to control the spaceship’s movement and button or UP key to fire lasers.
user testing:
Conclusion
This project demonstrates how to create and play games using p5.js and Arduino. The project includes three games that can be controlled using potentiometers and push buttons, and can be easily extended to include additional games and custom controller designs. We’re particularly proud of the aesthetics of the games – we were able to recreate what we initally had in mind. Furthermore, we had a lot of bugs which wouldn’t let the games run smoothly. We figured out how to implement the games smoothly by making changes in our algorithms and by handling specific types of errors which were mostly given by the arduino board. However, there is no proper restart function for flappy bird – if you lose, the canvas simply starts from scratch.
Work contributed: Car racing game, Space invaders, while helping in integrating the games together, and using my graphic design skills in making 1962 more user friendly. Also helped making the circuit necessary to send data.
The three of us collectively decided to make something similar to an arcade machine as it was a very exciting part of our childhoods. Therefore, the project is a collection of three games that are designed to be played using an Arduino controller. The games include Flappy Bird, a popular mobile game in which the player controls a bird and navigates through obstacles; Racing Game, in which the player controls a race car and avoids colliding with the randomly generated cars as the race car overtakes them; and Space Invaders, a classic arcade game in which the player controls a spaceship and fights against invading aliens.
Arduino to P5js communication:
int button = 2;
int pot = A0;
int button1 = 4;
int button2 = 7;
int button3 = 8;
int lastPotValue;
int lastbutton;
long previousmill = 0;
long timebutton = 500;
void setup() {
// put your setup code here, to run once:
Serial.begin(9600);
pinMode(button, INPUT_PULLUP);
pinMode(button1, INPUT_PULLUP);
pinMode(button2, INPUT_PULLUP);
pinMode(button3, INPUT_PULLUP);
pinMode(pot, INPUT);
}
int getpot(){
int potValue = analogRead(pot)/255 ;
int temp;
if(potValue == 2 || potValue == 1){
temp = 1;
}else if(potValue == 3 || potValue == 4){
temp = 2;
}else{
temp = 0;
}
return temp;
}
void loop() {
int potValue = getpot();
int buttonState = !digitalRead(button);
long currentmill = millis();
int buttonstate1 = !digitalRead(button1);
int buttonstate2 = !digitalRead(button2);
int buttonstate3 = !digitalRead(button3);
int game = 0;
if(buttonstate1 == 1){
game = 1;
}
if(buttonstate2 == 1){
game = 2;
}
if(buttonstate3 == 1){
game = 3;
}
Serial.println(String(buttonState) + "," + String(potValue) + "," + String(game));
if(buttonState == 1 && currentmill - previousmill >= timebutton){
previousmill = currentmill;
lastbutton = buttonState;
}
}
We implemented this using the web serial. Here is how it briefly works:
The user connects an Arduino board to their computer using a USB cable.
The user writes and uploads a sketch (see above for the code) to the Arduino board that defines the behavior of the board and the data that it will send to the computer.
The user opens a P5.js sketch in their web browser and includes the p5.webserial.js library in their code.
The user adds event listeners to their P5.js sketch that will be called when the user connects or disconnects from the Arduino board, when the Arduino board is ready to be connected to, when there is an error communicating with the Arduino board, or when data is received from the Arduino board.
The user calls the getPorts() method of the p5.WebSerial object to check for any available Arduino boards. If an Arduino board is available, the portavailable event listener is called, which can be used to open a connection to the Arduino board.
Once the connection to the Arduino board is established, the user can send data to the Arduino board using the send() method of the p5.WebSerial object. The user can also receive data from the Arduino board using the data event listener, which is called whenever data is received from the Arduino board.
The user can use the received data from the Arduino board to control the behavior and appearance of their P5.js sketch. The user can also send data from the P5.js sketch to the Arduino board to control the behavior of the Arduino board.
When the user is finished using the Arduino board, they can close the connection to the board using the close() method of the p5.WebSerial object.
The getRndInteger() function is a helper function that returns a random integer between two given values. This function is used to randomly generate the heights of the pipes in the game. The Bird and Pipes classes define the objects that appear in the game. The Bird class has a draw() method that is used to draw the bird on the screen, an update() method that is used to update the bird’s position and velocity, a flap() method that causes the bird to jump upwards, and a checkDeath() method that checks if the bird has collided with any of the pipes and ends the game if necessary. The Pipes class has an update() method that updates the positions of the pipes and a drawPipes() method that draws the pipes on the screen. Overall, the code defines a simple game in which the player controls a bird and must avoid colliding with pipes by jumping over them. The game keeps track of the player’s score and ends if the bird hits a pipe.
Racing Game:
The generateCars() function is used to randomly generate cars that appear on the screen and the displayCars() function is used to draw the cars on the screen. The displayScore() function is used to display the player’s current score on the screen. The potentiometer returns three readings: 0,1, and 2 based on the positioning. Based on the number being returned by the potentiometer – we handle the car movement.
Space Invaders:
// both games =================================== function score(){ textSize(32); fill(250); text("Score: "+currentScore,20,50); } function increaseD(){ if(currentScore === 1 + prevScore){ difficulty += 0.5; prevScore = currentScore; // console.log(difficulty); } return random(1,5)+difficulty; }
//space invadors ======================================
function startPage(){
textSize(27);
fill(250);
text("Space invador",27,250);
textSize(15);
text("press enter to start",52,290);
}
function removeRocks(){
rocks.splice(0,rocks.length);
rocksctr = 0;
}
function displaybullets(){
for(let i = 0; i < bullets.length; i++){
bullets[i].display();
if(bullets[i].y < 0){
bullets.splice(i,1);
numBullets--;
}
}
// console.log(numBullets);
}
function generaterocks(){
let rand = int(random(0, 100));
let rand2 = int(random(0, 100));
if(rand % 7 == 0){
if(rand2 % 3 == 0){
if(rand2 % 2 == 0 && rand % 2 == 0){
rocks[rocksctr] = new boulders();
rocks[rocksctr].display();
// console.log(rocksctr);
rocksctr++;
}
}
}
}
function displayrocks(){
for(let i = 0; i < rocks.length; i++){
rocks[i].display();
// console.log(">",rocks.length);
let temp = false;
for(let j = 0; j < bullets.length; j++){
if(bullets[j].didcollide(rocks[i])){
temp = true;
bullets.splice(i,1);
numBullets--;
}
}
if(mainship.didcollide(rocks[i])){
rocks.splice(i,1);
rocksctr--;
gamestatus = "end";
bomb.play();
losing1.play();
}else if(rocks[i].y > height || temp){
rocks.splice(i,1);
rocksctr--;
}
}
}
var timechecker = 0.5;
function makebullet(x,y){
// console.log(x);
bullets[numBullets] = new bulletClass(x,y);
m0 = millis(); //time when laser created
if(timechecker>0.3){
bullets[numBullets].display();}
numBullets++;
m = millis();
timechecker = m - m0;
} //tried to disable continous shooting maybe look into it later
// end space invadors ================================
// start racing car game functions=====================
function startPage2(){
textSize(27);
fill(250);
text("Car racing",63,255);
textSize(15);
text("press enter to start",52,290);
}
function generateCars(){
let rand = int(random(0, 100));
let rand2 = int(random(0, 100));
if(rand % 7 == 0 && carrs.length < 4){
if(rand2 % 3 == 0){
if(rand2 % 2 == 0 && rand % 2 == 0){
carrs[carsctr] = new cars();
carrs[carsctr].display();
// console.log(carsctr);
carsctr++;
}
}
}
}
function displayCars(){
for(let i = 0; i < carrs.length; i++){
carrs[i].display();
// console.log(">",carrs.length);
let temp = false;
if(maincar.didcollide(carrs[i])){
checklanes(0,carrs[i]);
carrs.splice(i,1);
carsctr--;
currentScore = 0;
// bomb.play();
gamestatus = "end";
losing2.play();
// gamestatus = "end";
// bomb.play();
}else if(carrs[i].y > height || temp){
checklanes(0,carrs[i]);
carrs.splice(i,1);
carsctr--;
currentScore++;
cargoing.play();
}
}
}
function checklanes(x,other){
if(x === 1){
if(lanes2[other.temp] === 1){
other.temp = int(random(0,4));
other.x = lanes[other.temp];
checklanes(1,other);
}else{
lanes2[other.temp] = 1;
}
}else if(x === 0){
lanes2[other.temp] = 0;
}
}
function removeCars(){
carrs.splice(0,carrs.length);
carsctr = 0;
}
Initialization:
we initialized a lot of variables that would be used by the serial and the three games. the pre load function was also used to prepare the necessary pictures and sounds as well as fonts. the we set up what was necessary in the set up function.
let whichgame = 0;
let whichgameprev = 0;
// space invadors
let mainship;
let bullets = [];
let numBullets = 0;
let arrRocks;
let rocks = []
let lasersound;
let rocksctr = 0;
let difficulty = 0; // both games
let currentScore = 0; //both games
let prevScore = 0; //both games
let gamestatus = "start"; // both games
let rate = 0; // both games
let widthh = 400;
//racing game
let arrCars;
let carrs = [];
let carsctr = 0;
let lanes = [8,88,168,248,328];
let lanes2 = [0,0,0,0,0];
//flappy bird
var menu = 0;
var SCROLL_SPEED = 4;
var SCORE = 0;
let oof;
let bruh;
let music;
var bird ;
var pipes;
var lives = 5;
const GRAVITY = 8.81;
const JUMP_HEIGHT = 6.0;
const GROUND_HEIGHT = 20;
const WIDTH = 600;
const HEIGHT = 550;
//-------------arduino----------------
// let x=0;
// var c;
let values = [];
// variable to hold an instance of the p5.webserial library:
const serial = new p5.WebSerial();
// HTML button object:
let portButton;
let inData; // for incoming serial data
let outByte = 0;
function preload() {
main = loadImage('mainpicture.png');
//space invadar
img = loadImage('Hs4QN2.gif');
startScreen = loadImage('startscreen.gif');
ship = loadImage('Untitled-1.png');
bullet = loadImage('bullet.png');
soundFormats('mp3', 'ogg');
lasersound = loadSound('lasersound.mp3');
bomb = loadSound('explo.mp3');
rock1 = loadImage('rock1.png');
rock2 = loadImage('rock2.png');
rock3 = loadImage('rock3.png');
gameoverpng = loadImage('gameover.png');
mainFont = loadFont('PressStart2P-vaV7.ttf');
alternateFont = loadFont('metal lord.otf');
losing2 = loadSound('losing2.wav');
arcade = loadSound('arcade.mp3');
sp = loadSound('space.wav');
//racing car game;
imgg = loadImage('maincar.png');
car1 = loadImage('car1.png');
car2 = loadImage('car2.png');
car3 = loadImage('car3.png');
backgroundd = loadImage('background.png');
backgrounddd = loadImage('final.gif');
cargoing = loadSound('mixkit-fast-car-drive-by-1538.wav');
startscreen = loadImage('startscreen.png');
done = loadImage('gameovercar.png');
losing1 = loadSound('losing1.wav');
carpassing = loadSound('carpassing.wav');
extraedge = loadImage('extraedge.png');
//flappy bird
music = loadSound("bgmusic.mp3");
bg = loadImage('bg11.png');
home = loadImage('homescreem.png');
b = loadImage('bird.png');
jump = loadSound('flap-1.mp3');
oof = loadSound('oof.mp3');
}
function setup() {
createCanvas(600, 550);
arcade.play();
if (!navigator.serial) {
alert("WebSerial is not supported in this browser. Try Chrome or MS Edge.");
}
// if serial is available, add connect/disconnect listeners:
navigator.serial.addEventListener("connect", portConnect);
navigator.serial.addEventListener("disconnect", portDisconnect);
// check for any ports that are available:
serial.getPorts();
// if there's no port chosen, choose one:
serial.on("noport", makePortButton);
// open whatever port is available:
serial.on("portavailable", openPort);
// handle serial errors:
serial.on("requesterror", portError);
// handle any incoming serial data:
serial.on("data", serialEvent);
serial.on("close", makePortButton);
//space invadors
mainship = new spaceship();
arrRocks = [rock1,rock2,rock3] ;
textFont(mainFont);
//racing
maincar = new main_car();
arrCars = [car1,car2,car3] ;
//flappy bird
bird = new Bird(WIDTH / 2, HEIGHT / 2, 30);
pipes = new Pipes(60, 200, 130);
}
The draw function: this is where all the functions get called:
The draw() function starts by clearing the background of the canvas with background(0), then it checks the value of whichgame and renders the appropriate game. The code uses several other functions, such as controls(), score(), and startPage(), to handle game mechanics and display game elements.
In the first if statement, the code checks if whichgame is equal to 0, and if so, it displays two images: main and extraedge and this is like the default screen. In the second if statement, the code checks if whichgame is equal to 1 and, if so, it displays the game for whichgame 1 which is space invadors. This game has several possible states (running, end, start) and the code uses if statements to handle each state. The third and fourth if statements do the same thing for games 2 and 3, car race and flappy bird respectively.
The boulders class represents the falling boulders, and the bulletClass class represents bullets that the player can shoot to destroy the boulders. The spaceship class represents the player’s spaceship, and the cars class represents the cars that the player must avoid. The main_car class is a subclass of spaceship, and it appears to have the same functionality as the spaceship class.
The boulders class has a display() function that is used to draw the boulder on the screen, a move() function that is used to update the boulder’s position, and a width() function that is used to determine the width of the boulder. The bulletClass class has a display() function that is used to draw the bullet on the screen, a move() function that is used to update the bullet’s position, and a didcollide(other) function that is used to check if the bullet has collided with another object.
The spaceship and main_car classes have a display() function that is used to draw the spaceship or car on the screen, a move() function that is used to update the spaceship or car’s position, and a didcollide(other) function that is used to check if the spaceship or car has collided with another object. The cars class has the same functions as the boulders class, but it is used to represent cars rather than boulders.
Flappy Bird: Use the button on the arduino or the UP key to jump.
Racing Game: Use the potentiometer or left and right keys to control the car’s steering.
Space Invaders: Use the potentiometers or left and right keys to control the spaceship’s movement and button or UP key to fire lasers.
Conclusion
This project demonstrates how to create and play games using p5.js and Arduino. The project includes three games that can be controlled using potentiometers and push buttons, and can be easily extended to include additional games and custom controller designs. We’re particularly proud of the aesthetics of the games – we were able to recreate what we initally had in mind. Furthermore, we had a lot of bugs which wouldn’t let the games run smoothly. We figured out how to implement the games smoothly by making changes in our algorithms and by handling specific types of errors which were mostly given by the arduino board. However, there is no proper restart function for flappy bird – if you lose, the canvas simply starts from scratch.
My contribution to the project was coding the handling the serial communication and the design of the physical user interface. I did some of the error handling and debugging towards the end as well.
As I was testing the game, I understood that one sensor is not enough as data gets into the game randomly depending on where is the user. That is why I decided to connect two sensors as well as get a board for jumping. I had a lot of stress getting data from two sensors as it was getting lost somewhere or as far as I understood, there was just an extra line.😭
Schematic
Circuit
User testing video
Results:
It seems like shoes can be a little obstacle on how legs are pressed on the board, as it might be difficult to center the pressure of the legs with shoes. If the legs are pressed correctly every time a person jumps, the game is pretty responsive.
For my final project, I made a radio that changes FM channels according to what value range you turn the potentiometer dial to. I inputted song files that play according to their designated serial monitor value and an array of different songs for each channel.
This includes:
51-240 (Hip Hop), which is designated the variable RED on p5
241-429 (History), variable YELLOW on p5
430-619 (Classic Rock), variable GREEN on p5
620-814 (Classical), variable TEAL on p5
815-1023 (Khaleeji), variable BLUE on p5
For 0-50 Radio is switched off (variable OFF on p5)
Implementation and Interaction
So, whenever you turn the dial to 267, for example, an audio file from the History channel will start to play, and the yellow LED will simultaneously light up to indicate the change in channels.
The interactive element comes from you turning the dial to change the channel, and being able to control the output according to what value you stop on.
Arduino Code
int potPin = A5; // potentiometer
int bluePin = 2;
int tealPin = 3;
int greenPin = 4;
int yellowPin = 5;
int redPin = 6;
int currentColor = 0; // current color
int OFF = 0;
int RED = 6;
int YELLOW = 5;
int GREEN = 4;
int TEAL = 3;
int BLUE = 2;
void setup() {
pinMode (potPin, INPUT);
pinMode (bluePin, OUTPUT);
pinMode (tealPin, OUTPUT);
pinMode (greenPin, OUTPUT);
pinMode (yellowPin, OUTPUT);
pinMode (redPin, OUTPUT);
Serial.begin (9600); // serial monitor count
}
void loop() {
// x == y (x is equal to y)
// x != y (x is not equal to y)
// x < y (x is less than y)
// x > y (x is greater than y)
// x <= y (x is less than or equal to y)
// x >= y (x is greater than or equal to y)
delay(100);
int potMeasure = analogRead (A5);
int mappedPot = map(potMeasure, 0, 1023, 0, 255);
Serial.println(currentColor);
if ((potMeasure > 0) && (potMeasure <= 50)) {
currentColor = 0;
digitalWrite (redPin, LOW);
digitalWrite (yellowPin, LOW);
digitalWrite (greenPin, LOW);
digitalWrite (tealPin, LOW);
digitalWrite (bluePin, LOW);
int OFF = 0;
}
else if ((potMeasure > 50) && (potMeasure <= 240)) {
currentColor = 6;
digitalWrite (redPin, HIGH);
digitalWrite (yellowPin, LOW);
digitalWrite (greenPin, LOW);
digitalWrite (tealPin, LOW);
digitalWrite (bluePin, LOW);
int RED = 1;
}
else if ((potMeasure > 240) && (potMeasure < 430)) {
currentColor = 5;
digitalWrite (yellowPin, HIGH);
digitalWrite (redPin, LOW);
digitalWrite (greenPin, LOW);
digitalWrite (tealPin, LOW);
digitalWrite (bluePin, LOW);
int YELLOW = 2;
}
else if ((potMeasure >= 430) && (potMeasure < 620)) {
currentColor = 4;
digitalWrite (greenPin, HIGH);
digitalWrite (redPin, LOW);
digitalWrite (yellowPin, LOW);
digitalWrite (tealPin, LOW);
digitalWrite (bluePin, LOW);
int GREEN = 3;
}
else if ((potMeasure >= 620) && (potMeasure < 815)) {
currentColor = 3;
digitalWrite (tealPin, HIGH);
digitalWrite (redPin, LOW);
digitalWrite (yellowPin, LOW);
digitalWrite (greenPin, LOW);
digitalWrite (bluePin, LOW);
int TEAL = 4;
}
else if ((potMeasure >= 815) && (potMeasure <= 1023)) {
currentColor = 2;
digitalWrite (bluePin, HIGH);
digitalWrite (redPin, LOW);
digitalWrite (yellowPin, LOW);
digitalWrite (greenPin, LOW);
digitalWrite (tealPin, LOW);
int BLUE = 5;
}
}
For my Arduino code, I essentially made it so that whenever the integer potMeasure (potentiometer value) is between a range of values, it would trigger the designated pin to light up.
For example, this segment of code here shows that whenever the potMeasure value is between or equivalent to 817 and 1023, all other pins but the bluePin are on LOW (to indicate them being switched off), and the bluePin is on HIGH (to indicate it being switched on). Each colored pin stands for each colored LED on the board.
The currentColor integer encompasses each of the potMeasure ranges for each LED to light up, hence why if the serial monitor is between or equivalent to 815 and 1023, the value 2 would pop up as it is the designated value for the blue LED.
int potMeasure = analogRead (A5);
int mappedPot = map(potMeasure, 0, 1023, 0, 255);
Serial.println(currentColor);
p5 Code
For p5, I first uploaded all the audio files for each channel to function preload and designated each channel an array of audio files that p5 can randomly choose from.
soundFormats("mp3");
// BLUE (KHALEEJI)
blueSong1 = loadSound ("MEHAD.mp3");
blueSong2 = loadSound ("UMY KM AHWAHA.mp3")
blueSounds = [blueSong1, blueSong2] // etc
I then designated a new variable called oldData in function draw, which holds the previous value of inData (serial monitor value/ potMeasure on Arduino). This was done so that whenever the user changes the channel, p5 would stop the song from playing so as to not interrupt the next song.
// oldData holds the *previous* value of inData
// if user changes station, p5 checks that oldData has changed into inData
// then prints the change
if (inData != oldData) {
// The station changed
console.log("changed from " + oldData + " to " + inData);
// stops all songs if inData (new station value) is not equal to oldData (old station value)
redSong1.stop();
redSong2.stop();
yellowSong1.stop();
yellowSong2.stop();
greenSong1.stop();
tealSong1.stop();
blueSong1.stop();
blueSong2.stop();
I then created an if statement for each inData value (BLUE in this context is the Khaleeji channel. Within each statement, there would be a new variable initializing the random selection of songs from the designated array, which it would then play.
if (inData == BLUE){
khaleejiChannel = random(blueSounds)
khaleejiChannel.play()
}
I then added more if statements further below within function draw, which consisted of the font and text to be displayed on the screen every time a channel is changed.
else if (inData == BLUE) { // KHALEEJI
// CHANNEL NAME TEXT
noStroke();
fill (128,172,57); // text color
textFont(radioFont, 45); // (font name, font size)
text("KHALEEJI", 178, 148); // (text, x, y)
glow(color(128,172,57), 19); // (color, intensity) calls glow function
}
The glow variable refers to the glow function further below, which makes the text look like it’s lit up.
Some other visual demonstration that indicates what channel you’re on. I would have liked for the background to possibly show the sound waves of the song playing and in the color of the designated channel.
I also wanted to do something more for the physical presentation of the Arduino board, though I was able to customize it just a little using some card paper.
