To be honest, I still don’t have a super clear idea of what I want to do with my project because I keep changing my mind, but currently I have the bare minimum elements working. For my project, I want to create a live version of Fruit Ninja, a classic mobile game that requires finger swiping interactions. For the live version, I will be using 4 different colored buttons, each button corresponding with the color of the fruit that appears on the screen. For example, if a red fruit shows up, the player should press the red button.

So far I have one of the 4 buttons working, just to test, but it doesn’t work the way it’s supposed to. It took me quite a while to figure out how to do the serial communication, but I realized that I need to send to Arduino the same amount of data that I want to receive on p5 (at least that’s what worked). At first I was only sending one thing, because technically I don’t use any information from p5 yet), but when I was receiving Arduino data it wasn’t working.

Physical Components

The physical components for my project are 4 buttons, a distance sensor, and one LED. The LED lights up when the distance is less than 10 cm. One thing I noticed is that the LED does not light up as bright sometimes when using the distance sensor, whereas when I used it to test if the buttons worked it was very bright (the testing was all in one program). So this is something to think about, because I tested both analog and digital write.

Screen/Digital Components

Here is a link to my work in progress p5 code. As I said earlier, I tested just the red button. Good news is that it removes the fruit when it is pressed, but the difficult thing to figure out is 1) how to remove only one fruit when it is pressed and 2) remove the fruit that corresponds with the button color. When I figure out these two issues, the rest of the project is mainly fine tuning.

I am still debating on whether or not I should add the distance sensor or not now. One idea that I came up with is to mark the floor where the player should stand and give then a “sword” to hit the buttons with. That will maybe make the game more challenging and eliminates the distance sensor.

Things I still need to work on:

• • Getting the color of the button to correspond with the color of the fruit.
  • Remove one fruit at a time when pressing a button.
  • Distance sensor to create an invisible barrier (maybe?).
  • Animate the fruit to look like it’s being thrown.
  • Aesthetics (but this is the very last thing to worry about).

 

 

I have admittedly fallen behind in terms of progress (in comparison to the current date), but I fortunately do have some updates to share! I have been able to commit to my chosen concept and start working on the primary components of the project.

The Concept, Revisited and Given Life

The base concept for my project involves an Arduino-based musical instrument of sorts that 1) the user can interact with to play notes and 2) is able to trigger a certain change in the p5js sketch when a correct tune—so a correct combination of notes—is played. As the general idea and its essential components were set in stone, I set out to first make the instrument itself. The plan was to have a device with five buttons that each play a different note through a speaker when pressed (D4, F4, A4, B4, D5). After some tinkering, I settled on a circuit that worked, the diagram of which looks like:

The actual circuit:

And now, the most prominent milestone so far: I was able to construct the code that I needed to achieve the desired effect from Arduino! Using arrays, I was able to simplify much of the repetitive code and condition checks, with the final result being:

#include "pitches.h"

int buttonPins[] = {
  2, 3, 4, 5, 6
};
int pinCount = 5;
int buttonStates[5];
int notes[] = {
  NOTE_D4, NOTE_F4, NOTE_A4, NOTE_B4, NOTE_D5
};
const int BUZZER_PIN = 8;
bool counter = false;

void setup() {
  Serial.begin(9600); // Initialize Serial
  for (int buttonPin = 0; buttonPin < pinCount; buttonPin++){
    pinMode(buttonPins[buttonPin], INPUT_PULLUP);
  }

  pinMode(BUZZER_PIN, OUTPUT); // Set Arduino Pin to Output Mode
  pinMode(13, OUTPUT);
}

void loop() {
  for (int buttonPin = 0; buttonPin < pinCount; buttonPin++){
    buttonStates[buttonPin] = digitalRead(buttonPins[buttonPin]);
  }
  for (int buttonPin = 0; buttonPin < pinCount; buttonPin++){
    if (buttonStates[buttonPin] == LOW) {
      if (counter == false) {
      Serial.println(buttonPin); //Sends Number of Button Being Pressed to p5js
      counter = true;
    }
    digitalWrite(13, HIGH);
    tone(BUZZER_PIN, notes[buttonPin], 500);
    delay(500);
    noTone(BUZZER_PIN); // Stop the tone playing:
  }
  else if (buttonStates[0] == HIGH) {
    digitalWrite(BUZZER_PIN, LOW);  // Turn off
    counter = false;
  }
  }
}

The use of for() loops in tandem with the buttonPins[] and notes[] arrays facilitates much of the input and output management here, with a notable line of code being Serial.println(buttonPin), which is used to send the “number” of the button being pressed to p5js. As I continue working on my project, I will be able to gather these outputs and run condition checks on p5js that determine whether the “correct” string of buttons (notes) has been played. Again, this will then trigger certain effects depending on the song. With the base Arduino-instrument completed, I can now move onto the next stages of the project—I look forward to what I can make of this vision!

The reading talks about how the design of products aids people with disability. It was really interesting as I never really thought of how certain designs do accommodate people with disabilities – I’m usually unaware of it.

The author mentions that the traditional views on design for disability have been to hide the disability which creates a negative connotation. Not only does it hide a part of the person, but it may also tell the person that they should be ashamed to publicly show and express their disability which could actually be a part of what makes them who they are.

An example of a design for the disabled they gave is eyewear. Glasses have shown the way some people may divert from this traditional view. There are many ranges and styles for eyeglasses giving people the choice to pick out what seems more comfortable and fashionable to them. There are also many people nowadays buying eyeglasses just for the aesthetic even if they have no prescription for them. However, if we look at another design they mentioned such as a prosthetic leg many people may associate negative connotations with it and may hide it due to it ‘not being goodlooking’ even though it is totally functional. I definitely think this needs to change. I don’t think anyone should have to feel that they are ashamed of their disability.

The last thing I want to mention is that I definitely agree with the fact that these designs should be as simple as possible. The main point of these designs is to help people who are disabled. The designs should be simple enough for the users to be able to use them to their full potential without stressing them out.

This was my hardest project yet. i spent a lot of time with my code and having a bunch of errors that I didn’t understand. after i tried to get some help from the internet i got a bit of a better understanding. what really helped is when I remembered what the professor said about old students records being on the WordPress site. this is my best attempt that kind of worked.

Task 1    

For the first task we were instructed to use a sensor on Arduino to make an ellipse in p5 move horizontally. And to have nothing on Arduino being controlled by p5.

After looking thru projects i decided to use a potentiometer to control the axis of the ellipse.

This was my p5 code.

function setup() {
  createCanvas(640, 480);
  textSize(18);
}

function draw() {
  background(54);
  map(100,0,1023,0,width);
  ellipse(100,height/2,100,37);
  
 

  if (!serialActive) {
    text("Press Space Bar to select Serial Port", 20, 30);
  } else {
    text("Connected", 20, 30);
  }

  
}

function keyPressed() {
  if (key == " ") {
    // important to have in order to start the serial connection!!
    setUpSerial();
  }
}

function readSerial(data) {
  ////////////////////////////////////
  //READ FROM ARDUINO HERE
  ////////////////////////////////////

  if (data != null) {
    ellipseXPos=data;
    }

  
}

My Arduino code:

I used it to just connect the sensor to the p5 sketch

void setup() {
  Serial.begin(9600);
}

void loop() {
  // wait for data from p5 before doing something
 
    
  
      int sensor = analogRead(A0);
      delay(1);
      Serial.println(sensor);
    
  
}

Task 2

For the second task we were initially asked to control the LED brightness. When i first took a stab at it i had no idea how to start. I initially tried to do it with mouse press but after looking at others projects  For inspiration I saw people used diffrent keys to control the brightness so i decided to do the same.

I was initially trying to use the B key for brightness and D for dull. for some reason i got an error. but after i tried the arrow keys like one of my colleges it seemed to work.  This is part of my code

let brightness = 0; 
let value=50;



function setup() {
  createCanvas(600, 600);

 
}

function draw() {
  background(255);
  
  if(value>255){
    value=255;
  }
  if(value<0){
    value=0;
  }
 
  
  brightness = value;

Arduino code:

int LED = 11;
void setup() {
 Serial.begin(9600);
 pinMode(LED, OUTPUT);
 while (Serial.available() <= 0) {
  Serial.println("Wait");  // send a starting message
   delay(300);               // wait 1/3 second
  }
 }
 void loop() {
  // wait for data from p5 before doing something
 while (Serial.available()) {
   int brightness = Serial.parseInt(); 
    if (Serial.read() == '\n') {
     analogWrite(LED, brightness); // turn on LED and adjusts brightness
     Serial.println("LIT"); 
    }
  }
 }

Task 3

This task started off as the easiest one but ended up being the most confusing.  I started of by looking at others projects like i did previously and i saw people using  the gravity wind example  so i took a look at that. I created. a p5 Code and didn’t get an error but when i hooked it up to Arduino it didn’t work.

 if (!serial Active) {
    text("Press Space Bar to select Serial Port", 20, 30);
  } else {
    text("Connected", 20, 30);
  }
  
  
  ellipse(position.x,position.y,mass);
  if (position.y > height-mass/2) {
      velocity.y *= -0.9;  // A little dampening when hitting the bottom
      position.y = height-mass/2;
      LEDbrightness=1;
      
    
    }
  else{
  LEDbrightness=0;
  }
  if(windSpeed<500){
  wind.x =-1;
  }
  else if(windSpeed>600){
    wind.x=1;
  }
  else{
    wind.x=0;
  }
  print(windSpeed);
  
  
}

Arduino

void setup() {
  Serial.begin(9600);
  pinMode(2, OUTPUT);
  // start the handshake
  while (Serial.available() <= 0) {
    Serial.println("0,0"); // send a starting message
    delay(300);            // wait 1/3 second
  }
}

void loop() {
  // wait for data from p5 before doing something
  while (Serial.available()) {
    int LEDstate = Serial.parseInt();
      
    
      digitalWrite(2, LEDstate);
      int sensor=analogRead(A0);
      
      
      Serial.println(sensor);
      
    
  }
}

Conclusion 

Although i spent a bunch of time with this project and it wasn’t perfect in the end I think I learned a lot out of it. This really helped me get coding on another level. Especially when it comes to Interactive Media. It humbled the level that i thought i was at( in a good way)

 

I decided for the final project to improve on my midterm project game, which is the pac-man game. In this post, I will discuss the role of p5.js and Arduino in my project and how the communications happen between them. Moreover, I will talk about the hardware stuff that I am using for this project.

Hardware devices

To build an interactive system that can control the pacman game in my laptop I will use an arduino Uno and a breadboard with wires. The major hardware component for my game is a joystick that I will attach to the breadboard to control the game. You can see this joystick attached below.

Furthermore, I want to add few indicators that will show the user their progress in the game. For example if the user needs 15 points to win the game then I will have three indicators and each indicator will light up if the user reaches 5 points. The three indicators will all light up together and start blinking when the user wins the game.

The role of p5.js

In my interactive system p5.js will contain all the commands needed to run my game on p5.js interface, which I have done for my midterm project. In addition to this p5.js should also read 4 different values from arduino which will be the UP position, DOWN position, LEFT position and RIGHT position. If any of these values is zero then in p5.js I will write if statements to move the pacman towards this direction. For instance, if UP is zero then pacman moves towards the up direction. In addition to this  p5.js should send the current score value to arduino so that the arduino IDE will deal with this value and light up the different indicators accordingly.

The role of Arduino Software

The arduino software will be the main bridge connecting my hardware stuff to the game on p5.js. The arduino should read 4 different values based on the joystick movements. After testing the joystick motions I figured out that the joystick returns 4 values and whenever I move it to one end, then one of the four values will return 0. Based on this observation I will be able to control the game on p5.js.

Furthermore, the arduino software will read value of score from p5.js and update the state of the indicators based on these lights.

I am very excited to complete this project but I am still a bit confused about how the physical structure for my project will look like and I hope to be able to figure out this challenging thing soon.

 

 

my idea: an imagine play interactive recipe.
current progress:

• • • • The basic structure is built
      • the communication between Arduino and p5js is done (all the information is sent back and forth as needed
      • The sensors and Arduino parts have been tested
    • An instructions page that uses all the sensors has been implemented(this is demonstrated later)

Arduino and Code

Data sent and received

Arduino would send the potentiometer, force sensor, tilt sensor, ultrasonic sensor, button, and limit switch data. The P5js would get that data and use it within the interactive recipe. p5js will also send whether the oven is on or off to the arduino so that it can turn the oven LED on.

force sensor – detect mixing motion
tilt sensor – detect pouring
ultrasonic sensor – detect person picking up ingredients
limit switch – to detect the oven door opening and closing

Testing sensors

I tested that the sensors did what I needed them to do.

I tried the force sensor and decided to use the square one but did the initial testing with the circle one because it was already soldered.

I also tested 2 tilt sensors(I’m not entirely sure what the difference between them is, but the first fit the use better fit my project in terms of it being easier to place on the prop).

then I tried to mimic how they would be used in the project. They seemed to work as intended. I implemented a simple p5js interface that displayed wether the sensor is triggered or not.

When I tried to add the ultrasonic sensor I couldn’t send the data to p5js. I checked that the ultrasonic sensor code works on it’s own and it did. I also checked that p5js was being sent a value and it was, except that value was always 0. I looked through the code line by line and realized that the pins I was using for the Ultrasonic sensor we’re being assigned as an output (but we’re never used as output, I think they we’re part of the initial code to test something but I never removed them). I removed this, and it worked.

Arduino assembly

I started with noting down the wires that need to be connected to each Arduino object so that the wire colors reflect that. (I did change a few wires later on, but those pictures were taken before that so they don’t reflect the changes)

I soldered the wires to the Arduino objects. (I used jumper wires to make the process quicker)

I 3D printed a knob to attach to the potentiometer.

I retested all the Arduino components together with the instructions p5js page.

The bread board looks a bit messy but I tried to keep all the components as neat as possible and ended up moving things around a bit later to make it neater.

also, the tilt sensor was not giving a constant value and kept moving between 1 and 0 when tilted, I realized that I wired the sensor the wrong way and forgot to add a resistor

Instructions page

I started with the instructions page because it would utilise all the Arduino functionalities which helps with the testing and would make the future pages easier to implement. The instructions page would have all the controls and allow the user to test them out to understand how to interact with the project. I thought that this way might be a better way to familiarize the user with the game controls, compared to only having text.
Here is how the screen looks like and the images change if the user interacts with the Arduino components.

building the physical structure

planning

I started to brainstorm the basic structure and decided that the main structure would be an oven with a door and a mixing bowl, a small container and a measuring cup on top. I also wanted to have a small compartment that would have the Arduino components.

I wanted to use wood for the structure. I created the design to cut the wood in and used the “finger joint” method(I believe this is what it is called based on what I found online) to make sure that the structure is strong. (the letters on the align with a sketch I did of the structure on paper and labeling a, b, c… helped me keep track of the measurements)

testing

I started with 1 cm between each comb and a 1 cm comb with a height of 0.3(the width of the material)

I tested that but it was a bit too loose(plot twist: it wasn’t, the little wiggle room is necessary). I settled for 1.03cm combs (with 1cm – 0.03/2 distance in between, not sure if this makes sense but this results in tighter fitting combs)

I also wanted to prepare the measurement for all the Arduino components that would be attached to the structure. I listed all the things that needed to be accounted for and measured all the components and their place on the structure.

All the testing was done a scrap piece of wood that was the same thickness as the wood I intend to use for the project. After I was done with the design and testing I tried to laser cut the design the wood I plan to use for the project. However, the laser cutter wasn’t cutting the wood, I realized that the wood had a different (“structure?”) than the wood I used for testing. I assumed that changing the laser cutter settings could help, but it either didn’t cut the wood, or burnt the edges to ashes.

I didn’t have more of the wood I used for testing but I did have acrylic sheets. I switched the material I was going to use(even though I reallyy wanted to used wood).
I cut the design on the acrylic but had a problem(I also changed the height of the combs to the width of the new material). Then I faced another problem, the pieces didn’t fit together, I think there was an error when I changed the width of the combs, because the acrylic fit in certain angles.

cutting and painting

I decided to redo the combs with the original measurements (1 cm between each comb and a 1cm comb). This worked!
since I was using transparent acrylic I decided to paint it since having all the walls of the structure being transparent might be confusing or too distracting for the user.
I tested out a few colors then painted the structure(I also painted the oven knob so I tested some colors for that too).


I also painted the oven door (the black part is a sticker to cover the parts I want to keep transparent.

assembly

I used super glue(and little white glue and glue gun in some parts) to stick all the parts together. There are a few structural things I would fix if I had time to make another prototype(for example I stuck the structure together in the “wrong” order, so it is slightly unaligned which affects the fit in some places.). I also used glue to add a hinge and stick some magnets to to the door. I also added magnets to the top back wall piece of the structure for easy access to the Arduino.

interface

I was not sure about how the interface would look like. I needed the task for each screen to be understandable with minimal text so that children who cannot yet read can still enjoy the game.

I decided to go with a front view, with a character that indicates what the current task is. I would also have the icons from the first instructions page to remind the child what to use for this task. Some of the pages also have  bars that indicate how long a task should be done for. I also need to consider that I have limited drawing skills, I tried to find game assets online, but didn’t find ones that I really liked. I started sketching a very basic representation of how I want each screen to look:

I tried this style but didn’t like it. I like the idea of the character wearing a Kandoora(Emirati traditional dress), but the eyes bordered on creepy and that was not the theme I was going for.

I asked my sister for input and she suggested a few things one of which is using colors without borders:

I liked this style more and I think I might stick with this style with a few adjustments.

Next steps

• • I will need to connect the arduino into the structure
  • I still need to finalize the p5js pages and find/make the graphics
  • user testing!