Author: Jun Ooi

Week 13 – Final Project – User Testing

Video

Are they able to figure it out? Where do they get confused and why? Do they understand the mapping between the controls and what happens in the experience?

Some users did not know when to start playing the Skeeball machine after interacting with the p5.js sketch.

Some users also did not know how to play the skeeball machine, and it was a little frustrating for them as they kept missing the targets.

What parts of the experience are working well? What areas could be improved?

Users loved the instant feedback from hitting a target in the skeeball machine, and in general most users loved the p5.js sketch as they liked the art style and animations. They also liked the high score list.

The ball would sometimes get stuck in the machine, so users would be confused when the ball would not return.

What parts of your project did you feel the need to explain? How could you make these areas more clear to someone that is experiencing your project for the first time?

I often had to manually remove the ball from the machine when it got stuck. I could improve this experience by making the ball return mechanism work better, but I was not able to figure out a way of fixing it 100% of the time in the 2 hours I’ve spent trying to fix this thing.

I also had to show many users how to roll the ball, as some users would try to throw the ball which is not the intended way of playing. I could improve this by adding a screen to the front of the targets, so the only way to hit the targets is by rolling the balls up the ramp.

Week 13 – Final Project – Skeeball Dungeon

Describe your concept

I wanted to make an arcade machine for my final project, and my initial idea was a pachinko machine. However, I pivoted away from that as I couldn’t think of a good way to integrate P5.js into that project, and decided to make Skeeball instead.

My project is basically skeeball, with a touch of an RPG battle concept, hence the very creative name ‘Skeeball Dungeon’. The player will have to fight 4 increasingly tougher opponents in P5.JS, and they the damage they do is equal to the amount of points they get in skeeball. For each opponent, they are given 5 balls, and they have to defeat the opponent in the number of given balls.

Include some pictures / video of your project interaction

p5.js battle! skeeball !

Description of interaction design

The p5.js sketch is used to display instructions, showing high scores, visualizing the battles and hold the graphics. The Arduino is used to obtain information from the user playing Skeeball, detecting when a target has been hit and how many points was obtained.

Description of Arduino code and include or link to full Arduino sketch

The Arduino was used to detect when targets has been hit in the Skeeball physical component. To detect when a target has been hit, a flex sensor is used for each hole. When the ball passes through the hole, the Arduino pin input that is connected through the flex sensor-voltage divider circuit will have a lower/higher reading. This change is reading is then used to detect when a target has been hit.

Description of p5.js code and embed p5.js sketch in post

https://editor.p5js.org/ojmjunming/sketches/V0ARmZA70
The p5.js code holds all of the battle logic and the screens. A state system is used to transition between each different ‘screen’ to make the code more readable. e.g the first screen, instructions and more.

if (currentScene === "highScore") {
    highScoreScreen();
  } else if (currentScene === "instructions") {
    instructionScreen();
  } else if (currentScene === "battle") {
    renderBattleScreen();
  }

The code also keeps track of the health of the monster, the current monster, the score and the number of balls. The Arduino does not know how many balls are left, it only communicates the ball hits to p5.js. The p5.js sketch also maintains a top 5 list of high scores.

Description of communication between Arduino and p5.js

The Arduino will send a message of length 2 a single time when a target is hit. Otherwise, it sends a garbage message of length 1. The message contains the amount of points that was scored with that hit. The p5.js will continuously handshake with Arduino, most of the time the data is useless (when it’s length 1), and when the length is 2, it means that a target has been hit and the message will be processed, dealing damage to the monster equivalent to the points scored.

What are some aspects of the project that you’re particularly proud of?

There were some really interesting problems that I was proud of solving:
1) Constructing the Skeeball machine.
I originally planned to use the laser cutter to build the physical component, but the laser cutter only managed to cut out one piece before it started acting up. I waited 3 hours and the laser cutter was still not working. So, I pivoted to making it with cardboard and wood pieces, using tons of hot glue but it turned out to be surprisingly stable so I’m pretty proud of the machine itself as it took me a lot of time.

2) 7 flex sensors, but only 6 analog inputs on the Arduino Uno.

I was using flex sensors to detect target hits, but I had 7 targets/holes but only 6 analog inputs. Remembering that a digitalPin reads HIGH or LOW voltages (2.5-5V, 0-2.5V), I constructed a voltage divider with 30K resistors, so the analog reading teetered on the edge of 2.4-2.6V (480-530 analogInput) depending on the flex of the flex sensor. Knowing this, I was able to use a digitalPin to read the flex of the sensor ( the other sensors were read with a analogPin, which was much easier ). When the sensor is flexed, the reading would change from HIGH to LOW.

What are some areas for future improvement?

The skeeball machine could have been constructed better, as I did not have time to paint or decorate it well. The ball also sometimes gets stuck while it’s attempting to return to the start tray and you have to manually push it out. I also had to scrap an idea I had of detecting missed ball hits, as I wasn’t able to figure out how to attach a sensor for that, so in the future it would be nice to be able to include that in my game.

Week 11 – Assignment

Video

P5.js code

let velocity;
let gravity;
let position;
let acceleration;
let wind;
let drag = 0.99;
let mass = 50;

let didTouch = 0;

function setup() {
  createCanvas(640, 360);
  noFill();
  position = createVector(width/2, 0);
  velocity = createVector(0,0);
  acceleration = createVector(0,0);
  gravity = createVector(0, 0.5*mass);
  wind = createVector(0,0);
}

function draw() {
  background(255);
  applyForce(wind);
  applyForce(gravity);
  velocity.add(acceleration);
  velocity.mult(drag);
  position.add(velocity);
  acceleration.mult(0);
  ellipse(position.x,position.y,mass,mass);
  if (position.y > height-mass/2) {
      didTouch = 1;
      velocity.y *= -0.9;  // A little dampening when hitting the bottom
      position.y = height-mass/2;
  } else {
    didTouch = 0
  }
}

function applyForce(force){
  // Newton's 2nd law: F = M * A
  // or A = F / M
  let f = p5.Vector.div(force, mass);
  acceleration.add(f);
}

function keyPressed(){
  if (keyCode==LEFT_ARROW){
    wind.x=-1;
  }
  if (keyCode==RIGHT_ARROW){
    wind.x=1;
  }
  if (key==' '){
    mass=random(15,80);
    position.y=-mass;
    position.x = width / 2
    velocity.mult(0);
  }
  
  if (key == 's') {
    setUpSerial();
  }
}

// This function will be called by the web-serial library
// with each new *line* of data. The serial library reads
// the data until the newline and then gives it to us through
// this callback function
function readSerial(data) {
  ////////////////////////////////////
  //READ FROM ARDUINO HERE
  ////////////////////////////////////

  if (data != null) {
    // make sure there is actually a message
    // split the message
    let fromArduino = split(trim(data), ",");
    // if the right length, then proceed
    if (fromArduino.length == 1) {
      // only store values here
      // do everything with those values in the main draw loop
      
      // We take the string we get from Arduino and explicitly
      // convert it to a number by using int()
      // e.g. "103" becomes 103
      windValue = int(fromArduino[0]);
      wind = createVector(map(windValue, 0, 1023, -2, 2), 0)
    }

    //////////////////////////////////
    //SEND TO ARDUINO HERE (handshake)
    //////////////////////////////////
    let sendToArduino = didTouch + "\n";
    print(sendToArduino)
    writeSerial(sendToArduino);
  }
}

Arduino code

// Week 11.2 Example of bidirectional serial communication

// Inputs:
// - A0 - sensor connected as voltage divider (e.g. potentiometer or light sensor)
// - A1 - sensor connected as voltage divider 
//
// Outputs:
// - 2 - LED
// - 5 - LED

int ledPin = 5;
int potPin = A2;

void setup() {
  // Start serial communication so we can send data
  // over the USB connection to our p5js sketch
  Serial.begin(9600);

  // We'll use the builtin LED as a status output.
  // We can't use the serial monitor since the serial connection is
  // used to communicate to p5js and only one application on the computer
  // can use a serial port at once.
  pinMode(LED_BUILTIN, OUTPUT);

  // Outputs on these pins
  pinMode(ledPin, OUTPUT);

  // Blink them so we can check the wiring
  digitalWrite(ledPin, HIGH);
  delay(200);
  digitalWrite(ledPin, LOW);



  // start the handshake
  while (Serial.available() <= 0) {
    digitalWrite(LED_BUILTIN, HIGH); // on/blink while waiting for serial data
    Serial.println("0"); // send a starting message
    delay(300);            // wait 1/3 second
    digitalWrite(LED_BUILTIN, LOW);
    delay(50);
  }
}

void loop() {
  // wait for data from p5 before doing something
  while (Serial.available()) {
    digitalWrite(LED_BUILTIN, HIGH); // led on while receiving data

    int ledLight = Serial.parseInt();
    if (Serial.read() == '\n') {
      digitalWrite(ledPin, ledLight);
      int sensor = analogRead(potPin);
      delay(5);
      Serial.println(sensor);
    }
  }
  digitalWrite(LED_BUILTIN, LOW);
}

 

Week 11 – Final project idea

I had two ideas for my final project but I’m still undecided on what I want to do

1. Pachinko machine 

Creating a pachinko machine that incorporates Arduino to add interaction, perhaps using servo motors to control some obstacles, or adding sound/lights whenever the ball hits a nail, or using Arduino to detect which hole the ball fell into. However, I can’t think of a good way of integrating P5.JS with this project without it being gimmicky like showing the score on P5.js, or shooting the balls from P5.js. I feel those interactions belong in the physical world and implementing them digitally takes away from it. Any interaction I can think of is better served physically, so I am not sure if I will proceed with this idea even though I personally like this idea more as it’s more fun.
Christmas Gift [Handmade DIY] Pinball Game Board Game Wooden Toy Handmade DIY - Shop jwoodgarden Wood, Bamboo & Paper - Pinkoi

2. Hello Kitty photobooth

A photobooth-type project, where the p5.js sketch is responsible for displaying the camera and taking the photos. The ‘twist’ is the usage of Arduino and the presence of a physical Hello Kitty that is sensed by the Arduino. By moving around the physical Hello Kitty it adjusts her location on the screen, so you can move her around and adjust her size, make her big so the viewer can stand next to her in the photo, or make her small and the viewer can hold her in the palm of their hands.

Week 11 – Reading

The debate between functionality and design has always been an interesting one, and though sometimes it feels like you can only pick one without compromising both, it is very much possible to accomplish both as what Apple did with the iPod. Though most engineers might argue for the case of functionality, I think one very beautiful part of the shared human experience is that humans appreciate beautiful things like a sunset or a well-designed product. Donald Norman has a teapot that is functionally unusable, yet has a place on his countertop because he likes how it looks. Perhaps not everyone is as invested in design as Donald Norman is, that they would give up all functionality for design, but a case can be made for ‘functional enough’ before people start looking at design. Ten different tote bags are functionally all the same, but there might be one specific tote that a person might gravitate to more often. A tote bag with a hole might not be used because it does not perform it’s function well even if it is well-designed, but other than that, the tote bag that is chosen is the one that suits the dress code the best.

In the case of disability, it is a little strange that design for disability has always been about ‘catching up to a normal human’, and not augmenting them to beyond the capabilities of an average person. Why are leg prosthetics designed to look like bad imitations of flesh, when it could be used as a fashion accessory that able-bodied people are not able to? Why not design hearing aids that can enable superhearing? An able-bodied person is not the standard, and it is always an option to design beyond that, to make disability less of a stigma and instead become something that opens up new possibilities instead of reducing them. Prosthetic covers: Interview with Emelie Strömshed from Anatomic Studios - theactiveamputee

Week 10 – Musical Instrument

Concept

I’ve been trying to learn the piano for the past year, so when we had to make a musical instrument I definitely knew I wanted to make something tangentially related to the piano.

Another thing I wanted to do with this week’s assignment was to add more ‘design wrapping’ to it. I liked what the professor said about Week 9’s projects — sometimes a well-wrapped simple project is much nicer than a messy technically impressive project, so before even designing what I want to do, I thought about the presence of materials and how I can use them to enhance the project.

I thought of something like Nintendo’s cardboard piano, and it sounded like it would be fun to implement. However, I looked at the documentation for Arduino Tones and learned that only 1 tone can be played at a time, even with multiple piezo buzzers/ I didn’t like the idea of creating a piano that can only play one note at a time, plus I thought this idea may have been overdone: creating a piano as a musical instrument for arduino.

Nintendo's cardboard piano for Switch is the most exciting gear of 2018

I still opted to create something piano-like, but instead of using touch as input, I decided to use the ultrasonic detector which makes it a theremin. This makes use of the 1-tone at a time limit as a feature rather than a bug, however during implementation I realized the ultrasonic detector isn’t very precise, so I wasn’t able to add the black keys to the piano without sacrificing accuracy. One more problem that came to mind when using the ultrasonic detector to determine which note to play was: “how often do I play the note”? To address this, I added a potentiometer which sets the tempo of the notes, so the theremin can play 1 note every 400ms-1000ms depending on the position of the position of the potentiometer. I realized I should also add some visual feedback to this as it might otherwise be difficult to set the desired value, so I added a blinking LED that blinks at the same rate the theremin is playing. If the LED is blinking at the same pace you want the song to be played, then you’re good!

Video

Week 10 – Reading Reflection

Picture under Glass requires more than just your hands — it requires your eyes too. You can’t navigate Picture under Glass without looking at where your fingers are as there is no touch feedback from the screen. With analog buttons, you can ‘feel’ your way around an interface, for example modern cars only have a mounted tablet as the car entertainment system, and sometimes when I’m driving and I want to skip the song, there is no safe way for me to do so without having to momentary take my eyes off the road to look at the tablet to see where the ‘>’ next song button is. In older cars with analog buttons, I don’t have to look at it to know where the button is, I can just remember that the next song button is the second from the right, and with one hand on the wheel and eyes on the road, I can interact with the car entertainment system.

I also found it interesting that in his follow up he brought up this hypothetical scenario of “Monet saying to his canvas, “Give me some water lilies. Make ’em impressionistic.”. Today with the existence of generative AI and advanced natural language processing units, it seems that the world of user interfaces has doubled down on the direction of Picture under Glass, further reducing the number of middlemen between the intent of the brain, and the wanted outcome. What I mean by this is that hands/physical movements are in some parts, a means to an end. To open a jar, we use our hands to open it because we can’t simply use our brain to open it. People spend years in art school to learn how to draw well, but generative art can accomplish what would take an untrained artist years of training to achieve in a singular sentence and a few minutes. ChatGPT further reduces the effort needed by humans to interface with the world — in the past, one would have to visit a library to read about a specific subject. Then came the internet, and using devices connected to the internet, one can search Google for the information they need. Now with ChatGPT, you can just simply ask it about whatever you need. The effort needed to accomplish anything is reduced, and I’m not sure if this is a good or bad thing. On this, the author says that “if they [brain interfaces] bypass the body, then we’ve just created a future where people can and will spend their lives completely immobile. Why do you want this future? Why would this be a good thing?”

Week 9 – Reading Response

I like the guidebook on commonly-made physical computing projects in class. I’ve thought about what it must be like to be a professor for an intro project-based class, like Intro to CS or Intro to IM, as there must be ideas that are constantly recycled each semester and the professor will have to pretend that they’re novel even when they’ve seen it a million times before. I assume a platformer game will always show up in Processing/P5.js class. Perhaps a paint/drawing program too, a particle system/ pattern animation, recreations of classic games like Pong or Snake. That’s not to say that they’re boring, as the reading also talks about how the cliches can sometimes be used as a base for something more interesting, but it’s fun to think about ideas that are strangely common.

Despite them being constantly recycled, it is possible to make interesting, unique projects out of them. I can think of a visual novel-esque game being one that is likely technically similar across most games of it’s genre, yet the deciding factor  between a good and bad visual novel is the story being told, not the  implementation of ‘click to see the next line of text’. Perhaps one of the best ways of being creative is to mix-up an existing overused form of media instead of trying to be both novel and interesting at the same time, as then you might be limited by your technical capabilities rather than your creative ones. In the case of the reading, video mirrors have been done. A Lot. Yet, this means that it is relatively easy to implement one thanks to the wealth of documentation and existing implementations on the internet, and you can focus instead on the ‘what’ you want to convey through your project, instead of the ‘how’.

The other reading compliments Don Norman’s chapters from the past few weeks really well. Show, don’t tell, in your projects. Students learn better when they make the intuition between two objects, rather than memorizing the two objects. Likewise, I assume that viewers of an art piece will enjoy it more when they come to their own conclusions about it, than being told what to feel. Of course, artists are always trying to convey something, and to have every viewer of their piece think wildly differently from their intent is probably not the artist’s intention. Through the design process itself, artists can guide viewers to their intent, to gently nudge them in the right direction without having to hold their hand. I like to think about Escape Rooms in this example. The goal of an Escape Room is to escape from it, yet players will find it unfun if the lock and key are both on the same table as they might feel it is too easy. Yet, if the puzzle is too difficult, players will feel frustrated and will not enjoy it either. The designers of Escape Rooms must strike a delicate balance between making a puzzle that is not easily solvable, yet guiding users delicately to their goal. For example, the usage of a padlock vs a combination lock will intuitively guide the players on what they should look for ( a key or a code ).

Week 9 – Simon Says on Arduino

Concept
With what we currently know about Arduino, there were many things I am technically incapable of making yet, such as using the servo/motors or the LCD screen. Given the lax requirements for this week’s project of having 1 analog and 1 digital button, and 2 LEDs, I wasn’t sure what I could do with it what I currently have ( momentary switches, light detecting diodes ) while still being creative, so I thought of making a game of sorts and I came up with a memory game with the buttons.

Implementation
The light detecting diode ( analog ) is used to start/restart the game. When the LDR is covered and it becomes dark, the game is started. The momentary buttons are used to play the game.

Once the game is started, the LEDs will blink in sequence, and you have to replicate the sequence with the aptly-colored buttons to light up the same LEDs in the same order. There are multiple rounds, starting with 1 LED, all the way up to 8 LEDs. The sequence will blink again at the start of each round, with the addition of a new LED in the sequence. The player will have to rely on their memory to press the 8 LEDs in the right order.

The LEDs also double both as the game mechanism, and information mechanism. I use the LEDs to display game state information too. When the user has lost ( pressed an LED in the wrong order) , the LEDs will all blink twice before turning off. When the user has won, the LEDs will blink in a fun pattern on repeat until a new game is started.

Video demo : In the first attempt, I made a mistake and the ‘loss sequence’ plays. The second attempt, I complete the challenge and the ‘victory sequence’ plays.

Expansion

It would be fun to use other forms of inputs other than buttons for this game! I was thinking of using the ultrasonic detector as a challenge too, e.g player has to remember the distance they have to trigger the ultrasonic detector. Perhaps once I learn about more sensors and inputs, I could think about implementing them into this game so there’s more variety rather than just a sequence of LEDs.

Week 8 – Unusual Switch – Wi-Fi switch

Preface: I realize that my idea ended up being more of a tech demo than a cool idea. Sorry!

Concept


My project was inspired off this tweet I saw. They were using a Wemos D1 Mini microchip which has a Wi-Fi module on it and can act as an access point ( other devices can connect to it ), and using the Wi-Fi capture portal to display a message/article/book/image/video ( up to 4MB ) to anyone who connects to it. I thought it was fascinating so I ordered the chip from Amazon and was playing with it, when I thought I could rig it up to the Arduino and use it as an unusual switch.

It ended up being more of a technical exploration than a artistic exploration, as I found that using microchips outside of the Arduino UNO we were provided was a little more challenging as not everything is available out of the box.

Implementation

 

Video Demo

My implementation was pretty simple — when you connect to the network, it will toggle the state of the LED between on and off, and the webpage that you see reflects that. There could have been more interesting applications of the switch, for example adding a Wi-Fi password so that the switch is only usable by certain users, or adding some HTML inputs to the webpage so that the LED is controllable from the webpage without having to disconnect/reconnect. Even if you’re in a receptionless area, this switch will still work ( as the chip is hosting it’s own Wi-Fi network, you do not need a network connection to use it ).

Reflections

Will try to do less tech-demo-ey projects for my future physical computing projects sorry!!!