Final Project Documentation

Concept:
My final project is a bomb defusal game inspired by Keep Talking and Nobody Explodes. Just like in the original game, the player has to disarm several modules on the bomb in order to successfully defuse it. Currently it includes three types of modules: The first is Simon Says, using four LED arcade buttons. The second is an adaptation of cutting wires, where the player will have to either disconnect or rearrange the wires correctly. The last module requires the user to use a potentiometer as a tuning knob and try to hone in on the correct frequency. Once all three modules are disarmed, the bomb is defused and the game ends.Image, Video

 

Implementation:

Arduino:

  • Reads button presses, potentiometer values, and wire states.
  • Blinks arcade button LEDs to display the current Simon Says sequence, and activates the green LED on each module to indicate that they have been disarmed. 
  • Uses a piezo buzzer to audibly indicate how much time remains, which helps add tension.
  • The UNO and two breadboards are contained inside the cardboard shell, and the inputs are mounted on top.

p5.js:

  • Renders a representation of the physical bomb, including module status.
  • Displays the countdown timer since the LCD screen from the kit was not used.
  • Handles initializing each new round, including options for difficulty.
  • Randomly generates the solution for each module (e.g. color sequence for Simon Says, sweet spot for potentiometer).

 

Interaction Design:

The player is presented with three modules, which can be independently disarmed in any order. As the modules are disarmed, green status LEDs light up to indicate that the user has succeeded and can move on. Once all three modules are disarmed, p5.js will halt the timer and display a win screen. If the player fails to defuse the bomb in time, they will instead see a loss screen.

  • Simon Says: Flashes a sequence of colors on the arcade buttons, increasing in length with each successful input. If the player makes an incorrect input or fails to respond within a set amount of time, the sequence will repeat. The length of the sequence is determined by the difficulty selected.
  • Tuning: A value within the potentiometer’s viable range is randomly chosen. The player moves the knob, and once it comes within a certain range of the target value it begins a short countdown while displaying a progress bar. Both the current and target values are visualized using the sin function. The leniency range is also determined by difficulty.
  • Wires: The player must figure out the correct sequence to connect the four wires. They are not penalized for attempts in this adaptation, so they are free to use trial-and-error. A rendered visual helps guide them towards the correct configuration.

 

Schematic:

 

Arduino Code:

The Arduino code is fairly straightforward. It has a few functions used in the main loop to send/receive control messages, check the relevant inputs, and handle timing for the audiovisual components.

/*
Final Project (WIP)
By Matthias Kebede
*/





// // // Global Variables
const int valueCount = 9;
int active = 0;

// // Inputs
const int potPin = A4;
const int wirePins[4] = {A0, A1, A2, A3};
const int blueButtonIn = 6;
const int redButtonIn = 7;
const int yellowButtonIn = 8;
const int greenButtonIn = 9;

// // Input stuff
int inputs[valueCount] = {potPin, wirePins[0], wirePins[1], wirePins[2], wirePins[3], blueButtonIn, redButtonIn, yellowButtonIn, greenButtonIn};
int inputVals[valueCount] = {-1, -1, -1, -1, -1, -1, -1, -1, -1};
float smoothVals[5] = {0, 0, 0, 0, 0};
char headers[valueCount][13] = {
  {"TUNE:POT"}, {"WIRES:W1"}, {"WIRES:W2"}, {"WIRES:W3"}, {"WIRES:W4"},
  {"SIMON:BLUE"}, {"SIMON:RED"}, {"SIMON:YELLOW"}, {"SIMON:GREEN"}
};

// // Outputs
const int speakerPin = A5;
const int blueButtonOut = 2;
const int redButtonOut = 3;
const int yellowButtonOut = 4;
const int greenButtonOut = 5;
const int simonLED = 10;
const int wiresLED = 11;
const int tuneLED = 12;

// // Output Information
const int beepFreq = 2000;   // hz
const int beepDur = 50;   // ms
int beepInterval = 500;   // ms
int lastBeepTime = 0;   // ms
const int simonBlink = 350;   // ms

// // Misc.
// Keep time for Simon Says lights
struct Blink {
  int pin;
  bool lit;
  long offTime;
};
Blink simonLights[] = {
  {blueButtonOut, false, 0},
  {redButtonOut, false, 0},
  {yellowButtonOut, false, 0},
  {greenButtonOut, false, 0}
};
// Wire thresholds
const int TH0 = (1000 + 928) / 2;   // 964
const int TH1 = (928  + 512) / 2;   // 720
const int TH2 = (512  +  92) / 2;   // 302
// For analog smoothing
const float alpha = 0.2;
const int potDelta = 4;





// // // Main Processes
void setup() {
  Serial.begin(9600);

  // // Inputs and Outputs
  pinMode(potPin, INPUT);
  pinMode(blueButtonIn, INPUT);
  pinMode(redButtonIn, INPUT);
  pinMode(yellowButtonIn, INPUT);
  pinMode(greenButtonIn, INPUT);
  for (int i = 0; i < 4; i++) {
    pinMode(wirePins[i], INPUT);
  }
  pinMode(LED_BUILTIN, OUTPUT);
  pinMode(speakerPin, OUTPUT);
  pinMode(blueButtonOut, OUTPUT);
  pinMode(redButtonOut, OUTPUT);
  pinMode(yellowButtonOut, OUTPUT);
  pinMode(greenButtonOut, OUTPUT);
  pinMode(simonLED, OUTPUT);
  pinMode(wiresLED, OUTPUT);
  pinMode(tuneLED, OUTPUT);

  // // Check built-in LED
  digitalWrite(LED_BUILTIN, HIGH);
  delay(200);
  digitalWrite(LED_BUILTIN, LOW);

  // // Temp check
  digitalWrite(blueButtonOut, HIGH);
  digitalWrite(redButtonOut, HIGH);
  digitalWrite(yellowButtonOut, HIGH);
  digitalWrite(greenButtonOut, HIGH);
  delay(200);
  digitalWrite(blueButtonOut, LOW);
  digitalWrite(redButtonOut, LOW);
  digitalWrite(yellowButtonOut, LOW);
  digitalWrite(greenButtonOut, LOW);

  // // Start handshake w/ p5.js
  while (Serial.available() <= 0) {
    digitalWrite(LED_BUILTIN, HIGH);
    Serial.println("Waiting for data..."); // identifiable starting number
    delay(300);
    digitalWrite(LED_BUILTIN, LOW);
    delay(50);
  }
}

void loop() {
  // // Wait for p5.js
  while (Serial.available()) {
    digitalWrite(LED_BUILTIN, HIGH);

    String target = Serial.readStringUntil('=');
    int value = Serial.parseInt();
    if (Serial.read() == '\n') {
      writeTarget(target, value);
    }

    digitalWrite(LED_BUILTIN, LOW);
  }

  // // Send data to p5.js
  for (int i = 0; i < valueCount; i++) {
    checkValue(i);
    delay(1);
  }

  // // Clear Simon Says lights
  clearSimon();

  // // Play beeps
  if (active) timerSound();

  // // // Temp read wires
  // Serial.print(analogRead(A0)); Serial.print(',');
  // Serial.print(analogRead(A1)); Serial.print(',');
  // Serial.print(analogRead(A2)); Serial.print(',');
  // Serial.println(analogRead(A3));
}





// // // Helper Functions
// // Check current input values and compare to last known value
void checkValue(int index) {
  // // Check value // Wires: 100=1000, 1k=928, 100k=91, 10k=512   <-- W1, W2, W3, W4
  int checking;
  if (index < 1) { // < 5
    // // Add delay and smoothing for analog reads
    delay(1);
    checking = analogRead(inputs[index]);
    smoothVals[index] = alpha * checking + (1 - alpha) * smoothVals[index];
    checking = int(smoothVals[index]);
    // // Check if pot has significant change
    if (abs(checking - inputVals[index]) >= potDelta) {
      inputVals[index] = checking;
      Serial.print(headers[index]);
      Serial.print('=');
      // // Send pot value
      if (index == 0) {
        Serial.println(checking);
      }
      // // Send index of wire connection
      // else {
      //   Serial.println(identifyWire(checking));
      // }
    }
  }
  // else if (index < 5) {
  //   delay(1);
  //   checking = analogRead(inputs[index]);
  //   smoothVals[index] = alpha * checking + (1 - alpha) * smoothVals[index];
  //   checking = int(smoothVals[index]);
  //   int binaryVal = digitalWire(checking);
  //   if (abs(checking - inputVals[index]) >= potDelta && binaryVal != inputVals[index]) {
  //     inputVals[index] = binaryVal;
  //     Serial.print(headers[index]);
  //     Serial.print('=');
  //     Serial.println(binaryVal);
  //   }
  // }
  else {
    checking = digitalRead(inputs[index]);
    // // Compare
    if (checking != inputVals[index]) {
      inputVals[index] = checking;
      Serial.print(headers[index]);
      Serial.print('=');
      Serial.println(checking);
    }
  }
}

// // Handle writing to the target pin
void writeTarget(String target, int value) {
  if (target == "ACTIVE") {
    active = value;
  }
  else if (target == "BUILTIN") {
    digitalWrite(LED_BUILTIN, value);
    delay(150);
    digitalWrite(LED_BUILTIN, LOW);
  }
  // // Change beep interval based on p5.js timer
  else if (target == "BEEP") {
    beepInterval = value;
  }
  // // Simon Says
  else if (target == "SIMON") {
    digitalWrite(simonLED, value); // // Simon Says = defused
  }
  else if (target == "BLUE") {
    flashSimon(blueButtonOut);
  }
  else if (target == "RED") {
    flashSimon(redButtonOut);
  }
  else if (target == "YELLOW") {
    flashSimon(yellowButtonOut);
  }
  else if (target == "GREEN") {
    flashSimon(greenButtonOut);
  }
  // // Wires
  else if (target == "WIRES") {
    digitalWrite(wiresLED, value);
  }
  // // Tune
  else if (target == "TUNE") {
    digitalWrite(tuneLED, value);
  }
}

// // Play beeping noise
void timerSound() {
  if (lastBeepTime > beepInterval) {
    // // Reset
    lastBeepTime = 0;
    noTone(speakerPin);
    // // Play
    tone(speakerPin, beepFreq, beepDur);
  }
  else {
    lastBeepTime++;
  }
}

// // Non-blocking flash for Simon Says
void flashSimon(int pin) {
  for (auto &btn : simonLights) {
    if (btn.pin == pin) {
      digitalWrite(pin, HIGH);
      btn.lit  = true;
      btn.offTime = millis() + simonBlink;
      break;
    }
  }
}
void clearSimon() {
  long now = millis();
  for (auto &btn : simonLights) {
    if (btn.lit && now >= btn.offTime) {
      digitalWrite(btn.pin, LOW);
      btn.lit = false;
    }
  }
}

// // // Determine wire connections
// int identifyWire(int val) {
//   // if (val < 25) return -1;   // unplugged or other issue
//   if (val > TH0) return 0;   // 100 ohm
//   else if (val > TH1) return 1;   // 1k ohm
//   else if (val > TH2) return 3;   // 10k ohm
//   else return 2;   // 100k ohm // remember bottom-up order is 100, 1k, 100k, 10k
// }
// int digitalWire(int val) { // 92, 512, 928, 1000
//   if (val < 50) return 0;
//   if (val <  110 && val > 70) return 1;
//   if (val < 530 && val > 480) return 1;
//   if (val < 950 && val > 905) return 1;
//   if (val < 1024 && val > 975) return 1;
//   return 0;
// }

 

p5.js Code:

p5.js handles the actual game logic, and generates unique solutions for the modules every time a game starts. The Game class contains an array of modules, and continuously calls their update methods. Each module has its own child class extending the Module class, and contains the code for its own specific mechanics. The user can select a difficulty level from the main menu, and start a game.

 

Serial Communication:

The protocol used here follows the basic idea from the in-class examples, reading up until it reaches a newline character. In order to avoid sending the state of every input device to p5.js with every message, I broke things down into messages of the format `HEADER=value`. I mainly used this to indicate which module was sending input data, and combined it with a switch statement to separate things out on the p5.js side. In terms of transmitting to Arduino, I followed a similar idea but only had to send messages to disarm modules (e.g. `SIMON=0`) or defuse the bomb itself to stop the beeping. I also used this to have p5.js increase the frequency of the beeping when its timer reached certain breakpoints.

 

What I’m Proud Of:

I was happy with a number of decisions I made. For one, I was able to cleanly use OOP to separate out the p5.js logic for my modules while including overlapping functionality like disarming. I was also proud of the protocol I came up with for serial communication. It gave me a lot of trouble at first, so it was very fulfilling to end up with a lightweight method where I could direct messages exactly where they needed to go. Lastly, I was proud of my Simon Says module in particular. I spent a lot of time on it early on since it was the first to be implemented, and I feel like it came out the best. I had to figure out how the arcade buttons work and soldered all the necessary wires, but it was worth it since it is probably the most engaging module.

 

Areas for Improvement:

In contrast to Simon Says, I was really disappointed by my Wires module. It was originally the one I was most excited about, since this gave me a chance to actualize the experience of the inspiring game in a unique way. However, I tried a number of different ways to implement it that all failed in the end. My first attempt was to use resistors of different values and use analogRead() to determine which wire was plugged in where. However, the floating values when the wires were unplugged threw things off too much.

Another area for improvement would be the design of the bomb. Using cardboard turned out just fine, but laser cutting a wooden box might have looked more professional. I put a lot of time and effort into the initial construction, especially since I cut the cardboard down by hand, but after that I became far too busy to add any decoration and finishing touches. The p5.js visuals also suffered a bit for the same reason. There was one choice I made that I’m still on the fence about, which was to omit an explosion sound when the player loses a round. It would have been a nice touch, but I already drove myself crazy listening to the beeping sound, and I felt that having the beep frequency pick up speed was sufficient by itself.

Week 12 – Final Project Proposal

My final project is a bomb defusal game inspired by Keep Talking and Nobody Explodes. Just like in the original game, the player has to disarm several modules on the bomb in order to successfully defuse it. Currently I plan to include four types of modules. The first is Simon Says, using four buttons and corresponding LEDs. The second is a single larger button connected to a multicolored LED, which will be disarmed differently depending on the color. The third is an adaptation of cutting wires, where the player will have to either disconnect or rearrange the wires correctly. The last module requires the user to use a potentiometer as a tuning knob and try to hone in on the correct frequency.

The Arduino will be responsible for reading the inputs from each module, and correctly lighting the LEDs where needed. I also intend to use the display screen from our kits to show the bomb’s timer if possible, and use a buzzer/speaker to create a beeping noise. The p5.js side will be responsible for initializing a new game and managing its state. It will receive the inputs from Arduino and verify them, and send back the confirmation if a module was completed. I also want to use it to render a representation of the bomb, including the correct LED lights and countdown timer. In line with the original game, it can also work to provide the information needed to disarm the more complicated modules. In terms of interaction, p5.js will be used to start the game and will display a win/loss screen after it ends.

I have started writing some basic code for both components:

/*
Final Project (WIP)
By Matthias Kebede
*/





// // // Global Variables
// // Inputs
const int tempIn = A1;

// // Outputs
const int tempOut = 8;
const int speakerPin = 10;

// // Communication and State Information
int sendInterval = 100;   # ms
int lastSendTime = 0;   # ms
int timer = [5, 0];   # minutes, seconds
int beepFreq = 1000;   # hz
int beepDur = 50;   # ms
int beepInterval = 100;   # ms
int lastBeepTime = 0;





// // // Main Processes
void setup() {
  Serial.begin(9600);

  // // Inputs and Outputs
  pinMode(tempIn, INPUT);
  pinMode(LED_BUILTIN, OUTPUT);
  pinMode(tempOut, OUTPUT);

  // // Check built-in LED
  digitalWrite(LED_BUILTIN, HIGH);
  delay(200);
  digitalWrite(LED_BUILTIN, LOW);

  // // Start handshake w/ p5.js
  while (Serial.available() <= 0) {
    digitalWrite(LED_BUILTIN, HIGH);
    Serial.println("123"); // identifiable starting number
    delay(300);
    digitalWrite(LED_BUILTIN, LOW);
    delay(50);
  }
}

void loop() {
  // // Wait for p5.js
  while (Serial.available()) {
    digitalWrite(LED_BUILTIN, HIGH);

    int firstVal = Serial.parseInt();
    if (Serial.read() == '\n') {
      analogWrite(tempOut, firstVal);
    }

    if (lastSendTime > sendInterval) {
      lastSendTime = 0;
      int sendVal = analogRead(tempIn);
      int mappedVal = map(sendVal, low, high, 0, 255);
      Serial.println(sendVal);
      delay(1); // stabilize ADC? check this
    } 
    else {
      lastSendTime++;
    }

    digitalWrite(LED_BUILTIN, LOW);
  }
}





// // // Helper Functions
void timerSound() {
  if (lastBeepTime > beepInterval) {
    // // Less than 30 seconds remaining
    if (timer[0] == 0 && timer[1] < 30) {
      tone(speakerPin, beepFreq + 500, beepDur);
    }
    // // Final minute but more than 30 seconds
    else if (timer[0] == 0) {
      tone(speakerPin, beepFreq + 250, beepDur + 25);
    }
    else {
      tone(speakerPin, beepFreq, beepDur + 50);
    }
  }
  else {
    lastBeepTime++;
  }
}
/*
Final Project
By Matthias Kebede
*/


// // // Global Variables
let debug = true;
let gameState = 'menu'; // menu, playing, win, lose
let game;
// // For display
let play, menuTitle;




// // // Main Processes
function preload() {
  
}

function setup() {
  createCanvas(600, 450);
  createMenu();
}

function draw() {
  background(220);
  
  switch (gameState) {
    case 'menu':
      showMenu();
      break;
    case 'playing':
      game.display();
      break;
    default:
      break;
      
  }
}



// // // Classes
class Game {
  constructor() {
    this.gameover = false;
    this.start = millis();
    this.time = 30;
    this.modules = [];
  }
  display() {
    text(`Time: ${Math.floor(this.time)}`, width/2, height/2);
    if (!this.gameover) {
      this.update();
    }
    else {
      text("Gameover", width/2, height*0.2);
    }
  }
  update() {
    for (let module in modules) {
      module.checkStatus();
    }
    // // Update timer
    if (this.time > 0 + 1/frameRate()) {
      this.time -= 1/frameRate();
    }
    else {
      this.time = 0;
      this.gameover = true;
    }
  }
}

class Module {
  constructor() {}
  checkStatus() {}
}



// // // Interaction
function mouseClicked() {}



// // // Displays
function createMenu() {
  // // Menu Title
  menuTitle = createElement('h1', "Main Menu");
  menuTitle.size(9*18, 100);
  menuTitle.position((width-menuTitle.width)/2, height*0.2);
  // // Play Button
  play = createButton("Play");
  play.size(width/5, height/15);
  play.position((width-play.width)/2, height/2);
  play.mousePressed(function() {
    game = new Game();
    gameState = 'playing';
    removeElements();
  });
}
function showMenu() {}


// // // Helper Functions


Week 11 – Reading Response

This week’s reading was very eye-opening, and addressed a number of design aspects that I had never really considered. The title Design Meets Disability left me with a preconception of what that entailed, which was quickly proven wrong. I’ve worn glasses since I was nine years old, so seeing one of the first sections focus on them came as a surprise to me. I am obviously aware of things like contacts and non-prescription lenses, but it was fascinating to look at the shift from medical spectacles to ‘eyeware’ and see how the perception completely changed. The hearing aid section served as a contrast to that change, where the emphasis is on hiding them from view. There have been a number of times where it took me years of knowing someone to ever notice that they wear hearing aids, which speaks to how far the technology has come. Another tidbit was the mention of using pink plastic to try and hide wearable devices like those, which stood out to me at the very beginning of the text and was addressed again here.

I found it harder to identify with the later portions on prosthetic limbs, mostly due to lack of exposure, but I also feel like it is harder to think about in general. Glasses and hearing aids just focus the ambient light/sound to fit your needs, but trying to design a fully-functional arm seems much more complex. The aesthetic side of things is also a bit strange. Instead of being worn on your head and framing your face, they are harder to work with in that they are simply much larger, but can also be obscured by clothing much more easily. The battle between functionality and aesthetic also becomes much more important, with some prosthetics aiming to look as realistic as possible versus using inorganic structures like hooks to maximize their usefulness. From there you can even ask whether we should allow prosthetics to be constrained by the rest of the body, or to instead push into the realm of transhumanism.

Week 11 – Final Project Concept

Concept:

While thinking about how to create something physically interactive, I was reminded of Keep Talking and Nobody Explodes. Player 1 is placed in a bomb-defusal scenario, while Player 2 is given a manual on how to defuse the bomb. Each bomb has different modules on it, and P1 has to describe them to P2. P2 will then instruct P1 on how to clear that module, and eventually defuse the bomb entirely. While the game is quite entertaining on a mouse and keyboard or in VR, it still isn’t the same as a physically interactive experience. Many of the modules involve pressing buttons or cutting wires, so I thought it would be perfect to adapt for an Arduino/breadboard setup.

Arduino:

The Arduino would be responsible for reading the physical inputs and passing the relevant data to p5.js for interpretation. For example, the user might have to press buttons in sequence or physically disconnect a wire in order to disarm a module. It could also have some outputs in the form of LEDs to indicate the status of each module (red for active, green for disarmed) and maybe use the LCD screen to display the serial number or a countdown.

P5.js:

The p5.js side would be responsible for managing the game state and interpreting the player’s inputs. The physical component would have to be static for the most part, but p5.js could maybe help with randomizing the behaviour of the modules for each new game. I’m not entirely sure on how to adapt the cooperative aspect of the original game, but one way could be to display the defusal instructions on the laptop screen and allow the player to interact with it there. Other display-related components could also go on this side, such as displaying a countdown and playing sound queues, or maybe having a representation of the physical ‘bomb’ and indicating the status of each module.

Week 11 – Exercises

1: Ellipse

/*
 * Week 11 Production (1)
 *
 * Inputs:
 *   - A1 - 10k potentiometer connected to 5V and GND
 *
 */

int interval = 100;
int lastMessageTime = 0;

int potPin = A1;

void setup() {
  Serial.begin(9600); // initialize serial communications
}
 
void loop() {
  // read the input pin:
  int potentiometer = analogRead(potPin);                  
  // remap the pot value to 0-255:
  int mappedPotValue = map(potentiometer, 0, 1023, 0, 255); 
  // print the value to the serial port.
  Serial.println(mappedPotValue);
  // slight delay to stabilize the ADC:
  delay(1);                                            
  
  delay(100);
}
let port;
let connectBtn;
let baudrate = 9600;
let lastMessage = "";
let currX;

function setup() {
  createCanvas(400, 400);
  background(220);

  port = createSerial();

  let usedPorts = usedSerialPorts();
  if (usedPorts.length > 0) {
    port.open(usedPorts[0], baudrate);
  }

  connectBtn = createButton("Connect to Arduino");
  connectBtn.position(80, height-60);
  connectBtn.mousePressed(connectBtnClick);

  currX = width/2;
}

function draw() {
  background("white");
  fill('grey');
  circle(currX, height/2, 100);
  
  let str = port.readUntil("\n");
  if (str.length > 0) {
    // console.log(str);
    lastMessage = str;
  }
  
  // Display the most recent message
  text("Last message: " + lastMessage, 10, height - 20);

  // change button label based on connection status
  if (!port.opened()) {
    connectBtn.html("Connect to Arduino");
  } else {
    connectBtn.html("Disconnect");
  }
  
  // // Move shape based on received value
  if (!lastMessage) {lastMessage = "127"}
  currX = map(int(lastMessage), 0, 255, 0, width);
  currX = floor(currX);
  // console.log(currX);
}

function connectBtnClick() {
  if (!port.opened()) {
    port.open("Arduino", baudrate);
  } else {
    port.close();
  }
}

2: LED Brightness

/* 
 * Week 11 Production (2)
 * 
 * Outputs:
 * - 5 - LED
 * 
*/

int ledPin = 5;

void setup() {
  Serial.begin(9600);

  pinMode(LED_BUILTIN, OUTPUT);
  pinMode(ledPin, OUTPUT);

  // Blink them so we can check the wiring
  digitalWrite(ledPin, HIGH);
  delay(200);
  digitalWrite(ledPin, LOW);

  // start the handshake
  while (Serial.available() <= 0) {
    digitalWrite(LED_BUILTIN, HIGH); // on/blink while waiting for serial data
    Serial.println("0,0"); // send a starting message
    delay(300);            // wait 1/3 second
    digitalWrite(LED_BUILTIN, LOW);
    delay(50);
  }
}

void loop() {
  // wait for data from p5 before doing something
  while (Serial.available()) {
    digitalWrite(LED_BUILTIN, HIGH); // led on while receiving data

    int brightness = Serial.parseInt();
    if (Serial.read() == '\n') {
      analogWrite(ledPin, brightness);
    }
  }
  digitalWrite(LED_BUILTIN, LOW);
}
let port;
let baudrate = 9600;

// Show button to connect / disconnect
let showConnectButton = false;

function setup() {
  createCanvas(640, 480);
  textSize(20);

  // Create the serial port
  port = createSerial();

  // If the user previously connected, reopen the same port  
  let usedPorts = usedSerialPorts();
  if (usedPorts.length > 0) {
    port.open(usedPorts[0], baudrate);
  }

  // any other ports can be opened via a dialog
  if (showConnectButton) {
    connectBtn = createButton('Connect to Arduino');
    connectBtn.position(80, 350);
    connectBtn.mousePressed(setupSerial);
  }
  
}

// Show serial port connection dialog in response to action
function setupSerial() {
  if (!port.opened()) {
    port.open('Arduino', baudrate);
  } else {
    port.close();
  }
}

function draw() {
  background('white');
  fill('black');
  
  if (showConnectButton) {
    // changes button label based on connection status
    if (!port.opened()) {
      connectBtn.html('Connect to Arduino');
    } else {
      connectBtn.html('Disconnect');
    }
  }

  if (!port.opened()) {
    text("Disconnected - press space to connect", 20, 30);
  } else {
    text("Connected - press space to disconnect", 20, 30);
    
    // // Transmit brightness based on mouse position
    mappedX = floor(map(mouseX, 0, width, 0, 255));
    console.log(mappedX);
    let sendToArduino = mappedX + "\n";
    port.write(sendToArduino);
  }
}

function keyPressed() {
  if (key == " ") {
    setupSerial();
  }
}

3: Wind Gravity

/* 
 * Week 11 Production (3)
 * 
 * Inputs:
 *   - A1 - 10k potentiometer connected to 5V and GND
 * 
 * Outputs:
 * - 5 - LED
 * 
*/

int potPin = A1;
int ledPin = 5;

int interval = 100;
int lastMessageTime = 0;

void setup() {
  Serial.begin(9600);

  pinMode(LED_BUILTIN, OUTPUT);
  pinMode(potPin, INPUT);
  pinMode(ledPin, OUTPUT);

  // Blink them so we can check the wiring
  digitalWrite(ledPin, HIGH);
  delay(200);
  digitalWrite(ledPin, LOW);

  // start the handshake
  while (Serial.available() <= 0) {
    digitalWrite(LED_BUILTIN, HIGH); // on/blink while waiting for serial data
    Serial.println("127"); // send a starting message
    delay(300);            // wait 1/3 second
    digitalWrite(LED_BUILTIN, LOW);
    delay(50);
  }
}

void loop() {
  // wait for data from p5 before doing something
  while (Serial.available()) {
    digitalWrite(LED_BUILTIN, HIGH); // led on while receiving data

    // blink LED based on p5 data
    int status = Serial.parseInt();
    if (Serial.read() == '\n') {
      digitalWrite(ledPin, status);
      delay(10);
      digitalWrite(ledPin, LOW);
    }

    if (lastMessageTime > interval) {
      lastMessageTime = 0;
      // send mapped potentiometer reading to p5
      int potentiometer = analogRead(potPin);                  
      int mappedPotValue = map(potentiometer, 0, 1023, 0, 255); 
      Serial.println(mappedPotValue);
      // slight delay to stabilize the ADC:
      delay(1);
    }
    else {
      lastMessageTime++;
    }

  }
  digitalWrite(LED_BUILTIN, LOW);
}
let velocity;
let gravity;
let position;
let acceleration;
let wind;
let drag = 0.99;
let mass = 50;

// // Arduino
let port;
let connectBtn;
let baudrate = 9600;
let lastMessage = "";
let showConnectButton = false;

function setup() {
  createCanvas(620, 400);
  noFill();
  position = createVector(width/2, 0);
  velocity = createVector(0,0);
  acceleration = createVector(0,0);
  gravity = createVector(0, 0.5*mass);
  wind = createVector(0,0);
  
  // // Arduino
  port = createSerial();
  let usedPorts = usedSerialPorts();
  if (usedPorts.length > 0) {
    port.open(usedPorts[0], baudrate);
  }
  if (showConnectButton) {
    connectBtn = createButton('Connect to Arduino');
    connectBtn.position(80, 300);
    connectBtn.mousePressed(setupSerial);
  }
}

function draw() {
  background(255);
  applyForce(wind);
  applyForce(gravity);
  velocity.add(acceleration);
  velocity.mult(drag);
  position.add(velocity);
  acceleration.mult(0);
  ellipse(position.x,position.y,mass,mass);
  if (position.y > height-mass/2) {
      velocity.y *= -0.9;  // A little dampening when hitting the bottom
      position.y = height-mass/2;
      flashLight();
    }
  
  // // Arduino
  if (showConnectButton) {
    if (!port.opened()) {
      connectBtn.html('Connect to Arduino');
    } else {
      connectBtn.html('Disconnect');
    }
  }
  fill('black');
  if (!port.opened()) {
    text("Disconnected", 20, 30);
  } else {
    text("Connected", 20, 30);
  }
  let str = port.readUntil("\n");
  if (str.length > 0) {
    // console.log(str);
    lastMessage = str;
  }
  
  // Display the most recent message
  text("Last message: " + lastMessage, 10, height - 20);
  
  // // Convert received value to wind.x value
  mappedPot = map(int(lastMessage), 0, 255, -1, 1);
  wind.x = mappedPot;
  let windSpeed = "Wind speed: " + wind.x
  text(windSpeed.substring(0,20), 10, height - 5);
  
  fill('white');
}

function applyForce(force){
  // Newton's 2nd law: F = M * A
  // or A = F / M
  let f = p5.Vector.div(force, mass);
  acceleration.add(f);
}

function keyPressed(){
  if (keyCode==LEFT_ARROW){
    wind.x=-1;
  }
  if (keyCode==RIGHT_ARROW){
    wind.x=1;
  }
  if (key==' '){
    mass=random(15,80);
    position.y=-mass;
    velocity.mult(0);
    port.write("0\n"); // reset light
  }
}

// // Arduino
function setupSerial() {
  if (!port.opened()) {
    port.open('Arduino', baudrate);
  } else {
    port.close();
  }
}

function flashLight() {
  if (port.opened()) {
    port.write("1\n");
    // port.write("0\n");
  }
}

Video (Google Drive)

Week 10 – Reading Response

These readings were very eye-opening in some regards, but I also felt that they were a bit closed-off in others. With our many discussions on what it means for something to be well-designed, we have had to start thinking about our works beyond just their function. Now we are bringing physiology into the equation on top of the prior psychological aspects. I have some idea of how miraculous the human body is and how much work it takes to try and replicate the simplest of actions via technology, but the examples posed about how we use our hands painted a much more intimate picture than I had considered. For example, I might think about how I struggle to play FPS games on a controller whereas several of my friends can’t stand using a mouse and keyboard. The broader significance of tactile feedback would be largely lost in that simple case, let alone the discussion on how input devices might look in 20 years.

Speaking of, I was much more uncertain about how we might work towards this brighter future. This post from 2011 has plenty of thoughts on the matter, but the landscape has changed greatly since then. For example, a good portion of the reading focuses on how touchscreens fall flat of delivering the true experience, but I would argue that many people feel more comfortable with an e-book than a real one, especially so for the younger demographic that grew up with a tablet in hand. The discussion of voice also seems rather shortsighted in hindsight, given that Siri first released in 2010 and was shortly followed by Alexa in 2014. Nowadays you can even push voice-based systems to the point of live-translating speech or leaning on LLMs to achieve goals in the realm of creating and understanding. Lastly, on the subject of brain interfaces, I felt that the author was a bit too pessimistic about things. Companies like Neuralink still fall short today, but they can be hugely impactful for people that can’t rely on their bodies. I can definitely agree that nature is the ultimate inventor, but I don’t think that brain-computer interfaces should be disregarded out of fear that we will all end up forsaking the real world. Either way, with the rate that certain technologies have progressed over the past few years, it’s hard to even imagine what direction interfaces might evolve in.

Week 10 – Musical Instrument

Design:
I used a slide switch as the digital sensor, and a potentiometer + photometer as the analog sensors. The slide switch was originally just an on/off switch, but now allows the instrument to switch between two modes (which currently shifts the octave).  The potentiometer acts as a volume control, while the photometer provides an additional input towards the tone frequency.
The physical component is a simple sort of keyboard made from paper and copper tape. It works by passing current through one of five different resistors (1k, 10k, 100k, 470k, 1m) and using analogRead() to identify which key was pressed. The five keys correspond to the notes A4/5 through E4/5. The initial steps of wiring the circuit and writing the code were fairly painless, which was not the case for converting it into its final result. I think it definitely suffered from my lack of ability with music as well as arts and crafts.

Diagram:

Code:

/*
  Musical Instrument
*/

#include "pitches.h"

// // Global Variables
const int LIGHT_PIN = A0;
const int TOUCH_PIN = A5;
const int MODE_PIN = 2;
const int SOUND_PIN = 8;
int low[] = {NOTE_A4, NOTE_B4, NOTE_C4, NOTE_D4, NOTE_E4};
int high[] = {NOTE_A5, NOTE_B5, NOTE_C5, NOTE_D5, NOTE_E5};



void setup() {
  Serial.begin(9600);
  pinMode(LIGHT_PIN, INPUT);
  pinMode(TOUCH_PIN, INPUT);
  pinMode(MODE_PIN, INPUT);
  pinMode(SOUND_PIN, OUTPUT);
}

void loop() {
  // // Read digital/analog
  int modeVal = digitalRead(MODE_PIN);
  // Serial.println(modeVal);
  int lightVal = analogRead(LIGHT_PIN);
  // Serial.println(lightVal);
  int touchVal = analogRead(TOUCH_PIN);
  Serial.println(touchVal);
  
  // // Choose note based on touch input
  // 1k = 932
  // 10k = 512
  // 100k = 92
  // 470k = 20
  // 1m = 8
  int x; int mapped;
  if (touchVal < 4) {noTone(SOUND_PIN); return;
  }
  else if (touchVal < 16) {x = 0;}
  else if (touchVal < 64) {x = 1;}
  else if (touchVal < 128) {x = 2;}
  else if (touchVal < 768) {x = 3;}
  else if (touchVal < 1024) {x = 4;}
  // Serial.println(x);

  // // Choose high/low range based on mode
  int note;
  if (modeVal) {
    note = low[x];
    mapped = map(lightVal, 400, 1200, 0, 2500);
  }
  else {
    note = high[x];
    mapped = map(lightVal, 400, 1200, 2500, 5000);
  }
  // // Play sound
  tone(SOUND_PIN, note);
}

 

 

Week 9 – Reading Response

Physical Computing:
It was really interesting to see all the different forms that physical computing pieces can take. The examples given, despite only being a small glimpse into the genre, cover a whole bunch of mediums and correspondingly provide entirely new experiences. In order to do so, they each have to take into account the physical and digital components for both input and output, and figure out how to blend them all together into one working whole. What stood out to me the most was all the little tricks used to make sense of intuitive human motions such as hand movements, as well as the variations of ways to do so. That particular point was also echoed in the introduction, where it mentioned that the same idea can be produced in varying ways. Hopefully as the semester progresses, my own pieces will similarly become more complex and interactive.

Interactive Art:
After the first half of this class and the resulting midterm project, I can definitely identify with what the reading is getting at. The previous readings on different aspects of design come into play once again, albeit with more emphasis on the physical aspect this time around. The listening aspect builds on that and turns the creation process into a more iterative version of itself to presumably beneficial effect. I liked the comparison to telling an actor how to act versus giving them the tools to perform their craft, which naturally leads into the next analogy of the piece being a performance involving the audience. Going forward I’d like to keep this perspective in mind, and endeavour to provide a more comprehensive experience in my work.

Week 8 – Reading Response 1 (Attractive Things Work Better)

I found the initial anecdote of the three teapots to be especially interesting. Norman spends nearly a third of the reading comparing and contrasting them, noting that he might choose one over the others depending on the occasion or his mood. In stark contrast, my first thought was to wonder why anybody would one teapot let alone three. Beauty is really in the eye of the beholder here, since on the rare occasion I make tea it involves a mug and a pre-packaged teabag. Even his mention of displaying them on the windowsill just made me question why he would waste space doing so.

As for the larger discussion on aesthetic/usability and affect/cognition, I have some mixed feelings. The explanation of how the chemicals in your brain can completely warp your perception was very well-done, what with the examples of tightrope walking or opening an emergency door. That being said, the claim that “attractive things work better” feels far too heavy-handed (although Norman himself notes that the claim is heretical). The preceding two sentences mention that a product that looks good will encourage the user to overlook its flaws, and to me that hardly means ‘working better’. You could definitely argue that the emotional aspect could improve the user’s enjoyment, but that is countered by claiming that performance would be better with an uglier, more efficient version. You could also argue that attractiveness helps the product sell, but then you are outright avoiding the point of debate. I would agree more if his claim was about how attractive things face less criticism, and are accepted more easily.

Week 8 – Reading Response 2 (Her Code Got Humans on the Moon)

When I first saw the title of this reading, I assumed that it referred to the three mathematicians that starred in Hidden Figures (2016). I was instead pleasantly surprised by a fresh perspective that was similar yet different, and happens to result in the creation of software engineering. As a Computer Science major, it was very cool to see how software was viewed before it became commonplace, and get to learn about the factors that shaped its development into what it is today. It’s still hard to imagine how they managed to put a man on the moon while being stuck in the era where the term ‘physical computing’ was all too literal.

In that sense there is a clear tie-in to our Arduino work, what with having to wire components in by hand and having actual limitations on memory, although thankfully we still have access to an IDE. Even aside from the Interactive Media perspective, I got a lot out of this piece that can apply to the software engineering class I’m taking. For example, having to make asynchronous calls to the backend of our web application and dealing with potential errors. We’re actually currently in the middle of writing system tests for our next deliverable, so reading through the ‘That would never happen’ section was particularly painful.