Author: Jhon

Concept 

My concept is a semi-thriller game that features two key sensors — the joystick, and the pulse sensor. It was inspired by my siblinghood. Growing up, I would often eat my sister’s snacks, of course, without her permission. This would give me a thrill that was akin to what someone would feel playing hide-and-seek with a murderer (I’m exaggerating here; I actually really love my sister haha). I settle on making it “slime-themed” since I wanted it to be more fun and gender-neutral to be inclusive to all players (possibly people that had older brothers, cousins, etc.). I mean certainly a slime character is non-specific enough I think.

How it works

Well, when you first start the game, you will be greeted with an eery background music and screen image with two options: “how to play” and “start game”. If you press on “how to play” with the joystick, you’ll be shown a very comprehensive manual on how to play the game. Then, if you press start, it’ll take you to the game dialog, and you can click through the dialog through the joystick clicks. After that, the game begins! When it starts, on the top right of the screen there is a icon of slime chips with a number on top, these are the number of chips you need to eat without getting caught and losing all your hearts. The number of hearts you have is on the top left. When the sister turns around and you’re eating your chips (i.e. your hands are touching; you can move the right hand with the chips left and right with the joystick), she gets mad and you lose a heart. Here are the game p5.js visuals:

(startscreen) (how to play) (dialog screen) (game started and sister turned around) (game started and sister caught you!!)(you won!!)

Implementation

1. Interaction Design:

I wanted it to have a flex sensor initially as the pseudo “joystick” but I realized that that wouldn’t really be intuitive or even enjoyable for the average user likely due to its foreign nature in games like these. So I decided to borrow a joystick from the IM lab, alongside a pulse sensor. I wanted to make the game take real-time input from the user and imitate the scenario of my heart racing as a kid when stealing my sister’s food. So, every 1.5 to 2.5 seconds, the game checks if the user’s heartbeat ‘jumps’ by more than 70 (imitating nervousness). If so, or with a 25% random chance, the sister turns around. I did this to simulate the tension of being caught, as the player’s nervousness (heartbeat jump) makes the slime sibling more likely to notice you with a bit of randomness. I added chip-eating sound effects among some other sounds to reflect what is happening on the screen.

2. Arduino Code:

My Arduino code controls three pairs of colored LEDs (red, yellow, green) based on commands received from a computer (via Serial). It also reads a joystick’s X-axis, a pulse sensor, and a joystick button (I put print statements for their values for debugging and monitoring). Only one color is lit at a time, and the code ignores repeated color commands, to make sure a state is reflected clearly for the player. I also added some helper functions to keep all LEDs off at the beginning and another one so that the same color LED is on at both sides of the box. Here is the code:

const int joyXPin   = A0;    // joystick a-axis
const int pulsePin  = A1;    // pulse sensor
const int selPin    = 8;     // joystick “select” button (digital)
const int baudRate  = 9600;

// LED pins
const int RED_L  = 7,  RED_R  = 5;
const int YEL_L  = 10, YEL_R  = 4;
const int GRN_L  = 12, GRN_R  = 2;

const int threshold = 50;    // dead-zone around center (512)

// remembering last LED state
String currentCmd = "";

void setup() {
  Serial.begin(baudRate);
  pinMode(selPin, INPUT_PULLUP);
  
  // LED pins
  pinMode(RED_L, OUTPUT);
  pinMode(RED_R, OUTPUT);
  pinMode(YEL_L, OUTPUT);
  pinMode(YEL_R, OUTPUT);
  pinMode(GRN_L, OUTPUT);
  pinMode(GRN_R, OUTPUT);

  // start with all off
  clearAllLeds();
}

void loop() {
  // checking for new command from p5.js
  if (Serial.available()) {
    String cmd = Serial.readStringUntil('\n');
    cmd.trim();

    // only accept valid colors
    if (cmd == "RED" || cmd == "YELLOW" || cmd == "GREEN") {
      // if different than last, update
      if (cmd != currentCmd) {
        currentCmd = cmd;
        applyLeds(currentCmd);
      }
    }
    // echo back for debugging
    Serial.print("Received: ");
    Serial.println(cmd);
  }

  // existing joystick/pulse prints
  int xRaw    = analogRead(joyXPin);
  int pulseRaw = analogRead(pulsePin);
  bool selPressed = (digitalRead(selPin) == LOW);

  int dirX;
  if (xRaw < 512 - threshold)      dirX = 0;
  else if (xRaw > 512 + threshold) dirX = 1;
  else                              dirX = -1;

  Serial.print("DIRX:");   Serial.print(dirX);
  Serial.print("  PULSE:");Serial.print(pulseRaw);
  Serial.print("  SEL:");  Serial.println(selPressed ? 1 : 0);
}

// helper to turn all LEDs off
void clearAllLeds() {
  digitalWrite(RED_L, LOW); digitalWrite(RED_R, LOW);
  digitalWrite(YEL_L, LOW); digitalWrite(YEL_R, LOW);
  digitalWrite(GRN_L, LOW); digitalWrite(GRN_R, LOW);
}

// helper to set exactly one color on both sides
void applyLeds(const String &cmd) {
  clearAllLeds();
  if (cmd == "RED") {
    digitalWrite(RED_L, HIGH);
    digitalWrite(RED_R, HIGH);
  } 
  else if (cmd == "YELLOW") {
    digitalWrite(YEL_L, HIGH);
    digitalWrite(YEL_R, HIGH);
  } 
  else if (cmd == "GREEN") {
    digitalWrite(GRN_L, HIGH);
    digitalWrite(GRN_R, HIGH);
  }
}

3. Schematic of Project:

4. p5.js Code:

The game features multiple states (start menu, gameplay, how-to-play screen) managed through a state machine architecture. When playing, users control a hand to grab chips while avoiding detection from the sister character who randomly turns around. The sister’s turning behavior is influenced by both randomness and the player’s actual heartbeat measured through a pulse sensor.

Key features in the p5.js include:

• Dialog system for narrative progression
• Character animation with multiple emotional states
• Collision detection between hands
• Health and scoring systems (hearts and chip counter)
• Two endings (happy and bad) based on gameplay outcomeThis is the code:

  let gameState = 'start'; // 'start', 'game', or 'howToPlay'
  let startImg;
  let bgImg;
  let howToPlayImg; 
  let bgMusic;
  let angry;
  
  // serial communication variables
  let rightX = 0;
  let pulseValue = 0;  
  let connectButton; // button to initiate serial connection
  
  // hoystick control variables for menu navigation
  let currentSelection = 0; // 0 = start game, 1 = how to play
  let selPressed = false;
  let prevSelPressed = false; // to detect rising edge
  let backButtonSelected = false; // tracking if back button is selected in how to play screen
  
  // hand objects
  let leftHand = {
    img: null,
    x: 0,
    y: 0,
    width: 150,
    height: 150,
    visible: false
  };
  
  let rightHand = {
    img: null,
    x: 0,
    y: 0,
    width: 150,
    height: 150,
    visible: false,
    currentImg: 1, // tracking current hand image (1, 2, or 3)
    animationStarted: false, // tracking if animation has started
    animationTimer: 0, // timer for animation
    lastLedState: "",
    sisterTurned: false, // tracking if sister is turned around
    sisterTurnStart: 0,      // ← when she last flipped to “turned”
    maxTurnDuration: 1700,   // ← max ms she can remain turned (1.7 s)
    lastTurnCheck: 0, // timer for checking sister's state
    heartDeducted: false, // flag to track if heart was deducted in current overlap
    lastSisterState: false, // tracking previous sister state to detect changes
    sisterAngry: false, // flag to track if sister is angry
    lastPulseValue: 0 // tracking last pulse value for heartbeat detection
  };
  
  let chipCounter = {
    img: null,
    x: 0,
    y: 0,
    width: 80,
    height: 80,
    count: 10
  };
  
  let hearts = {
    img: null,
    count: 5, 
    width: 30,
    height: 30,
    spacing: 10,
    x: 20, 
    y: 20 
  };
  
  let gameOver = false;
  let gameEndingType = ''; // 'good' or 'bad' to track which ending to show
  
  let dialogSounds = {};
  
  let startButton = {
    x: 0,
    y: 0,
    width: 150, 
    height: 30,
    text: 'start game',
    isHovered: false
  };
  
  let howToPlayButton = {
    x: 0,
    y: 0,
    width: 180, 
    height: 30,
    text: 'how to play',
    isHovered: false
  };
  
  let backButton = { 
    x: 0,
    y: 0,
    width: 100,
    height: 40,
    text: 'Back',
    isHovered: false
  };
  
  let instrumentSerifFont;
  let currentDialog = null; 
  
  let characters = {};
  
  function preload() {
    startImg = loadImage('Images/start.png');
    bgImg = loadImage('Images/bg.png');
    howToPlayImg = loadImage('Images/howtoplay.png'); 
    instrumentSerifFont = loadFont('InstrumentSerif-Regular.ttf');
    bgMusic = loadSound('Audios/bg.mp3');
    leftHand.img = loadImage('Images/left_hand.png');
    rightHand.img = loadImage('Images/right_hand1.png');
    rightHand.img2 = loadImage('Images/right_hand2.png');
    rightHand.img3 = loadImage('Images/right_hand3.png');
    chipCounter.img = loadImage('Images/slimechips.png');
    hearts.img = loadImage('Images/heart.png');
    happyEndingImg = loadImage('Images/happy_ending.png');
    badEndingImg = loadImage('Images/bad_ending.png');
  
    dialogSounds['chipbag.mp3'] = loadSound('Audios/chipbag.mp3');
    dialogSounds['chip_crunch.mp3'] = loadSound('Audios/chip_crunch.mp3');
    dialogSounds['angry.mp3'] = loadSound('Audios/angry.mp3');
  
    characters['sister_normal'] = new Character('sister_normal', 'Sister', 'Images/sister_normal.png');
    characters['sister_speaking'] = new Character('sister_speaking', 'Sister', 'Images/sister_speaking.png');
    characters['sister_turned'] = new Character('sister_turned', 'Sister', 'Images/sister_turned.png');
    characters['sister_upset'] = new Character('sister_upset', 'Sister', 'Images/sister_upset.png');
    characters['sister_angry'] = new Character('sister_angry', 'Sister', 'Images/sister_angry.png');
    characters['slime'] = new Character('slime', 'me', 'Images/slime.png');
    characters['slime_speaking'] = new Character('slime_speaking', 'me', 'Images/slime_speaking.png');
    for (let key in characters) {
      characters[key].preload();
    }
  }
  
  async function setUpSerialWithErrorHandling() { // handling errors with serial
    try {
      await setUpSerial();
    } catch (error) {
      console.log("Serial connection failed: " + error.message);
    }
  }
  
  function setup() {
    createCanvas(627, 447);
    noCursor(); 
  
    // playing background music
    bgMusic.setVolume(0.5); 
    bgMusic.loop(); 
  
    textFont(instrumentSerifFont);
  
    connectButton = createButton('connect serial');
    connectButton.position(25, 25);
    connectButton.style('font-size', '18px');
    connectButton.style('font-weight', 'bold');
    connectButton.style('color', '#000');
    connectButton.style('background-color', '#fff');
    connectButton.style('border', '2px solid #000');
    connectButton.style('border-radius', '8px');
    connectButton.style('padding', '8px 16px');
    connectButton.style('cursor', 'pointer');
  
      connectButton.mouseOver(() => {
        connectButton.style('background-color', '#000');
        connectButton.style('color', '#fff');
      });
      connectButton.mouseOut(() => {
        connectButton.style('background-color', '#fff');
        connectButton.style('color', '#000');
      });
  
    connectButton.mousePressed(setUpSerialWithErrorHandling); // using setUpSerial from p5.web-serial.js
  
  
    positionStartScreenButtons();
  
    positionBackButton();
  
    rightHand.x = width - 10;
    rightHand.y = height/2 - rightHand.height/2;
  
    chipCounter.x = width - chipCounter.width - 20;
    chipCounter.y = 20;
  
    const sampleDialog = [
      { speaker: 'sister_speaking', line: "Hey! Have you seen my chips?" },
      { speaker: 'slime', line: "Nope, I haven't seen them." , sound: "chipbag.mp3"},
      { speaker: 'sister_upset', line: "Hmmm... are you sure?" },
      { speaker: 'slime_speaking', line: "Yep...! Haven't seen them! I'll let you know if I see them!" },
      { speaker: 'sister_speaking', line: "Grrr! If I catch you with them..I'm going to kill you!" }
    ];
    currentDialog = new Dialog(sampleDialog); // passing the new structure to the Dialog constructor
  }
  
  function readSerial(data) {
    let val = trim(data);
    if (!val) return;
  
    let parts = val.split(" ");
  
    // parsing DIRX for both game movement and menu navigation
    let dirPart = parts.find(p => p.startsWith("DIRX:"));
    if (dirPart) {
      let dir = int(dirPart.split(":")[1]);
  
      // setting rightX for in-game hand movement
      if (dir === 0) {
        rightX = width * 0.25;  // move left
        
        // for menu navigation, select the left button (start game)
        if (gameState === 'start') {
          currentSelection = 0; // selecting start game button
        } 
        // for how to play screen, select the back button when moving left
        else if (gameState === 'howToPlay') {
          backButtonSelected = true;
        }
      } else if (dir === 1) {
        rightX = width * 0.75;  // move right
        
        // for menu navigation, select the right button (how to play)
        if (gameState === 'start') {
          currentSelection = 1; // selecting how to play button
        }
        // for how to play screen, deselect the back button when moving right
        else if (gameState === 'howToPlay') {
          backButtonSelected = false;
        }
      } else if (dir === -1) {
        rightX = width / 2;     // center (neutral)
      }
  
      console.log("Joystick Direction:", dir);
    }
  
    // parsing SEL for button selection
    let selPart = parts.find(p => p.startsWith("SEL:"));
    if (selPart) {
      let sel = int(selPart.split(":")[1]);
      prevSelPressed = selPressed;
      selPressed = (sel === 1);
      
      // if SEL button was just pressed (rising edge detection)
      if (selPressed && !prevSelPressed) {
        // handling button selection based on current screen
        if (gameState === 'start') {
          if (currentSelection === 0) {
            startGame();
          } else if (currentSelection === 1) {
            showHowToPlay();
          }
        } else if (gameState === 'howToPlay') {
          // only going back if the back button is selected
          if (backButtonSelected) {
            goBackToStart();
          }
        } else if (gameState === 'game' && currentDialog && !currentDialog.isComplete) {
          currentDialog.advanceDialog();
        }
      }
    }
  
    // parsing PULSE
    let pulsePart = parts.find(p => p.startsWith("PULSE:"));
    if (pulsePart) {
      pulseValue = int(pulsePart.split(":")[1]);
      console.log("Pulse Reading:", pulseValue);
    }
  }
  
  
  // helper that mirrors mouseClicked() logic but uses joystick coords
  function joystickClick(x,y){
    if(gameState==='start'){
      // is it on the “start” button?
      if(x > startButton.x && x < startButton.x+startButton.width &&
         y > startButton.y && y < startButton.y+startButton.height){
        startGame();
      }
      // how-to-play button?
      else if(x > howToPlayButton.x && x < howToPlayButton.x+howToPlayButton.width &&
              y > howToPlayButton.y && y < howToPlayButton.y+howToPlayButton.height){
        showHowToPlay();
      }
    }
  }
  
  
  function draw() {
    background(255); 
    
    switch (gameState) {
      case 'start':
        drawStartScreen();
        updateButtonHover(startButton);
        updateButtonHover(howToPlayButton);
        break;
      case 'game':
        drawGameScreen();
        updateButtonHover(backButton);
        break;
      case 'howToPlay':
        drawHowToPlayScreen();
        updateButtonHover(backButton);
        break;
    }
  
    if (!gameOver) {
      noCursor();
    }
  }
  
  function drawStartScreen() {
    image(startImg, 0, 0, width, height);
    
    // updating button hover state based on joystick selection instead of mouse
    if (gameState === 'start') {
      // hilighting the currently selected button based on joystick input
      startButton.isHovered = (currentSelection === 0);
      howToPlayButton.isHovered = (currentSelection === 1);
    }
    
    drawButton(startButton);
    drawButton(howToPlayButton);
    
    // hiding back button on start screen
    backButton.x = -backButton.width; // positioning off-screen
  }
  
  function drawGameScreen() {
    image(bgImg, 0, 0, width, height);
    
    // drawing dialog if it exists and isn’t complete
    if (currentDialog && !currentDialog.isComplete) {
      currentDialog.display();
      return;
    }
    
    // if dialog is complete but gameover, show ending
    if (gameOver) {
      drawGameOverScreen();
      return;
    }
  
    // --- sister character ---
    let sisterWidth  = 180;
    let sisterHeight = 200;
    let sisterX = width/2 - sisterWidth/2;
    let sisterY = height/2 - sisterHeight/2 - 30;
    
    let sisterImg = characters['sister_turned'].img;
    if (rightHand.sisterAngry)        sisterImg = characters['sister_angry'].img;
    else if (rightHand.sisterTurned)  sisterImg = characters['sister_upset'].img;
    
    // possibly flip her based on time+random+heartbeat
    if (!rightHand.lastTurnCheck 
        || millis() - rightHand.lastTurnCheck > random(1500, 2500)) {
          
      rightHand.lastTurnCheck = millis();
      let heartbeatTrigger = (rightHand.lastPulseValue 
        && Math.abs(pulseValue - rightHand.lastPulseValue) > 70);
      rightHand.lastPulseValue = pulseValue;
      
      if (random() < 0.25 || heartbeatTrigger) {
        rightHand.sisterTurned = !rightHand.sisterTurned;
        if (rightHand.sisterTurned) {
          rightHand.sisterTurnStart = millis();
        }
        rightHand.sisterAngry   = false;
        rightHand.heartDeducted = false;
      }
    }
    
    image(sisterImg, sisterX, sisterY, sisterWidth, sisterHeight);
  
    // enforcing max “turned” time
    if (rightHand.sisterTurned 
        && millis() - rightHand.sisterTurnStart > rightHand.maxTurnDuration) {
      rightHand.sisterTurned   = false;
      rightHand.sisterAngry    = false;
      rightHand.heartDeducted  = false;
    }
    
    // --- Collision ---
    leftHand.x = width/2 - leftHand.width - 50;
    leftHand.y = height/2 + leftHand.height/2;
    
    // compute & smooth right hand
    let targetX = constrain(rightX, width*0.1, width*0.9);
    rightHand.x += (targetX - rightHand.x)*0.2;
    rightHand.y  = height/2 + rightHand.height/2;
    
    let handsOverlap = checkHandCollision(leftHand, rightHand);
    let ledState = "";
    if (handsOverlap && rightHand.sisterTurned && !rightHand.heartDeducted) {
      // CAUGHT → angry → RED
      hearts.count--;
      rightHand.heartDeducted = true;
      rightHand.sisterAngry   = true;
      ledState = "RED";
      dialogSounds['angry.mp3'].play();
  
      if (hearts.count <= 0) {
        gameOver = true;
        gameEndingType = 'bad';
      }
    }
  
    else if (handsOverlap && rightHand.sisterTurned && rightHand.heartDeducted) {
      ledState = "RED";
    }
    else if (!rightHand.sisterTurned) { // sister is not turned
      // SAFE while she’s turned → GREEN
      ledState = "GREEN";
    }
    else if (rightHand.sisterTurned && !handsOverlap) { // sister is turned but hands are not overlapping
      // NORMAL (not turned) → YELLOW
      ledState = "YELLOW";
    }
  
    if (ledState && ledState !== rightHand.lastLedState) {
      writeSerial(ledState + "\n");
      rightHand.lastLedState = ledState;
    }
    
    // --- Eating Animation & Chips ---
    if (handsOverlap && !rightHand.animationStarted) {
      rightHand.animationStarted = true;
      rightHand.animationTimer   = millis();
      if (rightHand.currentImg === 3 && chipCounter.count > 0) {
        chipCounter.count--;
        if (chipCounter.count <= 0) {
          gameOver = true;
          gameEndingType = 'good';
        }
      }
      rightHand.currentImg = 1;
      dialogSounds['chip_crunch.mp3'].play();
    }
    else if (handsOverlap && rightHand.animationStarted) {
      let t = millis() - rightHand.animationTimer;
      if (rightHand.currentImg === 1 && t > 300) rightHand.currentImg = 2;
      if (rightHand.currentImg === 2 && t > 500) rightHand.currentImg = 3;
      if (rightHand.currentImg === 3 && t > 800) {
        rightHand.animationTimer = millis();
        rightHand.currentImg = 1;
        if (chipCounter.count > 0) {
          chipCounter.count--;
          if (chipCounter.count <= 0) {
            gameOver = true;
            gameEndingType = 'good';
          }
        }
        dialogSounds['chip_crunch.mp3'].play();
      }
    }
    else if (!handsOverlap && rightHand.animationStarted) {
      rightHand.animationStarted = false;
      dialogSounds['chip_crunch.mp3'].stop();
    }
    
    // drawing hands
    let handImg = rightHand.currentImg===1 ? rightHand.img
                : rightHand.currentImg===2 ? rightHand.img2
                : rightHand.img3;
    image(handImg, rightHand.x, rightHand.y, rightHand.width, rightHand.height);
    image(leftHand.img, leftHand.x, leftHand.y, leftHand.width, leftHand.height);
  
    image(chipCounter.img, chipCounter.x, chipCounter.y, chipCounter.width, chipCounter.height);
    push();
      fill(255); stroke(0); strokeWeight(2);
      textSize(24); textAlign(CENTER, CENTER);
      text(chipCounter.count, chipCounter.x+chipCounter.width/2, chipCounter.y+chipCounter.height/2);
    pop();
    for (let i=0; i<hearts.count; i++) {
      image(hearts.img, hearts.x+(hearts.width+hearts.spacing)*i, hearts.y, hearts.width, hearts.height);
    }
  }
  
  // function to check collision between hand objects
  function checkHandCollision(hand1, hand2) {
    // rectangular collision detection
    return (
      hand1.x + 10 < hand2.x + hand2.width &&
      hand1.x + hand1.width > hand2.x &&
      hand1.y < hand2.y + hand2.height &&
      hand1.y + hand1.height > hand2.y
    );
  }
  
  function drawHowToPlayScreen() {
      background(50); 
      let imgWidth = width * 0.9; // Use 90% of screen width
      let imgHeight = height * 0.7; // Use 70% of screen height
      image(howToPlayImg, width/2 - imgWidth/2, height/2 - imgHeight/2 + 30, imgWidth, imgHeight);
      
      fill(255);
  
      textSize(35);
      textAlign(CENTER, CENTER);
      text("How to Play", width/2, height/2 - imgHeight/2 - 15);
  
      backButton.isHovered = backButtonSelected;
      
      drawButton(backButton);
      
      startButton.x = -startButton.width; // Position off-screen
      howToPlayButton.x = -howToPlayButton.width; // Position off-screen
  }
  
  
  function startGame() {
    gameState = 'game';
    console.log("Game Started!");
  
    if (bgMusic.isPlaying()) {
      bgMusic.stop();
    }
  
    if (currentDialog) {
        currentDialog.currentLineIndex = 0;
        currentDialog.isComplete = false;
    }
  
    positionBackButton();
  }
  
  function showHowToPlay() {
    gameState = 'howToPlay';
    console.log("Showing How to Play");
     positionBackButton();
  }
  
  function goBackToStart() {
      gameState = 'start';
      console.log("Going back to Start Screen");
      positionStartScreenButtons();
  }
  
  
  // helper function to position buttons on the start screen
  function positionStartScreenButtons() {
      textSize(20); 
      startButton.width = textWidth(startButton.text) + 40; 
      howToPlayButton.width = textWidth(howToPlayButton.text) + 40; 
      startButton.height = 40; 
      howToPlayButton.height = 40; 
  
  
      let buttonY = height * 0.65; // Moved buttons higher (was 0.75)
  
      let buttonSpacing = 20;
      let totalButtonWidth = startButton.width + howToPlayButton.width + buttonSpacing;
      let leftShift = 15; 
      startButton.x = width / 2 - totalButtonWidth / 2 - leftShift;
      howToPlayButton.x = startButton.x + startButton.width + buttonSpacing;
  
      startButton.y = buttonY;
      howToPlayButton.y = buttonY;
  }
  
  function positionBackButton() {
      textSize(20); 
      backButton.width = textWidth(backButton.text) + 40;
      backButton.height = 40; 
      backButton.x = 20; 
      backButton.y = 20; 
  }
  
  
  // function to draw a button
  function drawButton(button) {
       if (button.x + button.width > 0 && button.x < width) {
           push(); // Save current drawing settings
  
          if (button.isHovered) {
              fill(255, 0, 0); 
          } else {
              fill(0); 
          }
  
          rect(button.x, button.y, button.width, button.height, 10); // 10 is corner radius
  
          fill(255);
          textSize(22);
          textAlign(CENTER, CENTER);
  
          text(button.text, button.x + button.width / 2, button.y + button.height / 2 - 5);
  
          pop(); // restoring drawing settings
       }
  }
  
  // helper function to update button hover state
  function updateButtonHover(button) {
      // inly update hover if the button is likely visible on screen
      if (button.x + button.width > 0 && button.x < width) {
          if (mouseX > button.x && mouseX < button.x + button.width &&
              mouseY > button.y && mouseY < button.y + button.height) {
              button.isHovered = true;
              cursor(HAND); // Change cursor on hover
          } else {
              button.isHovered = false;
              // only set cursor back to ARROW if NO button is hovered
               let anyButtonHovered = false;
               if (gameState === 'start') {
                   if (startButton.isHovered || howToPlayButton.isHovered) anyButtonHovered = true;
               } else if (gameState === 'howToPlay' || gameState === 'game') {
                   if (backButton.isHovered) anyButtonHovered = true;
               }
               if (!anyButtonHovered) {
                   cursor(ARROW);
               }
          }
      }
  }
  
  // helper function to update button hover state with joystick
  function updateButtonHoverWithJoystick(button, joyX, joyY) {
    if (button.x + button.width > 0 && button.x < width) {
      if (joyX > button.x && joyX < button.x + button.width &&
          joyY > button.y && joyY < button.y + button.height) {
        button.isHovered = true;
      } else {
        button.isHovered = false;
      }
    }
  }
  
  function mouseClicked() {
    switch (gameState) {
      case 'start':
        // Check for button clicks on the start screen
        if (mouseX > startButton.x && mouseX < startButton.x + startButton.width &&
            mouseY > startButton.y && mouseY < startButton.y + startButton.height) {
          startGame();
        } else if (mouseX > howToPlayButton.x && mouseX < howToPlayButton.x + howToPlayButton.width &&
                   mouseY > howToPlayButton.y && mouseY < howToPlayButton.y + howToPlayButton.height) {
          showHowToPlay();
        }
        break;
      case 'game':
        // handling dialog clicks in the game screen
        if (currentDialog && !currentDialog.isComplete) {
          if (mouseX > currentDialog.boxX && mouseX < currentDialog.boxX + currentDialog.boxWidth &&
              mouseY > currentDialog.boxY && mouseY < currentDialog.boxY + currentDialog.boxHeight) {
            currentDialog.advanceDialog();
          } else if (mouseX > backButton.x && mouseX < backButton.x + backButton.width && 
                     mouseY > backButton.y && mouseY < backButton.y + backButton.height) {
              goBackToStart(); // Go back if back button is clicked
          }
        } else { 
             if (mouseX > backButton.x && mouseX < backButton.x + backButton.width &&
                     mouseY > backButton.y && mouseY < backButton.y + backButton.height) {
              goBackToStart(); // Go back if back button is clicked
             }
        }
        break;
      case 'howToPlay':
          if (mouseX > backButton.x && mouseX < backButton.x + backButton.width &&
              mouseY > backButton.y && mouseY < backButton.y + backButton.height) {
              goBackToStart(); // Go back to start screen
          }
          break;
    }
  }
  
  
  class Character {
    constructor(key, displayName, imagePath) {
      this.key = key;
      this.displayName = displayName;
      this.imagePath = imagePath;
      this.img = null;
    }
    preload() {
      this.img = loadImage(this.imagePath);
    }
  }
  
  class Dialog {
    constructor(dialogLines) {
      // dialogLines: array of strings like 'sister_normal: "Hey!"' or just dialog text
      this.lines = dialogLines.map(lineObj => {
        if (typeof lineObj === 'string') {
          let match = lineObj.match(/^(\w+):\s*"([\s\S]*)"$/);
          if (match) {
            return { speaker: match[1], line: match[2] };
          } else {
            return { speaker: 'sister_normal', line: lineObj };
          }
        } else if (typeof lineObj === 'object') {
          return {
            speaker: lineObj.speaker || 'sister_normal',
            line: lineObj.line || '',
            sound: lineObj.sound || null 
          };
        }
      });
      this.currentLineIndex = 0;
      this.isComplete = false;
      this.boxWidth = width * 0.8;
      this.boxHeight = 100;
      this.boxX = (width - this.boxWidth) / 2;
      this.boxY = height - this.boxHeight - 20;
      this.textSize = 18;
      this.textMargin = 15;
      this.lineHeight = this.textSize * 1.2;
    }
  
    display() {
      if (this.isComplete) {
        return;
      }
      fill(0, 0, 0, 200);
      rect(this.boxX, this.boxY, this.boxWidth, this.boxHeight, 10);
  
      let currentLineObj = this.lines[this.currentLineIndex];
      let charKey = currentLineObj.speaker || 'sister_normal';
      
      let displayedCharKey = charKey; // Either 'me' or one of the sister variants
      
      let charObj = characters[displayedCharKey] || characters['sister_normal'];
      
      let charImgSize = (displayedCharKey === 'me') ? 150 : 220; 
      
      if (charObj && charObj.img) {
        let centerX = width / 2 - charImgSize / 2;
        
        let centerY = (displayedCharKey === 'me') 
          ? height / 2 - charImgSize / 2 - 20  
          : height / 2 - charImgSize / 2 - 45; 
        
        image(charObj.img, centerX, centerY, charImgSize, charImgSize);
        
        fill(255);
        textSize(18);
        textAlign(CENTER, TOP);
        text(charObj.displayName, width / 2, centerY + charImgSize + 5);
      }
  
      fill(255);
      textSize(this.textSize);
      textAlign(LEFT, TOP);
      let textY = this.boxY + this.textMargin;
      text(currentLineObj.line, this.boxX + this.textMargin, textY, this.boxWidth - this.textMargin * 2);
  
      if (this.currentLineIndex < this.lines.length - 1) {
        fill(255, 150);
        textSize(14);
        textAlign(RIGHT, BOTTOM);
        text("Click to continue...", this.boxX + this.boxWidth - this.textMargin, this.boxY + this.boxHeight - 5);
      }
    }
  
    advanceDialog() {
      if (this.isComplete) {
        return;
      }
      
      this.currentLineIndex++;
      
      if (this.currentLineIndex >= this.lines.length) {
        this.isComplete = true;
        console.log("Dialog complete!");
        return;
      }
      
      let currentLineObj = this.lines[this.currentLineIndex];
      if (currentLineObj.sound && dialogSounds[currentLineObj.sound]) {
        dialogSounds[currentLineObj.sound].play();
      }
    }
  }
  
  function drawGameOverScreen() {
    push();
    fill(0, 0, 0, 180); // dark overlay with 70% opacity
    rect(0, 0, width, height);
    
    let endingImg;
    if (gameEndingType === 'good') {
      endingImg = happyEndingImg;
    } else {
      endingImg = badEndingImg;
    }
    
    let imgWidth = 300;
    let imgHeight = 200;
    image(endingImg, width/2 - imgWidth/2, height/2 - 150, imgWidth, imgHeight);
    
    fill(255);
    textSize(40);
    textAlign(CENTER, CENTER);
    if (gameEndingType === 'good') {
      text("YOU WON!", width/2, height/2 - 30);
    }
    else {
      text("YOU LOST!", width/2, height/2 - 30);
    }
    textSize(24);
    text("Press R to restart", width/2, height/2 + 80);
    pop();
  }
  
  function keyPressed() {
    // Check if 'R' key is pressed to restart the game when game over
    if (key === 'r' || key === 'R') {
      if (gameState === 'game' && gameOver) {
        resetGame();
      }
    }
  }
  
  function resetGame() {
    hearts.count = 5;
    chipCounter.count = 10;
    gameOver = false;
    
    handsOverlap = false;
    rightHand.sisterTurned = false;
    rightHand.animationStarted = false;
    rightHand.currentImg = 1;
    rightHand.heartDeducted = false;
  }

   

5. Communication between Arduino and p5.js:

The Arduino and the p5.js sketch exchange data over a 9600 baud serial link, as we did in the sample homework 2 weeks back. On each loop, the Arduino reads the joystick X‐axis (A0), the pulse sensor (A1) and the select button (pin 8), then emits a line of the form “DIRX:<–1|0|1> PULSE:<0–1023> SEL:<0|1>
” (these were the specific format that I chose as it helped me debug as well). The p5.js sketch listens for these lines, parses out the numeric values for direction, heart rate, and button state, and uses them to drive menu selection, character motion, and game logic. When the sketch determines that the physical LED indicators should change color (based on the game state), it sends right back a single‐word command: “RED\n”, “YELLOW\n” or “GREEN\n” via writeSerial(), and the Arduino responds by calling applyLeds() to set the appropriate digital outputs. I’m glad that I was able to implement a bidirectional serial communication where not only the Arduino sends information to the p5 sketch but also vice versa.

Reflection

I was really proud of how seamlessly the physical box integrated into the game environment; it didn’t feel tacked on, but like a natural extension of the game world I created .  I’m also proud that I soldered the LEDs to the wires because I was really afraid of soldering in the beginning. However, after a couple of tries on a sample, I was able to solder all 12 legs . In the future, I’d look to add a custom cursor that follows the joystick movements and actually wire up the Y-axis input so users can move that cursor freely around the screen. During playtesting, at least two people instinctively pushed the stick up and down—and, of course, nothing happened (since I hadn’t implemented Y at all)—which led to a couple of awkward pauses. Enabling full two-axis control would make the interaction more intuitive and keep the game flow smooth.

Images from Showcase:

THANK YOU PROFESSOR & EVERYONE FOR AN AMAZING CLASS FOR MY LAST SEMESTER!

User Testing

So, I tried letting people play the game. I realized that most of them didn’t really look at the instructions. At first, I let them figure it out on their own, and after the first round, they usually managed to understand how to play, which was nice. However, starting with the third user, I started making more of an effort to explain things minimally, answer questions when they came up, or suggest they check the “How to Play” manual.

People really liked the design I was going for, so I kept that aesthetic and focused on making it pop and align with the theme I envisioned. One thing I noticed is that the wires could be better hidden with blue tape (to match the box), so I’ll likely add that along with other cosmetic improvements. I also want to tweak the win conditions, like changing how hearts are deducted or how chips are counted, because I felt it was a little too easy to win. I’d also like to add an LED that reflects the state of the game for clearer feedback.

One issue I noticed was that players didn’t fully understand the joystick controls, especially since there’s no visible cursor. I had to explain to at least two people that the joystick only moves left and right, not up and down. To help with this, I plan to make the “How to Play” manual more detailed so users can understand the controls better from the start.

Video 

(I know the video is after I added the LEDs..imagine them without because those were actually last minute additions + also for some reason I can’t attach the video as media so I’ll add a google drive link :>)

https://drive.google.com/file/d/1lNbEfx_gUZIfRPWL0JcJXUbqzv9c68sE/view?usp=sharing

 

Final Idea

After lots of thought (in which initially, I wanted it to be a I’ve settled on a concept for a game titled “Did You Eat My Chips?”. The premise is a semi-thriller, played in a lighthearted tone, where the objective is to consume chips surreptitiously while avoiding detection by your slime sibling. My aim was to strike a balance, creating an experience that is engaging without being overtly frightening, yet distinct from typical game scenarios. This approach was driven by the desire to incorporate a pulse sensor, using it to gauge a player’s physiological response during time-sensitive, high-pressure(?) situations within the game and observe the impact of these moments on their state. This was largely inspired by my experience growing up with my sister where we would sneakily eat each other’s food; it always felt like a thriller movie.

Progress

So far, I’ve been busy sketching out some ideas for how the game will look. I’ve been doing some image drafts in Figma and Procreate to get a feel for the design style and what I want to aim for when I start building things in p5.js. On the hardware side, I’ve also been playing around with the flex sensor. That’s going to be the main way you control the game, so I’m just getting the hang of how it works and how to use it.

Visuals

What is left?

I still have to test the pulse sensor and see how to interact with the game. I have to borrow it from the IM lab so I’ll be testing that out today.

So for my final project, I initially brainstormed with a classmate to possibly incorporate something that has to do with the arduino wifi board, a sort of multi-player, but tactile game, like Among us. However, I sort of branched out a bit after thinking about it a bit more.

So, I’m planning to build something a little… revealing. Inspired by those classic lie detector tests you see in movies (minus the intense interrogation lol), I want to create a physical system that reacts to some basic physiological signs – specifically, changes in skin conductivity and heart rate.

The core idea is to use copper tape on a surface connecting a circuit when a user’s two fingers touch. When a person places their fingers on the tape, they’ll complete a circuit. Now, the “lie detector” part comes in because, supposedly, when people are stressed or nervous (like when they’re lying!), their palms tend to get a little sweatier. This increased moisture actually makes their skin more conductive, which the arduino can sense as a change in the electrical signal going through the copper tape. I’m not too sure how accurate this might be so I’ll have to test it on a prototype before building it properly.

In addition to that, I want to incorporate a pulse sensor (as recommended by my classmate Ali) to get a reading on the user’s heart rate. This is another classic indicator that can fluctuate under stress.

In terms of the p5, I envision a game or visualization on the screen that reacts to these physiological inputs. The p5 sketch will be listening for data from the arduino about skin conductivity (how “connected” you are to the copper tape) and your pulse. The game aspect is still a bit in development, but I’m thinking of something where your “truthfulness” (based on the sensor readings) influences the outcome. Maybe there’s a visual element that changes based on how conductive your skin is, or a character that reacts to your heart rate. I’m thinking a cheesy romance game or one of those truth or dare type scenarios. I’m not too sure yet. I’m planning to build a prototype to test the sensor readings this weekend.

My goal is to make the interaction engaging and the responses clear and timely. I want the person interacting with it to feel like the system is really “listening” to them, even if it’s just through these simple sensor inputs. Looking forward to diving deeper into this and seeing how it evolves.

Reading about the intersection between design and disability really got me thinking about many facets of design that I often ignore in my day to day life as an able-bodied person. For example, when I purchase makeup, let’s say, a face cream, I see what the lid or container looks like and whether it looks easy to pack, carry, and open (while also being hygienic). However, recently I saw a video about how a rare beauty blush lid was disability friendly since it had a circular grip that made opening it much easier.

Similarly, in the reading they talk about something called trickle-down effect where designs are expected to eventually find its way to disabled people in a way that is accesible to them. It then mentions that the splint medical device to chair furniture pipeline challenges this idea. Usually we think innovation flows from mainstream products down to specialized disability products — like how smartphone tech might eventually make its way into accessible devices years later, but usually as watered-down technological versions. Although I understand that this is also a matter of making profit, it seems to me from the reading that this is actually a missed opportunity since looking at things from a different angle can also birth innovation for the masses in unexpected ways.

This makes me wonder about whether we approach software design in a very constrained way. For instance, we’re often taught to build for the “average users” first, then maybe add accessibility features later if we have time (which let’s be honest, most student projects never get to). But what if we started with accessible design principles from the beginning? Would we end up with more innovative solutions that actually work better for everyone? This is the question I was left with, that I still don’t have an answer to.

Before diving into the first task, we began by loading the sample Arduino and p5.js code from the previous class. We then read through each line to see how Arduino connects and communicates with p5. This served as a helpful foundation to jumpstart our workflow.

Task 1:

After reviewing the code and identifying the necessary components, we proceeded to build the circuit using a sliding potentiometer. Using analogRead from pin A0, we captured the potentiometer’s data and sent it to p5. The values ranged from 0 to 900, so we divided them by 2.25 to map them to the x-position on the canvas, ensuring smooth and accurate movement. A global variable ‘pos’ is updated and mapped into the x position of the ellipse.

Here is the p5.js code:

let pos = 0;
function setup() {
  createCanvas(400, 400);
}

function draw() {
  background(220);
  ellipse(pos,200,100,100);
}

function keyPressed() {
  if (key == " ") {
    setUpSerial(); // setting up connection between arduino and p5.js
  }
}

function readSerial(data) {

  if (data != null) {
    let fromArduino = trim(data) + "\n";
    pos = fromArduino/2.25; // to map 0 to 900 in the right range in p5.js (400 by 00) canvas
    writeSerial(sendToArduino);
  }
}

and the arduino code:

int sensor = A0;

void setup() {
  Serial.begin(9600);
  pinMode(LED_BUILTIN, OUTPUT);
  digitalWrite(led, HIGH);
  delay(200);
  digitalWrite(led, LOW);

  // starting the handshake
  while (Serial.available() <= 0) {
    digitalWrite(LED_BUILTIN, HIGH); // on/blink while waiting for serial data
    Serial.println("0"); // send a starting message
    delay(300);            // wait 1/3 second
    digitalWrite(LED_BUILTIN, LOW);
    delay(50);
  }
}

void loop() {
  digitalWrite(LED_BUILTIN, LOW);
  int sensor = analogRead(A0);
  Serial.println(sensor); // sending sensor information to p5.js
  
}

Here’s the video of it in action:

https://drive.google.com/file/d/1kT32H353kkMX_5HeKBphHf4Cxy-xhMF_/view?usp=sharing

 

Task 2:

We decided to create an input box where if the user inserted a number between 0-255 and pressed enter, it would then reflect the corresponding brightness onto the blue LED on the breadboard. It was a relatively simple implementation that required very minimal code changes.

Here’s the p5.js code:

let ledval = 0;
let input;

function setup() {
  createCanvas(400, 400);
  input = createInput('');
  input.position(120, 100);
}

function draw() {
  background(220);
}

function keyPressed() {
  if (key == " ") {
    setUpSerial(); // setting up connection
  }
}

  if (data != null) {
    let fromArduino = trim(data);
    let sendToArduino = input.value() + "\n";  
    writeSerial(sendToArduino);
  }
}

and the arduino code:

int led = 3;

void setup() {
  Serial.begin(9600);
  pinMode(LED_BUILTIN, OUTPUT);
  pinMode(led, OUTPUT);

  // Blink them so we can check the wiring
  digitalWrite(led, HIGH);
  delay(200);
  digitalWrite(led, LOW);

  // start the handshake
  while (Serial.available() <= 0) {
    digitalWrite(LED_BUILTIN, HIGH); // on/blink while waiting for serial data
    Serial.println("0,0"); // send a starting message
    delay(300);            // wait 1/3 second
    digitalWrite(LED_BUILTIN, LOW);
    delay(50);
  }
}

void loop() {
  // wait for data from p5 before doing something
  while (Serial.available()) {
    digitalWrite(LED_BUILTIN, HIGH); // led on while receiving data

    int ledVal = Serial.parseInt();
    if (Serial.read() == '\n') {
      analogWrite(led, ledVal);
      delay(5);
    }
  }
  digitalWrite(LED_BUILTIN, LOW);
  
}

and finally, the video of it in action:

https://drive.google.com/file/d/1eMi1d_3H6abYxtYwyEpybCnZB7-fTVXF/view?usp=sharing

Task 3:

For the last task, we needed to first open up and examine the given gravity wind code. We identified two key things we could alter that would complete the given task at hand: the “wind.x” variable and the “(position.y > height-mass/2)” IF statement. We could map the analog value we read in from pin A0 to the wind.x position to alter the ball’s position on the x axis and since the aforementioned IF statement indicates when the ball has touched the ground, we could simply sneak in a line that sets a boolean flag to true and sending this to arduino and performing a digitalWrite (replacing the previous analogWrite from the input()).

Here’s how we did it in p5.js:

let velocity;
let gravity;
let position;
let acceleration;
let wind;
let drag = 0.99;
let mass = 50;
let floor = false;

function setup() {
  createCanvas(640, 360);
  noFill();
  position = createVector(width/2, 0);
  velocity = createVector(0,0);
  acceleration = createVector(0,0);
  gravity = createVector(0, 0.5*mass);
  wind = createVector(0,0);
}

function draw() {
  background(255);
  applyForce(wind);
  applyForce(gravity);
  velocity.add(acceleration);
  velocity.mult(drag);
  position.add(velocity);
  acceleration.mult(0);
  ellipse(position.x,position.y,mass,mass);
  if (position.y > height-mass/2) {
      velocity.y *= -0.9;  // A little dampening when hitting the bottom
      position.y = height-mass/2;
      floor = true; // light up the LED!
    }
  
}

function applyForce(force){
  // Newton's 2nd law: F = M * A
  // or A = F / M
  let f = p5.Vector.div(force, mass);
  acceleration.add(f);
}

function keyPressed(){
  if (key == " ") {
    setUpSerial(); // setting up serial connection
  }
  
  if (keyCode==LEFT_ARROW){
    wind.x=-1;
  }
  if (keyCode==RIGHT_ARROW){
    wind.x=1;
  }
  if (key=='s'){ // changed from space to 's' since SPACEBAR is used to initiate serial connection pairing to arduino
    mass=random(15,80);
    position.y=-mass;
    velocity.mult(0);
  }
}

function readSerial(data) {
  if (data != null) {
    let fromArduino = trim(data);
    wind.x = map(fromArduino, 0, 912, -2, 2); // mapping sensor's analog value to ball's wind x axis value

    let sendToArduino = Number(floor) + "\n";
    
    writeSerial(sendToArduino);
    floor = false; // turning off blue LED
  }
}

*We used the Number() function to convert the boolean flag value to an integer value since initially we were encountering issues where it was not actually being send as a numeric value to turn on the LED in digitalWrite.

and the arduino code:

int sensor = A0;
int led = 3;

void setup() {
  Serial.begin(9600);
  pinMode(LED_BUILTIN, OUTPUT);
  pinMode(led, OUTPUT);

  // Blink them so we can check the wiring
  digitalWrite(led, HIGH);
  delay(200);
  digitalWrite(led, LOW);

  // start the handshake
  while (Serial.available() <= 0) {
    digitalWrite(LED_BUILTIN, HIGH); // on/blink while waiting for serial data
    Serial.println("0,0"); // send a starting message
    delay(300);            // wait 1/3 second
    digitalWrite(LED_BUILTIN, LOW);
    delay(50);
  }
}

void loop() {
  // wait for data from p5 before doing something
  while (Serial.available()) {
    digitalWrite(LED_BUILTIN, HIGH); // led on while receiving data

    int ledVal = Serial.parseInt();
    if (Serial.read() == '\n') {
      digitalWrite(led, ledVal);
      delay(5);
    }
  }
  digitalWrite(LED_BUILTIN, LOW);
  int sensor = analogRead(A0);
  Serial.println(sensor);
  
}

Finally, here’s the video of the final product (we have two to demonstrate both the analog and digital capacity):
1. https://drive.google.com/file/d/1TcwYwz7HcyUobzH0MwLQ3P1pbf2rw8BR/view?usp=sharing

2. https://drive.google.com/file/d/1Ydz9OjHuqt8VPypLTQhBYDtB-ShDBGmk/view?usp=sharing

VOILÁ!

“A Brief Rant on the Future of Interaction Design” made several points and rebuttals to responses that I resonated with. First, the rant itself reminded me of a user testing gig that I did back when I was in New York last semester, doing my studyaway. Although I don’t think I can disclose many details, it was vaguely about swipe mechanisms that would move screens based on a thumb and tap gesture I would do in the air with a watch on. Although I think it’s slightly different from the tech that is shown in Microsoft’s video, since it involves tactile elements, it still encapsulates similar sentiments of swipes essentially being the future of human interaction with tech. Before reading this article, I had never truly considered the idea that our “future” is something we actively choose. This opening thought lingered with me and prompted a deeper reflection. I realized that my perspective on innovation was inherently capitalistic. I had always viewed technological advancements as merely profit-driven responses to market demands. I’m drawn to the idea that we, as individuals within society, are the ones who shape market demand. It’s empowering to think about the influence we hold, and it makes me want to reclaim more autonomy in how I see my role. I’d like to challenge myself to think bigger—to strive to be, or at least attempt to become, more of a pioneer in shaping the world around me.

Furthermore, his responses to the comments on his rant were very coherent. I agree that you don’t need to propose an alternative to point out when something is wrong—doing otherwise risks complacency with the status quo and undermines critical thinking. This ties into a broader issue I’ve noticed: the way technology, particularly AI, is shaping our cognitive abilities. For instance, the quote about children mastering iPads but struggling with shoelaces, or his point on how it is like understanding “Cat in the Hat” but not “Hamlet”, highlights how our tools are often designed for simplicity rather than depth. While accessibility is important, oversimplifying tools for mass appeal can lead to shorter attention spans and a decline in critical thinking. This echoes his point in the article: tools are meant to amplify human abilities, yet the trend of dumbing them down risks doing the opposite—handicapping us rather than empowering us.

Concept

Our musical instrument is based off a combination of a both a mechanical and digital noise machine–controlled with a surprisingly satisfying potentiometer that allows us to modify the beats per minute (BPM), and a button that changes the octave of our instrument. We had fun with the relationship between the buzzer and our mechanical servo which produces the other noise, where the two are inversely proportional! In other words,  as the BPM of the buzzer increases, the servo slows down! And vice versa.

Figuring out the right code combination was a bit tricky at first, as we first made the mistake of nesting both the servo trigger and the buzzer tone in the same beat conditional. This meant that we fundamentally could not separate the motor from the buzzer tones. To resolve this, we ended up using two triggers–a buzzer and a motor trigger–which both run independently.

The code below shows how we ended up resolving this problem. Once we implemented this fix, we were able to celebrate with our somewhat perplexing instrument, that requires the user to complement the buzzers various beeping with the servos mechanical changes.

// --- 3. Perform Actions based on Timers ---

// --- Action Group 1: Metronome Click (Buzzer) ---
if (currentMillis - previousBeatMillis >= beatInterval) {
  previousBeatMillis = currentMillis; // Store the time of this beat

  // Play the click sound
  tone(BUZZER_PIN, currentFreq, CLICK_DUR);

  // Print beat info
  Serial.print("Beat! BPM: ");
  Serial.print(currentBPM);
  Serial.print(" | Freq: ");
  Serial.print(currentFreq);
  Serial.print(" | Beat Interval: ");
  Serial.print(beatInterval);
  Serial.println("ms");
}

// --- Action Group 2: Servo Movement Trigger ---
if (currentMillis - previousServoMillis >= servoInterval) {
  previousServoMillis = currentMillis; // Store the time of this servo trigger

  // Determine Target Angle
  int targetAngle;
  if (servoMovingToEnd) {
    targetAngle = SERVO_POS_END;
  } else {
    targetAngle = SERVO_POS_START;
  }

  // Tell the servo to move (NON-BLOCKING)
  myServo.write(targetAngle);

  // Print servo info
  Serial.print("Servo Trigger! Target: ");
  Serial.print(targetAngle);
  Serial.print("deg | Servo Interval: ");
  Serial.print(servoInterval);
  Serial.println("ms");

  // Update state for the next servo trigger
  servoMovingToEnd = !servoMovingToEnd; // Flip the direction for next time
}

 

Reading Tom Igoe’s articles helped me understand physical computing in a more thoughtful way, beyond just wiring things up and making LEDs blink (which to be fair, I do think is a preconceived notion that I have been gradually worn down through consecutive projects and readings in the class). In the first article (“Physical Computing’s Greatest hits and misses”), I appreciated how he explained that physical computing is really about building systems that can sense and respond to the world around them. This also reminded me of the discussion we had a couple of weeks ago on what defines interactivity. It also stood out to me because it made me think differently about my lie detector project — that I wasn’t just turning on lights but also creating a playful, real-world interaction in response to physical stimuli.

The second article (“Making Interactive Art: Set the Stage, Then Shut Up and Listen”) focused more on the nature of interaction, and it really resonated with me. He explains that real interaction happens when the user has control over the timing and nature of the input, rather than just pressing a button and triggering a fixed response. That reminded me of how, at first, my green LED would turn on as long as the button was pressed, even if there wasn’t enough pressure. It didn’t feel right, and reading this made me realize why. The system was reacting too easily, without enough real input from the user (this is also my re-interpretation of the event as well since this connection was mentally made a bit later). Adding a lower threshold fixed that and made the interaction more accurate. I think, overall, I’m left with concepts and ideas that can serve as tools to frame my projects and skills in this class, especially in the aspect of understanding interactivity and intentionality.

Ideation

When I first started thinking about what to do for this project, I wanted to create something that was a bit playful (a bit gamified?), while also showcasing how analog and digital inputs can work together in a meaningful way. I landed on the idea of a “Human Lie Detector” — a lighthearted concept that mimics stress or honesty detection based on physical responses. This was inspired by an existing device that I often saw being used in Korean drinking games. For that machine, small electric shocks are sent the person who is attached to the machine based on the sweat produced by someone’s hand. Since I don’t intend to cause pain to any body I decided to use a sensor that could imitate the just of the lie detector idea. So, the pressure sensor became the core of the project, as it allowed me to measure subtle changes in finger pressure, which I imagined could simulate a person’s nervousness or calmness when answering a question. By combining this analog input with a digital button (to simulate the moment a person gives an answer), I could create a clear trigger point and decision logic for controlling the LEDs. The red and green LEDs became symbolic indicators: red for “lie” and green for “truth,” depending on how much pressure was applied.

Implementation

To bring the idea to life, I started by wiring up the core components: a pressure sensor, a pushbutton, and two LEDs (one red and one green). I used a 10k resistor to build a voltage divider circuit with the pressure sensor, allowing me to read its analog values through pin A0 on the Arduino. The pushbutton was connected to digital pin 2 and configured with Arduino’s internal pull-up resistor to simplify the circuit. For output, I connected the red LED to pin 8 and the green LED to pin 9. You can see the demo video below (sound ON):
IMG_0052

Here’s the schema I drew :

Code Highlights

One small but important part of my code was adding a mid-range threshold to control the green LED. At first, I noticed that the green LED would turn on even when there was barely any pressure on the sensor, just because the button was pressed. To fix this, I added a lower bound to the pressure range. So now, the green LED only turns on if the pressure is somewhere between a low (50) and high threshold (500). This simple change made the interaction feel a lot more accurate and intentional. It wasn’t a complex fix, but it solved an annoying issue and made the project work the way I wanted.

int threshold = 500;         
int lower = 50;
.
.
.
if (buttonState == LOW) {
  if (pressureVal > threshold) {
    digitalWrite(redLED, HIGH);
    digitalWrite(greenLED, LOW);
  } else if (pressureVal > lower) {
    digitalWrite(redLED, LOW);
    digitalWrite(greenLED, HIGH);
  } else {
    digitalWrite(redLED, LOW);
    digitalWrite(greenLED, LOW);
  }
} else {
  digitalWrite(redLED, LOW);
  digitalWrite(greenLED, LOW);
}

initially wrong code:

if (buttonState == LOW && pressureVal > threshold) {
  digitalWrite(redLED, HIGH);
  digitalWrite(greenLED, LOW);
}
else if (buttonState == LOW && pressureVal < lower) {
  digitalWrite(greenLED, HIGH);   // always turned on green LED if button is pressed
  digitalWrite(redLED, LOW);
}
else {
  digitalWrite(redLED, LOW);
  digitalWrite(greenLED, LOW);
}

Reflection

Interestingly, the most challenging part of this assignment wasn’t the coding or wiring; it was actually drawing the schematic. I found it surprisingly difficult to sketch the circuit accurately without having the physical components in front of me. To overcome this, I brought the components to my workspace from the IM lab and started building the circuit first, using the multiple schematics we learned about in class as references. I looked closely at the examples for the LEDs, the button, and the pressure sensor, and then worked backwards from my working setup to draw the schema. While this might not have been the intended approach for the assignment, it turned out to be incredibly educational for me. I ended up researching circuit diagram symbols and layout conventions on my own, which gave me a much better understanding of how to read and create schematics. In a way, doing it “backwards” helped me build my confidence on circuit drawings. So next time, (hopefully) I’ll feel ready to start with the schematic before jumping into the circuit.