Category: Spring 2021

In summary, my final project was a series of failed experiments.

My original goal was to use the p5.speech library to create something for Zoom classes. The user would be alerted every few minutes if they ended up falling asleep on their pillow using a capacitative touch sensor. I also hoped to record the transcript of the meeting/class in session and play some sort of alarm if the user’s name was said, urging them to get up.

I ended up playing around with this idea, using the p5.serialcontrol program to communicate between p5.js and Arduino. While it is cool that this exists, it was a bit clunky in terms of workflow in terms of connecting ports, etc. Additionally, I wasn’t fully comfortable using the p5.speech library to record transcripts. Other than the technical problem of pipes breaking, I didn’t think it was okay to record the transcripts due to user privacy. Additionally, the speech recognition was a bit better than expected but probably still only a 65% recognition of the names of friends I tried, so I thought it would still make microaggressions in its failure to understand certain friends’ names, etc.

Thus, I needed to repivot my idea. I really wanted to keep the capacitative touch element of the project because this was something I had never been exposed to. I decided to maintain the original spirit of the project as well…something that could help me personally. I have lots of low energy periods due to insomnia and perpetual migraines, but I still have to do my best to complete my work. However, amidst the low energy, I need to make sure I don’t completely fall asleep. It’s not very predictable so setting an alarm ahead of time is not something I would know to do. Thus, I thought the capacitative touch sensor could be useful to see if I do end up falling asleep–having a sensor in my pillow could detect this and set into motion a way to wake me up.

From there I imagined having a combination of vibrations and sound to wake me up. I originally considered using an LED to show whether the sensor was working, but thought bright lights and migraines aren’t the greatest of combinations. In our class check-in, Aaron suggested using motors to create the vibrations.

I was also trying to brainstorm what the Processing program could look like.  Having structured work time and making sure I take breaks is important to make sure I don’t get too fatigued. Thus, I thought it could be interesting to make my own version of the Pomodoro method. Additionally, doing work tends to bring out a lot of negative thought patterns, so I wanted to integrate some positive energy into the program. I found this affirmation API that I could pull a random message from.

Creating the scrollbars was a bit manual, but I had help from the Scrollbar Processing documentation example. I modified it to make more simple and more to the aesthetic of the program and used a map function instead of calculating the ratio of the slider’s position to the slider’s width. I also added text to know what the current value of the slider is.

I designed a color palette using Coolors and tested it for accessibility for various visual abilities using their tool.

Creating the stopwatch display was a bit tricky because I wanted the new time to appear any time the scrollbar was moved. I ended up creating a helper function to calculate the new timer value. This would be used in the calculateTime function. Another issue I had was displaying the time in a way easy for the reader to understand. For instance, simply using the numerical value of minutes and seconds wouldn’t work for the 60 second mark (the user would expect to see 00 instead of 60) and for the second ticks less than 10 (the user would expect to see :01 instead of :1). I ended up just using an if statement to check for these edge cases, but I’m sure there is a smarter way to do this.

The final aspect of the front page of the program was the affirmations. Using the http.requests library was pretty straightforward, but the response was not in a form conducive for using JSON. I ended up just creating a substring of what was returned to include only the actual message part.

I also wanted to include another element of interaction with the Arduino. Thus, I thought I could create something for the break period of the program that involved using a hardware component. I considered doing a dance break–playing music with dancing LEDs, but had done something similar for one of the weekly projects and wanted to learn something new. I also thought it could be interesting to do a small mini game using the buttons on Arduino as controls, but I’m a bit tired of making games in all honesty. Thus, I decided to do something involving drawing. From the moment we starting working with potentiometers, they reminded me of the little knobs on those toys Etch-a-Sketch–I decided to build a mini Etch-a-sketch for the break part of the program!

In terms of my hardware, I used the capacitative touch sensor. I was inspired by the Love You Pillow project on the Arduino Hub to use foil for the sensor. I had to experiment a bit with finding the right threshold, but overall, it worked pretty smoothly and consistently.  I used the motor shield to connect two motors to create the vibrations…the code and wiring was heavily remixed from the class we spent on motors. I also used a piezo buzzer and two potentiometers.

I also experimented a bit with the LED display–I was thinking of randomly sending messages like “Hydrate” and “Stretch,” but I ran out of pins to include it in the final setup, already using some of the analog pins as digital pins.

Overall, I’m not very content with the project. I know I’m capable of better work, but for where I’m at in terms of energy, I think I did my best. There’s definitely a lot of room for improvement. It could be cool to include the LCD Display with user choice of messages on the Processing screen (hydrate, eye break, stretch, breathe deeply). It could be cool to have more options for what to do on the break. I realize I don’t really like drawing with the etch a sketch, so it could be more useful to explore other potential activities like making a mini piano or game, etc. I think the user interface of the etch a sketch could be made more interesting by having more user input in choosing a color etc.

Here’s a walkthrough of my project:

CODE:

Arduino:

#include <CapacitiveSensor.h>
#include <SparkFun_TB6612.h>
//#include <LiquidCrystal.h>      

#define AIN1 2
#define BIN1 7
#define AIN2 4
#define BIN2 8
#define PWMA 5
#define PWMB 6
#define STBY 9

 
//LiquidCrystal lcd(19, 18, 17, 16,11, 3);   
// these constants are used to allow you to make your motor configuration 
// line up with function names like forward.  Value can be 1 or -1
//from motor library example code
const int offsetA = 1;
const int offsetB = 1;
int duration = 2000;
Motor motor1 = Motor(AIN1, AIN2, PWMA, offsetA, STBY);
Motor motor2 = Motor(BIN1, BIN2, PWMB, offsetB, STBY);
//pressed boolean indicates whether the pillow sensor senses touch
int pressed = 0;
long timer;
//threshold for pillow sensor; if greater than threshold, there is touch
int threshold = 30;
//timeLength variable from first scrollbar in Processing program
int timeLength;

CapacitiveSensor   cs_4_2 = CapacitiveSensor(10,12);        //12 is sensor pin
//count indicates what level of intensity to be woken up
int count = 0;
int piezoPin = 3;

int pot1 = A4;
int pot2 = A5;

void setup()                    
{
   cs_4_2.set_CS_AutocaL_Millis(0xFFFFFFFF);     //from capacitive sensor library
   Serial.begin(9600);
   Serial.println(0,0);

//   lcd.begin(16, 2);                 //tell the lcd library that we are using a display that is 16 characters wide and 2 characters high
//  lcd.clear();                      //clear the display
}

void loop()  {
 while (Serial.available()) {
  //read from Processing
  timeLength = Serial.parseInt();
  //turns into minutes, change this value if you want the interval in seconds
  timeLength *= 60000;

   if (Serial.read() == '\n') {
    //sends potentiometer values to processing
  int val1 = analogRead(pot1);
  Serial.print(val1);
  Serial.print(",");
  int val2 = analogRead(pot2);
 
  Serial.println(val2);
   }  
  }

//   lcd.setCursor(0, 1);              
//  lcd.print(timeLength);
//  
    long start = millis();
    long cap =  cs_4_2.capacitiveSensor(30);
    //if the sensor value is greater than the threshold and has not yet been activated
    //starts sensor and level count
    if (cap > threshold && pressed != 1){
      pressed = 1;
      timer = millis();
      count = 0;

    } else if (cap > threshold && pressed == 1) {
      pressed = 1;
      //if sensor value is less than threshold, turn pressed off
    } else if (cap < threshold && pressed == 1) {
      pressed = 0;
    }
  if (pressed == 1) {
//   lcd.setCursor(0, 0);              
//  lcd.print("sensor on");
//  lcd.setCursor(0, 1);              
//  lcd.print(timeLength);

//after every X minutes
  if (millis() - timer >= timeLength) {
    timer = millis();
   level(count);
   count += 1;
  }
  }  
}
//function creates intensity of response to wake up depending on level
void level(int count) {
  switch(count){
  case 0:
    vibrate(0);
    break;
  case 1:
    vibrate(1);
    break;
  case 2:
    vibrate(2);
    break;
  case 3:
    vibrate(2);
    tone(piezoPin, 200, duration);
    break;
  default:
    vibrate(3);
    tone(piezoPin, 500, duration);
    break;
  }
}
//vibrate function to determine speed based on level
void vibrate(int level) {
  int sp = 0;
  if (level == 0) {
    sp = 75;
  } else if (level == 1) {
    sp = 120;
  } else if (level == 2) {
    sp = 180;
  } else {
  
    sp = 255;
  }
  //moves the motors
  motor1.drive(sp,duration);
  motor2.drive(sp,duration);
  //stops motors
  motor1.brake();
  motor2.brake();
}

Processing:

import http.requests.*;
import processing.serial.*;

Serial myPort;

Scrollbar bar1;
Scrollbar bar2;

int first = 0;
int second = 0;
float lastX = 0;
float lastY = 0;
//Button play;
//Button dance;
//Button draw;

int seconds;
int minutes;

int timer;
int startTime;

PFont lucida;
PFont avenir;
int state = 0;
String response;
void setup() {
    size(1200, 800);
    lucida = createFont("Lucida Bright Italic", 36);
    avenir = createFont("Avenir", 12);
    textAlign(CENTER);
    background(#D08C60);
    //requests affirmation from api
    GetRequest affirm = new GetRequest("https://dulce-affirmations-api.herokuapp.com/affirmation");
    affirm.send();
    response = affirm.getContent();
    //finds string of affirmation
    response = response.substring(12, response.length() - 3);
    
    //Scrollbar(x, y, width, height, min val, max val)
    bar1 = new Scrollbar(20, height/10, 300, 16, 1, 60);
    bar2 = new Scrollbar(20, height/10 + 40, 300, 16, 1, 60);
    //finds initial time of scrollbars
    adjustTime();
    
     //play = new Button("PLAY", width/2 - 150, height/3, 300, 80, 2);
     //dance = new Button("DANCE", width/2 - 150, height/3 + 120, 300, 80, 3);
     //draw = new Button("DRAW", width/2 - 150, height/3 + 240, 300, 80, 4);
     
     //sets up serial communication
     String portname=Serial.list()[2];
      myPort = new Serial(this,portname,9600);
      myPort.clear();
      myPort.bufferUntil('\n');
      delay(1000);

}

void draw(){
//if home screen
  if (state == 0) {
    background(#D08C60);
    //scrollbars - draw and update if user input
    bar1.update();
    bar2.update();
    bar1.display();
    bar2.display();
    fill(#F9F1DC);
    //displays affirmation
    textFont(lucida);
      text("A Reminder...", width/2, height - 140);
      text(response, width/2, height - 100);
    textAlign(LEFT);  
    textFont(avenir);
    text("intervals to be woken up in", 340, height/10 + 5);
    text("chunk of time to work for", 340, height/10 + 45);
    //calculates and displays time
    calculateTime();
  //} else if (state == 1) {
  //    background(#F1DCA7);
  //    textFont(avenir);
  //    textSize(72);
  //    fill(#997B66);
  //    text("Time for a break!", width/2, height/8);
  //    play.display();
  //    dance.display();
  //    draw.display();
  //    play.update();
  //    dance.update();
  //    draw.update();
  } else if (state == 2) {
    //drawing screen game
      textFont(avenir);
      textSize(42);
      fill(#997B66);
      text("Time for a break. Use the potentiometers to etch a sketch", width/2, height/8);
      textSize(36);
      text("Click ENTER to clear and SPACE to get back to work.", width/2, height/8 + 65);
      //maps potentiometer value to some coordinate on the screen
      float x = map(first, 0, 1023, 0, width);
      float y = map(second, 0, 1023, height/4, height);
      //finds random color for the line
      stroke(random(0,255), random(0,255), random(0,255));
      //idea for saving last coordinate from this project: http://graysonearle.com/edu/physcom/etch-a-sketch-arduino-processing/
      line(x, y, lastX, lastY);
      lastX = x;
      lastY = y;
  } 
}
//scrollbar class for user input on first screen
class Scrollbar {
  //slider width and height
  int sw;
  int sh;
  float xpos;
  float ypos;
  //the slider position (ellipse)
  float slider_pos;
  float newpos;
  float sliderMin;
  float sliderMax;
  boolean mouseOnTop;
  boolean moved;
  boolean posChanged;
  //min and max values of slider
  int sValMin;
  int sValMax;
  
  Scrollbar(float x, float y, int w, int h, int minVal, int maxVal) {
    sw = w;
    sh = h;
    xpos = x;
    ypos = y - sh/2;
    slider_pos = xpos + sw/2 + sh/2;
    newpos = slider_pos;
    sliderMin = xpos;
    sliderMax = xpos + sw;
    sValMin = minVal;
    sValMax = maxVal;
    posChanged = false;
  }
  
  void update() {
    //if user mouseX, mouseY is on top of the scrollbar
    if (overEvent()) {
      mouseOnTop = true;
    } else {
      mouseOnTop = false;
    }
    //if mouse held and on top, then the scrollbar is being moved
    if (mousePressed && mouseOnTop) {
      moved = true;
    }
    
    if (!mousePressed) {
      moved = false;
    }
    //keep within the range of the slider
    //code remixed from https://processing.org/examples/scrollbar.html
    if (moved) {
      newpos = constrain(mouseX - sh/2, sliderMin, sliderMax);
    }
    //if slider value has changed
    if (abs(newpos - slider_pos) > 1) {
      slider_pos += (newpos -slider_pos);  
      adjustTime();
    }
  }
  
  float constrain(float val, float smin, float smax) {
    return min(max(val, smin), smax);
  }
  boolean overEvent() {
    if (mouseX > xpos && mouseX < xpos + sw && mouseY > ypos && mouseY < ypos + sh) {
      return true;
    } else {
      return false;
    }
  }
  //creates the scrollbar
  void display() {
    noStroke();
    fill(200);
    rect(xpos, ypos, sw,sh);
    fill(#9B9B7A);
    //ellipse is the slider part of the bar
    ellipse(slider_pos, ypos + sh/2, 1.5 *sh, 1.5*sh);
    fill(255);
   textAlign(CENTER);
    textFont(avenir);
    //label of the actual value
    text(getVal(), slider_pos, ypos + sh/1.5);
  }
  
  int getVal() {
    float mapVal = map(slider_pos, xpos, xpos + sw, sValMin, sValMax);
    return int(mapVal);
  }
 
}
//get value from scrollbar and create a new start time
void adjustTime() {
  timer = bar2.getVal();
  startTime = millis();
}

void calculateTime() {
  seconds = 60 - ((millis() - startTime)/1000) % 60;
  minutes = timer - 1 - ((millis() - startTime) / 60000);
  //displays time in middle of screen
  textFont(avenir);
  textSize(240);
  textAlign(CENTER, BOTTOM);
  if (seconds == 60) {
    text(minutes + 1 + ":00", width/2, 2*height /3);
  } else if (seconds < 10) {
  text(minutes + ":0" + seconds, width/2, 2*height /3);
  } else {
    text(minutes + ":" + seconds, width/2, 2*height /3);
  }
  //when timer runs out, move to next screen
  if (seconds == 60 && minutes <= 0) {
    state = 2;
    background(#F1DCA7);
  }
}
//class Button {
//  String msg;
//  int xpos;
//  int ypos;
//  int w;
//  int h;
//  int cntl;
  
//  Button(String words, int x, int y, int w_, int h_, int st) {
//    msg = words;
//    xpos = x;
//    ypos = y;
//    w = w_;
//    h = h_;
//    cntl = st;
//  }
  
//  void display() {
//    noStroke();
//    if (mouseX > xpos && mouseX < xpos + w && mouseY > ypos && mouseY < ypos + h) {
//      fill(#7A6352);
//    } else {
//      fill(#997B66);
//    }
//    rect(xpos, ypos, w, h);
//    textAlign(CENTER, CENTER);
//    fill(#F1DCA7);
//    text(msg, xpos + w/2, ypos + h/3);
//  }
  
//  void update() {
//    //if button is clicked
//    if (mousePressed && mouseX > xpos && mouseX < xpos + w && mouseY > ypos && mouseY < ypos + h) {
//      state = cntl;
//      background(#F1DCA7);
//    }
//  }
//}

void keyPressed() {
  if (state == 2) {
    //returns to home screen if SPACE is clicked
    if (key == ' ') {
      state = 0;
    }
    //refreshes background if ENTER clicked to allow for new drawing
    else if (key == ENTER) {
    background(#F1DCA7); 
    }
  }
}
void serialEvent(Serial myPort){
 String s=myPort.readStringUntil('\n');
 println(s);
  s=trim(s);
  if (s!=null){ 
    //gets value of potentiometer
    int values[]=int(split(s,','));
    first = values[0];
    second = values[1];
  }
  //writes to arduino the scrollbar value
  myPort.write(bar1.getVal() + "\n");
}