Author: Fausto Fang

Idea 1:

For my first (and main) idea, I want to work with the concept of IOT and Home Automation. In this case, as cliche as it might sound, I would like to have a central control unit for every interactive part of our houses. What this allows is to have direct control over every item in the house, which at the same time allows me to make the main part of this project, which is an assistant that will be able to act on every interactive item of the house. The multiple ways of connecting to it will allow a variety of options and ideas.

Idea 2:

For this project, the idea is to create a combination between 2 of the most popular game genres this year: rhythm games and combination games. The concept is to create a rhythm game that every time you get a perfect hit, a  ball will fall, which will allow the user to drop the balls in order to combine them and achieve a higher score.

1. Make something that uses only one sensor  on Arduino and makes the ellipse in p5 move on the horizontal axis, in the middle of the screen, and nothing on arduino is controlled by p5

   let rVal = 0;
   let alpha = 255;
   let left = 0;
   let right = 0;
   
   
   
   
   function setup() {
   createCanvas(640, 360);
   noFill();
   //create an ellipse with x position changing and y (300) position constant
   }
   
   
   function draw() {
   background(255);
   
   
   
   
   
   
   if (!serialActive) {
   fill(0)
   text("Press Space Bar to select Serial Port", 20, 30);
   } else {
   fill(0)
   text("Connected", 20, 30);
   }
   let xPos = 300;
   xPos = map(alpha, 0, 1023, 0,640);
   ellipse(xPos, 150, 50, 50);
   
   
   console.log(alpha);
   }
   
   
   function keyPressed() {
   if (key == " ") {
   // important to have in order to start the serial connection!!
   setUpSerial();
   }
   }

    

   For this question, we decided to create a simple ellipse that changes direction based on the values from the potentiometer.

2. make something that controls the LED brightness from p5t
   function keyPressed() {
   if (key == " ") {
   // important to have in order to start the serial connection!!
   setUpSerial();
   }
   if (key== "1"){
   left = 50;
   }
   
   
   if (key== "2"){
   left = 100;
   }
   if (key== "3"){
   left = 255;
   }
   if (key =="4"){
   right = 50;
   }
   if (key =="5"){
   right = 100;
   }
   if (key =="6"){
   right =255;
   }
   }

    

   This one was pretty simple. Pressing the keys 1-6 increases/decreases the brightness of the left or right LED light. 1-3 correspond to the left, while 4-6 correspond to the right led.

3. take the gravity wind example (https://editor.p5js.org/aaronsherwood/sketches/I7iQrNCul) and make it so every time the ball bounces one led lights up and then turns off, and you can control the wind from one analog sensor.

let rVal = 0;
let alpha = 255;
let left = 0;
let right = 0;
let velocity;
let gravity;
let position;
let acceleration;
let wind;
let drag = 0.99;
let mass = 20;


function setup() {
createCanvas(640, 360);
noFill();
position = createVector(width/2, 0);
velocity = createVector(0,0);
acceleration = createVector(0,0);
gravity = createVector(0, 0.1*mass);
wind = createVector(0,0);
}


function draw() {
background(255);
applyForce(wind);
applyForce(gravity);
velocity.add(acceleration);
velocity.mult(drag);
position.add(velocity);
acceleration.mult(0);
ellipse(position.x,position.y,mass,mass);
if (position.y > height-mass/2) {
velocity.y *= -0.9; // A little dampening when hitting the bottom
position.y = height-mass/2;


}
console.log(position.y);
if(position.y >= 330){
left = 1;
}
else if(position.y <= 330){
left = 0;
}
console.log(rVal);
wind.x = map(alpha, 0, 1023, -1, 1);








if (!serialActive) {
fill(0)
text("Press Space Bar to select Serial Port", 20, 30);
} else {
fill(0)
text("Connected", 20, 30);
}
}
function applyForce(force){
// Newton's 2nd law: F = M * A
// or A = F / M
let f = p5.Vector.div(force, mass);
acceleration.add(f);
}


function keyPressed() {
if (key == " ") {
// important to have in order to start the serial connection!!
setUpSerial();
}
if(key == "1"){
left = 1;
}


}

For changing the wind direction, we followed similar logic to the first question. The ball changes direction based on values mapped from the potentiometer.

A Brief Rant on the Future of Interaction Design

From a personal point of view, I think that the main point of this text might be misunderstood by a lot of people. More than anything, because the follow-up article is a key part that adds context, depth, and information to the author’s rant.

In any case, I agree with a big part of the ideas presented in the text. However, it is not clear to me whether the problems presented should be taken with the same heaviness the author mentions. The reason behind this is none other than the feasibility of a solution to the problems that the author mentions. I believe the critique makes total sense because it gives value to the other mobility functions of the human body. However, finding new methods of interaction that the common user would prefer over traditional ones is not an easy task. If it helps, we never imagined leaving the mouse and keyboard behind, and here we are, using touch screens in everything. Considering how old this article is, it makes sense for the author to say that we should leave touchscreens behind eventually. But, after 10 years, we clearly haven’t moved on.

A follow-up article

As I mentioned before, the idea of a “rant” was confusing indeed. Most readers expect to find a solution to the problems presented in the previous text, but none were found. This is simply because most people will not get the main point of this text. The text does not imply that a solution will be found any time soon, but the opposite: the main objective is to inspire and motivate people to start researching for an answer.

I personally think he makes a great point in his ideas for the future. This doesn’t mean either I can provide a clear answer to the problems that he proposed back in his rant, but I can definitely say that reading his post encouraged me to start thinking about them.

Concept:

For this week, Izah and I decided to create a musical instrument based on our favorite analog sensor: the photoresistor. To do this, we decided to utilize an interesting concept to make the instrument work: an instrument that can only be played with 2 people. The idea here was to provide a completely different experience compared to traditional musical instruments. First of all, by allowing the notes only to be played with 3 hands and second, allowing to have different notes based purely on the placement of the hand (no touch). Because no traditional instrument requires 2 people to play it, we considered that this would be a greatly innovative idea.

Demonstration

Code highlight:

if (switchState ==1){
    //sensor 1
    if(sensorValue1 > 500){
      currentState+=100;
    }
    else if (sensorValue1>250 && sensorValue1 < 500) {
      currentState+=10;
    }

    else if (sensorValue1 < 250){
      currentState+=1;
    }

    //sensor 2
    if(sensorValue2 > 500){
      currentState+=100;

    }
    else if (sensorValue2>250 && sensorValue2 < 500) {
      currentState+=10;

    }

    else if (sensorValue2 < 250){
      currentState+=1;
    }

    //sensor 3
    if(sensorValue3 > 500){
      currentState+=100;

    }
    else if (sensorValue3>250 && sensorValue3 < 500) {
      currentState+=10;
    }

    else if (sensorValue3 < 250){
      currentState+=1;
    }

This code provides the change of currentState. Current State can be incremented in increments of 1, 10, or 100. This ensures that the state is always controlled in a way where we can always tell exactly in what level all sensors are working on. For example, if currentState = 111, then that means that there is one HIGH, one MID and one LOW state, regardless of which is which.

Challenges:

In this project, we faced multiple challenges that slowed down and hindered our development. For starters, because of the circuit that we built, being able to sustain the structure of the instrument was a big challenge we had to face. To solve this problem, we utilized a cardboard base, supported by the same paper cups we used to cover the light for the photoresistors.

Improvements:

In this case, the improvements would be related to polishing the implementation of the photoresistor detection. One of the biggest improvements we could apply would be to place all sensors at the same angle, in such a way that all of them captured the same light level initially. By doing this, the hand movements could be more streamlined and straightforward. Because we  utilized

Another possible improvement would be to solder the photoresistors and the cables together in order to provide a more solid connection and avoid some connection problems that happen every once in a while.

Concept:

For this week, our task was to get information from at least one analog sensor and at least one digital sensor (switch), and use this information to control at least two LEDs, one in a digital fashion and the other in an analog fashion, in some creative way.
My project is called “The Cookie Jar”, and it’s inspired by my love for chocolate chip cookies.

After recklessly spending my savings on this bucket of cookies, I decided to base my project on it. The idea was to turn a light on every time I opened the lid of this cookie bucket. To do this, I had to code the photoresistor in a way that it could take the difference between the light outside and the light inside. However, the problem relied on the fact that turning on the LED also generated light by itself, so I had to adapt the constraints of it in a way that would work properly. Also, I added a switch to turn on and off manually the other light inside, just in case someone wants to sneak a cookie in the middle of the night.

Breadboard and connections: 

Demonstration:

Code: 

void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);
  pinMode(5,OUTPUT);
  pinMode(9,OUTPUT);
  pinMode(2,INPUT);
  pinMode(A2,INPUT);
}

bool lightsOn = false;
void loop() {
  // put your main code here, to run repeatedly:
  int sensorValue = analogRead(A2);
  int switchValue = digitalRead(2);
  
  if(switchValue == 1){
    lightsOn = !lightsOn;
    delay(500);
  }
  digitalWrite(9,lightsOn);
// The brightness is the difference between inside and outside, but reversed (more light if the difference is low)
  int brightness = 255-(620-sensorValue);
  if(brightness<0){
    brightness = 0;
  }
  if(brightness>255){
    brightness = 255;
  }
  analogWrite(5,brightness);
  Serial.println(sensorValue);
  
}

Challenges:

The biggest challenge I faced was the shape and the fact that I had to connect the arduino UNO inside with the lid closed. I fixed this problem by reluctantly making a hole in my cookie bucket. I opened it just enough so it could fit the USB connector, which allowed me to insert the circuit inside even with the lid closed.

Making Interactive Art: Set the Stage, Then Shut Up and Listen

In this blog post, Tom Igoe talks about shifting the focus on what interactive art should be: instead of a complete piece, a finished artwork, we should approach it as if they were “performances”. His ideas, in my opinion, provide a valuable lesson on the final objective of interactive pieces. Every person might have their own vision and goals for their projects and art pieces, but at least for me, ultimately what I would like to achieve is to create something that people can see and be satisfied with their experience. I do not want to have to be present, or to have to talk to people explaining what they have to do in order to have the experience. In my opinion, that is the essence of interactivity and one of the key components of good design: the user should start, process and finish the experience by themselves.

Physical Computing’s Greatest Hits (and misses)

In this article, Tom Igoe discusses recurring themes in physical computing projects, highlighting popular ideas and their variations. In my opinion, I think that from this article, the author encourages creativity within these themes (physical computing recurring themes), emphasizing that even though certain ideas have been done before, there is room for originality and variation. And I personally want to emphasize this last part. “I don’t want do to that, it’s already done”, this quote resonated especially hard in my head. The reason behind this dates back to my highschool, where during a rough patch in my software development learning process, I struggled to think about projects that could be usable but would also help me learn. In my head, I was constantly repeating these exact words that the author included: “it has been done before, why care?”.

This mindset was what stopped me from improving as a creator and as a programmer. The principal of my highschool came up to me one day, and he clearly told me one example: “I do not want you to invent the wheel, and I do not want you to invent the suitcase. I want you to explore, and maybe you will end up creating the travel luggage”.

This article helped me remember those learnings, and I think the moral of the story is also one of the pillars of creativity: be able to push away your ego and work with things that are created, work on stuff that has been invented and do not shy away from these opportunities, because Rome was not built in one day, and definitely not from scratch.

Attractive Things Work Better by Donald A. Norman

This preview of Norman’s book “Emotion and Design” discusses the relationship between aesthetics and usability in design, starting with the example of teapots. While some critics argue that adhering to usability alone can result in ugly designs, the author suggests that beauty and usability should not be in conflict, as he describes in his phrase: “pleasurable designs are not necessarily usable”. However, from my point of view, the key point behind his idea is the fact that he relates cognitive studies that explore the concept of affect or emotional reactions in design. Affect has a significant impact on how people perceive and interact with products. In stressful situations, negative affect can lead to “tunnel vision”, while in positive or neutral situations, positive affect can promote “breadth-first thinking” and creativity, making users more tolerant of minor design issues and distractions. The author emphasizes that true beauty in a product should go beyond aesthetics and should be rooted in usability and functionality, and that design should combine and include various factors like marketing and aesthetics, and in my opinion, this is the most important learning we should take from this text.

Her Code Got Humans on the Moon – Robert McMillan

More than just an “outlier”, Hamilton was a pioneer in her field. Not even a pioneer, she was the founding pillar of modern software and the importance of it. The inhuman pressure that was born from the Apollo mission and its dependence on the software Hamilton was producing is, without doubt, one of the main reasons why companies, CEOs, and global leaders all around the world started to put their eyes (and resources) on software development. As she describes, “the original document laying out the engineering requirements of the Apollo mission didn’t even mention the word software”.

For me, it’s impressive how the software for this was produced at that time. Especially considering that the Apollo mission was a life or death matter for the astronauts, and definitely the most ambitious human project ever made at that time. The process in which Hamilton and the “Little Old Ladies” engraved the commands that made the Apollo mission such a famous and ultimately successful project was almost superhuman, and definitely an example for all aspiring software developers around the world. The story behind program PO1 and the “fool-proof” coding style that Hamilton proposed and was initially rejected is without a doubt a fundamental concept in today’s software development: secure, safe, and error-avoiding code.

Concept:

The idea behind this switch was to utilize something different than just our hands to control it. For this project, I decided to focus on one of the human body parts that we rarely include in our interactive projects: the feet.

To control this switch, the user will put together both feet in order to make a bridge between the two cables attached to the socks on each side. Something similar to how a pedal works. This switch would allow the user to use limbs that we usually do not include in our movements that intend interaction with a system: For example, a phone doesn’t need our feet, a camera doesn’t need our feet, and even computers usually don’t require our feet to be there.

Demonstration:

Challenges and improvements:

At first, I tried taping the tin foil to the socks. Surprisingly, because of how our feet are shaped, taping the cables was a tougher task than I expected because they would move and disconnect the moment I wore the socks. In addition to that, the tin foil is not exactly a strong material, which meant that the contact was interrupted sometimes. This was simply fixed by the most utilized solution in human history: just fix it with more tape, both in the cables and the tin foil surrounding the socks. Also, I taped the cables and the tinfoil after I wore the socks, which solved the foot shape problem.

Concept:

Link to the sketch: https://editor.p5js.org/ff2185/sketches/J9qU9jniU

The concept behind this idea was born from the coffee shop experience shown in class. When I saw it, I immediately recognized an opportunity to tinker around and produce an experience that would be similar to the one offered in the example. I was heavily interested in generating an interactive experience that would feature various types of “mini-games” or small interactive screens. In this case, I ended up choosing 3 options that would fit perfectly into the bar theme:

1. Piano Simulator: The first experience would be the piano, as it was something that I have been interested in since we started working with audio. Thankfully, I managed to implement a piano that I found in a sketch someone shared online (see reference). By adapting this piano into the system that my sketch was running, I was able to offer the user the capacity to play a fully functional piano made with the p5js.oscillator() in p5js.
2. Bartender Game: The bartender game was inspired by an old famous game for people born around the 2000’s. This game is called “The Right Mix”, and the idea is simple: Tap on a drink and add it to the mix. If the mix is good, then the score is high. If it isn’t, the score will be low.
3. Seated experience: Sit down and enjoy a drink at the bar. You will hear music playing in the background while you enjoy it and take it slow.
   

Design and Implementation:

For starters, the biggest challenge I knew I would face would be the state management. Having so many different interactive screens that do multiple varied functions is a big trouble considering how P5Js works. The reason behind this is mainly that P5JS is a framework that requires the user interaction to be screen-dependent: if you want to output something, it has to be relative to the coordinates. Same with the input. This is the only way to interact with the screen: that is why it’s called a sketch, it’s all about drawing.

The first part was to design the bar front, which I did manually by putting shapes in certain coordinates:

After this, I decided to start coding the piano. The piano sound is made naturally from p5js, as opposed to the other option which was to add the sounds of each key manually. As the second option was a nightmare in terms of implementation and code cleanliness, I decided to research how to generate it naturally. This is the result:

if(state == "piano"){
    if (keyCode === 65) { //osc refers to the oscillator
    rSide[0].red();
    osc[0].start();
    osc[0].freq(midiToFreq(root));
    envo[0].play();
  } else if (keyCode === 87) {
    black[0].red();
    osc[1].start();
    osc[1].freq(midiToFreq(root + 1));
    envo[1].play();
  } else if (keyCode === 83) {
    mid[1].red();
    osc[2].start();
    osc[2].freq(midiToFreq(root + 2));
    envo[2].play();
  } else if (keyCode === 69) {
    black[1].red();
    osc[3].start();
    osc[3].freq(midiToFreq(root + 3));
    envo[3].play();
  } else if (keyCode === 68) {
    lSide[2].red();
    osc[4].start();
    osc[4].freq(midiToFreq(root + 4));
    envo[4].play();
  } else if (keyCode === 70) {
    rSide[3].red();
    osc[5].start();
    osc[5].freq(midiToFreq(root + 5));
    envo[5].play();
  } else if (keyCode === 84) {
    black[3].red();
    osc[6].start();
    osc[6].freq(midiToFreq(root + 6));
    envo[6].play();
  } else if (keyCode === 71) {
    mid[4].red();
    osc[7].start();
    osc[7].freq(midiToFreq(root + 7));
    envo[7].play();
  } else if (keyCode === 89) {
    black[4].red();
    osc[8].start();
    osc[8].freq(midiToFreq(root + 8));
    envo[8].play();
  ............

The next step was to design the interior of the bar. After we linked the bar front door click with the interior, we had to design the background for it. For this, my approach was to utilize Ai generated pictures as a starter and work over that. After generating the background, I decided to add it in Photoshop and create the hover options by selecting the different elements and adding a shiny border to them:

For the drink game, I coded a simple interaction where the user can click on the cup and fill it. When it fills to the brim, the bartender will say thanks and the user can leave.

Lastly, the jukebox was made by utilizing the p5.TouchGUI library. This library allowed me to utilize buttons and generate a GUI with a volume slider in a very simple manner. By adding 2 lists, one containing the album covers and the other one containing MP3 files for every song, I managed to create a proper, functional music player with user-inputted files.

Reflections:

For this project, I am ultimately proud of my improvement on the creative side. The thing I struggled with the most was imagining the concepts and functions of my sketches, rather than how to do them. In this midterm assignment, I consider that finally challenged myself in terms of coding and creativity, and finally made an experience I find enjoyable.

Regarding improvements that I could make, I personally consider that there are multiple design decision that were a by-product of my coding decision rather than a conscious design choice. I would have like to utilize the TouchGUI library more to generate a better user experience around the bar (maybe jump through screens easily?).

In general terms, I have learned a lot doing this project and I hope to keep improving my coding and creative skills by the end of this semester.

Reference:

• Piano: https://editor.p5js.org/mrbombmusic/sketches/ryeLfZTd-
• TouchGUI: https://github.com/L05/p5.touchgui
• TouchGUI tutorial: https://www.youtube.com/watch?v=w1hEitRLKog
• Ai-generated images: https://playgroundai.com/
• Music: https://www.youtube.com/channel/UCGnClsipVxB9GhlX2W2ujBw

Sketch link: https://editor.p5js.org/ff2185/sketches/J9qU9jniU

After tinkering around with different ideas, I decided on the concept of my midterm project: The Interactive Bar.

Here, I will feature different parts of a bar that will allow the user to have 3 actions that intend to simulate a bar experience:
– Make your own drink.
– Play the piano.
– Sit down and enjoy a performance.

For this midterm progress, I have managed to do the front view of the bar and the piano, which was one of the most intimidating experiences.
The inspiration for the bar front was taken from “Moe’s Bar”, an iconic spot in the popular TV show “The Simpsons”

In addition to this, the piano is made using sounds generated directly from the oscillator function in P5js, which creates a signal that oscillates between -1.0 and 1.0. Using this, we can replicate the piano notes, at least the most basic ones.

for (var j = 0; j < 17; j++) {
    envo.push(new p5.Env()); //p5.Envelope,  controls the output volume of oscillator.
    envo[j].setADSR(0.01, 0.05, 1, 0.1);
    envo[j].setRange(1, 0);
    osc.push(new p5.Oscillator());
    osc[j].amp(envo[j]);
  }

  for (var i = 0; i < 10; i++) {
    rSide.push(new rSideKey(i, space, rKee[i]));
    black.push(new BlackKey(i + 0.667, space, blKee[i]));
    mid.push(new MidKey(i, space, midKee[i]));
    lSide.push(new lSideKey(i, space, lKee[i]));
  }

Potential challenges and next steps:

Starting from here, the most challenging parts of the project will be designing the interior of the bar. In order to make this functional, we have to allow the user to click in a certain object to start the experience. In addition to this, the project needs to include navigation options. By this I mean that every new screen should allow the user also to go back.
For this, I included a state management system to organize every screen and make navigation easier.