Author: Amina Magomedova

After reading the chapter from Graham Pullin’s book Design Meets Disability and thinking about my personal experiences related to this topic, I decided work on an unconventional model of eyeglasses. My final project will be a synthesis on an interactive artwork and an assistive device, inviting audience to think about the stigma around certain types of products for disabled people.

Drawing inspiration from Meta Ray Ban smart glasses,  I want to combine audio with visuals by adding small speakers to the ends of the temples, so that the user can hear the sound clearly only when they wear the product. In terms of visuals, I want to use camera mode in p5.js to detect the eye area of the user, with the help of glasses designed in a specific way to facilitate this. I will be modelling the glasses myself using Rhino and then printing them on 3D printer in order to create a precise structure that would fit desired electronic components. The speakers will be connected to Arduino, which in its turn will be connected to p5.js sketch using serial communication. Depending on the gazing sequences of the user, the visuals on the screen will be changing, depicting eyes of changing sizes. For user’s eye-tracking WebGazer.js library will be used.

From my individual experience and observations, one of the key reasons why people feel awkward about wearing glasses is because of the way they distort the size of their eyes depending on the prescription, hence altering the familiar look of the face. Such minor things are often invisible to strangers, but people wearing glasses can become extremely conscious about this. By providing an exaggerated illustration of such distortion, I wish to draw attention to the way people with disabilities perceive themselves.

Task 1

Prompt: Make something that uses only one sensor on Arduino and makes the ellipse in p5 move on the horizontal axis, in the middle of the screen, and nothing on Arduino is controlled by p5.

We decided to approach all the tasks following Christmas style.

P5 Code:

let positionX = 0; 
function setup() {
  createCanvas(600, 600);
  noFill();
}
function draw() {
  background("#bb010b");
  stroke("#ffcf00");
  // maping the values from light sensor to the width of the canvas
  ellipse(map(positionX, 0, 1023, 0, width), height / 2, 100, 100);
  if (!serialActive) {
  // the initial screen of p5, when it is not connected to arduino, this helps to start serial communication
    background("#2d661b");
    stroke("#white");
    textSize(50);
    text("Press Space Bar to select Serial Port", 20, 30, width - 30, 200);
  }
}
function keyPressed() {
  // starting serial communication when space bar is pressed
  if (key == " ") setUpSerial(); 
}
function readSerial(data) {
  if (data != null)
    // ensuring there is actual data, then storing it in a variable
    positionX = int(data);
}

Arduino Code

int lightPin = A0;

void setup() {
Serial.begin(9600);
pinMode(lightPin, INPUT);
}

void loop() {
int sensorValue = analogRead(A0);
Serial.println(sensorValue);
delay(5);
}

Task 2

Prompt: Make something that controls the LED brightness from p5

For this assignment, we decided to control LED brightness using 4 circles/buttons on p5 interface. By pressing different circles, you increased the brightness of LED.

P5 Code 

let LEDbrightness = 0; 
function setup() {
  createCanvas(600, 600);
  background('#2d661b');
  stroke ('#white');
  noFill();
}
function draw() {
  
  if (serialActive){  
  
  //circles-buttons
  circle (150, height/2, 50);
  circle (250, height/2, 50);
  circle (350, height/2, 50);
  circle (450, height/2, 50);
  
  // to understand whether user pressed specific circles, we used distance function. the LED brightness had 4 variations
  if (dist (mouseX, mouseY, 150, height/2) <= 50){
    LEDbrightness = 60;
  }
  if (dist(mouseX, mouseY, 250, height/2)<=50){
    LEDbrightness = 120;
  }
  if (dist(mouseX, mouseY, 350, height/2)<=50){
    LEDbrightness = 180;
  }
  if (dist(mouseX, mouseY, 450, height/2)<=50){
    LEDbrightness = 255;
  }
    }
  
  if (!serialActive) {
    // to understand if serial communication is happening
    textSize(50);
    text ("Press Space Bar to select Serial Port", 20, 30, width-30, 200);
    }
  }
function keyPressed() {
    if (key == " ")
      // starting the serial connection using space bar press
        setUpSerial(); 
}
function readSerial(data) {
  //sending data to arduino
    writeSerial(LEDbrightness);
}

Arduino Code

void setup() {
Serial.begin(9600);
pinMode(5, OUTPUT);
}

void loop() {
analogWrite(5, Serial.parseInt());
Serial.println();
}

IMG_9258

Task 3

Prompt: Take the gravity wind example and make it so every time the ball bounces one led lights up and then turns off, and you can control the wind from one analog sensor.

We decided to use light sensor as analog sensor, which controlled the wind on p5 that affected movement of ball.

P5 Code

let position, velocity, acceleration, gravity, wind;
let drag = 0.99,
  mass = 50,
  hasBounced = false;
function setup() {
  createCanvas(640, 360);
  noFill();
  position = createVector(width / 2, 0);
  velocity = createVector(0, 0);
  acceleration = createVector(0, 0);
  gravity = createVector(0, 0.5 * mass);
  wind = createVector(0, 0);
  textSize(18);
}
function draw() {
  background(255);
  if (!serialActive) {
    background("#2d661b");
    fill("white");
    text("Press F to select Serial Port", 20, 30, width - 30, 200);
  } else {
    applyForce(wind);
    applyForce(gravity);
    velocity.add(acceleration);
    velocity.mult(drag);
    position.add(velocity);
    acceleration.mult(0);
    if (hasBounced) {
      fill("red");
    } else {
      fill("white");
    }
    ellipse(position.x, position.y, mass, mass);
    if (position.y > height - mass / 2) {
      velocity.y *= -0.9; // dampening
      position.y = height - mass / 2;
      if (!hasBounced && abs(velocity.y) > 1) {
        hasBounced = true;
        setTimeout(() => (hasBounced = false), 100);
      }
    }
  }
}
function applyForce(force) {
  let f = p5.Vector.div(force, mass);
  acceleration.add(f);
}
function keyPressed() {
  if (key == " ") {
    mass = 20;
    position.set(width / 2, -mass);
    velocity.mult(0);
    gravity.y = 0.5 * mass;
    wind.mult(0);
  } else if (key == "f") {
    setUpSerial();
  }
}
function readSerial(data) {
  if (data != null) {
    let value = int(trim(data)); 
    wind.x = map(value, 0, 1023, -1, 1);
    let bounceState = 0;
    if (hasBounced) {
      bounceState = 1;
    }
    writeSerial(bounceState + "\n");
  }
}

Arduino Code

const int ledPin = 5;
const int lightPin = A0;
void setup() {
    Serial.begin(9600);
    pinMode(ledPin, OUTPUT);
    pinMode(lightPin, INPUT);
    while (Serial.available() <= 0) {
        Serial.println("0"); 
;    }
}
void loop() {
    while (Serial.available()) {
        int ledState = Serial.parseInt(); 
        
        if (Serial.read() == '\n') {
            digitalWrite(ledPin, ledState);
            Serial.println(analogRead(lightPin));
        }
    }
}

IMG_9260

Design Meets Disability

While reading the chapter from Graham Pullin’s book, I caught myself thinking about a recent case from my friend’s life – she is a student at Tandon, working on a project in collaboration with Langone Hospital. The theme of the project is development of products for accessible environment, specifically a belt that allows people with blindness to orient in space. My friend’s immediate suggestion was to move the sensors from the belt to hands, e.g. to gloves, so that calculation of the distance to objects was more precise – makes sense, right? However, her idea was met with criticism from the professor – she forgot about that very discretion.

“Our goal is to make sure that their limitation not only does not interfere with their lives, but also remains absolutely unnoticeable” – although I do understand the point of my friend’s professor, I would formulate this objective differently. From my understanding, discretion is not simply about hiding the product, but rather about its seamless integration into one’s life. Balancing both engineering and design aspects in one person’s mind is extremely complicated, since the “perfect” solutions in both categories of tasks can be mutually exclusive.

As a person who wears glasses and contacts, used to wear braces and orthopaedic insoles, I have been concerned with the image of such products since I was a child – when I was five, I refused to wear Birkenstock sandals because I was convinced that they were ugly, even though they were prescribed to me by a doctor. That fear of looking stupid and funny is relatable to many people, so I believe that there is a huge room for improvement in the field of design for disability.

Concept:

After playing with different songs from the GitHub folder, we have decided to include several of them at the same time into our interpretation of a musical instrument. One way to do so would be to switch between three songs back and forth, creating a version of a radio with changing stations. Three classical compositions were chosen: Ode to Joy, Fur Elise, and Badinerie. However, this idea could be realised simply with two digital switches, so we decided to push it further by adding photoresistors that would allow us to work with a spectrum of values.

In what ways can a melody be altered while playing? Its volume or playback speed can be changed – as such, we decided to take values from one photoresistor to set up how loud the song was playing, and the other to set up the speed. A combination of all these electrical components helped us to create an advanced version of a radio, resembling a music player that allows to modify certain qualities of the sound.

Highlight of the code:

:^)

Demonstration:

IMG_9132

Reflection:

Although we did not expect this idea to be so complex, we have faced serious difficulties in implementing the functions of switches – the audio kept repeating itself without any response to the change of digital or analog values. After troubleshooting the whole code using Serial.print(), we decided to ask the professor for help in order to finally tackle the issue.

The idea of developing immersive technology that goes beyond the limits of the 2D screen has been concerning me for a while already, so I got very involved into the topic of the rant by Victor Bret. Interactive design was an ambiguous term back in 2011, when the text was published, and remains often misunderstood now. How can we call “pictures under glass” truly interactive if we do not engage with them physically?

Haptics are one way to push the boundaries of human-computer interaction further. When our fingers get some sort of response, even a slight vibration, it makes our experience with gadgets more engaging on a cognitive level. Here is an example from my life: my phone is always in silent mode, so when I type something, I do not hear the artificial sound of keys clicking. At some point I stopped understanding whether I am actually typing – so I turned on the “haptic response” feature. This makes the keys vibrate slightly every time I press on them, which creates a more realistic experience of using a physical keyboard.

Nonetheless, I agree with Bret that interactivity can be pushed even beyond haptics. At the same time, it is still difficult to come up with a clear solution, if there is one. Reading Bret’s ideas almost 15 years later was interesting, considering the latest inventions in AI which, to some extent, contradict with his points about the use of voice – turns out it can be used to generate something beyond linguistics.

Concept

Using both digital and analog switches, I came up with a scheme that allows to change the colour and intensity of the RGB LED depending on the state of the button and values captured by the temperature sensor.

When the button is pressed, the RGB LED turns off, otherwise it is on – this is an example of working with digital switch, which works with high-low states. When the temperature sensor captures a value higher than 25C, the RGB LED lights up with red, and the light becomes more intense the higher the temperature increases. When the sensor captures a value lower than 25C, the RGB lights up with blue, and the light becomes more intense the lower the temperature decreases – this is an example of working with an analog switch, which allows to manipulate a spectrum of values.

Highlight of the code

It took me a long time to come up with the baseline temperature that would make the switch from red to blue light in room temperature conditions possible. In the end, I could use my own hands to make the sensor feel warmth, and then I used a frozen thermos wrapped in tissues to make the value decrease below 25 degrees.

if (switchState == LOW) {

  if (temperature > baselineTemp) {
   
    int redBrightness = map(temperature, baselineTemp, 30, 0, 255);
    redBrightness = constrain(redBrightness, 0, 0);
    analogWrite(redPin, redBrightness);
    analogWrite(greenPin, 0);
    analogWrite(bluePin, 0);
  } 
  
  else {
    
    int blueBrightness = map(temperature, baselineTemp, 25, 0, 255);
    blueBrightness = constrain(blueBrightness, 0, 0);
    analogWrite(redPin, 0);
    analogWrite(greenPin, 0);
    analogWrite(bluePin, blueBrightness);
  }
}

Demonstration

Amina_Temperature

Reflection

For future improvements of this idea, I could play with more shades of RGB LED light that would be associated with different values on the spectrum, since this electrical component allows to work with almost any colour of the light.

Physical Computing’s Greatest Hits (and misses)

As I was scrolling through the article by Tigoe before reading it, I immediately recognised several works I have read about or saw before. As I started reading it in depth, I understood why – these categories of physical computing projects are indeed very popular. Nonetheless, some of them were still unexpected, for example, I have never thought that a popular Dance Revolution arcade game is in fact an example of physical computing. This illustrates how many systems we interact with actually belong to one or several of the outlined types of projects, but we tend to overlook this. I plan to experiment with these categories further, possibly creating something on the edge of several ideas.

Making Interactive Art: Set the Stage, Then Shut Up and Listen

Coming from Visual Arts background, I am used to providing a description for each of my works, explaining what was the idea behind the piece and sometimes how to look at it. After reading the article by Tigoe, my perception on the explanatory part had been changed – I have not thought about the importance of open interpretation for interactive works before. From perspective of the creator, I often feel that I have to define the ways audience should observe my art, otherwise they might overlook something. As a viewer, I do feel that sometimes installations and other immersive works in museums and galleries can seem too predefined. As the author mentions, the basics should be outlined and the rest of the job should be left for the interactor – and I agree with this balance.

Norman,“Emotion & Design: Attractive things work better”

Don Norman, famous for outlining principles of user-friendly design, argues that aesthetics play an important role in the functionality of designs, impacting directly not only usability of the product but also user’s emotional response. An attractive object evokes positive emotions simply when looking at it, which further motivates the user to explore and engage. This is explained through the concept of emotional design, which highlights not only the decorative but also functional role of aesthetic in design.

I agree that it is important to think about the way your work looks like from the point of aesthetics – in both physical and digital works, “pretty” things catch user’s attention, which is then carefully navigated to functionality. Going back to the famous manifesto “form follows function”, in the context of Norman’s ideas I agree with it – attractive things do tend to work better, especially when the aesthetic and functionality of the product are intertwined.

Her Code Got Humans on the Moon

Margaret Hamilton is an incredibly important figure in computer science, and I am glad that I have learned about her work back in middle school. She is a motivating example of a person who has managed to combine her work and home duties back when the opportunities for women to enter technical fields were extremely limited.

Hamilton’s approach to error management is intriguing to me, since she highlighted the importance of simulating all experiences before bringing them to life. Planning potential errors in advance is crucial when it comes to such big inventions as Apollo. The example of tracking an accidental error and then resolving it under pressure says a lot about the importance of paying attention to all details and planning everything that can go wrong in advance.

In my projects, I wish to learn to pay more attention to such minor things that can potentially go faulty, especially since we have started working with physical electronic models – the risks are higher here compared to purely digital simulations.

Concept

I carry my mug with morning coffee to every class, and since it is not fully insulated but just spill-proof, I am often anxious about someone hitting it on the table accidentally and spilling the drink on my or someone else’s laptop. In order to prevent such accidents, I came up with a scheme that indicates when my mug is open and when it is closed using two LED – Red lights up when the mug is open and liquid can be spilled from it, and Green lights up when the mug is closed and there is no threat of spillage.

(For safety purposes, I did not pour any liquid into the mug during the demonstration to avoid risk of getting electrocuted)

Highlight of the code

https://github.com/am13870/UnusualSwitch

I have used the Button example of the code from the Basics category in Arduino IDE Application as the basis for my code. Two LED were used in my scheme, so I had to alter the code accordingly, considering that they have different conditions of being turned on and off.

void setup() {
  // initialize the LED pin as an output:
  pinMode(4, OUTPUT); //Red LED
  pinMode(6, OUTPUT); //Green LED

  // initialize the pushbutton pin as an input:
  pinMode(A1, INPUT);
}

void loop() {
  // read the state of the pushbutton value:
  int buttonState = digitalRead(A1);

  // check if the pushbutton is pressed:
  if (buttonState == HIGH) {
    digitalWrite(4, HIGH); //turn Red LED on
    digitalWrite(6, LOW); //turn Green LED off

  } else {
    digitalWrite(4, LOW); //turn Red LED off
    digitalWrite(6, HIGH); //turn Green LED on
  }
}

Demonstration

IMG_8859

Reflection

For future improvements, the scheme can be insulated from water to make sure that no liquid ruins the electrical components or the whole circuit. Furthermore, sensors can be added to make the construction even more safe – for example, when a hand approaches a sensor, Red LED lights up signifying that a potential spilling threat is too close to the mug.

Concept

When visiting different cafes, I often think how challenging it is for the waiters to keep in mind the orders of all people at the table – quite often they would place a dish your friend ordered in front of you instead, and then you will just exchange the plates by yourself. Not a big deal, right?  But visitors rarely think about the actual pressure in this kind of working environment – time, tastes, and preferences count for every customer.

I have decided to create a mini game that allows the player to experience how tricky it can actually be to serve the dishes during a rush hour. There is an extra challenge – the user is accompanying a table of two people on their first date, one of them is vegan and another loves meet. Will everything go as planned?

Sketch

https://editor.p5js.org/am13870/sketches/3_efwXDWQ

Highlight of the code

The most challenging part that I have managed to implement into my code was the functionality of the plates and the dishes on them. The falling dishes get attached to plate when caught by the user, and then the plate is dragged to the edge of the table. Vegetable-based dishes have to be dragged to the left, to Leonie, and meat-based dishes have to be dragged to the right, to Santi. If the player serves the dishes correctly, they get a tip – and I they don’t, they lose it.

Managing the interdependence of these objects was very tricky, but I have managed to defined specific functions within the classes so that everything appears and disappears when needed. Furthermore, setting up the winning and loosing conditions took time, so I have followed the guidelines from the class slides to add navigation through the screens.

  display() {
    image(plate2Patterned, this.x - this.radius, this.y - this.radius, this.radius * 2, this.radius * 2);

    // displaying the dish attached to the plate
    if (this.attachedShape) {
      this.attachedShape.x = this.x;
      this.attachedShape.y = this.y;
      this.attachedShape.display();
    }
  }

  mousePressed() {
    let d = dist(mouseX, mouseY, this.x, this.y);
    if (d < this.radius) {
      this.dragging = true;
      this.offsetX = this.x - mouseX;
    }
  }

  mouseReleased() {
    this.dragging = false;
  }

  attachShape(shape) {
    if (this.attachedShape === null) {
      this.attachedShape = shape;
    }
  }

  resolveShape() {
    if (this.attachedShape.type === 'vegetable' && this.x - this.radius <= 50) {
      score += 1;
    } else if (this.attachedShape.type === 'vegetable' && this.x + this.radius >= 750) {
      score -= 1;
    } else if (this.attachedShape.type === 'meat' && this.x + this.radius >= 750) {
      score += 1;
    } else if (this.attachedShape.type === 'meat' && this.x - this.radius <= 50) {
      score -= 1;
    }

    // removing the dish as it reaches the end
    this.attachedShape = null;
  }
}

Reflection

I have started the project by developing the “physics” part, so that all objects in a simplified style would move correctly. For example, the dishes were drawn as triangles and squares at first. After finalising this part, I have moved on to adding the detailed visuals – this was not the easiest part as well, but I enjoyed developing a certain aesthetic of the game.

I started the design stage by going back to the original reference for the game idea, the photograph by Nissa Snow, which depicts a long table with dishes on it. Using Adobe Illustrator, I have created the table for my game, adding and editing images of objects that follow an artistic, slightly random, almost incompatible aesthetic. Then I used Procreate to draw the introduction screen, the victory, and the loss slides that are shown depending on the outcome of the game. The illustrations were created in a minimalistic manner in order not to clash with clutteredness of the table.

Future improvements

To develop this mini game further, I would add more conditions for the player – for example, new sets of guests can come and go, their dietary preferences can change. This would require implementation of several more arrays and functions to set up the food preferences, and I think this is an interesting direction to dive into.