Author: Adeliya Aitpayeva

Concept & Progress

I’ve decided to change my initial idea with “physicalizing” my midterm project, cause I didn’t really think thoroughly of how would I completely change my initial workings. Therefore, I thought it would make more sense and it would give me more freedom and creativity, if I hit a fresh start.

I’ve stopped on the idea of recreating Kazakh cultural experience of sitting at the Dastarkhan (feast in Kazakh). Being a way from home, you naturally miss the traditions and rituals that your family used to follow for many years. Therefore, I wanted to somehow transfer this experience in physical and digital settings. I will create a physical model of the room with a Dastarkhan to inform people about our culture. This project will take not only emotional experience, but most importantly informative educational one. People of other nationalities will be able to learn about different table traditions and norms by interacting with a physical model of the Dastarkhan and also with the screen in p5, which will guide the user through these interactions and will complete them.

Here are the sketches that I’ve made to outline my project:

I’m going to be using cardboard to build the walls of the room. The main focus of this room is the table or Dastarkhan at the center, which I will laser cut from acrylic sheet and then cover with the texture of wood material, or I will use wood itself.

As you can see in physical model sketch, there are many components, let me explain all of them, which will be modes of interactions:

• Around the Dastarkhan there will be cloths, which will represent Korpe (Kazakh blankets). I’m planning to put buttons among those at specific points around the table to show different seatings and their meanings in Kazakh culture. So, by pressing at specific buttons, there will be a tab with explanation of specific seating appearing in p5.js
• I want to create a chandelier from LED lights. I want the turning on and off of the lights to be controlled with the Generic PIR Motion Sensor. In that way I want to signify that most importantly person’s presence highlights the Dastarkhan.
  
• Then, I will use flex sensors to introduce users to Kazakh musical instrument – Dombyra (You can see it hanging on the left wall). By flexing the strings of the Dombyra, users will initiate a video on p5 with prerecorded Dombyra performance, hence the sound will be introduced to this experience.
• There is also an LCD. I am going to ask the user to choose one of the offered holidays in the p5, which then will translated to LCD in physical model. In that way I want to encourage user’s awareness of different Kazakh holidays and overall appreciation of the feasts in Kazakh culture.
• And there is also a pulse sensor. I introduced it to reflect how important this culture of sitting at the Dastarkhan with a family is important to me. When the sensor will start tracking the values and sending it to p5, I want photos of my family and how we meet up on Dastarkhan to show up on the screen, so it would create a warm feeling in users (hopefully)
  
• Lastly, all of this will be connected by the p5 screen that will guide the user through this journey step by step by having 5 levels to complete the full experience. P5 will give specific directions to the user, like press the buttons and it will provide user with educational content on culture. There would be a progress bar in p5 to have a gamification element to this experience.

To summarize inputs an outputs
Arduino to P5:

• Buttons: Low or High
• Flex Sensors: 0 to 1023
• Pulse sensor: ~60 bpm

P5 to Arduino:

• LCD: string value (holiday name)

Concept

I have been thinking long on my final idea project and for now I stopped on idea of utilizing my midterm project and adding Arduino circuits to it.

Here is my previous Midterm project

The reason I decided to pursue my Friends Game House project was because I felt there is still room for its improvement in terms of interactivity. For example, I feel like the interfaces for the Ross, Joey, Monica characters can be designed more interestingly with greater room for unusual interactions. Therefore, I thought of transferring these interactions from software to real life. Currently, my focus stays on transferring important details from the Friends series into smaller things that I can build myself or recreate with elements that we learned to use in Arduino.
For example, in Phoebe interface, there is interaction with the user, where he has to press certain keys to voice specific lines of the songs. I think this can be transformed to the buttons on Arduino, especially bigger buttons that I saw in many final projects of previous students.

Here are some ideas that I’ve been noting to myself about the ways I can implement Arduino to the project:

• Joey’s interface – there was an idea initially that “Joey doesn’t share food”, so I could transfer it to Arduino, by making red LEDs go bright, when somebody is touching the plate with the food.
• Potentiometer to adjust the volume of the song on Phoebe’s screen or overall if the Friends theme song is implemented on the background.
• Using motors recreate the scene with umbrellas, so they could spin while the user is playing
  

Design Meets Disability

I always found the medical and disability assistance intended design very hard to work on and hence I never tried to evaluate it on the basis of design or aesthetics, rather than its usability and usefulness. I really liked how author approached the topic, seeing people with disabilities as a specific group of users whose needs and wants are usually not taken into account by the engineers or designers.

I think, like the author said, glasses is a great example of a product, which abandoned the association with disability, since from the moment I started wearing glasses, I never thought of them as a special needs device or assistance (even though they were), rather glasses were like my stylistic choice. When author discussed different cases, the idea of braces came to my mind, since is also became not only the medical tool to correct teeth, but some people are using braces to find a new fashion style for them.

It’s very disappointing to realize that users with disabilities have to compromise between more useful and technological product and product that provides discretion, since apparently those two objectives become conflicting in design. The approach “one fits all” also doesn’t work when we work in inclusivity, even though we seem to target one group – people with disabilities: we must recognize variance in disabilities as well.

In my view, the conviction that accessibility devices should prioritize discretion stemmed from current inability of engineers/designers to produce more confident and accomplished design, which would make users want to be seen.

The chapter introduced me to new concepts, such as appliance and new knowledge about dementia: I wasn’t aware that it is often accompanied by a heightened artistic appreciation and emotional response.

Task 1

Prompt: Make something that uses only one sensor on Arduino and makes the ellipse in p5 move on the horizontal axis, in the middle of the screen, and nothing on Arduino is controlled by p5.

We decided to approach all the tasks following Christmas style.

P5 Code:

let positionX = 0; 

function setup() {
  createCanvas(600, 600);
  noFill();
}

function draw() {
  background("#bb010b");
  stroke("#ffcf00");
  // maping the values from light sensor to the width of the canvas
  ellipse(map(positionX, 0, 1023, 0, width), height / 2, 100, 100);

  if (!serialActive) {
  // the initial screen of p5, when it is not connected to arduino, this helps to start serial communication
    background("#2d661b");
    stroke("#white");
    textSize(50);
    text("Press Space Bar to select Serial Port", 20, 30, width - 30, 200);
  }
}

function keyPressed() {
  // starting serial communication when space bar is pressed
  if (key == " ") setUpSerial(); 
}

function readSerial(data) {
  if (data != null)
    // ensuring there is actual data, then storing it in a variable
    positionX = int(data);
}

Arduino Code

int lightPin = A0;

void setup() {
Serial.begin(9600);
pinMode(lightPin, INPUT);
}

void loop() {
int sensorValue = analogRead(A0);
Serial.println(sensorValue);
delay(5);
}

Task 2

Prompt: Make something that controls the LED brightness from p5

For this assignment, we decided to control LED brightness using 4 circles/buttons on p5 interface. By pressing different circles, you increased the brightness of LED.

P5 Code

let LEDbrightness = 0; 

function setup() {
  createCanvas(600, 600);
  background('#2d661b');
  stroke ('#white');
  noFill();
}

function draw() {
  
  if (serialActive){  
  
  //circles-buttons
  circle (150, height/2, 50);
  circle (250, height/2, 50);
  circle (350, height/2, 50);
  circle (450, height/2, 50);
  
  // to understand whether user pressed specific circles, we used distance function. the LED brightness had 4 variations
  if (dist (mouseX, mouseY, 150, height/2) <= 50){
    LEDbrightness = 60;
  }
  if (dist(mouseX, mouseY, 250, height/2)<=50){
    LEDbrightness = 120;
  }
  if (dist(mouseX, mouseY, 350, height/2)<=50){
    LEDbrightness = 180;
  }
  if (dist(mouseX, mouseY, 450, height/2)<=50){
    LEDbrightness = 255;
  }
    }
  
  if (!serialActive) {
    // to understand if serial communication is happening
    textSize(50);
    text ("Press Space Bar to select Serial Port", 20, 30, width-30, 200);
    }
  }
function keyPressed() {
    if (key == " ")
      // starting the serial connection using space bar press
        setUpSerial(); 
}

function readSerial(data) {
  //sending data to arduino
    writeSerial(LEDbrightness);
}

Arduino Code

void setup() {
Serial.begin(9600);
pinMode(5, OUTPUT);
}

void loop() {
analogWrite(5, Serial.parseInt());
Serial.println();
}

IMG_9258

Task 3

Prompt: Take the gravity wind example and make it so every time the ball bounces one led lights up and then turns off, and you can control the wind from one analog sensor.

We decided to use light sensor as analog sensor, which controlled the wind on p5 that affected movement of ball.

P5 Code

let position, velocity, acceleration, gravity, wind;
let drag = 0.99,
  mass = 50,
  hasBounced = false;

function setup() {
  createCanvas(640, 360);
  noFill();
  position = createVector(width / 2, 0);
  velocity = createVector(0, 0);
  acceleration = createVector(0, 0);
  gravity = createVector(0, 0.5 * mass);
  wind = createVector(0, 0);
  textSize(18);
}

function draw() {
  background(255);

  if (!serialActive) {
    background("#2d661b");
    fill("white");
    text("Press F to select Serial Port", 20, 30, width - 30, 200);
  } else {
    applyForce(wind);
    applyForce(gravity);
    velocity.add(acceleration);
    velocity.mult(drag);
    position.add(velocity);
    acceleration.mult(0);

    if (hasBounced) {
      fill("red");
    } else {
      fill("white");
    }
    ellipse(position.x, position.y, mass, mass);

    if (position.y > height - mass / 2) {
      velocity.y *= -0.9; // dampening
      position.y = height - mass / 2;

      if (!hasBounced && abs(velocity.y) > 1) {
        hasBounced = true;
        setTimeout(() => (hasBounced = false), 100);
      }
    }
  }
}

function applyForce(force) {
  let f = p5.Vector.div(force, mass);
  acceleration.add(f);
}

function keyPressed() {
  if (key == " ") {
    mass = 20;
    position.set(width / 2, -mass);
    velocity.mult(0);
    gravity.y = 0.5 * mass;
    wind.mult(0);
  } else if (key == "f") {
    setUpSerial();
  }
}

function readSerial(data) {
  if (data != null) {
    let value = int(trim(data)); 
    wind.x = map(value, 0, 1023, -1, 1);
    let bounceState = 0;
    if (hasBounced) {
      bounceState = 1;
    }
    writeSerial(bounceState + "\n");
  }
}

Arduino Code

const int ledPin = 5;
const int lightPin = A0;

void setup() {
    Serial.begin(9600);
    pinMode(ledPin, OUTPUT);
    pinMode(lightPin, INPUT);

    while (Serial.available() <= 0) {
        Serial.println("0"); 
;    }
}

void loop() {
    while (Serial.available()) {
        int ledState = Serial.parseInt(); 
        
        if (Serial.read() == '\n') {
            digitalWrite(ledPin, ledState);
            Serial.println(analogRead(lightPin));
        }
    }
}

IMG_9260

Concept

After playing with different songs from the GitHub folder, we have decided to include several of them at the same time into our interpretation of a musical instrument. One way to do so would be to switch between three songs back and forth, creating a version of a radio with changing stations. Three classical compositions were chosen: Ode to Joy, Fur Elise, and Badinerie. However, this idea could be realised simply with two digital switches, so we decided to push it further by adding photoresistors that would allow us to work with a spectrum of values.

In what ways can a melody be altered while playing? Its volume or playback speed can be changed – as such, we decided to take values from one photoresistor to set up how loud the song was playing, and the other to set up the speed. A combination of all these electrical components helped us to create an advanced version of a radio, resembling a music player that allows to modify certain qualities of the sound.

Highlight of the code

:^)

Demonstration

IMG_9132

Reflection

Although we did not expect this idea to be so complex, we have faced serious difficulties in implementing the functions of switches – the audio kept repeating itself without any response to the change of digital or analog values. After troubleshooting the whole code using Serial.print(), we decided to ask the professor for help in order to finally tackle the issue.

—

Update

After a week of troubleshooting, changing the Arduino code and circuit, we still couldn’t figure out a way for project to work as we intended. Therefore, we introduced new idea of a player that will reflect the mood of the room based on its lightning. So we integrated photoresistor and a button. Based on how lighted the setting was, if moderate to highly lighted, the piezo buzzer would play major melody (happy), and if lighting was low, minor melody (sad) was played. The button was used to stop the play of the melody completely. When the button is pressed, melody is silenced.

Here is a link to the Arduino code.

We think the most complex part with working on this new idea, just like it was on the previous one was working with buttons, since it had to stop the play of the melody. We could finally resolve this by introducing boolean variables and setting them throughout the code.

void loop() {
  int lightValue = analogRead(lightPin);
  buttonState = digitalRead(buttonPin);

// checking if button is pressed
  if (buttonState == LOW && lastbuttonstate == HIGH) { 
    isPlaying = false;
    noTone(buzzerPin); // stop playing when button is pressed
  }
  lastbuttonstate = buttonState;

  if (!isPlaying) {
    if (lightValue > 600) {
      playMelody(melody1, duration1, melody1Length); //playing happy melody when sensor value is higher than 600
    } else {
      playMelody(melody2, duration2, melody2Length); //playing sad melody when sensor value is lower than 600
    }
  }

  delay(100);
}

void playMelody(int melody[], int duration[], int length) {
  isPlaying = true; // setting playing state of the melody
  for (int i = 0; i < length; i++) {
    if (digitalRead(buttonPin) == LOW) { 
      noTone(buzzerPin);
      isPlaying = false;
      return;
    }
    tone(buzzerPin, melody[i], duration[i]);
    delay(duration[i] * 1.3); //adding delay between notes
  }
  isPlaying = false; 
}

Here is the video of how it works:

IMG_9268

A Brief Rant on the Future of Interaction Design by Bret Victor

I usually feel very skeptical about videos, such as one that was presented by Microsoft on Productivity Future Vision. But, here are couple of points that I made on it:

• I really liked how gamification in education was presented in the video, as it feels less daunting to study for children and they feel more integrated in real life.
• It’s interesting that in all this digitalization, people still have to carry their suitcases physically, they have to wait in lines waiting for train and do other not so productive stuff. So, the video shows a reform in terms of digital devices solely. It’s a shallow vision of the future and our interaction.
• In the video, I saw future life, which lost its colors, because the main color that you can see are blue, green, white and black. There is no vividness.
• I didn’t recognize that so many devices convert what we can do to what we want to do.
• Also, you don’t really think about hands being central point of interaction. Hands our main sensors.
• Feedback from the objects that we are using usually is not verbal, but we can unconsciously feel things, especially if we remember or extensively use things. For example, I’m using a keyboard with arabic characters for writing in english, kazakh and russian languages, because my hands basically remember the location of each letter, which I’m not really aware of.
• The Pictures Under Glass concept can be applied to kids games applications, where they play being a hairdresser and cutting someone’s hair.
• After author of the article said it, I understood that the thing that seemed very strange to me about interactions shown in the video – they felt numb and senseless.
• I found pictures with hands and objects they carry kind of powerful.
• It occurred to me that dance as an activity can bring so much understanding about our daily interactions with our body, since we become more conscious about them.
• I strongly resonated with Bret’s saying that most artistic and engineering projects can’t just be described.
• I didn’t agree with one of the author’s points that if physical interactions should always be very informative and communicative, since I feel like in today’s interactions there is a lot of imagination employed by the users, hence physical may not be the only priority.
• I liked that the author addressed comments in separate article, where he also discussed concept of finger blindness, which is really fascinating.

Concept

When thinking about concept of my project, I decided to use a light sensor for controlling LEDs in an analogue fashion. I thought initially about making it as an interactive artwork, where the brightness of many LEDs will be controlled by the lighting of the room and the way the users manage light sensors themselves. However that idea didn’t work, because I made a mistake of connecting wires to wrong digital inputs (without PWM~), which led to unsuccessful fading of the LEDs. I realized my mistake too late, and already had switched my idea to something more fun and engaging. As lights were just switching on and off when I covered the light sensor with my finger, it reminded me of DJ Set where you press specific buttons and different sounds occur. I decided to do the same, but replacing sounds with LEDs. I had three LEDs that would be lighted using light sensors. And I also included an LED that was switched on using slide switch, to add digital inputs to the circuit, since light sensors were connected to analog inputs.

Arduino Setup

 

As you can see, I just followed the rhythm of the song with my DJ Set.

Highlight of the code

GitHub Link

int sensorValue = analogRead (A0);
Serial.println (sensorValue);
sensorValue = constrain (sensorValue, 10,200);
brightness = map (sensorValue, 15, 140, 0, 255);
analogWrite (ledPin2, brightness);
delay (10);

int sensorValue2 = analogRead (A1);
sensorValue2 = constrain (sensorValue2, 10,200);
brightness1 = map (sensorValue2, 15, 140, 0, 255);
analogWrite (ledPin3, brightness1);
delay (10);

int sensorValue3 = analogRead (A2);
sensorValue3 = constrain (sensorValue3, 10,200);
brightness2 = map (sensorValue3, 15, 140, 0, 255);
analogWrite (ledPin4, brightness2);
delay (10);

I think the hardest part of the coding was just with assigning correct numbers and pins to each function. I decided to use “constrain” and “map” functions that we learned on our lesson, because when building my project in IM Lab I found the light sensor values on Serial Monitor vary only from 15 to 140, hence I mapped them to 255 range, so that brightness could be manipulated. As it can be seen from the Arduino setup and the video, there are a lot of wires on the breadboard, which was another challenge of connecting them just right and not fail the code.

Reflection

In my next improvements of the project, I wish to experiment with different types of sensors, such as temperature, ultrasonic and sound sensors. It would give my project higher difficulty and also interest. Also, I am very eager to learn and employ Piezo Speaker in my next projects, as a further development of the DJ set. Moreover, it would make sense if I would use bigger breadboard and make connections more clear, reducing unnecessary  wires. I would also want to connect RGB LED, as it gives higher flexibility and room for experimentation.

 

 

Making Interactive Art: Set the Stage, Then Shut Up and Listen

I found this blog post insightful. At first, it made me feel very confused and upset, because the approach I was taking with the assignments that we’ve completed before is like with artworks, where I set the meaning and actions and people follow it. But, it actually makes such more sense when we say that it is not one sided interaction, we listen to how users speak with their actions and do they find their way by themselves. It was so surprising to me that the author related planning interactive artwork to the director working with actors, because this semester I’m taking a DIRECTING class too! After the paragraph about directing, the idea of the author became really clear to me and I actually got it. I resonated with his words of leaving half part of your work to the audience to figure out what they are feeling about it and what meaning they would find in it, because that’s what makes it an artwork. My professor in the Directing class used to tell us: “Love your audience”. Now, it could really apply to my interactive works, where the users would come first and the intentions and messages that I planned would go second. 

Physical Computing’s Greatest Hits (and misses)

In one of the first ideas of the author about not necessarily seeking originality, I agree with him, because I think that replication is key for learning and mastering skills. In art schools they teach artists to replicate the biggest art masterpieces, so afterwards there would be a room for original artworks creation.  

It’s my first encounter with the theremin instrument. And I like how Tom Igoe put the idea of interaction with instruments that when you are using them you essentially think about music and the process of using it. That leads us to the point that interaction should be simple, understandable and meaningful. 

When discussing dolls and pets, I remembered the viral dancing cactus toy, which replicates the sound and adds movements. The author mentions that “we like things that appear behave like us”, but the case with the cactus toy is definitely an exception, because in TikTok there is a growing trend of videos where these toys make babies cry after they hear a toy replicating their sound and movement. 

Moreover, even though dance is one of the most enjoyable physical expression forms, I don’t think the usage of it in physical computing expanded beyond the Step Maniax arcade, where you just have to step on right rectangles (or maybe I just didn’t see any other examples). 

When discussing the whole body as a cursor idea, I thought about using it for manipulating the computer screen. Also, in this case the concept of inclusivity really steps in, because different people have different abilities.  

The idea of flying seems very exciting in terms of interactions. 

Concept

I decided to create a switch that would be initiated by two cups touch. I was inspired by the “Emotions & Design: Attractive things work better” reading, as it focused on the affect and the emotional connection to the product. Then I thought of the importance of tea drinking in the Kazakh culture and how there is a clinking glasses ritual that makes you celebrate the moment. I believe that this ‘Cheering’ process is the one that bonds people and creates special atmosphere in the room. So, I wanted to highlight how special this moment is by lightning up the LED when cups touch.

I just attached the foil on two cups and connected it to the wires to the foil and arduino board. I used two LEDs of different colors to portray two different people and their lightning connection.

Coding

Arduino File on Github

void setup() {
  pinMode(11, OUTPUT); // led pin
  pinMode(2, INPUT);  // foil pin
  digitalWrite (2, HIGH);
}

void loop() {
  int cuptouch = digitalRead(2);

  if (cuptouch == LOW) { 
    digitalWrite(11, HIGH); 
  } else {
    digitalWrite(11, LOW);
  }
}

I don’t think there is a particular thing I am proud of about my code because the code is similar to what we did in the class. At first of my setups of the circuit, I forgot to add the digitalWrite function in the code, which made my circuit fail. But then I added it and the cups touching worked as a switch on to the circuit.

Setup

Reflection

I think that I could further work on how the clinking of glasses is portrayed on the breadboard: maybe the LEDs don’t light up immediately, but with a pace, which will remind of growing mood of the person.

I feel like my project could present a great potential to the experience of customers in coffeeshops, where people connect with each other while having cups of coffee. It could bring coffeeshops customers higher level of interaction with the products and a desire to come back to light up another LED with their cups.

Also, I would like to work on how aesthetically wire and foil is attached to the cup/glass (again coming back to Don Norman’s work). There are different designs of cupholders, and I think I could make one that would be created from metal or have foil attached more gracefully. Moreover the wire can be connected in a more hidden way and maybe incorporated to the coffeeshop tables, so that cups are kind of wired to the tables, but are still usable.

Her Code Got Humans on the Moon

I’ve heard about Margaret Hamilton and known her for this famous photograph, but I wasn’t familiar with her work specifically. And now reading the article, I am fascinated by the amount of dedication and commitment she put in development of the software as well as the scale of her creation and it’s very disappointing that not many people are familiar how she started the software engineering as a field.

I liked that in the article it was emphasized that we should approach coding by planning out and thinking through all possible mistakes and failures that could happen in the process of using the product. No one believed Margaret Hamilton’s saying that human error could happen among astronauts, but it did in one of the most unfavorable moments for the team. In the same way I feel like this attentiveness to details and thinking of possible outcomes beforehand helps to build usable and friendly product for the user, so that when the user fails, the application doesn’t fail as well.

Also I was shocked that copper “ropes” could store words.

Emotions & Design: Attractive things work better

The main takeaway for me from this article is about the importance of the impression and emotion that the user is left with after using the product. And basically that’s the reason why Don Norman buys three teapots, because each of them provides him an affect value. This kind of resonates with the idea of arts and also the teachings of my theater class that I’m taking this semester: everything falls to what kind of emotions artwork prompted you to. Hence, one of the ways to cause feelings in the person is by beauty and aesthetics of the product. That’s why attractive things work better, when you feel great you do great.

One of the things I noted for myself was that according to the Don Norman, design preferability depends upon the occasion, context, and above all, mood, hence people buy different things to suit different circumstances, which ultimately leads to overconsumption and waste.
Also I didn’t know that our choices and affective signals are so much biologically driven, because the way I thought about it was that it cannot be impacted in any way and it was purely intuitive.

Also, I believe that many people have a conviction that attractive things are less usable. I think it is a belief that was instilled to me by my community because too bright and vivid things were less desirable to have. But, the article completely proves otherwise. Overall, the trend described by Don Norman that people want to have more pleasure and joy in life kind of describes why there is now a great development to the entertainment field and why many designers prioritize the pleasure in using their product.