Author: Ryan Opande

Concept

My final project was  an interactive robot which someone can control. I was also inspired by the animation Despicable Me which can explain the design of my robot as a minion. The robot is a simple build with 4 tyres arranged in a + manner to allow the robot to move in all directions and also spin. It also has two hands which move up and down to depict the human hand movement. On movement of the robot the hands also move depending on the direction e.g when moving left the left hand rises. While creating the robot I had a dilemma on how to control the robot. However, I got inspired by previous students work which used the handspose library from ml5.js to control movement using hands. I went on to implement this to control my robot.

Interaction Design
The interaction was using ml5.js hanspose library which picks the users hands position to control the movement of the robot. I divided the screen into 4 sections such that having a hand in each section sends a certain command to Arduino to control the robot; a hand in the top left section moves the robot forward, top right section moves back, bottom left section turns left and bottom right turns right, to stop the robot one has to put both hands on the screen or remove all hands from the screen.

Arduino Code

My Arduino code included commands to control every movement of the robot; the motors, servos and LED’s. It receives data from p5 and depending on the data it executes the command on the robot.

Arduino code on GitHub

p5 Code

My p5 code picks data from user to move the robot. Using the average position of the users hand key points, the code checks where the hands are positions and stores this in as a defined data text which is sent to Arduino. Through serial communication the Arduino is able to process commands on the robot based on what it gets from p5.

Circuit Schematic

For the project I used the Adafruit motor shield for my connections as I initially wanted to use 4 motors. This component turned out to be very beneficial as it reduced the amount of wires that I needed to use if I would have opted for the small motor shield. I also didn’t have to use a breadboard which helped to minimise the size of my robot. I used KiCad to generate the schematic

Project Reflection and Future Improvements

Working on this project was a worthwhile experience. I was a good opportunity to put into work what I have learnt in the course. I had a couple of challenges while coming up with it. I initially wanted to use 4 DC motors for each individual wheel however this had a lot of issues as the robot couldn’t move in some directions. After spending a lot of time trying to fix this I finally got a solution by having to use only two motors and two support wheels which enabled the robot to move better. I also had an issue with picking commands from user. My initial idea was to have the user swipe the hands across the canvas but this had errors because the handspose library was too sensitive which resulted in clashing commands being picked. The project still has space for improvement especially in terms of the user interaction. The user experience can be enhanced by having both hands to control the movement since many users found it challenging to use only one hand. Everyone naturally started using two hands even after reading the instructions which might mean that including a two hand control would be better.All in all, I am proud of the whole project and how I was able to implement it.

User Testing Videos

 

 

 

Concept

I decided to settle with creating a simple robot for the final project. My idea was inspired by a robot waiter in one of the restaurants I visited back home. I found it cool how one can come up with such a thing and it pushed me to try and emulate it. I will be creating a simple robot that moves around on command of the user. The robot will be able to do a simple dance, spin around with the hands moving up and down, when the user commands it to. I would also like to have the robot to be able to do simple writing and also measure distance and give feedback to the user.

Arduino 

The Arduino will be the basis of the project as it will be getting input from p5 to move the robot. The robot is powered by four Arduino controlled DC motors which respond to inputs from p5 to move the robot. Additionally, there are two servo motors to control hand movements of the robot. I will also include a distance sensor that will allow the robot to read distance from objects and send the value to p5 for the user. I will also have LED lights which will represent the eye of the robot.

P5

I was conflicting on what my p5 would do but I settled on using the ML5.js Handpose to enable the user control the robot using their hands. Since I will also have two modes, one for moving the robot and one for drawing using the robot, I intend to have it that on the draw mode the robot can write on a white board and p5 can give a visual representation of what the robot is drawing. On the moving mode the p5 will be taking input from the user and sending it to Arduino to control the robot.

Progress

So far I have been able to make the base which is the wheels controlling the movement of the robot. I have also added the servo motors that will control the hands. I want to create a solid base that will hold the components and make the robot aesthetic. I also want to 3D print a body for the robot but I am yet to do that.

 

Design Meets Disability

The reading helped me understand the intersection go aesthetics and utility. Its emphasis on glasses changing from medical necessity to fashion accessory made me realise how deep the idea of “hiding” disability is in our culture. We tend to prioritise discretion in disability design over self-expression. This showed how good designs should not just accommodate but enhance and adapt to human experiences without bringing attention to differences. I now see how disability is more of a limitation by environments which aren’t designed inclusively rather than a limitation of a person.
I was particularly struck by the contrast between traditional medical design and the bold, unapologetic aesthetics introduced by people like Aimee Mullins. Her idea that a prosthetic leg can be glamorous and even a fashion statement reforms disability not as a limitation but as an opportunity for individuality. Another interesting point was the way simple designs such as the Muji CD player were framed as essentially inclusive. It made me question the products I use daily ,such as elevators without braille and phones with complex interfaces, and how they often don’t accommodate diverse needs. Some designs exclude certain groups by default by catering only to “average” users. In my opinion many innovations could be made if designers embraced the tension between accessibility and aesthetic appeal.

This reading changed my perspective on inclusivity in design, it’s not just about solving problems but embracing differences.

 

For my final project I had a couple of ideas of possible project. I am yet to narrow them down into one which I will work with as my project. I hope to do this as I progress to the next step of the assignment.

Idea 1

My first idea was to create a car with parking assistant simulator. My inspiration for this was trying to emulate the modern car parking assistance systems. My idea is to use distance sensors to detect how close the car is to surrounding objects. I would then use p5 to give a visual presentation of the parking scenario giving the user a visual feedback. I would also like to have alerts such as LED and buzzer warnings to give a better experience to the user

Idea 2

My second idea was to create a simple robot that can be controlled to move and possible wave and give data such as distance from objects around it. I plan to equip the robot with basic motion capabilities and a sensor to measure its distance from nearby objects. I intend to have the user interact with the robot through a controller or the keypad depending on what will be more convenient

Exercise 1.

p5 code:

let position;
let mass = 50;
let potValue = 0; 


function setup() {
  createCanvas(640, 360);
  noFill();

  // Initialize position of the ellipse
  position = createVector(width / 2, height / 2);

}

function draw() {
  background(255);

  // Map potentiometer value to horizontal position
  position.x = map(potValue, 0, 1023, 0, width);

  // Keep the ellipse in the middle 
  ellipse(position.x, height / 2, mass, mass);
}

function keyPressed(){
   if (key == " ") { //set up serial
     setUpSerial();
   } 
}

function readSerial(value) {
  potValue = int(value); // Parse the potentiometer value
}

Arduino Code:

void setup() {
  Serial.begin(9600); // Start serial communication 
}

void loop() {
  int potValue = analogRead(A0); // Read the potentiometer value 
  Serial.println(potValue);     // Send the value to p5.js
  delay(10);                    
}

Exercise 2.

p5 code:

function setup() {
  createCanvas(400, 400);
}

function draw() {
  background(225);

  // brightness based on cursor's x position
  let brightness = map(mouseX, 0, width, 0, 255);
  brightness = constrain(brightness, 0, 255); // Ensure brightness stays within range

  // Send brightness value to Arduino if serial is active
  if (serialActive) {
    writeSerial(`${brightness}\n`); 
  }
}

function keyPressed(){
   if (key == " ") { //set up serial
     setUpSerial();
   } 
}

function readSerial(value) {
  potValue = int(value); // Parse the potentiometer value
}

Arduino Code:

int ledPin = 9; 
int brightness = 0; 

void setup() {
  Serial.begin(9600); // Start serial communication
  pinMode(ledPin, OUTPUT); //LED pin as output
}

void loop() {
  if (Serial.available() > 0) {
    // Read the incoming brightness value
    brightness = Serial.parseInt();
    brightness = constrain(brightness, 0, 255);

    // Set the LED brightness
    analogWrite(ledPin, brightness);
  }
}

Exercise 3.

p5 code:

let velocity;
let gravity;
let position;
let acceleration;
let wind;
let drag = 0.99;
let mass = 50;
let potValue = 0; // potentiometer value
let serialConnected = false; 
let onGround = false; // whether ball is on the ground

function setup() {
  createCanvas(640, 360);
  noFill();
  position = createVector(width / 2, 0);
  velocity = createVector(0, 0);
  acceleration = createVector(0, 0);
  gravity = createVector(0, 0.5 * mass);
  wind = createVector(0, 0);
}

function draw() {
  background(255);
  if (!serialConnected) {
    textAlign(CENTER, CENTER);
    fill(0);
    text("Press 'Space' key on the keyboard", width / 2, height / 2);
    return;
  }

  applyForce(wind);
  applyForce(gravity);
  velocity.add(acceleration);
  velocity.mult(drag);
  position.add(velocity);
  acceleration.mult(0);
  
// check for bounce on the x axis
  if (position.x > width - mass / 2) {
    if(!onGround){
    position.x = width - mass / 2; 
    velocity.x *= -0.9;
    }
  }
    else if (position.x < mass / 2) {
      position.x = mass / 2; 
    velocity.x *= -0.9;
    }
  
  // check for bounce on the y axis
if (position.y > height - mass / 2) {
  velocity.y *= -0.9; 
  position.y = height - mass / 2; 
  if (!onGround) {
    sendBounceSignal(); 
    onGround = true; 
  }
} else {
  onGround = false; 
}

  wind.x = map(potValue, 0, 1023, -1, 1); 
  ellipse(position.x, position.y, mass, mass);

  // boundaries on x axis
  if (position.x > width - mass / 2 || position.x < mass / 2) {
    velocity.x *= -0.9; 
  }
}

function applyForce(force) {
  let f = p5.Vector.div(force, mass);
  acceleration.add(f);
}

// bouncing signal to Arduino
function sendBounceSignal() {
  if (serialActive) {
    let sendToArduino = "1\n"; 
    writeSerial(sendToArduino);
  }
}

// receive data from Arduino
function readSerial(data) {
    potValue = int(trim(data)); 
}

// initialize serial connection
function keyPressed() {
  if (key == ' ') {
    mass = random(10,100);
    setUpSerial(); 
    serialConnected = true; 
  }
}

function serialOpened() {
  serialActive = true; 
}

Arduino code:

const int potPin = A0; 
const int ledPin = 13; 
int potValue = 0; 
bool ledState;     

void setup() {
  pinMode(ledPin, OUTPUT);
  Serial.begin(9600);
}

void loop() {
  // read the potentiometer value
  potValue = analogRead(potPin);

  // send potentiometer value to p5
  Serial.println(potValue);

  // check for data from p5
  if (Serial.available() > 0) {
    char received = Serial.read(); 

    if (received == '1' && !ledState) {
      // turn on the LED 
      digitalWrite(ledPin, HIGH);
      ledState = true;
      delay(100); 
      // turn off the LED
      digitalWrite(ledPin, LOW);  
    }
    else if (received != '1'){
      ledState = false;
    }
  }
  delay(50); 
}

Video demonstration:

 

 

Concept

For this project, we thought of recreating a recognizable music theme from film/TV series that we could easily manipulate. We ended up on the eerie and haunting Stranger Things theme, as it is truly one of those few captivating intros that you would never skip. We also decided to dedicate this project to the upcoming final season of the series, which will be released next year.

By using the ultrasonic distance meter and tricolor LED, we made a musical instrument that is responsive to the distance of the hand: the music speeds up when the hand is closer and slows down when it goes farther, and the LED changes its color accordingly.

Sketch

Code Highlight

void loop() {
buttonState = digitalRead(buttonPin);

if (buttonState == LOW) { 
for (int thisNote = 0; thisNote < 48; thisNote++) {

// distance measured with ultrasonic sensor

digitalWrite(trigPin, LOW);
delayMicroseconds(2);
digitalWrite(trigPin, HIGH);
delayMicroseconds(10);
digitalWrite(trigPin, LOW);

duration = pulseIn(echoPin, HIGH);
distance = duration * 0.034 / 2;

// mapping the tricolor LED values

int redValue = map(distance, 5, 50, 255, 100); 
int greenValue = 0; 
int blueValue = map(distance, 5, 50, 100, 255);

analogWrite(redPin, redValue);
analogWrite(greenPin, greenValue);
analogWrite(bluePin, blueValue);

// mapping to adjust duration between notes

int adjustedNoteDuration = map(distance, 5, 50, StrangerThingsNoteDurations[thisNote] / 2, StrangerThingsNoteDurations[thisNote] * 2);

// playing with the new tempo

tone(buzzerPin, StrangerThingsMelody[thisNote], adjustedNoteDuration);

// a short pause

delay(adjustedNoteDuration * 1.3);
}

} else {

// when button is not pressed, turning off the LED and buzzer

analogWrite(redPin, 0);
analogWrite(greenPin, 0);
analogWrite(bluePin, 0);
noTone(buzzerPin);
}

delay(50); 
}

It was difficult at first to find the music in the proper format so that it can be implemented into the Arduino code. As we tried to somehow extract the notes from the .mxl (.musicxml) files, we couldn’t find a software that could do that. Hence, we ended up recreating the soundtrack by using the array of sequences (representing the 48 notes and their durations) that we luckily found on the GitHub.

We using mapping for the tricolour LED lights, keeping greenValue at zero so that we could get a somewhat “gradient” effect.

The key (and most challenging) aspect in the code was to speed up the music according to the varied distance. We used mapping to set the shortest and longest duration of the music, as well as implemented  a delay by multiplying the adjustedNoteDuration by 1.3 (a factor commonly used in Arduino to maintain a natural pacing) to add a slight pause between notes, making each note more distinct.

Embedded Sketch

GitHub

A  Brief Rant on the Future of Interaction Design

Bret Victor’s article was both frustrating and inspiring to me. I had not realised how number I’d become to the limitations of  “Pictures Under Glass” which he argues that it is a compromised way to interact with technology. His words made me think of how I have often felt detached from my daily devices like my phone. The smooth, untouchable screens which might seem convenient deny me any real physical feedback making interactions feel shallow.

The article also made me think about what “innovation” really means. To me progress meant having the devices faster and sleeker but now I wonder whether this progress is moving us further from natural, embodied ways of knowing. It leaves me wondering how we can design technology that will genuinely respect and use our physicality. Technology that will feel like extensions of ourself than barriers. Another weird realisation is the fact that the more advanced our devices become, the less they seem to engage with what makes us fundamentally human.

Concept

My idea for this project was to come up with something similar to an alarm system. I chose to work with the ultrasonic distance sensor to create an alarm based on proximity. I implement the system using to LED’s where one is for indicating the distance and is the analog control and the other is the alarm indicator, digital control. The latter LED gives a visual feedback of the distance from an oncoming object and acts as an alert. When the object is outside the 40 cm radius of the sensor the LED is completely off indicating no threat but when an object moves into the 40cm radius the LED starts to light gradually depending on the distance from the sensor; when distance is big the LED is dim and when distance continues to be shorter and shorter the LED also gradually increases brightness to alert someone to turn on the alarm.

Sketch/Setup

Code Highlight

void loop() {
  // Distance from sensor
  int distance = getDistance();

  // State of the switch
  alarmEnabled = digitalRead(switchPin) == LOW; 

  // Analog LED brightness based on distance
  int brightness = map(distance, 5, 50, 255, 0); // distances between 5 cm and 50 cm
  brightness = constrain(brightness, 0, 255);     // Limit brightness to range
  analogWrite(analogLedPin, brightness);

  // If alarm is enabled and distance is below threshold, blink alarm LED
  if (alarmEnabled && distance < distanceThreshold) {
    digitalWrite(digitalLedPin, HIGH);
    delay(200);
    digitalWrite(digitalLedPin, LOW);
    delay(200);
  } else {
    digitalWrite(digitalLedPin, LOW); // Keep alarm LED off
  }

  //Debug
  Serial.print("Distance: ");
  Serial.print(distance);
  Serial.print(" cm | Alarm: ");
  Serial.println(alarmEnabled ? "ON" : "OFF");

  delay(100); 
}

The above is the part of my code which I am proud of how I implemented. In the code I use the distance value which is picked from the sensor to control both LEDs. I control the analog LED based on the distance and to do this I limit the LEDs readings to the distance by mapping then to have a gradual increase in brightness I use constrain so that the difference can be visually perceived.

Arduino File on GitHub

Video demonstration

Reflection and Future improvements

Implementing the project was really fun as it gave me a space to actualise my idea of creating an alarm in an interesting way. I am proud of how I implemented the alarm system. However, there is still room for improvement as I would want  the system to have sound which goes off when the alarm is triggered. Overall, I enjoyed executing the project and further understanding Arduino.

Concept

My switch was a pretty simple implementation. I was fascinated by the light sensor and I thought I should use it to implement the switch. The idea was simple; turn on the LED when it is dark and off when there is light and using a light sensor was the best choice.

Setup

Code

const int ldrPin = 2;    // Pin connected to the LDR
const int ledPin = 13;   // Pin connected to the LED

void setup() {
  pinMode(ldrPin, INPUT);   // LDR pin as input
  pinMode(ledPin, OUTPUT);  //LED pin as output
}

void loop() {
  int lightStatus = digitalRead(ldrPin); // LDR value

  if (lightStatus == LOW) {
    // It’s dark, turn on the LED
    digitalWrite(ledPin, HIGH);
  } else {
    // It’s bright, turn off the LED
    digitalWrite(ledPin, LOW);
  }
}

My code was simple to implement as I just modified the example given in class. The code takes in the light sensor pin as input and LED pin as output then it reads the value of the light sensor which is dependent on if light is present or not. Depending on the light sensor value the code will turn on or turn off the LED.

Arduino file on GitHub

Sketch

IMG_3439

Reflection and Future improvements

Implementing the Arduino was really fun. I was able to learn more on connecting circuits properly and also using the digital read. I am proud of being able to use the light sensor which was very fascinating and fun to implement. For the future I hope to learn more about the capabilities of Arduino and possibly create a light sensitive switch which will light the LED depending on the amount of light in the room and not just light up and turn off.