“It’s not a weapon, it’s more of a highly advanced prosthesis.” ― Tony Stark (when the court asks if his Iron Man suit is a weapon)
Reflecting upon Graham Pullin’s Design Meets Disability, I find myself musing about this quote from Tony Stark in Iron Man. The suit, while seemingly a robust piece of technology, parallels a fashionable prosthesis, both functional and aesthetically pleasing with its intricate yet discreet components. Indeed, his suit is a prosthesis. This amusing interpretation not only highlights the suit’s dual nature but also aligns with broader design principles.
Transitioning from the fictional world of Iron Man to the real-world impact of design, Charles Eames’s perspective that “design depends largely on constraints” resonates deeply with me. This principle is vividly illustrated in the creation of the Eames leg splint during wartime—a necessity that spurred innovation. The splint was not only functional but also aesthetically pleasing, subsequently influencing mainstream furniture design. This example beautifully encapsulates the dance between limitation and innovation, challenging the preconceived notion that design for disability must forsake beauty for functionality.
The juxtaposition of fashion and discretion in disability design is equally eye-opening. The evolution of eyewear from a stigmatized medical appliance to a fashion accessory exemplifies a significant shift in societal perception. However, the underlying desire for discreet devices reveals a lingering societal discomfort with visible signs of disability. Graham Pullin’s insights provoke a reevaluation of this narrative, advocating for a shift from concealing disabilities to celebrating and empowering individuals.
Pullin’s text not only expands my understanding of functional design but also reaffirms the importance of aesthetics in user-centered design. Whether it pertains to physical objects like a leg splint or digital solutions like software interfaces, the essential principle remains the same: effective design must resonate with users on both functional and aesthetic levels, thereby enhancing their interaction and experience.
Make something that uses only one sensor on Arduino and makes the ellipse in p5 move on the horizontal axis, in the middle of the screen, and nothing on Arduino is controlled by p5
For this exercise, we used a potentiometer as our analog sensor. The Arduino program sends the reading from the sensor over to p5, where it is mapped to the range [ellipseWidth/2, canvasWidth-ellipseWidth/2]. The mapped value is then used to determine the x position of the center of the ellipse. Since nothing on Arduino is controlled by p5, p5 sends dummy data that is ignored by p5 to maintain the two-way handshake.
// ===== P5 =========
let diameter = 60;
let sensorVal; // read from p5 transmitted message
function setup() {
createCanvas(640, 480);
sensorVal = 1023/2;
}
function draw() {
background(220);
if (!serialActive) {
text("Press Space Bar to select Serial Port", 20, 30);
} else {
text("Connected", 20, 30);
}
fill(0);
ellipse(map(sensorVal, 0, 1023, diameter/2, width-diameter/2), height/2, diameter, diameter)
}
// === ARDUINO ====
void loop() {
// wait for data from p5 before doing something
while(Serial.available()){
digitalWrite(LED_BUILTIN, HIGH); // led on while receiving data
int _ = Serial.parseInt(); // get dummy variable sent over by p5
if(Serial.read() == '\n'){
delay(5);
int sensor = analogRead(A1); // read sensor value
delay(5);
Serial.println(sensor); // send sensor value
}
}
digitalWrite(LED_BUILTIN, LOW);
}
2. Make something that controls the LED brightness from p5
For this exercise, we created a slider that controls the brightness of the LED. The range of the slider matches that of the LED analog value ranges, from 0 to 255. Hence, we only had to send that value over to the Arduino. Once received, the Arduino program writes that value to the LED PWM pin in an analog fashion. For visualization, our p5 sketch draws a circle whose diameter is proportional to the LED brightness.
// ===== p5 ======
let slider;
function setup() {
createCanvas(640, 480);
sensorVal = 1023/2;
slider = createSlider(0, 255, 0);
}
function draw() {
background(0);
fill(0,0,255);
noStroke();
let diameter = map(slider.value(), 0, 255, 0, height);
ellipse(width/2, height/2, diameter, diameter);
fill(255);
if (!serialActive) {
text("Press Space Bar to select Serial Port", 20, 30);
} else {
text("Connected", 20, 30);
}
}
function keyPressed() {
if (key == " ") {
setUpSerial();
}
}
function readSerial(data) {
if (data != null) {
let sendToArduino = slider.value() + '\n';
print(sendToArduino)
writeSerial(sendToArduino);
}
}
// ===== ARDUINO =====
void loop() {
// analogWrite(leftLedPin, pwm);
// wait for data from p5 before doing something
while (Serial.available()) {
digitalWrite(LED_BUILTIN, HIGH); // led on while receiving data
pwm = Serial.parseInt(); // get dummy variable sent over by p5
if (Serial.read() == '\n') {
delay(5);
analogWrite(rightLedPin, pwm);
delay(5);
Serial.println("0");
}
}
digitalWrite(LED_BUILTIN, LOW);
}
3. Take the gravity wind example and make it so every time the ball bounces one LED lights up and then turns off, and you can control the wind from one analog sensor
We needed to establish bi-directional meaningful data exchange in this exercise. The first is to send the Arduino a signal from p5 when the ball hits the ground. This is done by checking if the bottom portion of the ellipse touches the canvas’s lower boundary position.y + mass/2 == height . If this evaluates to true, a signal of 1 is sent over. Otherwise, 0 is sent. When the Arduino program receives the signal, it turns the LED on or off accordingly. Similarly, the Arduino sends the value of the potentiometer (our chosen sensor) over to p5, where it gets mapped to the range [-1, 1]. The mapped value is set towind.x. The closer the potentiometer value is to the extremes, the stronger the wind effect on the ball. Sensor values below 512 also get mapped to negative values, and the ball motion trajectory goes leftward. Otherwise, the wind effect goes in the opposite direction.
// ===== p5 =======
let velocity;
let gravity;
let position;
let acceleration;
let wind;
let drag = 0.99;
let mass = 50;
let analogVal = 0;
let dir = 0;
function setup() {
createCanvas(640, 360);
noFill();
position = createVector(width/2, 0);
velocity = createVector(0,0);
acceleration = createVector(0,0);
gravity = createVector(0, 0.5*mass);
wind = createVector(0,0);
}
function draw() {
background(220);
applyForce(wind);
applyForce(gravity);
velocity.add(acceleration);
velocity.mult(drag);
position.add(velocity);
acceleration.mult(0);
if (!serialActive) {
text("Press S to select Serial Port", 20, 30);
} else {
text("Connected", 20, 30);
}
fill(0);
ellipse(position.x,position.y,mass,mass);
if (position.y > height-mass/2) {
velocity.y *= -0.9; // A little dampening when hitting the bottom
position.y = height-mass/2;
}
}
function applyForce(force){
// Newton's 2nd law: F = M * A
// or A = F / M
let f = p5.Vector.div(force, mass);
acceleration.add(f);
}
function keyPressed(){
if (keyCode==LEFT_ARROW){
wind.x=-1;
}
if (keyCode==RIGHT_ARROW){
wind.x=1;
}
if (key==' '){
mass=random(15,80);
position.y=-mass;
velocity.mult(0);
}
if (key == 's') {
// important to have in order to start the serial connection!!
setUpSerial();
}
}
function readSerial(data) {
if (data != null) {
// make sure there is actually a message
let fromArduino = data;
// if the right length, then proceed
if (fromArduino.length > 0) {
analogVal = int(data);
}
wind.x = map(analogVal, 0, 1023, -1, 1);
let sendToArduino;
if (position.y + mass/2 == height){
sendToArduino = 1 + '\n';
}
else{
sendToArduino = 0 + '\n';
}
writeSerial(sendToArduino);
}
}
// ======= Arduino ======
void loop() {
// wait for data from p5 before doing something
while (Serial.available()) {
digitalWrite(LED_BUILTIN, HIGH); // led on while receiving data
ledState = Serial.parseInt(); // get ball bounce indicator from p5.js
if (Serial.read() == '\n') {
// turn on LED accrodingly
if (ledState == 1){
digitalWrite(rightLedPin, HIGH);
}
else{
digitalWrite(rightLedPin, LOW);
}
currSensorVal = analogRead(A1); // read sensor value
Serial.println(currSensorVal); // send sensor value
}
}
digitalWrite(LED_BUILTIN, LOW);
}
Make something that uses only one sensor on Arduino and makes the ellipse in p5 move on the horizontal axis, in the middle of the screen, and nothing on arduino is controlled by p5
let rVal = 0;
let alpha = 255;
let left = 0; // True (1) if mouse is being clicked on left side of screen
let right = 0; // True (1) if mouse is being clicked on right side of screen
let xPos = 0; // Position of the ellipse
function setup() {
createCanvas(640, 480);
textSize(18);
noStroke();
}
function draw() {
// one value from Arduino controls the background's red color
background(255, 255, 255);
// Update the position of the ellipse based on the alpha value
// Mapping alpha to the width of the canvas
xPos = map(alpha, 0, 1023, 0, width);
// Draw an ellipse that moves horizontally across the canvas
fill(255, 0, 255, 255); // Same mapping as the text transparency
ellipse(xPos, height / 2, 50, 50); // Position ellipse at center height with a diameter of 50
if (!serialActive) {
text("Press Space Bar to select Serial Port", 20, 30);
} else {
text("Connected", 20, 30);
// Print the current values
text('rVal = ' + str(rVal), 20, 50);
text('alpha = ' + str(alpha), 20, 70);
}
}
function keyPressed() {
if (key == " ") {
// important to have in order to start the serial connection!!
setUpSerial();
}
}
function readSerial(data) {
if (data != null) {
let fromArduino = split(trim(data), ",");
if (fromArduino.length == 2) {
rVal = int(fromArduino[0]);
alpha = int(fromArduino[1]);
}
let sendToArduino = left + "," + right + "\n";
writeSerial(sendToArduino);
}
}
Production 2
Make something that controls the LED brightness from p5
let rVal = 0;
let alpha = 255;
let left = 0; // True (1) if mouse is being clicked on left side of screen
let right = 0; // True (1) if mouse is being clicked on right side of screen
let xPos = 0; // Position of the ellipse
let ledState = false; // To toggle LEDs
function setup() {
createCanvas(640, 480);
textSize(18);
noStroke();
}
function draw() {
// one value from Arduino controls the background's red color
background(255, 255, 255);
// Update the position of the ellipse based on the alpha value
// Mapping alpha to the width of the canvas
xPos = map(alpha, 0, 1023, 0, width);
if (!serialActive) {
text("Press Space Bar to select Serial Port", 20, 30);
} else {
text("Connected", 20, 30);
// Print the current values
text('rVal = ' + str(rVal), 20, 50);
text('alpha = ' + str(alpha), 20, 70);
}
text("Then, press A to flash the police lights.",20, 100)
//Edit the code below
// Toggle LED state every frame if space is held down
if (keyIsDown(65)) { // 32 is the ASCII code for space
ledState = !ledState;
if (ledState) {
left = 1;
right = 0;
} else {
left = 0;
right = 1;
}
}
// Edit the code above
}
function keyPressed() {
if (key == " ") {
// important to have in order to start the serial connection!!
setUpSerial();
}
}
function keyReleased(){
if (key == "a") {
left = 0;
right = 0;
}
}
function readSerial(data) {
if (data != null) {
let fromArduino = split(trim(data), ",");
if (fromArduino.length == 2) {
rVal = int(fromArduino[0]);
alpha = int(fromArduino[1]);
}
let sendToArduino = left + "," + right + "\n";
writeSerial(sendToArduino);
}
}
let velocity;
let gravity;
let position;
let acceleration;
let wind;
let drag = 0.99;
let mass = 50;
let rVal = 0;
let alpha = 255;
let left = 0; // True (1) if mouse is being clicked on left side of screen
let right = 0; // True (1) if mouse is being clicked on right side of screen
let serialSetup = false;
// Declare a variable to store the time at which to trigger the action
let triggerTime = 0;
function setup() {
createCanvas(640, 360);
noFill();
position = createVector(width / 2, 0);
velocity = createVector(0, 0);
acceleration = createVector(0, 0);
gravity = createVector(0, 0.5 * mass);
wind = createVector(0, 0);
// Set the trigger time to be 5 seconds after setup runs
triggerTime = millis() + 5000;
textSize(20);
}
function draw() {
// Check if the current time has passed the trigger time
if (millis() >= triggerTime && !serialSetup) {
serialSetup = true;
}
push();
fill(0);
if (!serialSetup) {
text("Simulation starts in 5 seconds. Press a to set up serial.", 20, 50);
}
pop();
if (serialSetup) {
background(255);
let windtext = "";
if (alpha < 300) {
wind.x = -1;
windtext = "Wind : Right";
} else if (alpha > 800) {
wind.x = 1;
windtext = "Wind : Left";
} else {
wind.x = 0;
windtext = "No wind.";
}
push();
fill(0);
text(windtext, 20, 50);
pop();
applyForce(wind);
applyForce(gravity);
velocity.add(acceleration);
velocity.mult(drag);
position.add(velocity);
acceleration.mult(0);
ellipse(position.x, position.y, mass, mass);
if (position.y > height - mass / 2) {
velocity.y *= -0.9; // A little dampening when hitting the bottom
position.y = height - mass / 2;
console.log(
"Bounce! Position Y: " + position.y + ", Velocity Y: " + velocity.y
);
left = 1;
} else {
left = 0;
}
}
}
function applyForce(force) {
// Newton's 2nd law: F = M * A
// or A = F / M
let f = p5.Vector.div(force, mass);
acceleration.add(f);
}
function keyPressed() {
if (keyCode == LEFT_ARROW) {
wind.x = -1;
}
if (keyCode == RIGHT_ARROW) {
wind.x = 1;
}
if (key == " ") {
mass = random(15, 80);
position.y = -mass;
velocity.mult(0);
}
if (key == "a") {
// important to have in order to start the serial connection!!
setUpSerial();
}
}
// This function will be called by the web-serial library
// with each new *line* of data. The serial library reads
// the data until the newline and then gives it to us through
// this callback function
function readSerial(data) {
////////////////////////////////////
//READ FROM ARDUINO HERE
////////////////////////////////////
if (data != null) {
// make sure there is actually a message
// split the message
let fromArduino = split(trim(data), ",");
// if the right length, then proceed
if (fromArduino.length == 2) {
// only store values here
// do everything with those values in the main draw loop
// We take the string we get from Arduino and explicitly
// convert it to a number by using int()
// e.g. "103" becomes 103
rVal = int(fromArduino[0]);
alpha = int(fromArduino[1]);
}
//////////////////////////////////
//SEND TO ARDUINO HERE (handshake)
//////////////////////////////////
let sendToArduino = left + "," + right + "\n";
writeSerial(sendToArduino);
}
}
Personally, I liked this reading. The article Design meets Disability, mentions points which I hadn’t previously considered but are intuitive. While the author gave multiple examples of designs that were made to be more accessible while not highlighting one’s disability there were a few that stood out to me.
The most obvious product which is meant to help people with a disability are glasses. Like the article mentioned people don’t view glasses the same way as a wheelchair or a crutch or an arm sling. People see glasses as aesthetically appealing and as someone that wears glasses I haven’t seen as it as a disability either. I believe that the reason for this is mostly that glasses are very commonplace but that also they can be both aesthetically appealing and discrete.
The theme of aesthetically appealing and discrete is consistent for designs that are meant to also help those with a disability. The author mentioned the iPod but really it made me consider a previous reading we had. The previous reading discussed how something that is aesthetically appealing can be helpful for creativity and that people are willing to overlook small details. I wondered if a product that may not be as technically advanced as another can still sell more if the user-experience is far greater. I think this has already been shown to be the case with the Macintosh, and Macs in general, but also the iPhone which has a cult-following (including myself). While some people stick to the iPhone because of the apple ecosystem being very convenient I think a lot of people (and I am speaking from anecdotal experiences) like the user-interface both because it is intuitive and aesthetically appealing whilst being discrete, and because they have been using it for such a long time that it’s inconvenient to change. Many android phones have better features and some are cheaper and yet people, like myself, still choose to buy an iPhone.
Going into the final project now I want to try to make the design aesthetically appealing and keep certain parts discrete depending on the function. For example any buttons or switches that should be used by the players will be easy to see and everything else will hopefully add to the aesthetic whilst not being distracting. Hopefully I can have this done before user-testing so that I can change it before the final deadline if needed.
Make something that uses only one sensor on Arduino and makes the ellipse in p5 move on the horizontal axis, in the middle of the screen, and nothing on arduino is controlled by p5
let rVal = 0;
let alpha = 255;
let left = 0; // True (1) if mouse is being clicked on left side of screen
let right = 0; // True (1) if mouse is being clicked on right side of screen
let xPos = 0; // Position of the ellipse
function setup() {
createCanvas(640, 480);
textSize(18);
noStroke();
}
function draw() {
// one value from Arduino controls the background's red color
background(255, 255, 255);
// Update the position of the ellipse based on the alpha value
// Mapping alpha to the width of the canvas
xPos = map(alpha, 0, 1023, 0, width);
// Draw an ellipse that moves horizontally across the canvas
fill(255, 0, 255, 255); // Same mapping as the text transparency
ellipse(xPos, height / 2, 50, 50); // Position ellipse at center height with a diameter of 50
if (!serialActive) {
text("Press Space Bar to select Serial Port", 20, 30);
} else {
text("Connected", 20, 30);
// Print the current values
text('rVal = ' + str(rVal), 20, 50);
text('alpha = ' + str(alpha), 20, 70);
}
}
function keyPressed() {
if (key == " ") {
// important to have in order to start the serial connection!!
setUpSerial();
}
}
function readSerial(data) {
if (data != null) {
let fromArduino = split(trim(data), ",");
if (fromArduino.length == 2) {
rVal = int(fromArduino[0]);
alpha = int(fromArduino[1]);
}
let sendToArduino = left + "," + right + "\n";
writeSerial(sendToArduino);
}
}
Production 2
Make something that controls the LED brightness from p5
let rVal = 0;
let alpha = 255;
let left = 0; // True (1) if mouse is being clicked on left side of screen
let right = 0; // True (1) if mouse is being clicked on right side of screen
let xPos = 0; // Position of the ellipse
let ledState = false; // To toggle LEDs
function setup() {
createCanvas(640, 480);
textSize(18);
noStroke();
}
function draw() {
// one value from Arduino controls the background's red color
background(255, 255, 255);
// Update the position of the ellipse based on the alpha value
// Mapping alpha to the width of the canvas
xPos = map(alpha, 0, 1023, 0, width);
if (!serialActive) {
text("Press Space Bar to select Serial Port", 20, 30);
} else {
text("Connected", 20, 30);
// Print the current values
text('rVal = ' + str(rVal), 20, 50);
text('alpha = ' + str(alpha), 20, 70);
}
text("Then, press A to flash the police lights.",20, 100)
//Edit the code below
// Toggle LED state every frame if space is held down
if (keyIsDown(65)) { // 32 is the ASCII code for space
ledState = !ledState;
if (ledState) {
left = 1;
right = 0;
} else {
left = 0;
right = 1;
}
}
// Edit the code above
}
function keyPressed() {
if (key == " ") {
// important to have in order to start the serial connection!!
setUpSerial();
}
}
function keyReleased(){
if (key == "a") {
left = 0;
right = 0;
}
}
function readSerial(data) {
if (data != null) {
let fromArduino = split(trim(data), ",");
if (fromArduino.length == 2) {
rVal = int(fromArduino[0]);
alpha = int(fromArduino[1]);
}
let sendToArduino = left + "," + right + "\n";
writeSerial(sendToArduino);
}
}
let velocity;
let gravity;
let position;
let acceleration;
let wind;
let drag = 0.99;
let mass = 50;
let rVal = 0;
let alpha = 255;
let left = 0; // True (1) if mouse is being clicked on left side of screen
let right = 0; // True (1) if mouse is being clicked on right side of screen
let serialSetup = false;
// Declare a variable to store the time at which to trigger the action
let triggerTime = 0;
function setup() {
createCanvas(640, 360);
noFill();
position = createVector(width / 2, 0);
velocity = createVector(0, 0);
acceleration = createVector(0, 0);
gravity = createVector(0, 0.5 * mass);
wind = createVector(0, 0);
// Set the trigger time to be 5 seconds after setup runs
triggerTime = millis() + 5000;
textSize(20);
}
function draw() {
// Check if the current time has passed the trigger time
if (millis() >= triggerTime && !serialSetup) {
serialSetup = true;
}
push();
fill(0);
if (!serialSetup) {
text("Simulation starts in 5 seconds. Press a to set up serial.", 20, 50);
}
pop();
if (serialSetup) {
background(255);
let windtext = "";
if (alpha < 300) {
wind.x = -1;
windtext = "Wind : Right";
} else if (alpha > 800) {
wind.x = 1;
windtext = "Wind : Left";
} else {
wind.x = 0;
windtext = "No wind.";
}
push();
fill(0);
text(windtext, 20, 50);
pop();
applyForce(wind);
applyForce(gravity);
velocity.add(acceleration);
velocity.mult(drag);
position.add(velocity);
acceleration.mult(0);
ellipse(position.x, position.y, mass, mass);
if (position.y > height - mass / 2) {
velocity.y *= -0.9; // A little dampening when hitting the bottom
position.y = height - mass / 2;
console.log(
"Bounce! Position Y: " + position.y + ", Velocity Y: " + velocity.y
);
left = 1;
} else {
left = 0;
}
}
}
function applyForce(force) {
// Newton's 2nd law: F = M * A
// or A = F / M
let f = p5.Vector.div(force, mass);
acceleration.add(f);
}
function keyPressed() {
if (keyCode == LEFT_ARROW) {
wind.x = -1;
}
if (keyCode == RIGHT_ARROW) {
wind.x = 1;
}
if (key == " ") {
mass = random(15, 80);
position.y = -mass;
velocity.mult(0);
}
if (key == "a") {
// important to have in order to start the serial connection!!
setUpSerial();
}
}
// This function will be called by the web-serial library
// with each new *line* of data. The serial library reads
// the data until the newline and then gives it to us through
// this callback function
function readSerial(data) {
////////////////////////////////////
//READ FROM ARDUINO HERE
////////////////////////////////////
if (data != null) {
// make sure there is actually a message
// split the message
let fromArduino = split(trim(data), ",");
// if the right length, then proceed
if (fromArduino.length == 2) {
// only store values here
// do everything with those values in the main draw loop
// We take the string we get from Arduino and explicitly
// convert it to a number by using int()
// e.g. "103" becomes 103
rVal = int(fromArduino[0]);
alpha = int(fromArduino[1]);
}
//////////////////////////////////
//SEND TO ARDUINO HERE (handshake)
//////////////////////////////////
let sendToArduino = left + "," + right + "\n";
writeSerial(sendToArduino);
}
}
One of the many things I appreciated about this week’s reading is how expansive and detailed its exploration of the relationship between design and disability is. Indeed, for a long time, designing for disability, as a discipline or practice, was brushed aside and accessibility was achieved merely by asking what augmentations need to be introduced to allow certain groups of people with disabilities the ability to use a platform or a product. The chapter begins by examining products built for those with disabilities and the role of fashion and artistic designers in normalizing and destigmatizing these products, effectively engulfing designs for those with disabilities into the wider culture of “mainstream” fashion. This latter inclusion affords people with disabilities both the luxury of choice and the long-awaited pleasure of finally being seen, both of which are things most people take for granted. We can see this sentiment reflected in the desire of athlete and model Aimee Mullins to find prosthetics that are “off-the-chart glamorous.” Incorporating artists in conversations about design for disability, which have been largely dominated by those with clinical and engineering backgrounds, is imperative in finding the sweet spot between functionality and aesthetics. I believe it is, however, important that those who are designing for disability are keen on involving the target user in their design process – the same way they would be if they were designing for any other target audience.
Interweaved within this aforementioned discussion is an emphasis on language. Integrating such designs into fashion means integrating them into a larger system of cultural artefacts, one of which is language. I enjoyed the delineation of the linguistic evolution of “spectacles” to “eyewear”. “Patients” become “wearers”. The term “HearWear” is proposed in place of “hearing aids.” These are illustrations of how designers can actively contribute to changing cultural and societal outlooks, which is an essential prerequisite for actualizing inclusion.
One belief I held prior to reading this chapter was that designs had to be universally inclusive. After all, what is so difficult about ensuring that we have a multimodal interface that can accommodate all users? What I failed to realize is that additive complexity, as was pointed out by the article, would potentially create unintended inaccessibility. The idea of designing appliances and platforms that are effectively “flying submarines” is bound to leave us with an influx of products that are overly intricate, but with subpar performance in any singular task. It seems like what we need is a diversity of robust designs and products, for each type of user, that can perform their intended tasks optimally with elegant simplicity. We can only begin to reframe how we think of designing for disability – designing for inclusion – by inviting more than just engineers, whose aim is merely to achieve functionality and problem-solving, into the conversation.
For my final project, I am creating a cooking game that engagingly explores Palestinian culture. The dish created is “Mansaf” with the help of “Teta” meaning grandma. The game will consist of several tasks for the dish. The visuals will be on p5 and so will the instructions. I will use several sensors such as a button, joystick, and motion sensor… to complete the cooking tasks. Through audio, visuals, and the hands-on game experience I think the outcome will be very interesting. Of course, some challenges will include p5 and Arduino communication which is why I’ll address them early on.
We connected the Arduino to the P5 sketch using a serial monitor. We did this by downloading a Javascript app from the P5 serial library which acts as an intermediary to allow p5 to access hardware via serial port: available here. This has both access to the javascript browser and connected Arduino. The code below is responsible for declaring variables, handling serial communication (events such as (connection, port listing, data reception, errors, port opening, and closing.), and storing data. To complete the connection we used the code below:
let serial;
let latestData = "waiting for data";
let ellipseX; // Variable to store the x-coordinate of the ellipse
function setup() {
createCanvas(400, 400);
serial = new p5.SerialPort();
serial.list();
serial.open("/dev/tty.usbmodem101");
serial.on('connected', serverConnected);
serial.on('list', gotList);
serial.on('data', gotData);
serial.on('error', gotError);
serial.on('open', gotOpen);
serial.on('close', gotClose);
}
function serverConnected() {
print("Connected to Server");
}
function gotList(thelist) {
print("List of Serial Ports:");
for (let i = 0; i < thelist.length; i++) {
print(i + " " + thelist[i]);
}
}
function gotOpen() {
print("Serial Port is Open");
}
function gotClose() {
print("Serial Port is Closed");
latestData = "Serial Port is Closed";
}
function gotError(theerror) {
print(theerror);
}
function gotData() {
let currentString = serial.readLine();
trim(currentString);
if (!currentString) return;
console.log(currentString);
latestData = currentString;
}
The x-coordinate of the ellipse is specified by ellipseX, which is updated based on the sensor data in the gotData() function. This allows the ellipse to move horizontally across the canvas based on the sensor readings.
// Update the x-coordinate of the ellipse based on the sensor data
ellipseX = map(parseInt(latestData), 0, 1023, 0, width);
}
function draw() {
background(255);
fill(0);
// Draw the ellipse at the middle of the screen with dynamic x-coordinate
ellipse(ellipseX, height/2, 50, 50);
}
Demo:
Task 2: Use p5 to change LED brightness.
et rVal = 0;
let alpha = 255;
let upArrow = 0;
let downArrow = 0;
function setup() {
createCanvas(640, 480);
textSize(18);
// Replaced Space bar with an actual Button for Serial Connection c
const connectButton = createButton('Connect to Serial');
connectButton.position(10, height + 30);
connectButton.mousePressed(setUpSerial); // Call setUpSerial when the button is pressed
}
function draw() {
background(map(rVal, 0, 1023, 0, 255), 255, 255);
fill(255, 0, 255, map(alpha, 0, 1023, 0, 255));
if (serialActive) {
text("Connected", 20, 30);
let message = upArrow + "," + downArrow + "\n";
writeSerial(message);
} else {
text("Press 'Connect to Serial' button", 20, 30);
}
text('rVal = ' + str(rVal), 20, 50);
text('alpha = ' + str(alpha), 20, 70);
}
// Decided on keyPressed as it is more straight forward
// Up Arrow Key turns the light on
function keyPressed() {
if (keyCode === UP_ARROW) {
upArrow = 1;
} else if (keyCode === DOWN_ARROW) {
downArrow = 1;
}
}
// Down Key turns the light off
function keyReleased() {
if (keyCode === UP_ARROW) {
upArrow = 0;
} else if (keyCode === DOWN_ARROW) {
downArrow = 0;
}
}
// This function will be called by the web-serial library
// with each new *line* of data. The serial library reads
// the data until the newline and then gives it to us through
// this callback function
function readSerial(data) {
////////////////////////////////////
//READ FROM ARDUINO HERE
////////////////////////////////////
if (data != null) {
// make sure there is actually a message
// split the message
let fromArduino = split(trim(data), ",");
// if the right length, then proceed
if (fromArduino.length == 2) {
// only store values here
// do everything with those values in the main draw loop
// We take the string we get from Arduino and explicitly
// convert it to a number by using int()
// e.g. "103" becomes 103
rVal = int(fromArduino[0]);
alpha = int(fromArduino[1]);
}
//////////////////////////////////
//SEND TO ARDUINO HERE (handshake)
//////////////////////////////////
let sendToArduino = left + "," + right + "\n";
writeSerial(sendToArduino);
}
}
Demo:
Task 3:
Code Snippet:
if (!bounced) {
bounced = true; // Update bounce state
} else {
bounced = false; // Reset bounce state
}
} else {
ledState = 0;
}
I have three ideas for my final project, each emerging from my interdisciplinary interests.
The first idea I would like to propose is creating a Sign Language Glove, aimed at facilitating communication and improving accessibility for individuals who are deaf or hard of hearing. I shall limit it to fingerspelling using the alphabet for now. The glove will incorporate flex sensors on each finger to detect bending movements. Arduino will process this data and send the finger configurations to a p5.js sketch, which will interpret the gestures and recognize the corresponding letters of the alphabet.
The p5.js screen will display the recognized letters visually and audibly using text-to-speech. Additionally, users will have the option to input letters via a keyboard to display the corresponding Sign for it on the screen. This interactive system enables individuals that use sign language to have two-way communication with non-sign language users effectively.
I initially thought of using American Sign Language (ASL), but the issue is a lot of the signs have the same finger positions, and it will be difficult to differentiate the signs.
An alternative is using Indian Sign Language, which uses two hands, but can overcome the above issue. However, this adds complexity of checking 10 finger configurations.
My second idea is conducting a psychology experiment utilizing p5.js for the visual presentation of stimuli and Arduino for participant response collection. I aim to design either Perception experiments, such as Visual search tasks, which involve participants searching for a target stimulus among distractors, or Cognition experiments, which may involve memory tasks, where participants memorize and recall sequences of stimuli presented, or face recognition tasks, where participants identify familiar faces. In these experiments, the p5.js sketch will display visual stimuli, while Arduino buttons will serve as response inputs.
Eg, in the visual search tasks, the p5.js screen will display each of the trials and participants will use buttons connected to the Arduino board to indicate when they have found the target stimulus. Arduino will record response times and accuracy.
At the end of the experiment session, participants will be able to view their performance metrics and compare them to group averages or previous trials. This setup allows for the seamless integration of psychological experimentation with interactive technology, facilitating data collection and analysis in a user-friendly manner.
For my third project idea, I propose creating an interactive system that generates music from art! The user will be able to draw on the p5.js canvas, creating their unique artwork. The system will then analyze this artwork pixel by pixel, extracting the RGB values of each pixel. These RGB values will be averaged to create a single value for each pixel, which will then be mapped to a musical note. Additionally, the system will detect sharp changes in color intensity between adjacent pixels, indicating transitions in the artwork. These transitions will determine the length of each note, with sharper changes resulting in shorter notes. The coordinates of each drawn point can influence the tempo or volume of the music, to make it sound better. Once the music composition is generated in p5.js, it will be sent to Arduino, where a piezo buzzer will play the music in real-time. This interactive system lets users create their own art and music.
While I found the reading’s focus on human capabilities to be a very insightful perspective concerning interaction design, I found its focus on the use of hands to be both limiting and outdated.
In the past several decades, technologies have developed with the consideration of hands as human’s main ‘capability’. We type, swipe, scroll, hold and even move devices with our hands in order to generate a given output – this has become second nature to us.
However, I believe that the future of immersive and seamless human-centred design revolves around moving beyond this. I feel that the utilisation of other physical human inputs can be used to maximised interaction design, both from the perspective of immersion and ease of use.
An example of this being used to provide seamless experiences for users is the use of facial recognition to unlock smartphones. By taking advantage of the front camera’s natural position when a user picks up their device, designers have been able to eliminate the tedious action of typing in a passcode or scanning a thumbprint.
Conversely, full-body immersion has been utilised in game consoles such as the Xbox Live and Wii. In these cases, sensors were put in use to revolutionise how players interact with games, effectively deconstructing the notion that playing video games is a lazy and inactive process. Despite the minimal success of these consoles, the application of full body immersion seen in them can be used as a reference for other interactive experiences such as installations and performances.
