Author: Sneheel Sarangi

Example 1:

// Map the sensor value to the width of the canvas
  ellipseX = map(latestData, 0, 1023, 0, width);

  // Draw the ellipse at the mapped position
  ellipse(ellipseX, height/2, 50, 50);

Example 2:

I simply map the mouseX coordinate to the light’s brightness!

if (mouseIsPressed) {
light=map(mouseX,0,width,0,1023);
}

 

Example 3:

let velocity;
let gravity;
let position;
let old_position;
let acceleration;
let wind;
let wind_arduino=0;
let drag = 0.99;
let mass = 50;
let light =0;

function setup() {
  createCanvas(640, 360);
  noFill();
  position = createVector(width/2, 0);
  velocity = createVector(0,0);
  acceleration = createVector(0,0);
  gravity = createVector(0, 0.1*mass);
  wind = createVector(0,0);
  noLoop();
}

function draw() {
  background(255);
  
  if(wind_arduino<512)
    wind.x=-1;
  else
    wind.x=1;
  applyForce(wind);
  applyForce(gravity);
  velocity.add(acceleration);
  velocity.mult(drag);
  old_position=position.y;
  position.add(velocity);
  acceleration.mult(0);
  ellipse(position.x,position.y,mass,mass);
  if (position.y > height-mass/2) {
      velocity.y *= -0.9;  // A little dampening when hitting the bottom
      position.y = height-mass/2;
    }
   print(height-mass/2);
  if (position.y >= 334)
  {
    light=1;
  }
  else
    light=0;
  if(old_position==position.y)
    light=0;
}

function applyForce(force){
  // Newton's 2nd law: F = M * A
  // or A = F / M
  let f = p5.Vector.div(force, mass);
  acceleration.add(f);
}

function keyPressed(){
  if (key==' '){
    loop();
    setUpSerial();
  }
}

// This function will be called by the web-serial library
// with each new *line* of data. The serial library reads
// the data until the newline and then gives it to us through
// this callback function
function readSerial(data) {
  ////////////////////////////////////
  //READ FROM ARDUINO HERE
  ////////////////////////////////////

  if (data != null) {
    // make sure there is actually a message
    // split the message
    let fromArduino = split(trim(data), ",");
    // if the right length, then proceed
    if (fromArduino.length == 1) {
      // only store values here
      // do everything with those values in the main draw loop
      
      // We take the string we get from Arduino and explicitly
      // convert it to a number by using int()
      // e.g. "103" becomes 103
      print(fromArduino[0]);
      wind_arduino = int(fromArduino[0]);
    }

    //////////////////////////////////
    //SEND TO ARDUINO HERE (handshake)
    //////////////////////////////////
    let sendToArduino = light+"\n";
    writeSerial(sendToArduino);
  }
}

My idea for the final project involves building a robot like character that can do simple interaction with a person in front of it. On the Arduino side, this will involve the use of several sensors such as the distance sensor, and the buttons.

This project can be as complicated as necessary and the interactions can be altered at any point.

Some example interactions include petting the robot – and it will make a sound. and talking to it.

Another interaction could be using 2 distance sensors as the eyes and then calculating an object’s distance – then using a servo motor to follow the object.

On the p5 side, I plan to use the p5.speech library that has both speech transcription and text to speech functionality. Additionally, if possible I’d like to use the ChatGPT API to generate the speech, and additionally keep track of each interaction that can be used to define the robot’s current ‘mood” -which decides the further interactions.

Design Meets Disability

Reflecting on “Design and Disability,” it’s clear that the approach to designing for disabilities is evolving. The shift from a medical to a social model highlights that disability is shaped by societal attitudes, not just medical conditions. This is exemplified by the transformation of eyewear from medical devices to fashion statements, challenging traditional views of assistive devices.

However, the integration of disability-focused design into mainstream design must be handled carefully. There’s a need to balance inclusivity with recognizing the unique needs of people with disabilities, avoiding a one-size-fits-all approach.

The role of constraints in driving innovation is also key. Design limitations, especially in disability contexts, can lead to creative, functional, and aesthetically pleasing solutions.

The reading broadens this perspective, urging us to rethink design beyond functionality, considering cultural and aesthetic impacts. Design isn’t just about creating objects; it’s about shaping cultural values and perspectives.

Lastly, design plays a crucial role in influencing societal attitudes towards disabilities. Embracing inclusive design is a step towards a more equitable society, where creativity caters to everyone’s needs. The readings collectively underscore the importance of considering both functional and societal aspects in designing for disability.

A Brief Rant About The Future of Interactive Design

It was fascinating to read an article about the future’s vision and direction – 12 years into the future.  Comparing Bret Victor’s ideas and complaints to what actually ended up transpiring, I am struck by both how correct he was and how much improvement is still possible.

In 2011, Victor dreamed of a future with interactivity design that involved more than just sliding and tapping on pictures behind a screen. Today, while we still continue to do  so (albeit with a few haptic gimmicks like he puts) – it is also true that we may directly be moving towards a future quite unlike this. Personally, my first experience with any kind of haptics or virtual movement simulation was the Nintendo Wii with motion detection. Today the technology has not just improved but we seem to be on the cusp of a virtual reality revolution. Virtual Reality systems have improved by leaps and bounds year upon year and soon we may reach a world where there is mainstream adoption of such technologies in everyday use.

I believe that while the future we have today would be immensely disappointing to the Bret Victor who wrote this post, the immediate future seems to be much more exciting. I am excited to see the digital future of mankind move towards a completely new direction!

Concept:

While working on this project, we went over several ideas on how we would make an instrument. One thing we discovered in our ideation process was that both of us had previously made projects that centrally involved cats. Thus, with some more tinkering around – we came up with “The Cat Composer”! The Cat Composer is an all in one musical instrument that can play a variety of simple tunes such as “Hot Cross Buns” and “Mary Had a Little Lamb”. It consists of 2 switches to control 2 beat-making servo motors, a distance sensor to control the notes (C, D, E, and G), and a turning potentiometer to toggle between octaves. Additionally, we incorporated a speaker from the IM Lab inventory to make a better sound than the provided buzzer. This instrument is best played with two people, one to play percussion/toggle between octaves, and one to play the notes.

However, with a bit of practice it is completely possible to play it by oneself! Note: In order for the distance sensor to receive a steady input and play the correct notes, it is best to play not with one’s hands, but with a larger piece of material.

Demonstration Video:

Code & Highlights:

The toughest part in the coding process was to ensure that the distance sensor worked exactly as we intended. For example, an issue that we ran into early was the abrupt changing of tune at the border values. Since the sensor isn’t accurate to the exact cm. It would then fluctuate between two tunes. We corrected this by instead using a 5-value moving average. This makes transitions significantly smoother (and the experience much more enjoyable!!)

unsigned int measureDistance() {
  const int numReadings = 5;  // Number of readings to average
  const int maxChange = 150;   // Maximum acceptable change in cm between readings
  static unsigned int lastAverage = 0;  // Store the last valid average distance
  unsigned long totalDuration = 0;


  for (int i = 0; i < numReadings; i++) {
    digitalWrite(trigPin, LOW);
    delayMicroseconds(2);
    digitalWrite(trigPin, HIGH);
    delayMicroseconds(10);
    digitalWrite(trigPin, LOW);


    totalDuration += pulseIn(echoPin, HIGH);
    delay(10); // Short delay between readings
  }


  unsigned int currentAverage = (totalDuration / numReadings) * 0.034 / 2;


  // Check if the change from the last average is within the expected range
  if (abs((int)currentAverage - (int)lastAverage) <= maxChange || lastAverage == 0) {
    lastAverage = currentAverage;  // Update the last valid average
    return currentAverage;
  } else {
    return lastAverage;  // Return the last valid average if the current reading is an outlier
  }
}

Reflections and Improvements:

We can improve our project significantly given more time!

Firstly, we would love the diversify the sounds our project can generate. In our research we discovered that instead of simply using tone() we could perhaps use some other sound generating function. We would love to try this!

Regarding the hardware implementation, the provided potentiometer is too hard to turn and often messes with the wiring. Instead we would love to use a better/larger potentiometer that allows us better access.

Similarly, another change we would like to do is to use a single Arduino Board and breadboard rather than our current 2 board solution. This will make the project more cohesive. Even though this seems easy enough to implement, we let our current design be as of now to simplify our approach.

Lastly, the ultrasonic distance sensor often gives outlier readings. As discussed in the highlight section, we tried our best to resolve this issue, however it still persists. We have some more ideas to remedy this. But we believe that given the scope of this project this was unnecessary. However, we would love to do this in the future.

Making Interactive Art: Set the Stage, Then Shut Up and Listen

This piece highlights the difference in making interactive art and making art. While artists (and viewers alike) generally believe that a work of art is an expression or a statement – interactive art is much different in several aspects. The artist’s role isn’t to make a statement but rather start a conversation. The viewer will then interact with this and have an experience that ideally will communicate what you want it to.

Here Tom Igoe gives practical advice as to how to make interactive art – that ultimately boils down to setting the stage, shutting up and listening to the audience. At the start of the course, this idea would have seemed foreign to me, but through both the readings and personal experiences I have come to realize what Tom intends to say here. Additionally, I love the example of the director Tom uses here, and his overall writing style!

Lastly, I believe interactive art – as the way Tom puts it – in a way serves to liberate the artist. It suggests that as an artist, one doesn’t have to bear the entire burden of meaning or impact. Instead, by creating a framework for interaction and then stepping back, an artist can allow the artwork to breathe, grow, and morph through each interaction. At the same time, it also poses a challenge: can an artist resist the urge to dictate, to instead become a facilitator of experience?

Physical Computing’s Greatest Hits (and misses)

Here Tom Igoe provides us with several examples of the kinds of physical computing works that show up frequently. Of these, I would like to reflect on 3 ideas in particular:

I particularly liked the idea of a “meditation helper”. Beyond directly building something to help with meditation. What really interests me is the concept of reading a person’s heart rate, breathing rate, etc. These factors are easily accessible these days, and it will be very interesting to build technology infused clothing that is practical and useful.

The Scooby Doo painting art type seems overdone but something about it still remains intensely creepy. I would love to incorporate something of that manner in a project I do going forwards.

Lastly, floor pads are also exciting. I am curious as to if we can somehow engineer them in a way that lets us walk in place (like a treadmill and have that actually simulate a walk for a character in real-time etc. Moreover, there’s a lot more that can be done with such pads if we keep adding layers of complexity!

Approach and Concept:

It took me a long time to come up with the idea for this project given the requirements it had. But at the end of our previous lecture, I realized I could make a night-light+morning alarm system.

For my analog input I use the light sensor we employed in class. In the morning, this turns on the buzzer (digital output 1) and the red LED light( digital output 2). While it’s night (low light) the blue LED (analog output) blinks in a slow and satisfying manner. A button (digital input switch) can be used to turn the whole system off or on at any given time.

I believe this whole system, if refined further a bit has actual practical utility, and I’m really happy I could make something like this on my own!

Highlights:

I would like to highlight both the coding and the hardware work for this project:

Coding: The system uses a state management system. When I tried to use code from the examples we used in the lecture. This became complicated because doing things like making the analog output led blink slowly couldn’t be done through for loops – or the delay would mess up with the state switching mechanism. To navigate through this, I had to remove the for loops and instead work directly with the loop function. Similarly, I faced this issue again with the buzzer where the delay’s blocking nature was causing some problems – leading me to use the external ezBuzzer library. Beyond this, joining together so many components itself was a fun challenge!

void loop() {
  buzzer.loop();
  // read the state of the pushbutton value:
  buttonState = digitalRead(buttonPin);
  int analogValue = analogRead(A0);
  // check if the pushbutton is pressed. If it is, the buttonState is HIGH:
  if (buttonState == HIGH && led_state==0) {
    while(buttonState==HIGH)
    {
      buttonState = digitalRead(buttonPin);
    // turn LED on:
      led_state=1;
    }
  }
  else if(buttonState == HIGH && led_state==1)
  {
    while(buttonState ==HIGH)
    {
      buttonState = digitalRead(buttonPin);
      led_state=0;
    }
  }
  Serial.println(led_state);
  digitalWrite(ledPin,led_state && led_stateD);
  if(led_state==0 || led_stateD==0)
  {
    if (buzzer.getState() != BUZZER_IDLE) {
      buzzer.stop() ; // stop
    }
  }
  else 
  { if (buzzer.getState() == BUZZER_IDLE) {
      int length = sizeof(noteDurations) / sizeof(int);
      buzzer.playMelody(melody, noteDurations, length); // playing
    }
  }
  if (analogValue < 300) {

    led_stateD=0;
    led_stateA=1;
  }
  else {
    led_stateD=1;
    led_stateA=0;
  }
  if(led_state==1 && led_stateA){
  // fade in from min to max in increments of 5 points:
  if(millis()%10==0) {
    // sets the value (range from 0 to 255):
    fadeValue+=increment;
    analogWrite(ledPinAnalog, fadeValue);
    if (fadeValue==255 || fadeValue==0)
      increment*=-1;
  }}
  if(led_state==0 ||led_stateA==0)
    analogWrite(ledPinAnalog, 0);

}

 

Hardware: The connections itself were simple enough in themselves – since I had knowledge of what we had done in class to borrow from. However, given the many elements the board itself got messy – making navigation a hassle.

Reflections and Future Work:

There are several improvements I would like to make on this work. On the technical implementation aspect, I can possible manage the wires I use much better than I have done here. And on the coding aspect, the code can definitely be made more streamlined and easier to read.

In terms of future work, I would like to add more lights to the nightlight aspect, and make the blinking pattern more complex to have a more soothing experience. Similarly, the buzzer noise and the morning alarms can also be improved aesthetically. In terms of functionality, it would be nice to have another button than can function as a snooze button.

 

 

Approach and Concept:

After spending a lot of time brainstorming ideas – I found one I was particularly proud of. The phrases “it happened in the blink of an eye “, “blink and you miss it” etc. are often used to talk about phenomena that happen really quickly. The literal meaning behind them is that it may as well happen that the phenomenon will start and end in the time it takes for you to finish a blink – leaving you with nothing to observe.

So how about we build a system that only does ‘something’ when you close your eyes? Thus now, every time you blink, or close your eyes – you miss ‘it’. You never actually have the experience but everyone around you does. I was personally satisfied with the concept but it was quite difficult to actually implement it given the very limited range of motion of our eyelids. Nonetheless, I ended up succeeding in doing so:

The mechanism that connects the switch to the eyelid movement. (double sided tape connects the thread to your eyelid. a piece of cardboard covered with aluminum foil and weighed down further using a coin serves as the conducting bridge.)

The simplified circuit

The switchboard. Two wires are taped to a piece of cardboard and aluminum foil is used to increase the surface area of conduction.

Demonstration:

Highlights:

The part I’m the most proud of in this project is devising the exact mechanism for the switch to work. Unlike the p5.js coding assignments, arduino and physical hardware is something I have no experience in – so this was my first time actually working with these elements and I had a lot of fun finding solutions to problems as they came up.

For example, just taping the wires to the cardboard and using the aluminum foil as the switch worked – but it wasn’t as reliable due to the uncertain points of contact between the wire ends and the foil. I managed to remedy this by putting a layer of aluminum foil on the wire ends.

Next, the switch mechanism itself didn’t have enough weight so it wouldn’t rest properly on the wire ends. I changed this by putting a small coin as to add some weight to the mechanism – and this ended up working spectacularly.

Reflections and Future Work:

I’m proud of several decisions I made in the design of this switch as I stressed earlier. However, there are quite a few things I would do differently if I had to redo the project:

First, I would find a more effective way to connect my eyelids to the switch. The current method using double-sided tape works but is very inefficient. I am still unsure as to how I would do this though.

Second, the switch itself is fairly robust to perturbations – but I believe it can be improved further aesthetically and functionally. I can use a more stable base and find better configurations to place the aluminum foil such that it covers the best area possible.

Lastly, since all my time was spent focusing on the switch, my actual design for the lights can definitely improve by a lot. I would like to add more LEDs to make my design prettier.

 

On Norman and the Importance of Attractive Design:

In this follow-up reading we had to Norman’s previous work – Norman feels the need to clarify that when he didn’t mention aesthetics in his previous writing “The Design of Everyday Things” he didn’t put functionality above aesthetics, but rather on the same level. I never got the impression this was the case, but nonetheless, it’s a welcome clarification.

Throughout the reading, Norman stresses the synergy of affect and cognition – not only are emotions driven by cognition,  cognition is also influenced by affect. I particularly love the rigorous way in which Norman points out how aesthetics can make things more ‘functional’ by themselves. Moreover, going into this reading, I did not expect to read an analysis of how mental/emotional states affect our mental processing – Norman’s discussion of depth first processing etc. – a pleasant surprise as this is a topic I have read about before!

On a more practical level, working to make your designs pleasant and relaxing for the user (or in general being conscious of the emotional state we want the user to be in when experiencing our work) seems extremely important. I have personally experienced this when even though my website designs were often technically better than my peers, I would myself prefer theirs because they looked pretty. Moreover, (though not recommended) even implementation flaws can be hidden by an aesthetic design – but are glaring when your designs are not aesthetic. From the perspective of psychology, perceptual sets, and first impressions of an aesthetic design deciding the overall experience more than the technical details seems obvious too!

I want to end this reflection with this beautiful quote by Norman: “True beauty in a product has to be more than skin deep, more than a facade. To be truly beautiful, wondrous, and pleasurable, the product has to fulfill a useful function, work well, and be usable and understandable”

 

On Hamilton’s Contributions to the Space Mission:

Margaret Hamilton’s narrative with the Apollo program is not just a testament to her pioneering efforts but a remarkable moment in the history of software. On a personal level, it echoes some sentiments I’ve personally experienced in my coding journey.  For example, the evolution of software from an afterthought to a critical component during the Apollo moon landings parallels many projects I’ve encountered, where the true importance of effective software design only shows up later.

Moving on, this deep-dive into Hamilton’s contributions also reveals an essential trait in software design: predictive error handling. I can recall several instances in my own projects where I had neglected potential edge cases. When a user unexpectedly entered data in a manner I hadn’t anticipated, causing the software to crash. While this is an important concept drilled into students who learn coding today – I imagine the roots of how it came to be thought of as a fundamental concept can be traced back to Hamilton. Additionally Hamilton’s championing of asynchronous processing in the Apollo computer also sounds way ahead of the time they were operating in and is incredibly impressive.

In sum, Margaret Hamilton’s journey with the Apollo program, filled with foresight, innovation, and resilience, is not just historical but a beacon for every coder. Her story doesn’t just echo the challenges and triumphs in the software domain but resonates with the intricacies of my personal coding experiences. It’s a reminder that in the realm of coding, challenges are but stepping stones to innovation.

 

 

Overall Concept:

Through this project, I take a chance to reflect on my work and progress during the first half of the semester. In broad terms, it is a compilation of my work in this class, designed as a mini-game where the user themselves has to jump over obstacles I faced. The idea for this concept came to me when I realized how happy I was with my evolution in terms of artistic expression as the class went on, and I wanted everyone to get a glimpse of this evolution.

The game itself is simple enough and can be thought of as a simplified version of the Google Chrome Dinosaur game – with only ground obstacles. (it is deceptively hard!). However, the design is more complex. For example, since I want my viewers to wait and watch the artworks unfold – I made it such that no new obstacles spawn when the cat is stationary. Lastly, I went for a dreamy – after image trail- look for aesthetic purposes.

Working and Strengths:

As I stated earlier, the game itself can be thought of as a simplified version of the Chrome Dinosaur game. Your journey starts the moment you step into the cat’s paws. The left and right arrow keys move the cat, and the spacebar allows it to jump. Additionally, pressing 1 lets you click a screenshot and save it to your device. Whereas, pressing 2 lets you restart the game. You also have the option to toggle the audio on/off.

The progression of the game happens at the user’s own pace. Obstacles only spawn as the cat keeps moving, and they are randomized in size – being one of three variants: “Error”, “Deadline” and “Breakdown”. The cat’s movements include a sitting animation cycle and a walking animation cycle. Aesthetically, I went for a simple color pallet making my main character – a cat white, and the background black, that complimented my color usage in previous projects. The font style I chose resembles handwritten text – a decision made to further highlight the personal tone of the work. I also went for a generally upbeat theme to add to the work!

 

My cat sprites in motion!

 

On a technical level, the collision detection and mechanics were surprisingly hard to implement due to the length of the cat. Even when I got the implementation right, it was nigh impossible to play the game as intended (as it would lead to a continuous triggering of the collision mechanism). I needed to find a way to make this mechanism work without increasing the cat’s speed and worsening the overall experience.

Ultimately, I realized I could also increase the speed on the X axis while jumping, that made the game a lot smoother.

function jump() {
  if (catY >= groundHeight - catHeight / 2) {
    velocity += jumpForce;
    speed = boostSpeed; // boost the speed during the jump
  }
}    //jump function

function checkCollision(obs) {
  return (
    camX + width / 2 + (catWidth / 2) * 0.6 -50 > obs.x &&
    camX + width / 2 - (catWidth / 2) * 0.6 +40< obs.x + obs.width &&
    catY + (catHeight) * 0.6+30 > obs.y &&
    catY - (catHeight) * 0.6 -30< obs.y + obs.height
  );
}

Another area whose technicality I am proud of is the cat animations. I only had a sprite sheet for a right walk cycle. In code, I used this to also work for a left walk cycle (by flipping). But more importantly, I managed to find a way to connect my sitting cycle with my walk cycle. The cat now sits down when you are not moving, and then gets back up (in reverse the sit cycle) when you start again!

/ Check direction of camera movement to adjust cat's direction
    if (camX > lastCamX) {
      direction = -1;
    } else if (camX < lastCamX) {
      direction = 1;
    }

    // If camera is not moving and the cat was walking previously
    if (camX == lastCamX && wasMoving) {
      wasMoving = false;
      isWalking = false;
      isSitting = true;
      sitFrames.reverse();
      currentFrame = 0;
    }
// If camera starts moving and the cat was sitting previously
    if (camX != lastCamX && !wasMoving) {
      wasMoving = true;
      isSitting = false;
      isWalking = true;
      sitFrames.reverse();
      currentFrame = 0;
    }

// Draw the cat
    let frameToDraw;
    if (isWalking) {
      frameToDraw = walkFrames[currentFrame % 12];
      if (frameCount % 2 == 0)
        //making the animation slower
        currentFrame = (currentFrame + 1) % 12;
    } else if (isSitting) {
      frameToDraw = sitFrames[currentFrame];
      if (frameCount % 2 == 0 && currentFrame < 6) {
        currentFrame++;
      }
    }

On a more fun note, while working on the project I realized a peculiarity of the code – if you try to run away from the obstacles – they start bunching up and appear together more frequently – a fact we all know to be true from experience.

Reflections and Future Work:

Throughout this project, I ran into several problems – a couple of which I already highlighted in the above section. In general, staging the whole game was quite the task (as it involved basically a moving background created by a camera movement (that we simulate by a constant translation based on arrow keys)). This could only be resolved with meticulous testing.

Moving on, I’ve identified two crucial components of my project that might require some improvement: the way it appears and the way it functions. I believe I can further improve the project’s aesthetics and visual appeal. To make it more appealing, the background and cat’s appearance can be improved. Additionally, I want to make the text on the obstacles more legible. On a functional level – the hitboxes of the collision functions aren’t yet perfectly set – I’d like to explore using an elliptical hitbox than a rectangular one.

Finally, I would love to include my expectations and plans for the second half of the semester, as well as any abstract ideas or goals I may have as the ending scene – something that I didn’t have time to implement.

 

References:

Music:

Voxel Revolution – Kevin Macleod

Digital Lemonade – Kevin Macleod