Category: S2026 – Aya

“Vision of the Future” videos have been a thing since the early 21st century, the amazing hologram screen, or people waving their hands in the air to move digital windows around (I mean we have augmented reality for that now). It looks futuristic, sophisticated and innovative, but from an interaction perspective, it is actually incredibly timid.

I agree with what Bret is saying, a tool is supposed to amplify human capabilities, converting what we can do into what we want to do, and if that entire principle is gone, then it isn’t just not a good tool, it is not a tool at all.

Most modern devices ignore the two things hands do best, feeling and manipulating, so how can we call these ideas revolutionary if its going backwards?

Bret’s point about “finger-blindness” is actually terrifying to think about, what we do for granted the future generation may struggle with. If we do not use our hands to feel texture, weight, and pliability, we lose the ability to understand the “inner meaning” of objects. We are building a world where we can spend our entire lives immobile, starting at a “hokey visual facade” that has no physical connection to the work we are doing.

If the future of interaction does not let us see, feel, and manipulate space simultaneously, then it is not a future worth building or investing it.

Demo Below:

Concept:

We came up with a piano that utilizes your keyboard presses and a buzzer, using keys from A-L allows you to play 9 different notes. We added an LCD that displays the note of each key and the frequency of that note.

Implementation:

Schematic:

The components used are pretty simple, just being an LCD device and a buzzer. We wrote 2 files of code for this, a python file and an c++ file. As typing the letter into the serial monitor every time you wanted to play a note would be counter-intuitive, we wrote a python file that listens to your key presses, and if you press a key between A and L, then it will send that key press to the arduino which ends playing the note that correlates to that key press.

try:
    arduino = serial.Serial('COM11', 9600, timeout=1)
    ...
except:
    ...

while True:
    if keyboard.is_pressed('a'):
        arduino.write(b'A')
        time.sleep(0.15) 
    elif keyboard.is_pressed('s'):
        arduino.write(b'S')
        time.sleep(0.15)
    elif keyboard.is_pressed('d'):
        arduino.write(b'D')
        time.sleep(0.15)
    elif keyboard.is_pressed('f'):
        arduino.write(b'F')
        time.sleep(0.15)
    elif keyboard.is_pressed('g'): 
        arduino.write(b'G')
        time.sleep(0.15)
    elif keyboard.is_pressed('h'):
        arduino.write(b'H')
        time.sleep(0.15)
    elif keyboard.is_pressed('j'):
        arduino.write(b'J')
        time.sleep(0.15)
    elif keyboard.is_pressed('k'):
        arduino.write(b'K')
        time.sleep(0.15)
    elif keyboard.is_pressed('l'):
        arduino.write(b'L')
        time.sleep(0.15)

    if keyboard.is_pressed('esc'):
        print("Closing...")
        break

arduino.close()

Here we first try to connect to the arduino using the port and the buad rate that is on the arduino IDE, then until the program stops, we check for any key presses, and if it matches one of our conditional statements, we write that letter to the arduino.

switch (key) {
  case 'A': frequency = 262; noteName = "C4"; break;
  case 'S': frequency = 294; noteName = "D4"; break;
  case 'D': frequency = 330; noteName = "E4"; break;
  case 'F': frequency = 349; noteName = "F4"; break;
  case 'G': frequency = 392; noteName = "G4"; break;
  case 'H': frequency = 440; noteName = "A4"; break;
  case 'J': frequency = 494; noteName = "B4"; break;
  case 'K': frequency = 523; noteName = "C5"; break;
  case 'L': frequency = 587; noteName = "D5"; break;
  default: return; // Ignore any other keys
}

Here we have a switch statement which checks whether we got a matching letter, which then returns the respective frequency and note. We got these frequencies for each note from here: https://en.wikipedia.org/wiki/Piano_key_frequencies as we wanted it to sound as similar as possible to a piano. The LCD shows the note you play and the frequency of that note when you play it. A potentiometer is used to control the contrast of the LCD!

GitHub Link!

Reflection:

Currently this is a single press piano meaning you can’t play multiple multiple notes at once, so an improvement that can be made is to find some way to be able to play multiple notes at once, otherwise this works perfectly, and is simple and accessible to anyone!

I found this week’s readings quite eye-opening. The video at the beginning of the article was accurately how I imagine the “future” of technology: exactly what it is now, but unlocks more gesture possibilities and projects onto anything. This reminds me of a rumored concept for the iPhone 6 that I got quite excited over in 2013, deeming it to be innovative and futuristic.

While I was upset that Apple did not end up releasing this concept as the iPhone 6, this article made me hope they never release such a phone.

I feel like the article made me realize innovation has kind of paused in the last few years. My friends and I often discuss this question of “if we wanted to invent something new, what could we invent?” because we feel like everything we need, that has a realistic solution, has already been invented. However, this article made me think, what if videos and visions like the ones presented in the article are the reason we are unable to think outside of the box and invent new ideas. The vision for the future often consists of simply combining different existing technologies into one, or expanding current technologies, halting the invention of ground-breaking inventions. It almost feels like phones with a touch screen was the last time humanity witnessed a truly impressive and shocking invention.

It was interesting to see that the author wasn’t trying to provide a solution, but instead wanted to raise awareness of the problem and push further research. He emphasizes that current tools aren’t fully taking advantage of human capabilities, and when discussing things like physical keyboards or styluses, he dismisses them as not being “dynamic mediums.” This made me think about interactive media, where we experiment with more dynamic forms of interaction by combining physical inputs, sensors, and outputs to create systems that respond in real time.

However, I don’t fully agree with his view on voice. While he limits it to simple commands, I think voice has potential as a more dynamic form of interaction, especially when combined with other inputs. When he talks about the three categories of gestures, it reminded me of things like voice assistants, where we use a kind of “vocabulary” to communicate with the system, similar to saying commands like “Hey Siri.” The idea of spatial gestures at a distance also connects to examples we briefly saw in class, where movement controls objects on a screen, even if it can feel indirect. The last example, directly manipulating a virtual 3D space like Iron Man, was a bit confusing to me, but it shows a more ambitious and futuristic idea of interaction. Overall, even though he critiques these methods, I think they still have potential, especially when combined together, rather than being dismissed individually.

What stood out to me was the idea that tools should be designed around what humans are actually capable of doing, not just what technology can do. When he said “if a tool isn’t designed to be used by a person, it can’t be a very good tool” it made me think about how often we accept interfaces that don’t fully match how we naturally interact with the world.

For example, with touchscreens, most interactions are reduced to tapping or sliding, which feels limited compared to how expressive our hands actually are.

This connects to what we’ve been doing in class with circuits and Arduino, where interaction feels more physical and responsive. When building circuits, we’re not just coding something to appear on a screen we’re creating systems where human actions, like touching foil or changes in light, directly affect outputs like LEDs. In my project, I used inputs and conditions to map real world interactions to responses, which felt more aligned with how we naturally engage with objects. This reflects the idea of amplifying human capabilities, because the system responds to touch and environmental changes rather than limiting interaction to a flat surface. It also made me realize that even simple projects like ours explore more meaningful interaction than typical touchscreen interfaces, since they involve feedback, physical input, and a closer connection between the user and the technology.

Arduino GitHub File:

https://github.com/MouzaAlMheiri/Intro-to-IM/blob/main/sketch_ap12_Week10.2.ino

Arduino Set-up:

Project Demo:

IMG_0334

Images

Concept

This project is a simple musical instrument that uses both digital and analog input at the same time.

The button acts as the digital sensor because it only has two states, either pressed or not pressed. When we press it, the sound plays, and when we release it, the sound stops.

The potentiometer acts as the analog sensor because it gives a continuous range of values instead of just two states. We used that range to select different musical notes, so turning the knob changes the pitch.

What we liked about this setup is that both inputs have completely different roles. The button controls when the instrument is played, while the potentiometer controls what sound is produced. It made the difference between digital and analog feel really clear and actually useful.

Code Snippet We’re Proud Of

int index = map(potValue, 0, 1023, 0, 7);

if (buttonState == LOW) {

  tone(buzzerPin, notes[index]);

} else {

  noTone(buzzerPin);

}

This part is where everything comes together. The potentiometer gives a value from 0 to 1023, and we use map() to convert that into a smaller range that matches the number of notes we have. Then we use that number to pick a note from the array.

At the same time, the button decides whether the note should actually play. So one input controls the pitch, and the other controls when the sound happens, which made it feel more like a real instrument instead of just a buzzer making random noise.

Problems Encountered

The biggest challenge was honestly the wiring. Even when everything looked right, one wire in the wrong row would break the whole circuit. We had to be really precise with the breadboard and double check every connection.

The button also gave us trouble at first. It either didn’t work or stayed on all the time, and we realized it was because of how it was placed across the gap and which rows we were using. Once we fixed that, it started behaving correctly.

Another challenge was understanding how the potentiometer connects to the sound. At first it felt random, but once we understood that the Arduino reads values from 0 to 1023 and that we needed to map that to our notes, it made a lot more sense.

Reflection

This project helped us actually understand the difference between digital and analog input instead of just memorizing it. The button made sense as something binary, while the potentiometer showed how values can change continuously.

It also made us more comfortable working with sound. Before this, the buzzer just felt like something that makes noise, but now we understand how pitch is controlled and how different inputs can affect it.

If we were to improve this project, we would probably expand the number of notes or organize them into a more structured scale so it feels more like a playable instrument.

Overall, it was a really successful assignment and working in pairs made it a lot easier to think and refine ideas and carry each other throughout the trial and error process of the entire project!

References

https://projecthub.arduino.cc/SURYATEJA/use-a-buzzer-module-piezo-speaker-using-arduino-uno-cf4191

Arduino Project | Play Melody with Passive buzzer using Arduino UnoYouTube · IoT Frontier3 Jul 2023

YouTube · Tech Explorations1.3K+ views  ·  1 year ago[240] Arduino Getting Started: Make noise and beeps with the passive buzzer

Arduino GitHub File

(pitches.h) file

Arduino Set-up:

Arduino Illustration:

Project Demo:

IMG_0396

Concept

This project is a simple musical instrument that uses both digital and analog input at the same time.

The button acts as the digital sensor because it only has two states, either pressed or not pressed. When we press it, the sound plays, and when we release it, the sound stops.

The potentiometer acts as the analog sensor because it gives a continuous range of values instead of just two states. We used that range to select different musical notes, so turning the knob changes the pitch.

What we liked about this setup is that both inputs have completely different roles. The button controls when the instrument is played, while the potentiometer controls what sound is produced. It made the difference between digital and analog feel really clear and actually useful.

Code Snippet We’re Proud Of

int index = map(potValue, 0, 1023, 0, 7);

if (buttonState == LOW) {

  tone(buzzerPin, notes[index]);

} else {

  noTone(buzzerPin);

}

This part is where everything comes together. The potentiometer gives a value from 0 to 1023, and we use map() to convert that into a smaller range that matches the number of notes we have. Then we use that number to pick a note from the array.

At the same time, the button decides whether the note should actually play. So one input controls the pitch, and the other controls when the sound happens, which made it feel more like a real instrument instead of just a buzzer making random noise.

Problems Encountered

The biggest challenge was honestly the wiring. Even when everything looked right, one wire in the wrong row would break the whole circuit. We had to be really precise with the breadboard and double check every connection.

The button also gave us trouble at first. It either didn’t work or stayed on all the time, and we realized it was because of how it was placed across the gap and which rows we were using. Once we fixed that, it started behaving correctly.

Another challenge was understanding how the potentiometer connects to the sound. At first it felt random, but once we understood that the Arduino reads values from 0 to 1023 and that we needed to map that to our notes, it made a lot more sense.

Reflection

This project helped us actually understand the difference between digital and analog input instead of just memorizing it. The button made sense as something binary, while the potentiometer showed how values can change continuously.

It also made us more comfortable working with sound. Before this, the buzzer just felt like something that makes noise, but now we understand how pitch is controlled and how different inputs can affect it.

If we were to improve this project, we would probably expand the number of notes or organize them into a more structured scale so it feels more like a playable instrument.

Overall, it was a really successful assignment and working in pairs made it a lot easier to think and refine ideas and carry each other throughout the trial and error process of the entire project!

References 

https://projecthub.arduino.cc/SURYATEJA/use-a-buzzer-module-piezo-speaker-using-arduino-uno-cf4191

Arduino Project | Play Melody with Passive buzzer using Arduino UnoYouTube · IoT Frontier3 Jul 2023

YouTube · Tech Explorations1.3K+ views  ·  1 year ago[240] Arduino Getting Started: Make noise and beeps with the passive buzzer

When I watched the 6-minute video from the article, I immediately thought of a scene from an Avengers movie. In that scene, Tony Stark, also known as Iron Man, works with his technology. He manipulates a floating, transparent screen that he can see through and control with his hands. When I was younger, I remember imagining myself owning the same technology from the movie. I also wondered how many years I would have to wait for that kind of technology to be released, especially since technology keeps evolving so quickly. I almost forgot the article’s main point: technology becomes more meaningful when we can interact with it ourselves, rather than just swiping or looking at it. 

As Bret Victor explains, our hands are meant to feel and handle things, not just tap on a flat surface. I liked the examples he gave about how there is almost nothing in the natural world that we interact with by just sliding. In real life, we do not just slide our hands on things. We hold them, adjust our grip, and feel their weight, which helps us control them better. That kind of interaction is missing when we only slide on a flat screen.

Thinking about it now, even if something like Iron Man’s interface existed, it might still feel limited if it only relied on gestures in the air without any real sense of touch or resistance. This led me to think that even if something looks advanced, it can still miss the deeper idea of creating something that we can fully see, feel, and physically interact with.

Concept

In this project, a simple interactive mini piano was created using Arduino. The system combines both analog and digital inputs to control LEDs and sound output. The main goal was to explore how analog sensor data can be translated into both visual and auditory feedback in a way that feels responsive and engaging.

The project brings together multiple components working at the same time: 

• Analog input through Force Sensitive Resistors (FSRs)
• Digital input using a switch for overall control
• Analog output through a PWM-controlled LED for brightness variation
• Digital output using a red LED and a buzzer for clear feedback

Each FSR functions like a piano key. When pressure is applied, it changes the resistance, which affects the analog value being read. This value is then used to control both the pitch of the sound from the buzzer and the brightness of the LED, making the interaction feel more dynamic depending on how hard the user presses.

All FSRs are connected using a voltage divider with a 10kΩ resistor, and their values are read through analog pins A0 to A3. The system also includes several indicators:

• The yellow LED shows when the system is turned on
• The green LED represents the analog output, adjusting brightness based on the input
• The red LED lights up when a stronger pressure threshold is reached
• The buzzer produces a sound corresponding to the input
• The switch acts as the main ON/OFF control

How It Works

The system starts by checking the switch. When it is OFF, everything stays inactive, so no LEDs light up, and no sound is played. Once the switch is turned ON, the yellow LED lights up to show that the system is active, and the Arduino begins reading the values from all the FSR sensors. It then compares these readings to figure out which sensor is being pressed the most.

This core logic, added by my group partner, is what allows the system to function like a piano:

// ---------- Read + smooth sensors ----------
for (int i = 0; i < 4; i++) {
  int raw = readAverage(FSR_PINS[i]);   // read sensor
  smoothVals[i] = (smoothVals[i] * 3 + raw) / 4;   // smoothing
}

// ---------- Find strongest press ----------
int strongestIndex = 0;
int strongestValue = smoothVals[0];

for (int i = 1; i < 4; i++) {
  if (smoothVals[i] > strongestValue) {
    strongestValue = smoothVals[i];
    strongestIndex = i;
  }
}

// ---------- Determine pressure level ----------
 int level;

 if (strongestValue <= LEVEL_1_MAX) {
   level = 0;   // light press
 } 
 else if (strongestValue <= LEVEL_2_MAX) {
   level = 1;   // medium press
 } 
 else {
   level = 2;   // hard press
 }

 // ---------- Select note ----------
 int nextNote = NOTES[strongestIndex][level];

 // ---------- Play sound ----------
 if (nextNote != currentNote) {
   tone(BUZZER, nextNote);   // play note
   currentNote = nextNote;   // update
 }

Based on how much pressure is applied, the input is grouped into three levels: light, medium, and hard. At the same time, the outputs respond to these changes. The green LED adjusts its brightness using PWM depending on the pressure, the red LED turns on when a strong press is detected, and the buzzer plays the corresponding note.

Tinkercad Link | Demo | Code

Reflection

At first, my partner and I both thought about adding a piano-like element to our Arduino for this week’s project, so we then decided to move forward with that idea. It was really interesting to see how different inputs could change the sound and how that made the interaction feel more alive. Instead of just having one tone, being able to play different pitches made it feel more responsive and a bit more like an actual instrument.

I really liked how the project turned out. Setting up the code both in Arduino and Tinkercad helped us see how the system fits together step by step. It was also fun to be able to actually try it out after. Overall, this project gave us a better sense of how input values can be interpreted and used in different ways. It made us more aware of how even simple changes in input can lead to noticeable differences in how a system responds. Because of this, I would definitely consider integrating sound into my future projects as well.

References

• Arduino Documentation
https://www.arduino.cc/reference/en/
• Analog Input Tutorial
https://www.arduino.cc/en/Tutorial/BuiltInExamples/AnalogInput
• PWM (analogWrite)
https://www.arduino.cc/en/Tutorial/PWM
• Voltage Divider Explanation
https://learn.sparkfun.com/tutorials/voltage-dividers

https://youtu.be/JZ44h-jy0p4?si=LTeRxI9Gy2SYuQWJ

https://youtu.be/QYYigxRwXfc?si=fmBr9DmXWSOYV1rm

https://youtu.be/CvZ-SGJ8fGo?si=kRXJ7upESpJA1Qsh