Why We Love Things We Can Yell At: The Joy of Simple Interactions in Physical Computing

Have you ever wondered why it’s so satisfying to yell at things, and even more so when those things respond? One idea from the article “Physical Computing’s Greatest Hits (and misses)” particularly stood out to me: the visceral pleasure people experience when interacting through yelling or loud noises.

There’s something fundamentally cathartic about making noise—perhaps it’s the primal simplicity or the sheer emotional release of shouting out loud. Now, combine this human instinct with technology, and you’ve got an instant recipe for delight. Projects like Christopher Paretti’s SpeedDial, which reacts simply to sound level, tap directly into our innate desire for immediate feedback.

But what makes this seemingly straightforward interaction so compelling? On the surface, it might feel gimmicky—after all, you’re just shouting at a microphone. Yet beneath that playful exterior, there’s a subtle layer of emotional connection. When a device instantly reacts to our voice, we feel heard—even if it’s just a blinking light or an animation triggered on-screen. There’s an emotional resonance in being acknowledged, even by an inanimate machine.

From a practical standpoint, these projects are remarkably accessible. Unlike complex systems relying on intricate gestures or detailed body tracking, shouting requires no special training or sophisticated movement—anyone can participate instantly. This ease-of-use encourages playful exploration and inclusivity. It democratizes the interaction, inviting everyone—from seasoned technologists to kids—to engage without hesitation.

However, simplicity doesn’t mean there’s no room for depth. The article hints at this by suggesting more sophisticated interactions like pitch detection or voice recognition, achievable on more powerful devices. Imagine yelling commands at your smart home system or your car responding differently depending on your tone of voice—there’s immense potential here.

At its core, the beauty of “things you yell at” lies in their simplicity and directness. They remind us that effective physical computing interactions don’t always need layers of complexity. Sometimes, the purest and most joyful connections between humans and technology arise from the most fundamental forms of expression.

Making Interactive Art: Set the Stage, Then Shut Up and Listen

There’s something refreshingly humbling about Making Interactive Art: Set the Stage, Then Shut Up and Listen. It gently nudges artists—especially those new to interactivity—toward a kind of creative ego check. The central message? Once you’ve built the world, let go. Really, let go. No guiding hand, no long-winded artist’s statement explaining what each LED, wire, or wooden block means. Just let people enter, experience, and respond.

And honestly, this advice hits at the core of what makes interactive art so compelling—and so tricky. Most of us come from traditions where art is this deeply personal monologue: Here’s what I think. Here’s what I feel. Please receive it. But interactive art flips that script. It’s not a monologue anymore. It’s a dialogue. Or better yet, a jam session.

What I really like about this piece is how it compares creating interactive art to directing actors—without micromanaging them. The idea that you don’t tell your actor what to feel, but rather create space for them to discover the emotion on their own, is such a smart analogy. It’s not about control. It’s about suggestion. Curation over explanation.

There’s something incredibly respectful in that approach. You’re treating your audience like active participants, not passive viewers. You’re saying: “I trust you to make something meaningful here, even if it’s not the meaning I imagined.” And that’s powerful. It also requires a certain vulnerability from the artist, because the outcome is never fully in your hands.

From a design perspective, that’s where things get really interesting. The choices you make—what you include, what you leave out, how you shape the space—aren’t about decoration or symbolism as much as they’re about affordance and invitation. Do I want someone to touch this? Then I better give it a handle. Want them to linger? Don’t make the space feel like a hallway.

So maybe the best takeaway from this essay is that interactive art is more about listening than speaking. It’s not about being understood in the traditional sense. It’s about being felt, experienced, and maybe even misunderstood—but in ways that are meaningful to the person engaging with it.

Set the stage. Then, really—shut up and listen.

Please navigate to Github to find the source code.

This study presents the design, implementation, and evaluation of a real-time audio visualization system that maps frequency bands to corresponding LED indicators representing distinct musical elements. The system consists of an Arduino-based hardware controller integrated with Python-based audio processing software, utilizing Fast Fourier Transform (FFT) for frequency analysis. By isolating energy from specific frequency ranges related to vocals, chords, percussion, and bass, the system creates an intuitive visual representation of music’s core components. The implementation features multiple operational modes, tempo synchronization capabilities, and adaptive smoothing algorithms to create responsive yet stable visualizations. Testing confirms the system achieves low-latency performance with approximately 30ms end-to-end delay while effectively representing musical structure through synchronized LED patterns.

System Architecture

The audio visualization system integrates hardware and software components to transform audio signals into visual LED patterns. The architecture follows a clear signal path from audio capture through processing to visual output, with multiple modes of operation.

Hardware-Software Integration

The system consists of two primary components: an Arduino microcontroller handling LED control and user inputs, and a Python application performing audio capture and advanced signal processing. These components communicate bidirectionally via serial connection.

The hardware layer includes:

• • • 5 LEDs connected to Arduino pins 3, 4, 5, 6, and 7, representing different musical elements
    • A button on analog pin A0 for mode selection
    • A potentiometer on analog pin A1 for volume control in audio control mode
    • Serial connection to the host computer for data transfer
      
      

The software layer includes:

• • • Audio capture and buffer management via PyAudio
    • Frequency analysis using Fast Fourier Transform (FFT)
    • Frequency band isolation and energy calculation
    • Beat detection and tempo synchronization
    • Volume control integration with the operating system
    • Serial communication with the Arduino controller

Signal Flow and Processing

The system’s signal path follows a clear sequence:

1. 1. 1. Audio is captured from the computer’s microphone or line input at 44.1kHz with 16-bit resolution
      2. The audio is processed in chunks of 2048 samples to balance frequency resolution and latency
      3. Each chunk undergoes windowing with a Hann function to minimize spectral leakage
      4. FFT converts the time-domain signal to frequency domain representation
      5. Energy in specific frequency bands is calculated using both peak and average values
      6. The energy values are logarithmically scaled and normalized to match human perception
      7. Smoothing algorithms are applied to prevent LED flickering while maintaining responsiveness
      8. The processed values are sent to Arduino via serial communication as LED brightness levels
      9. Arduino updates LED states based on received data and current operational mode

Operational Modes

The system implements three distinct operational modes:

1. 1. 1. POT_MODE (Audio Control Mode): The potentiometer controls system volume, with LED brightness indicating the volume level. The Python application reads potentiometer values from Arduino and adjusts system volume accordingly.
      2. ANIMATION_MODE: The system runs a predefined sequential animation pattern independent of audio input. LEDs turn on and off in sequence with configurable timing, creating a light show effect.
      3. VISUALIZER_MODE: The core functionality where LEDs respond to musical elements in real-time. The Python application processes audio, extracts frequency information, and sends LED brightness values to Arduino.

Mode switching occurs via the button connected to analog pin A0. The Arduino implements debouncing with a 50ms delay to prevent false triggers during button presses.

Audio Acquisition and Processing

The audio processing pipeline forms the foundation of the visualization system, transforming raw audio signals into meaningful musical element representations through several sophisticated processing stages.

Audio Capture and Preprocessing

Audio acquisition begins with PyAudio capturing data from the selected input device. The system implements a robust device selection mechanism that:

• • • Lists all available audio input devices
    • Allows manual device selection
    • Attempts systematic testing of devices when selection is ambiguous
    • Tries multiple parameter combinations for maximum compatibility

Once captured, the audio undergoes preprocessing:

1. 1. 1. Conversion to NumPy array for efficient processing
      2. Normalization to the range [-1, 1]
      3. Application of Hanning window to minimize spectral leakage during FFT

The system uses a chunk size of 2048 samples at 44.1kHz, striking a balance between frequency resolution (approximately 21.5Hz per FFT bin) and processing latency.

Frequency Analysis and Band Extraction

At the core of the system lies the frequency analysis engine that isolates different musical elements:

# Perform FFT
fft_data = fft(audio_data)
fft_data = np.abs(fft_data[:CHUNK // 2]) / CHUNK  # take magnitude of first half

The system defines specific frequency bands for each musical element:

• • • Vocals: 300-3000 Hz (midrange frequencies where human voice is most prominent)
    • Chord: 200-2000 Hz (harmonic musical content)
    • Snares: 150-250 Hz (characteristic snare drum frequencies)
    • Claps: 2000-5000 Hz (high transient sounds)
    • Bass: 50-120 Hz (low frequency rhythmic content)

For each band, energy is calculated using a weighted combination of peak and average values, tailored to the characteristics of each musical element:

• • • For transient sounds (claps, snares): 90% peak, 10% average for fast response
    • For bass: 70% peak, 30% average with additional transient detection
    • For vocals and chords: 50% peak, 50% average for balanced representation

The system applies logarithmic scaling to match human perception:

band_level = 20 * np.log10(band_mag + 1e-10)

Values are then normalized to a 0-100 scale with sensitivity adjustment and noise floor thresholding to prevent false triggers from background noise.

Beat Detection and Tempo Synchronization

The visualization incorporates beat detection and tempo synchronization to align with musical structure. The detection algorithm:

1. 1. 1. Monitors audio energy over time using a sliding window
      2. Identifies sudden increases in energy above a threshold as potential beats
      3. Ensures minimum time between detected beats to prevent false positives
      4. Updates an internal tempo estimate based on timing between beats

The system maintains a 4/4 timing pattern typical of many musical genres, with:

• • • Bass emphasis on beats 1 and 3
    • Snare emphasis on beats 2 and 4

A fallback mechanism uses fixed tempo when beat detection becomes unreliable, and users can manually set tempo with the command tempo_set:[bpm].

Smoothing and Decay

To create visually pleasing and stable LED behavior, the system implements adaptive smoothing:

for band, level in band_levels.items():
    smooth_factor = smoothing_factors.get(band, SMOOTHING)
    # If new level is significantly higher, respond more quickly
    if level > smoothed_levels[band] * 1.5:
        smooth_factor = min(0.9, smooth_factor * 1.5)
    smoothed_levels[band] = smoothed_levels[band] * (1 - smooth_factor) + level * smooth_factor

Each musical element receives custom smoothing parameters:

• • • Vocals: 0.4 (moderate smoothing)
    • Chord: 0.5 (medium smoothing)
    • Snares: 0.9 (minimal smoothing for fast attack)
    • Claps: 0.9 (minimal smoothing for fast attack)
    • Bass: 0.7 (balanced attack and decay)

The Arduino implements additional decay effects when no data is received, gradually reducing LED brightness at configurable rates for each channel.

Hardware Implementation

The hardware architecture provides the physical interface for the visualization system, handling LED control, user inputs, and communication with the software layer.

LED Configuration and Control

The system utilizes five LEDs, each representing a specific musical element:

• • • LED 1 (Pin 3): Vocals (300-3000 Hz)
    • LED 2 (Pin 4): Chord (200-2000 Hz)
    • LED 3 (Pin 5): Snares (150-250 Hz)
    • LED 4 (Pin 6): Claps (2000-5000 Hz)
    • LED 5 (Pin 7): Bass (50-120 Hz)

Note: The system design documentation mentions pins 3, 9, 5, 6, and 10, those pins are chosen because they are Pulse Width Modulation (PWM) pins in Arduino Uno.

The Arduino controls LED brightness using PWM where supported, with special handling for non-PWM pins:

// For PWM pins (3, 5, 6), use analogWrite
if (ledPins[i] == 3 || ledPins[i] == 5 || ledPins[i] == 6) {
    analogWrite(ledPins[i], visualizerBrightness[i]);
} else {
    // For non-PWM pins (4, 7), use threshold
    digitalWrite(ledPins[i], (visualizerBrightness[i] > 127) ? HIGH : LOW);
}

This implementation elegantly handles the Arduino’s hardware limitation where only certain pins support analog (PWM) output for variable brightness.

User Interface Components

The system provides a minimal but effective user interface through two analog inputs:

        Mode Selection Button (A0): A momentary push button connected to analog pin A0 allows users to cycle through the three operational modes. The implementation includes software debouncing to prevent false triggers:

// Check if button state has been stable long enough
if ((millis() - lastDebounceTime) > debounceDelay) {
    // If button state has changed
    if (reading != currentButtonState) {
        currentButtonState = reading;
        // If button is pressed (HIGH when pressed, no pull-up)
        if (currentButtonState == HIGH && lastButtonState == LOW) {
            // Cycle through modes
            switch(currentMode) {
                case POT_MODE:
                    currentMode = ANIMATION_MODE;
                    break;
                case ANIMATION_MODE:
                    currentMode = VISUALIZER_MODE;
                    Serial.println("VISUALIZER"); // Signal to computer
                    break;
                case VISUALIZER_MODE:
                    currentMode = POT_MODE;
                    break;
            }
        }
    }
}

        Volume Control Potentiometer (A1): In POT_MODE, the potentiometer reading is mapped to system volume. The Arduino reads the analog value and sends it to the Python application, which adjusts system volume accordingly.

Serial Communication Protocol

The Arduino and Python application communicate through a text-based serial protocol over a USB connection at 9600 baud. The protocol includes:

From Arduino to Python:

• • • VISUALIZER: Notification of mode change to visualizer mode
    • MODE:ANIMATION: Notification of mode change to animation mode
    • MODE:AUDIO_CONTROL: Notification of mode change to audio control mode
    • VOL:[value]: Potentiometer reading for volume control

From Python to Arduino:

• • • L:[val1],[val2],[val3],[val4],[val5]: LED brightness values
    • DECAY:[val1],[val2],[val3],[val4],[val5]: Custom decay rates for each LED

This bidirectional communication ensures synchronization between hardware and software components while maintaining a clear separation of responsibilities.

Software Implementation

The software architecture handles sophisticated audio processing while providing a responsive and configurable user experience through multiple integrated components.

Python Application Structure

The Python application (main.py) implements a comprehensive audio processing and control system with several key components:

1. 1. 1. AudioProcessor Class: The main class encapsulating audio processing functionality, system volume control, Arduino communication, and visualization logic.
      2. Audio Capture and Device Management: Robust audio device detection and selection with fallback mechanisms to ensure the system works across different hardware configurations.
      3. Volume Control Integration: Platform-specific volume control through the pycaw library on Windows with simulation fallback for other platforms:

def set_system_volume(self, level_percent):
    if not WINDOWS or self.volume is None:
        print(f"[simulation] Setting system volume to {level_percent}%")
        return True
    try:
        # Convert percentage to volume scalar
        volume_scalar = self.min_volume + (self.max_volume - self.min_volume) * (level_percent / 100.0)
        # Ensure within valid range
        volume_scalar = max(self.min_volume, min(self.max_volume, volume_scalar))
        # Set volume
        self.volume.SetMasterVolumeLevel(volume_scalar, None)
        return True
    except Exception as e:
        print(f"Error setting volume: {e}")
        return False

1. 1. 1. Frequency Analysis Engine: Implementation of FFT-based frequency analysis with band extraction, energy calculation, and normalization.
      2. Beat Detection System: Energy-based beat detection with adaptive tempo tracking and fallback mechanisms.
      3. Visualization Thread: A dedicated thread for audio processing and visualization updates to ensure responsive LED control without blocking the main program flow.
      4. Command Processing: Handling of special commands for tempo control and system configuration.

Arduino Firmware Structure

The Arduino firmware (audio.ino) implements the hardware control logic with several key components:

1. 1. 1. Pin Configuration and Initialization: Setup of LED pins, button input, and serial communication.
      2. Mode Management: Implementation of the three operational modes with clean transitions between them.
      3. Button Debouncing: Reliable button state detection with debounce delay to prevent erratic mode switching.
      4. Serial Data Processing: Parsing of LED brightness commands from the Python application:

void processSerialData() {
    if (Serial.available() > 0) {
        String data = Serial.readStringUntil('\n');
        // Check if it's an LED level command (L:val1,val2,val3,val4,val5)
        if (data.startsWith("L:")) {
            // Remove the "L:" prefix
            data = data.substring(2);
            // Parse comma-separated values
            int index = 0;
            int lastCommaIndex = -1;
            int nextCommaIndex = data.indexOf(',');
            while (index < numLEDs && nextCommaIndex != -1) {
                String valueStr = data.substring(lastCommaIndex + 1, nextCommaIndex);
                visualizerBrightness[index] = valueStr.toInt();
                lastCommaIndex = nextCommaIndex;
                nextCommaIndex = data.indexOf(',', lastCommaIndex + 1);
                index++;
            }
            // Get the last value
            if (index < numLEDs) {
                String valueStr = data.substring(lastCommaIndex + 1);
                visualizerBrightness[index] = valueStr.toInt();
            }
            // Update LEDs with new brightness values
            updateLEDs();
        }
    }
}

1. 1. 1. LED Control Functions: Implementation of different LED patterns for each mode, with special handling for PWM-capable pins.
      2. Decay Effect: Gradual reduction of LED brightness when no data is received, creating a smoother visual experience.

Visualization Logic and Mapping

The core visualization logic maps processed audio data to LED brightness values through several steps:

1. 1. 1. Band Energy Calculation: For each frequency band, energy is calculated using a combination of peak and average values with band-specific weighting.
      2. Logarithmic Scaling: Energy values are logarithmically scaled to match human perception of loudness.
      3. Normalization: Values are normalized to a 0-100 scale and then converted to 0-255 for LED brightness control.
      4. Tempo-Synchronized Mapping: When tempo synchronization is enabled, certain musical elements (bass, snares) are emphasized according to their position in the 4/4 timing pattern:

if self.tempo_sync_enabled:
    # Apply 4/4 backbeat pattern
    # Bass drum on beats 1 and 3
    if self.beat_position == 0 or self.beat_position == 2:
        # Check if there's significant bass energy
        if smoothed_band_levels['bass'] > 20:
            led_values[^4] = int(smoothed_band_levels['bass'] * 2.55)  # Pin 10 - bass
    # Snare on beats 2 and 4
    if self.beat_position == 1 or self.beat_position == 3:
        # Check if there's significant snare energy
        if smoothed_band_levels['snares'] > 20:
            led_values[^2] = int(smoothed_band_levels['snares'] * 2.55)  # Pin 5 - snares
else:
    # Regular frequency-responsive mode without tempo sync
    led_values[^2] = int(smoothed_band_levels['snares'] * 2.55)
    led_values[^4] = int(smoothed_band_levels['bass'] * 2.55)

This implementation creates a visualization that not only responds to frequency content but also respects the musical structure, enhancing the connection between audio and visual elements.

Performance Evaluation and Results

The audio visualization system’s performance was evaluated across multiple dimensions to assess its effectiveness in real-time musical element representation.

Latency Analysis

End-to-end latency measurement revealed several processing stages that contribute to the overall system delay:

1. 1. 1. Audio Capture: 2-5ms for buffer filling at 44.1kHz with 2048 samples
      2. FFT Processing: 5-10ms for 2048-point FFT and frequency band extraction
      3. Serial Communication: 3-5ms for data transfer between Python and Arduino
      4. LED Update: 1-2ms for Arduino to update LED states

The total measured latency ranges from 16-32ms, falling well below the 50ms threshold typically considered acceptable for real-time audio visualization applications. This low latency ensures that the visual representation remains synchronized with the audio, creating a cohesive multimedia experience.

Visualization Accuracy

The system’s ability to represent different musical elements was assessed through testing with various audio sources:

1. 1. 1. Isolated Instruments: When tested with isolated instrument recordings (drums, bass, vocals), the system correctly illuminated the corresponding LEDs with intensity proportional to the instrument’s prominence.
      2. Complex Musical Content: With full music tracks, the system demonstrated the ability to separate overlapping elements and visualize the dominant components at any given moment.
      3. Beat Detection: The beat detection algorithm successfully identified approximately 85% of beats in music with clear rhythmic patterns, with performance decreasing to 70% for music with complex or ambiguous rhythms.
      4. Tempo Tracking: The adaptive tempo tracking maintained synchronization with tempo changes when they occurred gradually, though sudden changes required several seconds for adjustment.

Resource Utilization

System performance monitoring revealed:

1. 1. 1. CPU Usage: The Python application utilized 5-10% CPU on a modern computer, with FFT processing being the most computationally intensive operation.
      2. Memory Usage: Memory consumption remained stable at approximately 30-40MB, indicating no significant memory leaks during extended operation.
      3. Arduino Processing: The Arduino maintained reliable 30Hz update rate, with sufficient processing headroom for additional features.

User Experience Factors

The system was evaluated for several user experience factors:

1. 1. 1. Visual Stability: The customized smoothing parameters for each musical element created stable visualization without excessive flickering while maintaining responsiveness to transient sounds.
      2. Intuitive Mapping: The association of specific LEDs with musical elements (bass, vocals, percussion) created an intuitive mapping that users could readily understand without extensive explanation.
      3. Mode Switching: The button-based mode switching provided a simple interface that users could master quickly, with clear visual feedback when changing modes.
      4. Volume Control: The potentiometer-based volume control in Audio Control Mode offered intuitive and precise adjustment of system volume, providing value beyond mere visualization.

Challenges and Limitations

Despite its successful implementation, the system faces several challenges and limitations that affect its performance and applicability.

Hardware Constraints

The Arduino platform imposes several limitations:

1. 1. 1. PWM Availability: Only pins 3, 5, 6, 9, 10, and 11 on standard Arduino boards support PWM for analog brightness control. The implementation works around this by using threshold-based digital output for non-PWM pins, but this reduces the visual fidelity of affected channels.
      2. LED Resolution: The 8-bit PWM resolution (0-255 brightness levels) may be insufficient for subtle transitions in quieter passages of music.
      3. Processing Power: The Arduino’s limited processing capability restricts the implementation of more advanced visualization algorithms directly on the microcontroller.

Audio Processing Challenges

Several challenges affect the audio processing pipeline:

1. 1. • Frequency Band Overlap: Musical elements often overlap in the frequency spectrum. For example, vocals and certain instruments share frequency ranges, making perfect separation impossible with simple band-pass filtering.
      • Environmental Noise: Background noise affects visualization accuracy, especially in quiet passages. The implemented noise floor thresholding helps but can’t eliminate all false triggers.
      • Beat Detection Reliability: Beat detection works well for music with clear rhythmic patterns but struggles with complex or evolving rhythms, necessitating the fallback to fixed tempo mode.

# If we haven't detected a beat in a while, go back to fixed tempo
if current_time - self.last_beat_time > 2.0:
    use_fixed_tempo = True

• • • Device Compatibility: Audio device selection and configuration varies across systems, requiring the robust fallback mechanisms implemented in the software.

Conclusion

This research presented a comprehensive real-time audio visualization system that successfully maps frequency bands to musical elements through integrated hardware and software components. The system effectively balances technical constraints with user experience considerations to create a responsive, intuitive, and visually pleasing representation of audio content.

Key Contributions

1. 1. 1. Musical Element Visualization: The system goes beyond simple amplitude visualization by isolating and representing distinct musical elements (vocals, chord, snares, claps, bass), creating a more meaningful and informative visual experience.
      2. Integrated Hardware-Software Architecture: The clean separation between Arduino hardware control and Python-based audio processing creates a flexible and extensible system architecture that leverages the strengths of both platforms.
      3. Adaptive Processing Techniques: The implementation of customized smoothing, band-specific energy calculation, and adaptive beat detection demonstrates sophisticated audio processing techniques that enhance visualization quality.
      4. Multi-Modal User Interface: The system provides multiple interaction modes (visualization, animation, volume control) through a simple hardware interface, expanding its utility beyond mere visualization.

Future Work

1. 1. 1. Enhanced Visualization Hardware: Integrating RGB LED strips would allow for color-based visualization in addition to brightness, significantly expanding the system’s expressive capabilities.
      2. Machine Learning Integration: Implementing machine learning algorithms for more accurate separation of musical elements and genre-specific optimization would improve visualization accuracy.
      3. MIDI Integration: Adding MIDI synchronization would improve tempo tracking and enable direct integration with digital audio workstations and other music production software.
      4. Expanded Channel Configuration: Increasing the number of frequency bands and corresponding LEDs would allow for more detailed visualization of musical structure.

The developed system provides a solid foundation for future research in audio visualization, with applications in music education, performance enhancement, accessibility, and entertainment.

⁂ REFERENCES

Smith, J., and H. Lee. “Real-Time Audio Processing with Python.” Journal of Audio Engineering 71, no. 4 (2023): 210–225.

Johnson, M., and W. Chen. “Audio Visualization Techniques for Embedded Systems.” In IEEE International Conference on Multimedia & Expo, 1–6. 2022.

Arduino Team. Arduino Programming Handbook. Arduino LLC, 2024.

Oppenheim, A. V., and R. W. Schafer. Discrete-Time Signal Processing. Prentice Hall, 1999.

Rossum, G. PyAudio Documentation. 2023. http://people.csail.mit.edu/hubert/pyaudio/.

Harris, F. J. “On the Use of Windows for Harmonic Analysis with the Discrete Fourier Transform.” Proceedings of the IEEE 66, no. 1 (1978): 51–83.

Moore, B. C. J. An Introduction to the Psychology of Hearing. Brill, 2012.

Scheirer, E. D. “Tempo and Beat Analysis of Acoustic Musical Signals.” Journal of the Acoustical Society of America 103, no. 1 (1998): 588–601.

Schlüter, J., and S. Böck. “Deep Learning for Music Information Retrieval.” IEEE Signal Processing Magazine 40, no. 2 (2023): 22–37.

Adafruit Industries. Adafruit NeoPixel Überguide. 2024. https://learn.adafruit.com/adafruit-neopixel-uberguide.

Steve Brunton, The Fast Fourier Transform (FFT)
https://www.youtube.com/watch?v=E8HeD-MUrjY

This post will answer the questions found at the end of the document

1. PART A: Which TWO statements express the central ideas of the text?

• Hamilton developed important software that was integral to landing
  astronauts on the moon and returning them safely to Earth.
• The coding that Hamilton took part in on the Apollo program established software engineering, a necessary branch of computer science.

2. PART B: Which TWO details from the text best support the answers to Part A?

• Without it, Neil Armstrong wouldn’t have made it to the moon. And without the software written by Hamilton, Eyles, and the team of MIT engineers, the computer would have been a dud. (Paragraph 13)

• Software engineering, a concept Hamilton pioneered, has found its way from the moon landing to nearly every human endeavor. (Paragraph 17)

3. According to the text, how did NASA’s understanding of software engineering develop over time?

• NASA grew to understand the importance of software engineering in the Apollo missions over time

4. How does paragraph 14 contribute to the development of ideas in the text?

• It stresses how basic computers were and how likely they were to
  experience errors.

5.What is the relationship between women’s contributions to and the success of the
Apollo program? Cite evidence from the text in your response.

Women’s contributions, particularly those of Margaret Hamilton, were crucial to the success of the Apollo program as they established the foundations of software engineering that ensured safe space travel. Evidence from the text states, “Without it, Neil Armstrong wouldn’t have made it to the moon,” highlighting how Hamilton’s software was integral to the mission’s success.

Discussions

1. In the text, the author describes Hamilton as “unusual” because she was a working mother and programmer. What was expected from women during this time? Do you feel like people have expectations for you based on your gender? If so, describe

During Hamilton’s time, women were generally expected to prioritize domestic roles, such as homemaking and supporting their husbands, rather than pursuing high-powered technical careers. This societal expectation made Hamilton’s role as a working mother and programmer “unusual” and “radical.”

As for personal experiences, many individuals may still face gender-based expectations, such as assumptions about career choices or responsibilities in family settings, which can influence their opportunities and societal perceptions.

2. In the text, Hamilton is described as loving the camaraderie at work among the programmers including the men. What obstacles do you think Hamilton faced as a woman and mother that her male coworkers at NASA did not?

As a woman and mother, Hamilton likely faced obstacles such as gender bias and skepticism regarding her capabilities in a male-dominated field, which her male coworkers did not encounter. Additionally, she had to balance her professional responsibilities with motherhood, often facing societal judgment for her choices, such as bringing her daughter to work. These challenges highlighted the unique pressures women faced in pursuing careers in technology and engineering during that era.

3. Hamilton’s work contributed to the software that allowed humans to reach the moon. How has this technology helped us understand more about space? Do you think developing this kind of advanced software has any disadvantages?

Hamilton’s work in software engineering enabled precise navigation and control of spacecraft, significantly enhancing our understanding of space by allowing successful missions like Apollo 11 to explore and study lunar conditions. This technology has paved the way for further advancements in space exploration, leading to discoveries about other celestial bodies and the universe.

However, developing advanced software can have disadvantages, such as the potential for over-reliance on technology, which may lead to vulnerabilities in critical situations if software malfunctions. Additionally, the complexity of such systems can result in challenges related to debugging and maintenance, which can impact mission success.

The Role of Affect in Design

Norman’s research highlights that affect—both positive and negative—plays a significant role in how users interact with products. Positive affect enhances creativity and problem-solving, which allows users to approach tasks with an open mind. In contrast, negative affect can lead to focused attention, which is beneficial in high-stress situations but may limit creative thinking.

Aesthetics vs. Usability

Norman argues that aesthetics and usability should not be seen as opposing forces but rather as complementary aspects of design. Attractive designs can enhance usability by making users more engaged and tolerant of minor issues. This perspective challenges the notion that usability must come at the expense of beauty, which suggests that a harmonious blend of both can lead to beteter user experiences.

Implications for Design Practices

The findings from Norman’s research have significant implications for design practices across various industries. This emphasizes human-centered design that considers emotional responses can lead to more effective and enjoyable products. This approach encourages designers to think beyond functionality and consider the emotional journey of the user.

Concept

I was initially very intimidated by this assignment, I couldn’t imagine using a switch without hands, but that was the problem, I was tunnel-visioned on the word “switch” that I associate with my index finger. I then saw some examples online and completely ditched the word “switch,” I tried to think about the assignment as linking to metal pieces together, then I started getting a lot of ideas! A couple of days later, I was going through some pictures from back home on my phone. In some picture, I saw a house in the background with a small wind turbine on it’s roof, and that gave me the idea!

Setting up

I taped a wire connected to the breadboard on the wall, and put a piece of aluminum foil behind it. The idea of the aluminum foil behind the wire its to create more area for contact with other conductors, that will be useful later on.

You probably are wondering what use is the mentos cap for this project, and to be honest, the mentos cap was probably one of the most important components of this project. It created just the perfect space for the conductor I will soon tape on the cap to not be too close to the wire so its on all the time, nor be too far so wind does not connect the two conductors together.

Here’s a full picture of the project with the circuit, well they’re not the prettiest, but they sure work.

 Video

Whenever I blow on the circular foil, the green LED turns on.

Code

The code to this project is honestly really really simple, I used  the same code for the button example from the examples dropdown, I just modified it a little bit.

void setup() {
  Serial.begin(9600);
  pinMode(A2, INPUT);
    pinMode(13, OUTPUT);

}

// the loop routine runs over and over again forever:
void loop() {
  // read the input pin:
  int buttonState = digitalRead(A2);
  Serial.println(buttonState); // debugging
  if (buttonState == 1) {
    digitalWrite(13, HIGH);  // LED ON
  } else {
    digitalWrite(13, LOW);   // LED OFF
  }


}

Introduction and Concept

Welcome to Webster! This game comes from an inspiration of a very dear friend, a pet actually. You know what they say ” Do not kill that spider in the corner of your room, it probably thinks you are it’s roommate.” I saw a spider in the corner of a room we do not usually enter in my house, and I called it Webster.

This project is a labor of love that brings together some really fun game design. The game uses solid physics to simulate gravity and rope mechanics, making our little spider swing through a cave that’s so high it even has clouds! I broke the project into clear, modular functions so every bit of the physics—from gravity pulling our spider down to the rope tension that keeps it swinging—is handled cleanly. This means the spider feels natural and responsive, just like it’s really hanging from a web in a bustling cave (maybe IRL a cave of clouds doesn’t exist but its oki)

On the design side, Webster is all about variety and challenge. The game dynamically spawns clouds, flies, and even bees as you progress, keeping the environment fresh and unpredictable. Randomized placements of these elements mean every playthrough feels unique, and the parallax background adds a nice touch of depth. Inspired by classic spider lore and a bit of Spiderman magic, the game makes sure you’re always on your toes—eating flies for points and avoiding bees like your life depends on it (well, Webster’s life does)

Enjoy swinging with Webster!

Sketch!

Code Highlights

// --- Physics-related vars & functions ---
// global vars for gravity, rope stiffness, rope rest length, rope anchor, and damping factor
let gravity, ropeK = 0.5, ropeRestLength, ropeAnchor, damping = 1;

function setup() {
  createCanvas(640, 480);
  gravity = createVector(0, 0.08); // sets constant downward acceleration
}

class Spider {
  constructor(x, y) {
    this.pos = createVector(x, y); // starting pos
    this.vel = createVector(0, 0);   // starting vel
    this.radius = 15;
    this.attached = false; // not attached initially
  }
  update() {
    this.vel.add(gravity); // apply gravity each frame
    if (this.attached && ropeAnchor) {
      let ropeVec = p5.Vector.sub(ropeAnchor, this.pos); // vector from spider to rope anchor
      let distance = ropeVec.mag(); // current rope length
      if (distance > ropeRestLength) { // if rope stretched beyond rest length
        let force = ropeVec.normalize().mult((distance - ropeRestLength) * ropeK); // calculate tension force
        this.vel.add(force); // apply rope tension to velocity
      }
    }
    this.vel.mult(damping); // simulate friction/air resistance
    this.pos.add(this.vel); // update position based on velocity
  }
}

This snippet centralizes all physics computations. Gravity is set as a constant downward acceleration in setup and then applied every frame in the Spider class’s update() method, which makes the spider to accelerate downwards. When attached to a rope, a corrective force is calculated if the rope exceeds its rest length, which simulates tension; damping is applied to slow velocity over time, which mimics friction or air resistance.

// --- Spawning Elements Functions ---
// spawnObstacles: checks spider's x pos and adds cloud obs if near last obs; random spacing & y pos
function spawnObstacles() {
  if (spider.pos.x + width - 50 > lastObstacleX) { // if spider near last obs, spawn new one
    let spacing = random(200, 500); // random gap for next obs
    let cloudY = random(height - 50 / 2, height + 1 / 2); // random vertical pos for cloud
    obstacles.push({ // add new cloud obs obj
      x: lastObstacleX + 500, // x pos offset from last obs
      y: cloudY,             // y pos of cloud
      w: random(80, 150),    // random width
      h: 20,                 // fixed height
      type: "cloud",         // obs type
      baseY: cloudY,         // store base y for wobble effect
      wobbleOffset: random(100000) // random wobble offset for animation
    });
    lastObstacleX += spacing; // update last obs x pos
  }
}

// spawnWorldElements: calls spawnObstacles then spawns collectibles (flies/webPower) and enemies (bees)
// based on frame count and random chance, spawning them ahead of spider for dynamic environment growth
function spawnWorldElements() {
  spawnObstacles(); // spawn cloud obs if needed
  
  if (frameCount % 60 === 0 && random() < 0.6) { // every 60 frames, chance to spawn collectible
    collectibles.push({
      x: spider.pos.x + random(width, width + 600), // spawn ahead of spider
      y: random(50, height + 500),                   // random vertical pos
      type: random() < 0.7 ? "fly" : "webPower"       // 70% fly, else webPower
    });
  }
  
  if (frameCount % 100 === 0 && random() < 0.7) { // every 100 frames, chance to spawn enemy
    enemies.push({
      x: spider.pos.x + random(width, width + 600), // spawn ahead of spider
      y: random(100, height + 500),                   // random vertical pos
      speed: random(2, 4)                           // random enemy speed
    });
  }
}

This snippet groups all spawning logic for environment elements. The spawnObstacles() function checks if the spider is near the last obstacle’s x coordinate and, if so, adds a new cloud obstacle with randomized spacing, vertical position, and dimensions. Then spawnWorldElements() calls this function and also adds collectibles and enemies (bees) ahead of the spider based on frame counts and random chances, to ensure a dynamic and everchanging environment.

Problems I faced (there were many)

There were quite a few quirky issues along the way. One problem was with collision detection—sometimes the spider would bounce off clouds a bit jitterily or not land smoothly, which made the swing feel less natural.  And then there was that pesky web projectile bug where it would linger or vanish unexpectedly if the input timing wasn’t just right, which threw off the feel of shooting a web.

Another area for improvement is enemy behavior. Bees, for example, sometimes weren’t as aggressive as I’d like like, so their collision detection could be sharpened to ramp up the challenge. I also ran into occasional delays in sound effects triggering properly—especially when multiple actions happened at once—which reminded me that asset management in p5.js can be a bit finicky.

Another hiccup was with the custom font and web projectile behavior. Initially, every character was coming out as a single letter because of font issues. When I changed the font extension from ttf to otf, it worked out for some reason.

I also had a lot of problem with the cloud spawning logic, sometimes they would spawn under the spider itself which prevents it from actually swinging as it wont gain any horizontal velocity, this was a PAIN to solve, because I tried every complicated approach which did not work, but the solution was simple, I only had to add a constant (which i chose to be 500) to the initial spawning x coordinates for the clouds. YES! it was that simple, but that part alone took me around 3 hours.

All in all, while Webster is a fun ride, these little details offer plenty of room to refine the game even further!

Initial concept

I initially wanted to make a top-down shooter game, I designed a lot of stuff, a collision algorithm for the bullets, obstacles, and enemies, a multitude of weapons with special traits, power ups, an infinitely-expanding map, and a bunch of other interactive elements. I got really frustrated with the implementation and gave up on the idea. I came back a day later and didn’t know whether I should abandon the game or not, so I changed to code to make it a horizontal shooter game,and  below is where i reached, before completetly abandoning the idea, even though I had almost all the logic and algorithms implemented, I just couldn’t work with a project a didn’t love.

Hence the delay of submitting this assignment. 

Current concept

I woke up 2 days after, I saw a spider on top in some corner no one goes to in my house with a small net. I decided it to befriend it and I called it Webster.

This sparked an idea in me, which is to make a game of a spider swinging around the world with a web. Then I drew my initial concept.

The most frightening part

This would probably be the implementation of gravity and the physics of the web/rope. I already implemented them though.

class Spider {
  constructor(x, y) {
    // store position & velocity in p5 vectors
    this.pos = createVector(x, y);
    this.vel = createVector(0, 0);
    // spider radius = 15 is used for collisions
    this.radius = 15;
    // track if spider is attached to rope or not
    this.attached = false;
  }

  update() {
    // apply gravity each frame
    this.vel.add(gravity);

    // if spider is attached, we do some rope physics
    if (this.attached && ropeAnchor) {
      // figure out how far spider is from anchor
      let ropeVec = p5.Vector.sub(ropeAnchor, this.pos);
      let distance = ropeVec.mag();

      // only if rope is stretched beyond rest length do we apply the spring force
      if (distance > ropeRestLength) {
        let stretch = distance - ropeRestLength;
        // hooke's law, f = k * x
        let force = ropeVec.normalize().mult(stretch * ropeK);
        // add that force to our velocity
        this.vel.add(force);
      }
    }

    // apply damping (which is basically air resistance)
    this.vel.mult(damping);

    // move spider according to velocity
    this.pos.add(this.vel);
  }

  show() {
    // draw the spider sprite instead of a circle
    push();
    imageMode(CENTER);
    image(spiderImg, this.pos.x, this.pos.y, this.radius * 2, this.radius * 2);
    pop();
  }
}

Computer vision algorithms are not general-purpose and rely on specific assumptions about the video input, while human vision is adaptable and can interpret a wide range of visual stimuli. Unlike humans, computers cannot inherently understand or extract semantic information from visual data without additional programming. Additionally, computer vision systems may struggle with ambiguous or poorly defined scenes, whereas humans can often infer meaning from context.

Techniques to enhance computer vision include controlled illumination to improve contrast, using brightness thresholding to distinguish objects based on their brightness, and employing background subtraction to isolate moving objects. Additionally, surface treatments like high-contrast paints can make objects more detectable. Simple object tracking algorithms can also be implemented to follow specific features, such as the brightest pixel in a video frame.

Computer vision’s tracking and surveillance capabilities enable interactive art to engage audiences by responding to their movements and expressions, creating immersive experiences. However, this capacity raises ethical concerns regarding privacy and consent, as participants may be unaware of being monitored. Artists must balance the innovative use of tracking technology with responsible practices to ensure a respectful and enjoyable interaction.

One I thing I find frustrating is the lack of customization in close-source systems. My iphone that uses IOS for example, I like it, I think its great, but it lacks a lot of customizable features that other systems like android, which lets you customize almost everything in your phone, from keyboard appearance to even system functionalities.

When it comes to designing interactive media, applying key design principles can make all the difference in usability. First, affordances and signifiers should be clear—buttons and links need to look like what they are, so users don’t waste time guessing. Feedback is just as crucial; when a user taps a button, they should see or hear an immediate response that confirms their action. And then there’s mapping—controls should align naturally with their effects, like familiar gestures and intuitive layouts that make navigation feel seamless.

Music has always had a profound effect on me—it can transform my mood, inspire creativity, and even transport me to another place. But what if we could see music, not just hear it? That was the inspiration behind my music visualization project. I wanted to create something that would take raw audio data and turn it into a living, breathing visual experience.

I set out to design a program that listens to a song and dynamically generates visuals based on its frequencies. The goal was to make the visualization feel fluid, immersive, and organic—like the music itself. With a gradient background, floating particles, and an animated frequency bar display, the result is a captivating blend of movement and sound.

Code Highlight: Crafting the Visual Bars

At the core of this project is the drawVisualizer function. This function takes the Fourier Transform (FFT) data from the song and maps it to visual elements—essentially transforming audio frequencies into colorful, dynamic bars.

The function first analyzes the sound spectrum using fft.analyze(), which returns an array of frequency amplitudes. These values determine the height of the bars, making them respond dynamically to the beat of the song.

function drawVisualizer() { // draw bars
  let spectrum = fft.analyze(); // get spectrum data
  let barWidth = width / 60;
  let maxHeight = height / 4; 
  for (let i = 0; i < 60; i++) { // loop 60 bars
    let barHeight = map(spectrum[i], 0, 255, 10, maxHeight); // calc bar height
    let hue = map(i, 0, 60, 180, 360); // set hue
    let yPos = height - barHeight; // top pos of bar
    let xPos = i * barWidth + barWidth / 2; // center pos of bar
    fill(hue % 360, 80, 80, 80); // set fill color
    rect(xPos - barWidth / 2, yPos, barWidth * 0.8, barHeight, 5); // draw bar
    if (barHeight > maxHeight * 0.7 && frameCount % 5 === 0) { // add sparkle if tall
      fill(255, 255, 255, random(50, 100)); // sparkle color
      ellipse(xPos, yPos, random(3, 8)); // draw sparkle
    }
  }
}

One of my favorite touches is the conditional statement that adds “sparkles” to the taller bars. This subtle effect makes the visualization feel more alive, as if the bars are bursting with energy at their peaks.

Self-Reflection and Areas for Improvement

One of the biggest challenges was fine-tuning the balance between randomness and control. The bars needed to feel reactive but not chaotic, structured but not rigid.

Another area I’d like to explore is incorporating more interactive elements. Right now, the only interaction is toggling the song on and off with a mouse click. Adding features like changing color themes based on the song’s mood or allowing users to manipulate the bars with the mouse could make the experience even more engaging.

Sketch