Author: Tengis Temuulen

Concept

Embark on an exciting treasure hunt adventure in a mystical world filled with hidden riches and formidable foes. In this immersive game, you’ll take on the role of either a skilled hunter or a stealthy ninja, depending on your taste.

Your quest begins in a green plain, where ancient treasures lie scattered, waiting to be discovered. However, time is of the essence, as you’ll have a limited duration to uncover the hidden treasures . Utilize your character’s agility and the radar to navigate through the treacherous terrain, avoiding obstacles and evading any lurking dangers.

As you progress through, the challenge intensifies. In level 2, you’ll find yourself navigating through a desolated space, where slimes, slimy creatures with a voracious appetite, roam freely. These formidable foes will stop at nothing to impede your treasure-hunting endeavors, so you’ll need to stay vigilant and employ strategic maneuvers to outmaneuver them.

To aid you in your quest, you’ll have access to a powerful radar system. By activating the radar, you’ll receive visual cues indicating the proximity of nearby treasures. However, use this ability judiciously, as it has a limited cooldown period, during which you’ll be vulnerable to potential threats.

Will you have what it takes to overcome the obstacles, outsmart the slimes, and emerge victorious with a bountiful collection of treasures? Prepare yourself for an adventure like no other, where quick reflexes, strategic thinking, and unwavering determination will be the keys to your success.

Immerse yourself in this captivating treasure hunt, where every step brings you closer to uncovering the secrets of a long-forgotten civilization. Embrace the thrill of the chase, and let your adventurous spirit guide you through this extraordinary journey.

Implementation

Interaction design:

In terms of interaction design, I had to consider two types of interaction, use interaction with the game, and the interaction between the arduino board the p5js. In terms of the serial communication, arduino sends 4 different values: buttonX, buttonY, positionX and positionY of the thumb joysticks. In the P5js code, the readSerial function splits and trims the incoming data, and each of the variables assigned to the respective incoming value. As for the p5js part, it sends the detected value, which then controls the led.

As for user interaction, I attempted to make the instructions as clear and concise as possible, but even then there were certain aspects of the game that were slightly vague. Inside the game, the user goes through 3 instruction panel, one introducing the game and stating the objectives, second one showing the remote controller instructions and the third one between the 2 levels that explains the content of level 2. Then there are 2 options waiting for the user for winning and losing condition. Furthermore, there is another panel after instructions that allows the user to choose from two given characters. Within the game states, there were also elements of interaction that were needed, for instance to represent the cooldown of the radar, I made a pulsing effect that only starts when the radar is in cooldown.  Also, when the character is in contact with the slime, their animation changes for a brief second to let the user know they are defeated. When the treasure is found, it will stay on the map for brief period of time and then vanishes, increasing the counter.

P5JS Code:

P5js code is where the main game mechanics are implemented. Firstly, in the preload function, I had to load the sprites, sound, background picture and slime array. In the setup function, I configured the hunter, ninja and the slime sprites’ animations and other necessary changes. In the draw function, there are multiple game states such as intro, choose, level1, between, level2, win, and lose. Moreover, there are several helper functions at the bottom part of the code, the helps with the in game mechanics.

1. Game State “intro”:
   • Displays the background image (bg1).
   • Shows the title “Treasure Hunt” and a description of the game with instructions on how to play.
   • Prompts the user to press the space bar to start the game.
2. Game State “instructions”:
   • Displays the background image (bg2).
   • Shows the title “Instructions” with details about controls (joystick for movement, green button for confirmation, red button for radar activation with cooldown).
   • Prompts the user to press enter to continue.
3. Game State “choose”:
   • Displays the background image (bg3).
   • Presents the options to choose a character between “Ninja” and “Hunter” and how to select (using the green button and joystick button).
   • Allows the player to choose a character by moving the joystick and pressing the green button.
   • Transitions to “level2” if a character is selected.
4. Game State “level1”:
   • Displays the background image (bg4).
   • Updates the timer and displays the remaining time.
   • Spawns treasure chests and updates the treasure count based on collected treasures.
   • Handles character movement and treasure detection based on the chosen character (Ninja or Hunter).
   • Updates the game state to “between” when all treasures are collected.
5. Game State “between”:
   • Displays the background image (bg3).
   • Presents a transition message to indicate the start of the next level (“Level2”).
   • Waits for the player to click to continue and transitions to “level2” when clicked.
6. Game State “level2”:
   • Displays the background image (bg1).
   • Resets the timer and displays the remaining time for the new level.
   • Spawns new treasure chests and updates the treasure count based on collected treasures.
   • Handles character movement, treasure detection, and enemy interactions (slimes).
   • Updates the game state to “win” if all treasures are collected or to “lose” if the timer runs out or the character is attacked by slimes.
7. Game State “win”:
   • Displays the background image (bg2).
   • Stops the game’s background sound.
   • Congratulates the player on winning, displays the time taken, and prompts to restart by clicking the green button.
8. Game State “lose”:
   • Displays the background image (bg3).
   • Stops the game’s background sound.
   • Informs the player of losing and prompts to restart by clicking the green button.

P5js Code Snippet:

if (chosenCharacter === "ninja"){
      hunterSprite.position.x = -200;
      updateSprite(ninjaSprite, "walk", 2);//function that enables the movement of sprite
      removeFoundTreasure(ninjaSprite, treasures);//function that removes the found treasures
      if (buttonY && canDetectTreasure) {
        detected = checkSurroundingForTreasure(ninjaSprite);
        canDetectTreasure = false; // Disable treasure detection for cooldown
        setTimeout(() => canDetectTreasure = true, 3000); // Enable detection after 3 seconds
  }
    } else if (chosenCharacter === "hunter"){
      ninjaSprite.position.x = -200;
      updateSprite(hunterSprite, "walk", 2);
      removeFoundTreasure(hunterSprite, treasures);
      if (buttonY && canDetectTreasure) {
        detected = checkSurroundingForTreasure(hunterSprite);
        canDetectTreasure = false; // Disable treasure detection for cooldown
        setTimeout(() => canDetectTreasure = true, 3000); // Enable detection after 3 seconds
  }
    }

Arduino:

The Arduino code sets up a system to interface with a thumb joystick, two buttons with LED indicators. The pins for these components are defined at the beginning: `D_pin` for the joystick switch, `Y_pin` for the Y output of the joystick, and `X_pin` for the X output. Two digital pins, `button1Pin` and `button2Pin`, are designated for the buttons, while `gledPin` and `rledPin` are used for green and red LEDs, respectively. In the setup function, the pins are configured accordingly – `D_pin` is set as an input for the switch, and the LED pins are set as outputs. Additionally, the serial communication is initialized at a baud rate of 9600.

The main loop of the code continuously reads the analog values from the X and Y outputs of the joystick, as well as the digital states of the two buttons. It also listens for incoming serial data. When serial data is available and ends with a newline character, the code parses the data into an integer (`serialValue`). If `serialValue` is 1, it turns on the green LED and turns off the red LED; otherwise, it does the opposite. The joystick position (X and Y values) and button states are then sent to the serial port in a comma-separated format for external processing or monitoring. Finally, a small delay of 200 milliseconds is included before the loop repeats, allowing for consistent and controlled operation of the system.

Arduino Code Snippet:

void loop() {
  int posX = analogRead(X_pin);
  int posY = analogRead(Y_pin);

  button1State = digitalRead(button1Pin);
  button2State = digitalRead(button2Pin);
  
  if (Serial.available()) {
    int serialValue = Serial.parseInt();//incoming detection value
    if (Serial.read() == '\n'){
      if (serialValue == 1) {
        digitalWrite(gledPin, HIGH); // Turn LED on
        digitalWrite(rledPin, LOW);
    } else {
        digitalWrite(gledPin, LOW);  // Otherwise, turn LED off
        digitalWrite(rledPin, HIGH);
      }
    }
  }
  //sending data to p5js
  Serial.print(posX);
  Serial.print(",");

  Serial.print(posY);
  Serial.print(",");

  Serial.print(button1State);
  Serial.print(",");

  Serial.println(button2State);

  delay(200); // create a small delay before repeat loop
}

Part That I Am Proud Of:

Within this project, I am proud of the different sprite interactions that I was able to utilize. Whether it is founding the treasure or being in contact with the sprite, they were all very interesting to think and code. Furthermore, I am also proud of the hardware construction of the project, especially implementing joystick and building the outer box, the hard part was putting and sticking the board inside the ox, but luckily I had relatively bigger box.

Resources used:

https://support.arduino.cc/hc/en-us/articles/4408887452434-Flash-USB-to-serial-firmware-in-DFU-mode

https://p5js.org/reference/

https://www.arduino.cc/reference/en/libraries/

Challenges faced:

One of the biggest challenges I faced was rather mysterious one. My laptop was not recognizing the arduino board. I have tried different cords, laptops, microcontrollers but it was not working, but apparently the problem was the wire that was connected to the 5v in the microcontroller. It started working after I changed the wire. Another aspect of the project that was difficult for me was soldering the led buttons. Somehow the melted alloy kept falling of after some time, which made me quite frustrated. Due to this, I spent relatively longer time soldering the wires and making sure that they are well connected.

Future Improvement:

There are definitely some improvements needed, whether it is in game mechanics or hardware aspect. For instance, there could be more variety of characters added to the game, so that users would have more variety. Since it is a level based game, there are always the opportunity to make more levels. Also, it could be better to use the collected treasures to procure something, whether it is a good boots to speed up or a better radar with wider range, making the game smoother and coherent. As for the hardware aspect, I would make more input possibilities like piezo speaker in order to let the user know if the radar detects a treasure or not. This could be much better choice than led lights. Furthermore, I would update the box so that it would look like and actual gaming console like xbox or playstation.

Project Images:

Project Videos:

IMG_4601

IMG_4602

User Testing:

IMG_4595

Im Show Documentation:

The IM show went well. Several people played the game, but due to the small radar and large map, people took a bit long time to win the game, otherwise it went very smoothly. Because I keep the serial communication too long, there were times were it was not responding to the buttons, so I had to constantly refresh it, in case it did not work.

IMG_4608

IMG_4603

IMG_4605

IMG_4606

IMG_4607

For the user testing, I asked two of my coursemates and asked for their feedback. There were several aspects in the game that they and I had differing opinions but there were also several agreements in terms of improvement of the game. Initially, I had the game with the objective of collecting 10 treasures on both level 1 and level 2, but due to the map size, it seems just 3 to 5 treasures is enough as it would take too much time and might bore the players. Another advice they gave me was to decrease the range of the radar, which actually surprised me, as I thought that the range was too small. Furthermore, we thought it would also be good idea to make the pulsing effect much more slower and inline with the cooldown effect, so that the users will know the cooldown of the radar. I found their opinions in this regard to be most useful and seeing as I also had to explain to them certain instructions and game mechanics.

IMG_4595

Exercise 1

The Arduino continuously reads the electrical signal from a potentiometer. This signal represents the potentiometer’s position based on its rotation. The code then scales this raw sensor reading (ranging from 0 to 1023) to a new range of 0 to 400 using a function called map. This scaled value becomes the control signal for the circle’s movement on the web page.

 The scaled sensor value (now within the 0-400 range) is sent from the Arduino to the p5js code running in the web browser. When data arrives to p5js, the code assigns it to a variable named circleX. This variable essentially controls the circle’s position on the screen. Finally, the code uses this value to dynamically adjust the horizontal position (X coordinate) of the circle drawn on the p5js canvas.

Hardware

• Arduino Board
• Potentiometer
• Computer
• Wires
• Breadboard

Picture: 

Video: 

IMG_4574

Schematics:

Exercise 2

The p5js code continuously tracks the mouse cursor’s position on the canvas (represented by mouseX and mouseY variables). The p5js code directly assigns the mouseX value to the brightness 1 variable and the mouseY value to the brightness2 variable. These variables essentially store the desired brightness levels for the two LEDs. The readSerial function in p5js combines the brightness1 and brightness2 values with a comma (“,”) separator and adds a newline character (“\n”) to create a formatted message. This formatted message is then sent to the Arduino.

Once the data (representing the formatted message) are available, the code reads them using Serial.parseInt. This separates the combined brightness values stored in brightness1 and brightness2. The map function then scales both brightness values from the p5js range (0-400) to the appropriate range (0-255) for controlling the LEDs. The code includes error handling by checking for a newline character (\n) after reading the brightness values. This ensures complete data reception before setting the LED brightness. If no data is received, the Arduino prints an error message “No signal received” to the serial monitor. Finally, the Arduino sets the brightness of each LED (LED 1 on pin 10 and LED 2 on pin 11) based on the corresponding received values (brightness1 and brightness2).

Hardware

• • Computer
  • Arduino Board

• Two LEDs

• Connecting Wires

Picture: 

Video: 

IMG_4574

Schematics:

Exercise 3

The p5js code establishes the core mechanics for simulating a bouncing ball on the screen. It defines various physics concepts like: Gravity, Drag, Acceleration and Wind. The code continuously updates the ball’s position and velocity based on the applied forces and drag. When the ball hits the bottom of the canvas, its vertical velocity is reversed, simulating a bounce. At this point, a variable named ledOn is set to 1, indicating the LED should be turned on. If the serial connection is active, the code sends the ledOn value (0 or 1) as a string followed by a newline character (“\n”) to the Arduino using writeSerial.

The readSerial function gets called whenever new data arrives from the Arduino. Here, it parses the received data and assigns it to the windVale variable. This value updates the wind vector, influencing the ball’s horizontal movement in the simulation. The Arduino code continuously reads the analog value from a potentiometer connected to pin A0. It then maps this value (ranging from 0 to 1023) to a new range of -10 to 10 using the map function. This mapped value represents the wind force affecting the ball in the p5js simulation. The Arduino transmits this wind force value to the p5js code. The Arduino constantly checks for incoming data on the serial port. If data is available, it reads the first character and checks its value. If the character is ‘1’, the LED connected to pin 13 is turned on. If the character is ‘0’, the LED is turned off.

Hardware

• Computer
• Arduino Board
• Potentiometer
• LED
• Connecting Wires
• Bread board

Picture: 

Video: 

Schematics:

Code and other resources: 
Exercises 1 & 2: https://drive.google.com/drive/folders/1sVlmQsPKM8jgOwRl0b_gx8DIoMTCjYt0?usp=share_link

Exercise 3:

https://drive.google.com/drive/folders/1Ro-Iw_UmvcQimw7nk7MeqBhqhtzyy95X

Design meets disability

As I have stated in the previous reading response, one of the main priorities of design should be inclusivity. However, how we approach inclusivity also matters. The way eyeglasses have mutated from medical tool to fashion entity clearly demonstrates the multiple forms it can take in various social models. Ideally speaking, the design of the product should be implemented in a way that it does not imply any negative interpretations in different social models, but this is easier said than done. In any way the design attempts to cover or camouflage the disability would imply the idea that the disability should be something to be ashamed of, but highlighting the disability would also be counterintuitive as it could support segregation.

So how exactly should we approach inclusivity? I believe that the optimal way is to design the object so that it prioritizes functionality but also offers variety. Similar to glasses, regardless of the product in question, there should be different models or designs that could be utilized. This way, the consumer/user would have the opportunity to select according to their liking. This could also be used to counter argue against negative interpretations as it would mainly depend on the user. Design, in this sense, should be flexible, so that it could provide sense of compatibility to the user, which I believe is the ultimate goal of design.

The Chroma Cassette is a fun and interactive project that plays pre-loaded songs and allows users to control the playback speed using a distance sensor. The name “Chroma” refers to the project’s ability to switch the color of LEDs based on the song being played, while “Cassette” reflects the inspiration behind the variable playback speed control, similar to the fast-forwarding feature of cassette tapes.

• Hardware Components:
  • Arduino Uno microcontroller
  • Piezo Speaker for playback
  • Distance sensor (ultrasonic sensor) to detect distance
  • Button for manual song switching
  • LEDs (Red, Green, Blue) for colorful song indication
  • Jumper wires for connecting components
  • Breadboard
• Software (Code):
  • An array named songNames stores the titles of the pre-loaded songs (Game of Thrones, Imperial March, Pirates of the Caribbean, Silent Night).
  • Each song melody is defined as an array representing musical notes and their corresponding durations.
  • A function named playSong iterates through each note in the current song and plays it based on its duration. The function also calculates a speedFactor based on the distance measured by the sensor. This speedFactor is used to adjust the note duration, essentially changing the playback speed. Higher distances from the sensor result in slower playback, mimicking the fast-forwarding effect.
  • The setRGBColor function assigns specific colors to the LEDs based on the current song being played, adding a visual element to the project.
  • An interrupt service routine is triggered when the button is pressed, and a flag named switchSong is set to true, indicating the need to switch to the next song in the playlist.

The Chroma Cassette project, initially, embarked on a path paved with frustration. Our initial goal was to directly control the volume of the pre-loaded songs. Countless hours were spent crafting code, only to be met with a difficulty: directly manipulating volume on the Arduino platform proved to be an insurmountable hurdle. This limitation stemmed from the inherent architecture of the Arduino kit, lacking dedicated hardware components for fine-grained volume control.

We brainstormed alternative approaches to achieve a dynamic audio experience, eventually agreeing on varying the playback speed of the music. This approach, however, presented its own set of challenges. After lots of trial and error, adjusting code and testing countless iterations. This phase, though time-consuming, ultimately yielded a solution that met our exacting standards.

The foundation of the Chroma Cassette lies in its pre-loaded song library. Each song, be it the epic theme from Game of Thrones or the whimsical melody of Pirates of the Caribbean, was meticulously chosen to complement the project’s functionality. Once the song selection was finalized, we embarked on a critical step: adjusting the speedFactor. This variable acts as the heart of the speed control mechanism. Meticulous adjustments were made to the speedFactorfor each song, ensuring that even at faster playback speeds, the music retained its integrity and remained pleasant to the ears.

The distance sensor served as the conductor in this symphony of sound and speed. It was calibrated to operate within a specific range, from 1 centimeter to 30 centimeters. This precise calibration ensured a smooth and responsive adjustment curve. As the distance between the sensor and an object increased, the playback speed would gradually slow down, mimicking the fast-forwarding effect of a cassette tape.

To enhance user interaction, we incorporated additional functionalities. Pressing the designated button would seamlessly switch between songs in the playlist. To provide a visual cue for song changes, an LED was integrated into the system. Whenever the user switched songs, the LED would illuminate briefly, acknowledging the user’s input.

The RGB LED added a captivating layer of visual flair to the project. This versatile LED, capable of displaying a spectrum of colors, was linked to the songIndex variable. As the user cycled through the song playlist, the RGB LED would change color, reflecting the currently playing song. This color association wasn’t random – it drew inspiration from the source material of each song. For instance, the vibrant hues of green, purple, yellow, and orange adorned the LED when playing the Harry Potter theme, a subtle nod to the four Hogwarts houses.

Faced with an initial hurdle, we pivoted our approach and ultimately delivered a unique and engaging audio experience. The project seamlessly blends pre-loaded songs, dynamic speed control based on sensor input, intuitive user interaction, and a captivating visual element through the RGB LED. 

The Chroma Cassette might be a interesting blend of sound and light, but there’s always space to make it even better. For Instance, enhancing the audio quality, especially at faster playback speeds, could be a priority. Techniques like utilizing digital signal processing libraries on the Arduino might help reduce pitch shifting and distortion. Imagine users having the ability to upload their own personal soundtracks! This could be achieved by incorporating an SD card or a Bluetooth module, significantly expanding the song library and personalizing the experience for each user. The distance sensor integration could be taken a step further. By using a more advanced sensor with a wider range, users would have finer control over the playback speed across a larger distance. This refinement could create a more intuitive user experience. Another exciting possibility is an interactive light show. The RGB LED could be programmed to react to the music’s rhythm and melody, creating a dynamic visual spectacle that complements the audio. This would undoubtedly add a whole new captivating dimension to the user experience.

Picture: 

Video:

Code Snippet:

void playSong(int *melody, int melodyLength) {
  // Check if the speaker is turned off
    if (digitalRead(SPEAKER_SWITCH_PIN) == LOW) {
        Serial.println("Speaker is turned off.");
        return; // Exit the function if the speaker is turned off
    }

    // Iterate through each note in the melody
    for (int noteIndex = 0; noteIndex < melodyLength; noteIndex += 2) {
        float distance = getDistance();  // Update distance with each note

        // Adjust speedFactor based on the song and distance
        float speedFactor;
        if (currentSong == 1) { 
            // Slower scaling for Imperial March
            speedFactor = 1.1 + (distance / 30.0); //  slows down the max speed
        } else if (currentSong==2){
            speedFactor = 0.6 + (distance / 30.0);
        } else if (currentSong==3) { 
            speedFactor = 0.4 + (distance / 30.0);
        } else {
            speedFactor = 1.2 + distance / 30.0;
        }

         // Calculate the note duration based on the speed factor
        int noteDuration = (int)(1000 / melody[noteIndex + 1] * speedFactor);

        // Check if the song should be switched or the speaker is turned off
        if (switchSong || digitalRead(SPEAKER_SWITCH_PIN) == LOW) {
            noTone(SPEAKER_PIN); // Stop tone when switching off
            break;
        }

         // Play the note
        tone(SPEAKER_PIN, melody[noteIndex], noteDuration);
        setRGBColor(currentSong, melody[noteIndex]);
        delay(noteDuration * 1.30);
        noTone(SPEAKER_PIN);

        Serial.print(songNames[currentSong]);
        Serial.print(": Playing note: ");
        Serial.print(melody[noteIndex]);
        Serial.print(" at duration: ");
        Serial.println(noteDuration);  
        Serial.print("Distance: ");
        Serial.print(distance);
        Serial.println(" cm");
    }
}

 

Schematics:

Additional link:

https://drive.google.com/file/d/1Hl3iAj1yXwIkOQnEi8e_lYcrHDTGKorh/view?usp=sharing

A Brief Rant on The Future of Interactive Design

I partially agree with the author about the vision of the future of interaction.  I do agree that the picture under glass interactivity narrows our capability in terms of hand usage, but it is wrong to just identify it as such. First of all, why was “picture under glass” chosen as a primary interactivity of the modern times in the first place? The main reason was because it is simple, therefore making the interactive process much more easier. Due to its simplicity, people can interact with it more efficiently, making it less time consuming. Other interactive designs that demands the user more hand capabilities may be interesting or more diversified but ultimately, they are inefficient in comparison to “picture under glass”.

Another point is inclusivity. The author talks about various capabilities of the hand, but what about those who are incapable of them? As an interactive product, they must be as inclusive as possible, otherwise it is difficult to classify it as a good design. “Picture under glass” utilizes touch and drag, which could be argued as one of the simplest actions that human can take, therefore making it much more inclusive than other interactive designs. As seen from the responses as well, the author completely dismisses this situation.

Another point in the responses section was about Dr.Seuss and Shakespeare. Maybe Shakespeare’s work is regarded as a marvel but when you think about who can understand it, Dr.Seuss has much wider range of people. Shakespeare’s works, while literary masterpieces, are certainly exclusive on who can understand and who cannot. This leads to the point of vision. The advancement of technology and the vision for interactive design always had inclusivity and ease of use in mind, which is why better tools appear in the end. Some tools will make life easier compared to other tools, therefore making it better. Making tools that requires higher level of capabilities would only encourage segregation, which is definitely unnecessary.

For this week’s assignment, I used potentiometer as an analogue sensor, and a tactile button as a digital sensor (switch) and two LEDs. For the LED that is controlled by the tactile button, I made it so that it blinks twice and the potentiometer for brightness control.

Code:

int led = 11;
void setup() {
  // initialize digital pin LED_BUILTIN as an output.
  Serial.begin(9600);
  pinMode(led, OUTPUT);
  pinMode(8, OUTPUT);
  pinMode(A2, INPUT);
}

// the loop function runs over and over again forever
void loop() {
  int button1State = digitalRead(A2);
  int sensorValue = analogRead(A1);
  
  Serial.println(sensorValue);

  analogWrite(led, sensorValue/4);
  delay(30);

  if (button1State == HIGH) {
    digitalWrite(8, HIGH);
    delay(1000);
    digitalWrite(8, LOW);
    delay(1000);
    digitalWrite(8, HIGH);
    delay(1000);
    digitalWrite(8, LOW);
    delay(1000);

  } else if (button1State == LOW){
    digitalWrite(8, LOW);
  }

}

Video:

Physical Computing’s Greatest Hits (and misses)

Among the physical computing projects, Fields of Grass piqued my interest the most. Thinking about it, I believe that the sensors could be arranged in various ways so that the output changes based on not only the position of the hand, but also the pressure applied. Imagine creating a virtual landscape where the terrain shifts depending on how firmly you press your hand! Delicate touches could reveal hidden paths or trigger calming sounds, while heavier presses might activate bolder visuals or more dramatic effects. This additional layer of interaction would add a whole new dimension to the experience. Beyond pressure, the sensor arrangement could be tweaked to respond to other hand interactions. For instance waving your hand across the field to control the movement of virtual birds, or gently cupping your hand to scoop up shimmering virtual butterflies. Moreover, Fields of Grass could be adapted to respond to footsteps, creating a truly immersive experience where walking through the installation ripples the virtual landscape. In a museum setting, the project could be transformed into an educational tool. Visitors could “grow” different virtual plants by placing their hands in designated areas and manipulating pressure or movement to influence factors like sunlight or water. The applications could even reach the field of physical therapy, with the virtual world responding to specific hand motions or muscle control, providing a visually engaging way to track progress.

Making Interactive Art: Set the Stage, Then Shut Up and Listen

Integrating this reading’s notion into the Fields of Grass projects, the users will be free to interact with the grass, but the output will be in the prescribed yet non-deterministic way. If we assume each user’s interaction to be a performance, we can also assume collaborative performances between 2 users generating new form of performance between multiple users and the project. Their combined hand movements and pressure could trigger entirely new visual and audio responses, fostering a collaborative performance unlike anything seen before. The project becomes a bridge, translating individual actions into a shared, ever-evolving experience. This opens doors for fascinating possibilities. Friends could create synchronized “dances” with the field, therapists could use it for collaborative movement exercises, or even strangers could stumble upon unexpected moments of artistic synergy.