Author: Steven Wyks

I am very happy with how the showcase went for what I created! I thought that it was a very interesting user testing session as well. I previously thought that the instructions and flow of the game were intuitive and easily flowed together. However, when people who had zero context of the game or what it was and the physical interface, they sometimes struggled in knowing what to do.

I think that this project really forced me to reflect on what makes an interactive media project, or any entertaining product successful, and I think I realized that people don’t necessarily find the most complex of interactions entertaining, but sometimes these are just simple things with a clear feedback loop. The part of the game that had the most success was probably just people walking up to it and petting the stuffed toy’s head and seeing hearts coming out of the horse or hitting the padded force sensor with a hammer. People understood the deeper meaning of the game that I was attempting to convey, and I got responses like “oh yeah that’s cool,” but they were nowhere near as excited after petting the animal. It now seems so obvious that that interaction would evoke such a strong reaction because of how intuitive and personal it felt.

There were four stages to the game: a forested level, a desert area, an ice area and a wooded area. The idea is that the objective of each other remains the same: the player is able to interact with the animals and harvest resources in order to build a piece of technology. As the player progresses through the game, the animals become increasingly scared of the player and the amount of available resources begins to dwindle, the environment changes and the music becomes more dramatic and tense. The whole idea is that the human’s requests from nature for things to improve their livelihoods are initially meager, like the train and the airplane necessary for movement and communication, but as the human creates these things, their requests become more elaborate and arguably more unnecessary.

All of the assets that I took for the game were from: itch.io and some of the other assets were taken from Minecraft.

I also spent a lot of time on the sound and music used during the game and since this style of the character and assets was reminiscent of older 2D games like Pokemon, I used music from Harvest Moon and Pokemon, as well as sound effects from Minecraft.

Here is a link to the github repository with all of the assets and code:

https://github.com/slw515/introToIM/tree/master/finalIntro%20copy%203/finalIntro

A few pictures and a video of my project’s setup and someone playing it during the showcase:

It was super awesome to see this all come to life even though it did not turn out like how I envisioned it! Thank you to Aaron and everyone who helped, supported or just simply saw my project, it has been an awesome semester!

I had Dhabia, Nick and Pangna user test my project. I think that the key takeaways that I got were the following:

• Adding signifiers or instructions of some sorts so users would know when to pet the animal or harvest resources and how they would do this.
• Featuring my (somewhat vague) instructions in a more prominent location on the screen.
• Fixing boundary issues between the player and the resources and animals.
• Some issues with the touch sensor, thinking of possibly making the hit location of the “hammer” smaller.

I think that the most important thing that can contribute to user experience is my use of instructions. Furthermore, people did not engage with the game as I thought they would. I wanted the users to pet the animals and then harvest the resources at each level, to fully see how the environment changes throughout the game, however, they often just wanted to complete the game as fast as they could.

User testing was extremely useful because a lot of what I have been doing for the past few days has become very narrow. I think it has been really hard to step out of my biases, my experiences and my intentions for the game that I neglected instructions and very basic things. I am lucky because there is very little that I have to do for the project to be complete by Thursday so I can really focus on fine-tuning the little things before the show.

Here are the videos of their user testing:

I have continued to work on my final project. However, after attempting to solder additional components (a FSR) to a circuit board, my project no longer uses any soldered components for the time being. Aaron advised me to use capacitive touch instead of a FSR with the stuffed animal, and I have tested this yesterday and it works fine and responds well to touch. I eventually plan on sewing this to the animal. As I move into finalizing the project this weekend.

I have refined my code slightly and have fixed a few technical issues revolving around changing stages and some of the animations.

I plan on adding another FSR, which I will attach to a block of wood. I will also create a tool (most likely a hammer) that will have a foam cushion at the end. If the FSR’s reading goes above a certain value, the player will harvest the resources. I plan on going into the scene shop to quickly building these tools and testing out the FSR tonight.

The game is playable and a few of my friends have tested out the game. However, some refinements still need to be made in progression.

 

Here is a video of the prototype in action:

I worked intensively on my final project this weekend and I think I am about 80% done with the project, the rest of what I need to complete being fine tuning animations and creating a few physical components to be used along with my joystick. I initially used a button at the beginning of my creation process, but found that it was not engaging enough. I am now using two force sensors: one for harvesting resources and the other for interacting with the animals.

I plan to put one sensor into a stuffed animal’s head requiring the user to “pet” the animals. In order to harvest materials, users must hit the force sensor, which I plan to hide under a piece of wood, with a mallet I plan to construct.

I have completed the most difficult aspects of the project by coding nearly all the stages and the mechanics and animations. Although some of the timings and animations are not what I intended, I think I can complete this within a few days. I have also completed the Arduino circuit to receive information from the joystick and the two force sensors which are completely usable in the game.

 

Although I am not incorporating computer vision into my final project, this is still a field of interactive media that I am very interested in. What I found most interesting was that this technology has been used for art since the 1960s/1970s. I thought that computer vision’s use for art and media was relatively new, and that computer vision in general was a field that has only become useful in the past two decades. The reading mentions that its use has been limited to  law-enforcement and military purposes, so this really makes me wonder how much computer vision for art would change if this technology was declassified.

I am extremely interested in how computer vision can be further used for media and art, but I cannot help but feel intimidated by the sophistication of it. At least to me, computer vision seems like technology out of a Star Wars movie, but this article allowed me to gain insight into how this technology is not as new as I thought and how it is very accessible to interactive media artists.

I tried to use the get() function, but it did not really work as well as I wanted. Here is the code and the result.

I wanted to create a collage where different parts of the baboon would be at intervals on the screen just to play with the get() function. 

I have decided to scrap my initial idea for a final project, and have instead decided to create a simple multistage game.

This game is intended to be an educational experience about the environment and human history. The game starts with a player in a forest, the style and POV of which is similar to an early Pokemon game, and the player is able to navigate the forest, which is also populated by animals. The player must walk about and interact with the trees, which will be “chopped” down to make a vehicle. The player can also interact with the animals, who will react positively.

Once the vehicle is completed, the player is taken to a world map and can go to three other stages, collecting resources for more complex and modern vehicles. However, as the player moves on to the other stages, interacts with more animals and constructs more vehicles, the environment reacts more and more negatively, with the animals becoming hostile and the landscape burning and becoming desolate. Eventually, the player is left alone on the planet, free to explore the desolate landscapes of the world alone.

I plan on creating a physical controller, similar to the nunchuck on the original Wii with a x-y joystick and one button.  I also hope to create the controller using clear acrylic, with LEDs inside that change according to how much destruction the player has caused the environment, starting with a green controller,  to a yellow one and finally to a red one.

The materials I need are:

• Clear Acrylic
• Several RGB LEDs
• One X-Y Joystick
• One Push Button
• One Arduino
• One Prototyping Board
• Screen or TV to present game

I think that the most challenging parts of creating this project will be creating the game, because I am not so confident in coding the game mechanics and am sometimes not confident in my programming logic skills. I am also worried about the game design and the assets I use, especially the visuals and the music/audio. I want this game to be simple but effective in immersing the user into the meaning of this game. Lastly, I am in the process of creating the game and everything is going relatively smooth. However, I want the game to be coded efficiently so that I do not have hundreds or even thousands of lines of code. I want to be able to replicate stages and objects’ behavior properly so I do not have to copy code between different classes.

 

For this week’s assignment I built upon the assignment I created two weeks ago for the Object Oriented Programming game. I added a joystick to control the square’s movement, as well as a potentiometer so the user can choose a color for the square throughout the game.

It was challenging to figure out how to send the information from the Arduino to Processing, but I am a lot more confident now in understanding how the Arduino sends information to Processing and how Processing uses this information.

The user uses the potentiometer attached to a breadboard, while they can use the joystick to control the customized square’s movement.

Here is a video of the joystick and potentiometer in action:

Below is the code for the Processing sketch:

import processing.serial.*;
Serial myPort;
int xPos, yPos;
int xMove;
int bgValue;

float monsterSpotY = random(300, 560);
float monsterSpotX = random(200, 560);
float monsterRadius = 40;
int gravity = 1;
int coloramt = 0;

class Player { 
  float x;
  float y;
  float up;
  float down;
  float left;
  float right;
  float speed;
  float maxJump;
  float side;
  float prevY;
  Player() {
    x = 0;
    y = 580;
    up = 0;
    down = 0;
    left = 0;
    right = 0; 
    speed = 4;
    maxJump = 400;
    side = 20;
    prevY = 1;
  }    
  void display() {
    fill(255, 100, finalbgValue);
    rect(x, y, side, side);
  }    
  void update() {
    x = x + (xPos) * speed;
    if (prevY == 1) {
      gravity = 1;
      y = y + (yPos + gravity) * speed;
    }
    //prevY = y;
    println(prevY);
    println("ypos is " + yPos);
    if (x <= 10) {
      if (y >= height - 10) {
        x = 10;
        y = height - 10;
      } 
      else if (y < height - 10) {
        x = 10;
      }  
    } else if (x >= 590) { 
       if (y >= height - 10) {
        x = 590;
        y = height - 10;
       } else if (y < height - 10) {
        x = 590;
      }   
      } 
     else if  (y <= 0) {
      y = 0;
    } else if (y >= 590) {
      y = 590;
      prevY = 1;
    } 
    if (y <= 480) {
      prevY = 2;
    }  
    if (prevY == 2) {
      gravity = 3;
      y = y + (yPos + gravity) * speed;
    }   
  }
} 
class Monster {
  float x;
  float y;
  float radius;
  float ySpeed;  
  float monsterSpotY;
  float monsterSpotX; 
  color monsterColor;
  Monster() {

    radius = monsterRadius;
    ySpeed = random(2, 6);
    monsterSpotX = random(0, width);
    monsterSpotY = random(0, 100);
  }    
  void show() {
    //monsterColor = color(map(monsterSpotY, 0, height, 0, 255), map(monsterSpotY, 0, height, 100, 150), map(monsterSpotY, 0, height, 150, 255));      

    fill(coloramt);
    noStroke();
    ellipse(monsterSpotX, monsterSpotY, monsterRadius, monsterRadius);
  }
  void fall() {
    monsterSpotY = monsterSpotY + ySpeed;
    if (monsterSpotY > height) {
      monsterSpotY = random(-200, 300);
      monsterSpotX = random(0, width);
    }
  }
  void run() {
    fall();
    show();
  }
}
int counter = 0;
int gameState = 0;
int finalbgValue;
Player myPlayer;
Monster myMonsters[];
void setup() {
  size(600, 600);
  myPlayer = new Player();
  int monsterCount = 100;
  myMonsters = new Monster[monsterCount];
  for (int i = 0; i < monsterCount; i++) {
    myMonsters[i] = new Monster();
  }
  printArray(Serial.list());  
  String portname=Serial.list()[69];
  println(portname);
  myPort = new Serial(this, portname, 9600);

  //myPort.bufferUntil('\n');
}    

void draw() {
  background(255);
  
  // configuration state; user can select color. 
  if (gameState == 0) {
    textAlign(CENTER);
    fill(0);
    text("Color your hero!", width / 2, 200);
    fill(255, 100, bgValue);
    rectMode(CENTER);
    stroke(0);
    rect(width / 2, 290, 120, 120);
    fill(100);
    rect(width / 2, 395, 140, 30);
    fill(0);
    text("Click here to play!", width / 2, 400);
    println(mouseX);
    println(mouseY);
    if (mousePressed == true && mouseX > 230 && mouseX < 370 && mouseY > 383 && mouseY < 414) {
      gameState = 1;
      finalbgValue = bgValue;
    }
  } else if (gameState == 1) {
    myPlayer.display();
    myPlayer.update();    
    gameOver();
    fill(0);
    text(counter, width - 40, 30);
    int counterIncrement = counter /7;
    for (int i = 0; i < counterIncrement / 20; i+= 1) {
      myMonsters[i].run();
      if (i == counterIncrement / 20) {
        i = 0; 
        println(i);
      }
    }  
    counter++;
  } else if (gameState == 2) {
    textAlign(CENTER);
    background(255);
    text("You Died! Your score is: " + counter, width/2, height / 2);
    text("Click here to play again", width / 2, height / 2 + 30);
    if (mousePressed == true && mouseX > 230 && mouseX < 360 && mouseY > 320 && mouseY < 340) {
      gameState = 1;
      counter = 0;
      myPort.stop(); 
      // ^ myport clears serial before running setup again
      setup();
    }
  }
}  


void serialEvent(Serial myPort) {
  String temp=myPort.readStringUntil('\n');
  temp=trim(temp);
  if (temp != null) {
    int values[]=int(split(temp, ','));
    //split string of inputs by commas and end at line break "/n"
    if (values.length == 3) {
      xPos=(int)(values[0]);
      yPos=(int)(values[1]);
      bgValue = (int)(values[2]);
      xPos =(int) map(xPos, 0, 1023, -3, 3);
      yPos =(int) map(yPos, 0, 765, 3, -3);
      bgValue = (int) map(bgValue, 0, 1023, 0, 255);
    }
  }
}

void gameOver() {
  for (int i = 0; i < 30; i++) {
    if (myPlayer.x + 20  > myMonsters[i].monsterSpotX - 20 && myPlayer.x - 20 < myMonsters[i].monsterSpotX && myPlayer.y - 20 < myMonsters[i].monsterSpotY && myPlayer.y + 20 > myMonsters[i].monsterSpotY - 10) {
      gameState = 2;
    } else {
    }
  }
}

Below is the code for the Arduino:

void setup()
{
  Serial.begin(9600); 
  pinMode(4, INPUT); 
  pinMode(A1, INPUT);
  pinMode(A2, INPUT);
  pinMode(A0, INPUT);

}

void loop() {
  int xValue = analogRead(A1);
  int yValue = analogRead(A2);
  int sensor = analogRead(A0);
   
  Serial.print(xValue);
  Serial.print(",");
  Serial.print(yValue);
  Serial.print(",");
  Serial.println(sensor);
}

 

For my final project, I hope to create an interactive installation that combines both light and sound that uses analog input that influences both LED and a generative art piece created using Processing.

Here is a rough sketch of the project.

I realize that it is hard to see these sketches, but essentially I plan to have one large projection of a Processing sketch, and in front of that, six beams that hold LEDs, and finally a control panel for the user.

The Processing sketch will be able to be controlled by the user through a large touch sensor, or a series of touch sensors, and effects will happen in the corresponding spot on the sketch that the user touches on the touch pad. I am still not sure how to do this, or if this is even possible to do considering that I want the correspondence to be very precise.  I am in the process of looking at pieces of generative, interactive art created using Processing that I might be able to gain inspiration from.

Secondly, I will create a second control panel with six touch sensors that will correspond to each of the LED beams hanging in front of the user. These will be clear acrylic beams with three RGB LEDs inside. When there is no touch or force detected, the LEDs will remain at a certain brightness. When a touch is detected, the LED will light up exactly how strong the force the user puts into the sensor. Each LED will have exactly one color that the user is able to manipulate. A touch of the sensors will also result in a corresponding sound being played aloud.  The beams will also appear to “wrap around the user,” with the closest beams being directly to the right and left of the user.

The Processing sketch will also have a sound being played, probably a more bass-like, ambience noise, while the beams and their sensors will produce sounds that correspond to musical notes.  This will allow the user to be immersed in a sound and light piece that they are able to see direct feedback from.

As of now, I believe that these are the parts I require:

• 18 RGB LEDs
• 6 panels of transparent acrylic
• 6 sound amplifiers
• 6 force/touch sensors (small)
• 1 large touch sensor with the ability to detect where the touch occurred (I do not know if this is possible)
• 1 projector