Final Project: Formosa!

Concept

The game is inspired by the history of Taiwanese rock, which developed not as a direct product of American cultural imperialism but as a manifestation of our people’s desire to preserve our cultural identity through these modern forms of music; many Taiwanese rock bands write their lyrics in Taiwanese Hokkien and are very involved in social movements outside of their music. These are elements of Taiwanese culture that were suppressed during the martial law era, and the vibrant pop punk/punk rock scene embodies the resilience of our people. Taiwanese rock is an amalgamation of local Taiwanese, Chinese, American, and Japanese influences that reflects our complex history as a colonial subject of many but also our present and future as a country.

The game’s title Formosa! refers to the name “Ilha Formosa,” which was given to Taiwan by Portuguese sailors in the 16th century, meaning “beautiful island.” The song that plays throughout the game is “人間條件 Human Condition” by Fire EX., a band that I consider a symbol of Taiwanese resilience due to their outspoken support of Taiwanese independence and other causes.

I intend for the game to be a light-hearted introduction to this aspect of Taiwan. In the spirit of the wacky, energetic aesthetic of the Scott Pilgrim franchise, the player controls a guitarist who is running late for a gig and has to get there as soon as possible, evading cops and picking up pirated music tapes to gain experience and recharge. My final project is a game centered around a guitar repurposed as a gamepad: each string, when strummed, triggers a specific movement for the protagonist.

How the game works

  • 3 of the guitar’s frets are covered with red strips. These are buttons that players press to either start, jump or attack.
  • Players will have to jump to collect power-ups in the form of vinyls and cassette tapes in order to build up an attack charge (red bar on the top right corner of the screen). The maximum charge value is three.
  • When the attack charge is greater than zero, players can attack to destroy enemies (cops and cop cars). Cops take one attack to destroy while cars take two.
  • When an enemy is destroyed, the score goes up. Contact with enemies will cause the score to go down.
  • Each game lasts 60 seconds. The game automatically ends when the countdown timer reaches zero.

Arduino + p5.js Setup

When players press on a “button” (the red strips on the guitar neck), a circuit is closed between the metal guitar fret and its strings, triggering in-game actions such as jumping and attacking depending on which button is pressed.

The Arduino code simply takes input from the digital pins connected to the three guitar frets; the p5.js code takes this input (0 or 1 depending on whether the circuits are closed or not) and translates it into triggers for character actions.

Arduino code on Github 

The p5.js code has a separate function for each game state — start, instructions, main game and end screen — that is called on in an if-else statement in the main draw function.

The properties of characters and power-ups are set in their individual classes. The hit(entity) function in the Protag class set conditions that are then used to check if the Protag has come into contact with enemies or power-up collectibles in the handleCollisions() and handleAttacks() functions of the main sketch.

Problems Encountered + Solutions

Personally, I’m proud of managing to successfully set up the serial connection between p5.js and Arduino. For a good amount of time, the connection wasn’t working, and I was panicking until I realized that p5.js wasn’t responding as it should be because in all the “if” statements for each in-game action, I put the button state “1” as a string instead of an integer, as they were specified as in the readSerial() function. After simply removing the quotation marks, the serial connection was up and running successfully.

As for the hardware, I also had to figure out new solutions after initial configurations didn’t work as I expected. While i had originally planned to attach 4 wires to individual guitar strings and use a wired conductive guitar pick to create a controller in which strumming each string triggers a different in-game action, this failed to work as all the strings were connected to a metal saddle, which made all the strings one conductive body and therefore could not be utilized for individual actions. I ultimately decided to turn the guitar’s metal frets into the buttons of the controller. I covered three separate frets with copper tape for enhanced visibility and connected them to wires, doing the same for the metal saddle; when the player presses any string against the taped frets, a circuit is completed and a signal is sent from arduino to p5js to trigger 3 actions — start, jump, and attack — depending on which fret is pressed.

Showcase

Future Improvements

The game interface could definitely use a bit for polishing; if I had more time, I’d want to try making more advanced visual and sound effects for when an enemy is attacked or when the protagonist successfully collects a power-up so that players don’t have to keep checking their score and attack charge bar to keep track. It would also be interesting if I could somehow incorporate haptic feedback on the guitar controller– perhaps have the strings vibrate when an attack is used — so that the experience is more immersive and engaging.

Final Project: Don’t Strangle Daisy!!

Concept:

For my final project, the idea switched from a wholesome cuddly teddy bear that gets happier the tighter you hug it, to a tragic ending for Daisy the duck, where she gets sadder the harder users strangle her up until the point where she dies. My intention was to create a wholesome stress reliever for finals season, but it ended up still working in that many users found it to be stress relieving, but it was in a very aggressive manner. This was due to the struggle to find a proper spot for the flex sensors to sit inside of her, so I found it easier to put it around her neck. Depending on the amount of pressure the user puts onto Daisy as they strangle her, her emotions get more and more depressing the stronger the pressure is, which is shown through the p5 display.

Picture:

Implementation:

The physical aspect was relatively simple before I had to figure out how to attach the flex sensors onto Daisy. I connected two flex sensors to the Arduino board, and with the help of several male-to-male and female-to-female jumper wires, I was able to lengthen the “legs” of the flex sensors from the bread board, which allowed for more flexibility when working with the placement of the flex sensors. The difficult part was figuring out how to neatly link the flex sensors to Daisy.

The initial idea was to cut her back open and place the sensors and possibly the breadboard and Arduino inside of her, and then sew her shut. This ended up being problematic because from test hugs, a lot of people hugged her in different ways. Some hugged her with pressure on her stomach while some hugged her with pressure on her head. This would’ve caused issues with the project functioning or not, because that depends on whether users accurately apply force on the flex sensors. Additionally, I had some concerns over the wires getting loose and detached while inside of Daisy. It would’ve also made her really heavy with a thick inconvenient cable coming out of her back, which is the one that connects the Arduino board to my laptop.

I ended up revamping the idea into placing the flex sensors around her neck. Not only was this much easier to implement as I just used some strong double sided stick tape, it was also a lot easier for people to understand what to do without having to give precise detailed instructions on where to hold her. The double sided stick tape was a lot stronger than expected, and still held up after the showcase, although I’m not so sure how well the flex sensors work now. I think they’ve become a little desensitized to all the pressure.

To cover the flex sensors to make it look more neat, I initially made a paper bow to attach to her, but after looking around the IM Lab, I found a small piece of mesh fabric that made a perfect scarf for her, which covered the flex sensors really naturally.

As for the P5 side of things, I took a lot from what I learned while making my midterm project, where I layered different pictures together. These layered backgrounds and pictures of Daisy would represent and visualize her mood, as each layer would show only if the pressure received from Arduino was between a certain amount. For example, if the pressure average was greater than 965, p5 would layer and display Daisy in a happy mood alongside a sunny rainbow background. She has 4 moods/stages. Happy, neutral, sad, and dead. These stages also have corresponding music that plays when the level is unlocked. The p5 also has a mood bar that matches the intensity of the pressure and also changes color to match the mood it reaches. The Arduino code calculated the average of the two flex sensors, so the p5 would display different images depending on the pressure number.

Arduino Code:

Final Project Arduino Code

Schematic:

Challenges and Proud of:

I struggled a lot with, as mentioned earlier, the placement of the flex sensors. I really wanted to keep the wholesome concept of the initial idea I had, so I really had to brainstorm ways to execute it, but I ended up changing it. I found it to be for the better, as the idea seemed to grab people’s attention a lot more than I expected. It was really popular at the IM Showcase and everyone wanted to strangle poor Daisy, up to the point where the flex sensor pressure wouldn’t go lower than roughly 650, which meant it was really difficult for people to reach her “Sad” stage and even harder to kill her. I ended up tweaking the numbers a bit during the showcase so people could feel satisfied with reaching her “Dead” stage, but it wasn’t that way at the start of the showcase.

I was really proud of this project overall and I was satisfied with the way it turned out even though it was a complete 180 from my idea. It did relieve a lot of people’s stress, just not in the way I intended or expected it to be. It was really funny to watch everyone strangle this poor plushy just to reach the “Dead” stage, and to see the pure joy on people’s faces once they managed to kill her. I was also proud that I properly managed to add that “Dead” stage and also complementary music a bit before the showcase.

Future Improvement:

In the future, I would want to figure out how to prevent the flex sensors from “wearing out” so that it wouldn’t affect the sensitivity of it. I would also want to add more flex sensors so that people could strangle Daisy however they want without having to stay on one specific line in order for it to work from there being more surface area to work with.

Showcase:

IM Showcase Interaction

The tragic aftermath of Daisy by the end of the IM Showcase. Very stiff.

 

Final Project | Cooking Nana

Concept

This project is a cooking simulation game, inspired by the DS game Cooking Mama. Players interact with the game using physical buttons connected to an Arduino, allowing them to select ingredients and create virtual dishes across three courses: starter, main course, and dessert. Visual feedback is provided through animations (GIFs) that I created using Procreate.

How it works

  1. Starting the Game:
    • The game begins on a start screen, with an animated GIF displayed.
    • The player presses the space bar to advance to the menu.
  2. Choosing a Course:
    • The player selects a course (starter, main course, or dessert) using a keyboard key (1, 2, or 3).
    • The game transitions to the selected course with an animated visual prompt.
  3. Making Ingredient Choices:
    • For each course, the player makes three ingredient choices (e.g., base, protein, garnish) by pressing buttons:
      • Button 1 selects the first option.
      • Button 2 selects the second option.
    • The game responds by showing an animated GIF representing the selected ingredient.
  4. Final Dish Display:
    • After three choices, the game displays a final animated dish based on the player’s selections. I had to draw 8 versions for each course to account for all outcomes. I did 53 GIFs for the whole thing.
    • Players can restart the game by pressing the reset button.
  5. Reset Functionality:
    • Pressing the reset button at any point restarts the game to the start screen.

Project Demo

user testing video 

Code

P5: link to the P5 fullscreen page, GitHub

Arduino:  Github

Digital Arduino and Schematic

 

 

 

 

 

 

 

Reflection

Overall I’m really happy with my final project and I’m proud of myself for getting it done. I like how the animations came out and I’m glad I got the logic down to work regarding the user choices for ingredients.

challenges: 1. I originally wanted to use LEDs to inform the users of their progress throughout the game. But one of the LEDS didn’t work so I had to scratch that idea. 2. The animations took way longer than I anticipated so it set me back a lot with this project. 3. I struggled a lot trying to make the serial communication happen the way I wanted to. 4 I wanted to include sound effects for the cooking  because I think sound is a really big part in feeling immersed in something, but it was too much on p5 since I had 12 different audios with the 53 GIFs; it kept lagging and wasn’t worth it, so I just stuck with background audio.

future improvements: If I had more time I would’ve added better instructions on how to play the game, to make it more user friendly.

screen shots of the P5 interface GIFs

Final Project | The Others (Eye tracking & LED strip)

Concept

This project explores the theme of gaze—the act of looking and being looked at—and its tangible representation. Usually the male gaze has been viewed as a source of objectification, but gaze itself can carry a range of meanings: it can be powerful, harmful, or even kind. This project aims to make the intangible concept of gaze perceptible through physical means, by offering the dual identity of seen and being seen.

Code & Schematics

P5.js

Arduino

User Test

Initially, the prototype used LEDs on a breadboard to represent the gaze (forgot to record a video but breadboard pic can be found below). While functional, the LEDs were not as bright as desired. To achieve better visual appeal, NeoPixels were chosen for their brightness and flexibility in creating specific shapes. This decision also eliminated the need for a shift register.

Video: IMG_5236

The project was inspired by works such as Behnaz Farahi’s Caress of the Gaze, which explores the physical and emotional connection created by gaze.

 

Challenges

This project was heavily focused on troubleshooting, with many technical and physical challenges arising despite the relatively simple underlying logic:

  1. Webcam captures the gaze.
  2. P5.js maps the zones based on the gaze.
  3. Arduino sends the data to control the NeoPixels.

While the concept is straightforward, implementation involved resolving numerous small but critical details. It was rewarding to achieve the desired effect: NeoPixels that “breathe” in response to the user’s gaze.

Physical Challenges

  • Cutting and Soldering NeoPixel Strips: To achieve specific shapes, the NeoPixel strips had to be cut and soldered back together. The small connection points made soldering difficult, as wires often disconnected or the solder failed to hold. Using a glue gun improved the stability of the connections.
  • Attaching to the Dummy: The dummy’s soft fabric surface posed challenges in attaching the NeoPixels. Taping the dummy first created a sturdier surface for application.

Technical Challenges

  • P5.js:
    • Managing sound (loadSound issues): Ensuring the sound file was correctly placed and loaded at the right stage of the code.
    • Resolving unexpected errors (no readSerial(), unexpected token, etc.).
    • Wrapping text within certain areas of the canvas.
  • Arduino:
    • Voltage insufficiency: The NeoPixels required more power than the current setup provided.
    • Addressing only one LED lighting up by ensuring proper data propagation across the strips.
  • General Issues:
    • Balancing power requirements and ensuring stable performance without burning out components

Future Improvements

There are so many things that I can change for this project.

  1. Color Customization:
    • Enable the NeoPixels to display colors that align more closely with the intended vision.
  2. Wearable Integration:
    • Draw inspiration from Farahi’s Caress of the Gaze by combining NeoPixels and mirrors into a wearable piece, offering a dynamic, interactive experience.
  3. Refining the Setup:
    • Integrate a more discreet camera to make the setup less obvious.
    • Explore new materials for the dummy to simplify the attachment process.
  4. Power Optimization:
    • Use a larger power supply or distributed power injection points to ensure consistent brightness across all NeoPixels.
  5. Software Refinement:
    • Improve error handling in both P5.js and Arduino code to address recurring issues like serial communication failures and unexpected tokens.

Final Project – Blue Lovable Utilitarian Reading Buddy (B.L.U.R.B.)

p5.js Sketch   |   GitHub URL   |   Schematic Diagram

Inspiration

For all my assignments so far, I’ve tried to do something that’s personal and meaningful to me in some way, and this project is no different. My favorite hobby, without a doubt, is reading—it’s something I’ve loved for as long as I can remember. So, when I started brainstorming for this project, I decided I wanted to create something that revolves around the joy of books and the experience of reading as a whole.

Now, of course I’ve always thought there was something magical about getting lost in a story and exploring new worlds through words. But as much as I love reading, it can sometimes feel like a solitary experience, especially when I come across a thought or idea I want to share but no one’s around to talk to. That’s what inspired me to create a book buddy robot—something that could bring a sense of companionship to reading, making it feel a little less lonely while still keeping the focus on the stories I love so much.

Concept

The concept behind my book buddy robot, B.L.U.R.B. i s to create a small, interactive companion that enhances your reading experience. Made using p5.js and Arduino, B.L.U.R.B. is designed to be there with you as you dive into a good book, offering a few fun features along the way.

  • Flip Pages (Arduino): Using Arduino, B.L.U.R.B. has the ability to automatically flip the pages of your book. This page-flipping function helps keep the flow going without having to pause and turn the page yourself. It gives the feeling of having a personal assistant to handle the mundane task, allowing you to stay immersed in your story.
  • Play Ambient Music (p5.js): B.L.U.R.B. uses p5.js to play soft, ambient music tailored to enhance your reading experience. Whether it’s a gentle instrumental tune or nature sounds, the music helps set the atmosphere, making it easier to focus and get lost in your book. You can choose different genres or moods depending on the type of book you’re reading, helping you stay engaged and relaxed.
  • Track Reading Time (p5.js): With p5.js, B.L.U.R.B. can track how much time you’ve spent reading, giving you a sense of accomplishment as you progress through your book. It can display a timer on the screen, helping you set reading goals or simply monitor how long you’ve been immersed in the story. It’s a subtle way to keep track of time without taking away from the reading experience.
  • Give Break Suggestions (p5.js): Sometimes, reading for too long can leave you feeling drained. When you take a break, B.L.U.R.B. suggests helpful activities that could refresh your mind. These suggestions might include stretching, taking a short walk, drinking water, or even doodling on a piece of paper. It’s a fun way to make sure you’re taking care of yourself while you dive into your books.
  • Display Messages (Arduino): B.L.U.R.B. features an Arduino powered LCD screen that displays small, personalized messages. These messages could tell you when B.L.U.R.B. says “Hello” and also display whether you’re currently reading or on break. The idea is to add a little more interaction to the reading process, keeping you engaged and providing a sense of companionship while reading.
  • Move Head (Arduino): B.L.U.R.B.’s head movement is controlled by Arduino, reacting to your actions. Whenever you switch between reading and break modes,  B.L.U.R.B. moves his head from side to side. It’s a fun way to make him feel more like a real companion that’s in tune with your reading habits.
  • Light Up Reading Lamp (Arduino): B.L.U.R.B. has a small reading lamp that can be activated by tapping its foot. This feature adds a practical element to your reading experience. When the lights are dim or you need a little extra light, a simple tap on B.L.U.R.B.’s foot will turn on the lamp, giving you the perfect amount of illumination to read comfortably.

Implementation

B.L.U.R.B’s Design

B.L.U.R.B.’s design started with a cute 3D model found on Thingiverse. However, 3D printing issues led to the need to split the model into multiple parts, which were then glued together later. After printing and assembling the pieces, I spray-painted all the components in a soft pastel blue, matching B.L.U.R.B.’s calming design.

This is a video I took after the initial 3d printing stages, while trying to work on making his head move from side to side:

On the p5 side, B.L.U.R.B. is seen sitting on a cozy sofa in a softly lit library room. The setting is designed to create a comfortable atmosphere, enhancing the user’s reading experience. B.L.U.R.B. interacts with the user through animated speech bubbles, where the text appears in a typewriter effect, adding a dynamic element to the conversation. This combination of design and animation gives B.L.U.R.B. a friendly, approachable personality while keeping the focus on a relaxing, cozy environment.

Flipping Pages

To figure out how to flip pages mechanically, I watched a few videos online for inspiration. My initial idea was to use two servo motors and a temporary adhesive, like Blu Tack, that doesn’t dry out too easily. I built several prototypes and tested different adhesives, but none worked perfectly. Strong adhesives stuck too firmly to the paper and couldn’t unstick, while weaker ones didn’t have enough grip to flip a page.

The adheseive would be stuck on the lower end of the shorter stick of the prototype below:

Eventually, I decided to try a different approach. This involved using a combination of a wheel and a stick attached to a servo motor. The wheel made contact with the page to be flipped, rotating clockwise for some time to lift the page. Then, the stick would get into the gap that was created by this page lift and swing 180 degrees to complete the flip, before returning to its original position. To ensure the next page lay flat, the wheel briefly rotated counterclockwise to smooth it down.

The wheels available in the lab didn’t work well for this setup, so I had a custom wheel 3D printed. This improved the mechanism somewhat, but it still wasn’t perfect. The main issue was that the wheel would sometimes lift multiple pages at once, causing the system to lack the power to flip them all successfully. This happened because the wheel’s motion often lifted more than one page during its run.

It took a lot of iterations to figure this out but the mechanism worked much better at the end with a small change: in the initial few seconds of the wheel running (when it is likely to have slightly lifted the first page), the stick moves slightly inward to fill this gap. The wheel then runs a little more and the servo motion happens as it did before. This ensures that the stick (more likely than not) gets into the right spot (under just one page) and the servo motion causes the page above it to flip.

Playing Ambient Music

For this feature, integrated into the p5 part of the project, I added three different sound files: Lofi, Nature, and Piano. These give the user the option to choose between them whenever they want to listen to ambient music while reading. The user can easily switch between the sounds or stop the music entirely whenever they wish, giving them full control over their listening experience. These sounds were specifically chosen because they are calming and help create a focused atmosphere without being distracting.

Tracking Reading Time

To help users monitor how long they spend reading, I added a reading timer to the project. The timer starts automatically when the user clicks the “Begin Reading” button in p5. If the user decides to take a break, the timer pauses and saves the current reading time. Once the user clicks to continue reading, the timer resumes from where it left off.

This feature is designed to work in the background, ensuring the user can focus on their reading without worrying about keeping track of time manually. It provides a simple and efficient way to measure reading sessions, making it easy for users to reflect on their habits or set reading goals.

Giving Break Suggestions

Whenever the user takes a break, the robot, B.L.U.R.B., adds a personal touch by saying, “Enjoy your break! I suggest you [insert break suggestion] :)”. (all of these messages show up on p5). The break suggestion is randomly selected from a list of  about 40 healthy activities stored in a CSV file. These activities are thoughtfully curated to encourage wellness and productivity during breaks.

The suggestions include ideas like:

  • Doing a quick exercise routine
  • Grabbing a healthy snack
  • Tidying up your desk
  • Meditating for a few minutes
  • Sketching or doodling
  • Taking a short walk

This feature not only adds personality to B.L.U.R.B. but also helps users make the most of their breaks by promoting healthy habits and keeping their minds refreshed.

Displaying Messages

On the p5 side, B.L.U.R.B. introduces himself to the user and provides a brief explanation of the features he offers and how he can assist with reading.

On the Arduino side, an LCD screen is attached to B.L.U.R.B.’s neck to display messages and add personality.. Initially, the screen shows a cheerful “Hellooo!” to greet the user.

  • When the user begins reading or resumes after a break, the screen updates to display “Reading Time” along with a beating heart animation. This beating heart adds a sense of life to the robot and are also in tune with the passing seconds on the reading timer.
  • When the user takes a break, the screen switches to display a friendly “Break Time :)” message, creating a fun and relaxing vibe during the pause.
Moving Head

To make the experience more interactive, B.L.U.R.B. moves his head from side to side whenever the user switches between reading and break modes. This movement is controlled by a servo motor attached to B.L.U.R.B.’s head, adding a playful and lifelike element to the robot.

Light Up Reading Lamp

To enhance the reading experience, a blue NeoPixel ring is used as a reading lamp, positioned near the book to provide sufficient lighting. The color of the light matches B.L.U.R.B.’s blue design, creating a cohesive look. The lamp is controlled through a simple and interactive mechanism: a piezo pressure sensor placed on B.L.U.R.B.’s foot, which is marked with a lamp sticker. Tapping the foot toggles the light on or off. This setup makes it easy for the user to control the lamp without interrupting their reading. The soft blue light from the NeoPixel ring adds a calming and functional touch to the environment.

Initial prototype:

Some Work In Progress Pictures


Overall Setup

The overall setup was designed with a focus on functionality and aesthetic appeal. A wooden plank of the right size was covered with brown paper, creating a smooth surface to lay the book and house the flipping mechanism. The mechanism includes a wheel with a motor attached to a block of wood, which is fixed to the plank, along with a servo mounted on the bottom edge to control the page flipping.

To keep the wiring organized, I used another box, also covered in brown paper, to house all the electronics. I carefully cut slits in the box to allow the wiring to extend outside when necessary. To add a touch of personality, I printed out cute book-related quotes that resemble stickers, cut them out, and stuck them to the box. The robot, B.L.U.R.B., was placed on top of the box, completing the setup and making it visually appealing while still functional. This design not only keeps everything neatly organized but also enhances the overall aesthetic of the reading space.

Final Setup in Progress

User Testing

This project was displayed in the Fall 2024 end-of-semester Interactive Media Showcase at NYU Abu Dhabi. Here are some photos/videos of interaction from the showcase:

Potential Improvements

There are a few areas where I could improve the project to make it even more engaging. One idea is to increase interaction with B.L.U.R.B. by having him move his head randomly to keep the user engaged. Additionally, if the screen in p5 is left idle for a while, I could make everything except the timer disappear, creating a more minimalistic and focused reading environment. Lastly, positioning the reading lamp above the book would provide better coverage of light across the pages, improving the overall reading experience. These changes would enhance both the functionality and the user experience, making the project even more dynamic and enjoyable :))

Reflections

I’m really proud and happy with how this project turned out! When I first decided on the idea, I had no clue whether it would be possible to bring everything together and make it all work. But I was determined to give it my best and worked on it every day, tackling one challenge at a time. The most difficult part, of course, was the page-flipping mechanism—I spent several days experimenting and testing different approaches before I finally got it to work.

In the end, seeing all the different components come together felt incredibly rewarding. Presenting it at the showcase was a really proud moment for me. Watching people interact with B.L.U.R.B. and enjoy the experience I had created made all the effort worth it. I’m happy with how the project reflects my passion for reading and combines functionality with a cute, thoughtful design. It’s been a journey full of challenges, but the result really does feel like an accomplishment :))

Final Project User Testing

User Testing

For the user testing, I think the fact that the same plushy in real life was on the p5 screen kind of hinted at what users were supposed to do, especially since the sensors were not so subtly taped onto the back of the plushy. The physical aspect of the project is still in the makes, which is why the sensors were on the back, because I’m still trying to figure out the best way to implement the sensors without taking away from the capabilities of the flex sensors. Ideally, I wanted to stuff the flex sensors as well as the arduino board and bread board into the plushy, but I realized how heavy it would be and also the struggle of the jumper wires potentially disconnecting. I think that if I stuck with the plan where the only thing visible was the arduino cable to connect to my laptop, it would be less self-explanatory and a bit harder to figure out since the sensors wouldn’t be in plain sight.

I think there was some confusion on how exactly the plushy should be squeezed. I don’t think the user knew it was supposed to just be hugs, but he probably hugged the plushy simply because it’s a cozy plushy. If it were a different object, I don’t think his first instinct to interact with it would be to hug it. He ended up squishing the plushy between his palms as well, which was a really interesting way of playing with it.

I think the overall concept is working well, where the pressure value and p5 connection is working properly, but there could be major improvements in where the sensors/arduino components could be within the plushy, as well as better p5 display and design with instructions and maybe some context on the project. I think I would have to explain the purpose of it because knowing your tight hugs make the plushy’s day would definitely add more positivity into the interactive experience. I would also have to explain how exactly it would work, which is a bit complicating because saying to just hug it is a bit vague since I only have two flex sensors, so if they hug an area without the sensor, the p5 screen wouldn’t reflect anything, which would be a pretty disappointing outcome. Having just brief words on the p5 screen before or maybe during its start would help make sense of the idea

Final Project Progress: User Testing

Progress

Over the break, I made quite a few changes to the hardware component. While i had originally planned to attach 4 wires to individual guitar strings and use a wired conductive guitar pick to create a controller in which strumming each string triggers a different in-game action, this failed to work as all the strings were connected to a metal saddle, which made all the strings one conductive body and therefore could not be utilized for individual actions. I ultimately decided to turn the guitar’s metal frets into the buttons of the controller. I covered three separate frets with copper tape for enhanced visibility and connected them to wires, doing the same for the metal saddle; when the player presses any string against the taped frets, a circuit is completed and a signal is sent from arduino to p5js to trigger 3 actions — start, jump, and attack — depending on which fret is pressed.

The overall structure of the p5 game is now complete. The only work left to do is to finish the visuals of the game, as some of the power-ups and enemies are displayed as basic black placeholders. A system also has to be set up to keep track of high scores over the course of multiple games.

User Testing

At first, it was confusing for users to figure out how to operate the guitar controller, as I haven’t created any signage to attach to the frets and indicate their functions. It took them a few seconds of fiddling before getting to the actual gameplay. The gameplay also wasn’t as straightforward as I thought it was; users were confused as to how to increase their score (which is by collecting power-ups to build up attack charge and attack enemies). I will have to work on the visual components of the guitar controller (adding small labels, etc.) and adding an instruction menu before the game starts.

Final Project User Testing

Without clear instructions the user was confused between how the buttons and notes corresponded to each other, so I had to explain the game before the user tested it. I think everything will be fine when I combine the game and the tutorial and maybe if I add an instructions page if the user is still confused with the tutorial.

Everything in my experience is working well, the buttons are corresponding to the notes and sound, and if a wrong button is pressed 5 points are deducted and if you press the correct button corresponding to the note you get 10 points, I might want to add a library to save the highest score so far so it becomes competitive.

What I want to improve in my experience is the overall look, I want to laser cut the box that would hold the Arduino and I want to paint some of the buttons to match the music therapy note colors.

Final project User testing

Video: 

IMG_4223

 

User testing analysis: 

The basics of my project were easily understood by users, without needing much instructions, they all understood that the buttons corresponds with the graphics that are shown on p5. There was nothing that particularly confused them regarding the mapping of the controls and the experience. However, the drum selection dropdown was mostly ignored by everyone that I have shown the project to. I felt the need to tell the users that that was a feature that was built in, so I think i can improve on making it more clear to the users that the dropdown menu is available on for them choose different drum kits from.

 

 

 

Assignment 13: Final Project User Testing

Progress

I finally finished with the hardware aspect of my project, in terms of making the head body and arm of the cat (the ears are still missing though..). During the process, I encountered some issues with my project, mainly with the pressure sensor. First, I made my own pressure sensor using velostat and that works as shown in the final project proposal for Assignment 12. However this was too small for what I actually need for my final piece, so I tried making it on a larger scale but the pressure sensor did not give me a variety of readings and only gave me high values of readings (in the 1000s) and it rarely varies. I tried using the piezo sensor, but it also does not vary. So at the moment, the pressure sensor is not a functioning part of my project.

Other than that, I have made progress with the interface on the p5 side, finished building the arm and have tested it so that it moves with the servo motor and I have also implemented the lcd to reflect what is written on the input box in the p5 sketch.

User Testing Video

Feedback and Reflections

For this user testing, I asked one of my friends to test my project. He was able to figure it out because I was debugging right in front of him, so he saw what actions I made and knew what to do. But if he didn’t see me do it, he said he would have been confused because I didn’t give instructions for how to trigger the cat’s hand to move.

There were also parts where I couldn’t finish in time to implement, that being illustrating the different images I want to pop up on the p5 sketch whenever the cat’s hand move. So he said the cat’s hand moving right now seems a bit pointless (and I had to explain what was meant to happen).

After this user testing, I realized there are still a lot more improvements and edits that I need to make to my project. One being to give clear instructions on the p5 sketch so that users know exactly what to do. Another thing is to find a way to get the pressure sensor working. And finally, finish with the illustrations so that I can implement it to my p5 sketch.