Midterm: DanceVille

Link to my sketch: https://editor.p5js.org/is2587/full/Bn466sW2p

Concept

My midterm project began as a whimsical idea: imagine a disco party where your favorite cartoon characters hit the dance floor. But this idea seemed too simple and I could not figure out how to make it more interactive and engaging for the user. As I began coding, I stumbled upon Dan Shiffman’s Teachable Machine playlist on YouTube. His playlist sparked a new inspiration – why not infuse ML into my project as well? Plus, a repeated suggestion I got from my friends regarding making the project more engaging was to somehow make the users mimic the dancing characters movement.

This fusion of disco vibes and machine learning transformed my simple “disco experience” into an engaging game. Now, users don’t just watch the dance moves; they become the stars of the show, mimicking poses detected by the webcam. To enhance the scene, I added dancing characters, some spotlights, and floor highlights.

Sketch

Problems/WHAT IM PROUD OF

Since working with a machine learning model was a new experience for me, I faced quite a lot of challenges throughout the coding process. One significant hurdle was integrating my trained machine learning model to recognize user poses. I had a hard time figuring out how I can increment the users score according to the poses detected by the model.

function classifyVideo() { 
classifier.classify(video, gotResults); 
}

function gotResults(error, results) {
  if (error) {
    console.error(error);
    return;
  }
  // Store the label and classify again
  label = results[0].label;
  classifyVideo();
}

function updateScore() {
  if (currentImage == clappingImage && label == "Clapping") {
    score += 1;
  } else if (currentImage == dabbingImage && label == "Dabbing") {
    score += 1;
  } else if (currentImage == handsUpImage && label == "Holding hands up") {
    score += 1;
  } else if (currentImage == armRollImage && label == "Rolling arms") {
    score += 1;
  }
}

Here’s how it works: when the machine learning model processes the user’s dance moves, it returns a label indicating the detected pose, like “Clapping” or “Dabbing.”

Then, the updateScore function comes into play. It’s responsible for comparing the detected pose to the pose displayed on the screen (like clapping or dabbing images) and, if they match, incrementing the user’s score.

Another puzzle was dealing with user stillness. Initially, even if users stood frozen, the machine learning model would guess a dance move. This unexpected stillness would inadvertently increase the user’s score. The solution I came up with was to train my model to identify moments of “no movement,” ensuring it could distinguish between action and inaction.

Since I could not find the right poses online that were also consistent with the overall aesthetics of the game, I decided to sketch the different poses on my ipad and integrated that into my sketch which was another aspect I really enjoyed working on.

What I like the most is how little features such as the dancing characters or the music choice really add to the whole experience. Before you enter the disco, I added a special touch to the music. At the beginning, it’s like you’re standing right outside a disco, and you can hear the muffled sounds of music from inside. It sets the stage and builds anticipation. But as soon as you enter, the music changes to something lively and funky, perfectly matching the energetic and vibrant atmosphere of the game.

Reflections

I knew working with machine learning would not be easy however, taking on this challenge working on this project all the more exciting. It was a blend of fun and anxiety I would say. I’m particularly proud of how I managed to keep track of scores and how I elevated the game’s aesthetics. Details like animated characters grooving on the dance floor and hand-illustrated dance move images added an extra layer of vibrancy.

If I could revise one aspect of my project, it would be fine-tuning my model to better recognize distinct dance poses. This, I believe, would further enhance the user’s dance-off experience, making it even more enjoyable and accurate.

REFERENCES

Dan Shiffman’s teachable machine playlist: https://www.youtube.com/watch?v=kwcillcWOg0&list=PLRqwX-V7Uu6aJwX0rFP-7ccA6ivsPDsK5&index=3