Concept:
For my final project, I want to develop an interactive system which mixes emotional machine learning, environmental sensing, and generative art. The core idea is to create an ambient environment that responds dynamically to human emotions and gestures. Using ML5.js, I aim to analyze real-time inputs like facial expressions (via FaceAPI) or body movements (via PoseNet) from a webcam feed, translating these into evolving light patterns using Arduino-controlled LEDs and abstract visualizations in p5.js.
Challenges:
First, I need to determine how to reliably connect ML5.js (running in a browser) with Arduino, maybe through the WebSerial API, while ensuring real-time synchronization between emotion detection, visualization, and lighting. Latency concerns me, as even minor delays could break the immersive experience. Second, mapping emotional states (e.g., “happy” or “angry”) to artistic outputs feels subjective; I’ll need to experiment with parameter mappings (color, motion, sound).