Images from Sketch
Concept
ArtfulMotion is an innovative fusion of interactive art creation and musical immersion, designed as a captivating digital canvas for users to unleash their creativity. Grounded in the p5.js framework, the program offers a dynamic platform where art and music coalesce seamlessly. A richly textured backdrop, comprising the fifth print from Casey Reas’s Tissue prints, sets the visual stage, creating an aesthetic foundation for users to craft their masterpieces. The harmonious integration of various audio tracks and intuitive music controls allows users to select a musical score that perfectly complements their artistic journey, cultivating a multi-sensory and immersive experience.
The project pushes the boundaries of interaction with the implementation of machine learning through ml5.js. Leveraging the handpose model, users can shape their art through real-time hand gestures, offering a tangible link between the digital canvas and the physical world. A diverse array of artistic modes empowers users to explore varied styles and tools, encouraging experimentation and artistic expression. The user-friendly interface facilitates smooth navigation through different sections, while responsive design ensures the project adapts flawlessly to different screen sizes and resolutions. This program aspires to inspire creativity, fostering an environment where art and music converge to create a captivating and enjoyable artistic experience.
Implementation
In developing this project, I adopted a systematic approach that involved several key steps. First, I conducted an initial assessment to distinguish between frames containing a hand and those without. Subsequently, I delved into verifying the accuracy of the model’s key points. To ensure that these points corresponded correctly to the hand’s landmarks, I employed green ellipses as visual indicators. During this process, I identified a mirroring issue, which arose from my attempt to mirror the webcam feed. To address this, I placed the drawing function for the points within the push() and pop() functions, effectively correcting the mirroring problem.
A pivotal element of my approach was the creation of a dedicated Point class, designed to render the model’s points onto the screen with precision. I seamlessly integrated this class into the existing sketch responsible for drawing the hand’s landmarks. I also fine-tuned the drawKeyPoint() function to generate points that were stored in an array, allowing for streamlined rendering on the canvas.
For the project’s second mode, I took a different artistic approach by implementing curveVertex() in the draw function of the Point class. This choice lent a unique aesthetic to this mode, differentiating it from the first. Furthermore, I embarked on the design of the program’s landing page and other sub-interfaces. These elements were connected through functions I defined in separate JavaScript files. The collective result of these steps was a visually engaging project, which not only translated logic into code but also integrated modularity, object-oriented programming, event-driven programming, and image handling.
[Sketch of planning phase]
Highlight of Code
The draw() function plays a pivotal role in the code, as it manages the application’s state and navigation. It renders different pages, such as the menu, instructions, color selection, mode selection, and the art creation canvas, by utilizing a switch statement to determine the current state. It also incorporates a back button for easy navigation between pages, excluding the menu and art creation canvas. Additionally, the function is responsible for rendering music control buttons, allowing users to adjust the soundtrack as they create art. The code’s modularity and clear separation of rendering logic for distinct states contribute to improved code organization and maintenance.
function draw() {
// determines which page to display
switch (state) {
case "menu":
drawMenu();
break;
case "instructions":
drawInstructions();
break;
case "select_colors":
drawSelectColors();
break;
case "select_mode":
drawSelectMode();
break;
case "create_art":
drawArt();
break;
}
// adds back button if on any other page except menu and main sketch
if (state != "menu" && state != "create_art") {
drawBackButton();
}
// draws music control buttons
drawMusicControls();
}
One of the primary technical challenges encountered in the project was related to resizing the video feed within the drawArt() function while preserving its original aspect ratio and ensuring it was correctly positioned. The challenge involved intricate mathematical calculations to determine the new dimensions and position for the scaled video feed. Additionally, it required addressing the mirroring of the feed to ensure that user interactions remained intuitive and natural. The process demanded rigorous testing and iterative adjustments to strike the right balance and achieve the desired outcome, ensuring that the video feed displayed in the intended location on the canvas while preserving its correct proportions.
function drawArt() {
// scaling video to avoid distortion
let aspectRatio = video_feed.height/video_feed.width;
// mirroring feed from the webcam
// -- beginning of transformation
push();
translate(width, 0);
scale(-1, 1);
if (drawing_flag) {
loadKeyPoints();
drawKeyPoints();
}
image(video_feed, width - ( width / videoScale ), 0,
width / videoScale , (width /videoScale) * aspectRatio);
pop();
// -- ending of transformation
// print button
updatePrintButton();
// end button
createEndButton();
}
Areas for Improvement
In future iterations of ArtfulMotion, key considerations include implementing real-time collaboration, providing an intuitive and secure environment for multiple users to collectively create art; incorporating interactive art elements, enabling users to seamlessly add animations, dynamic effects, or physics simulations, with a focus on user-friendliness and real-time previewing; and achieving music and art synchronization, allowing users to link music parameters to visual elements for a multimedia experience. Furthermore, the project could offer customizable brushes and visual effects, giving users the tools to add unique artistic touches to their creations through an accessible interface that balances customization and usability. These enhancements have the potential to expand the project’s creative horizons and provide users with a more versatile and engaging platform for artistic expression.