Concept
For my Final Project, I don’t have a concrete idea yet that I have committed to but I know the general area which I want to work on as well as some potential projects.
I want to explore ML5.js which is a machine learning designed for the web. In short, it utilizes the graphical powers of the browser to do machine-learning calculations. It works well with P5.js which is why I want to utilize it. It’s open source library which is beginner-friendly providing high-level interface to TensorFlow. As someone with no machine learning experience, it will be very interesting to explore it in depth. It has some available pre-trained models which work for image classification and recognition. For example, HandPose model would work for detecting hands and doing some action with it.
The initial concept idea would be to create a canvas where you can draw with your hands in P5js with the help of Handpose. I could utilize arduino to add additional functionality. For example put sensors in a cardboard which would act as brush settings. For example, various sensors would change various brush properties. Cut out for ultrasonic sensor could potentially change size of the brush by moving your finger. Ideally, I would like to make the interface very seamless.
I could expand this idea for user to create generative art by varying different mathematical properties using their hands, it does not have to be limited to just drawing.
Challenges
I would face many challenges going with this approach as I have no prior experience. Moreover, I would have to combine the machine learning aspect with Arduino to create a seamless experience. My plan to overcome these challenges is to start learning as early as possible. This includes going through the library, learning the basics and finalizing my plan which I can stick with and dedicate much more time to. It should be challenging, yet achievable in the timeframe we are given. I am optimistic that this project will not only be very beneficial by learning lots of new things but will also be quite fun!