Are they able to figure it out? Where do they get confused and why?
My friends were a bit confused what to do in first place. So, I included instructions on top left corner, that by doing a fist they could transition between different 3D models and by using index finger and thumb they could manipulate the sketch.
Do they understand the mapping between the controls and what happens in the experience?
They do understand it. Controls how to use it is already given in the top left corner as instructions, so no problem with that.
What parts of the experience are working well? What areas could be improved?
I’m still working on adding new 3D models and adding explanation to each one and how it is specifically tied to Kazakh culture. I want people to learn about Kazakh culture and feel the items, observe them, and etc. I want to do a menu screen, and another instructions screen. Add 3 more Kazakh Cultural elements. Also, more features that would allow to do something interesting with the sketch line draw lines and oscillations and something interactive like this on the background of 3D model. In addition, I will working heavily on Arduino side from now on, adding arcade buttons for transitions between different 3D models. and building an engraved casing using probably ply wood.
What parts of your project did you feel the need to explain? How could you make these areas more clear to someone that is experiencing your project for the first time?
I’m expecting that engraved casing for the project with arcade buttons would make more sense on how to transition between different aspects of Kazakh culture rather than making a fist. But, I think using a fist pretty cool idea, and I get to explore hand gestures more. I will try to make instructions constant on the page, in case user forgot, they will always be there. Also, in main menu I will try to explain a bit about the project itself.
The project teaches people about Kazakh culture through interactive 3D objects. The user moves their hand in front of the webcam and controls a 3D asyq and a 3D yurt in real time. Hand rotation and hand openness change rotation, scale, and animation of the models. The goal is to make cultural objects feel alive and playful.
Handpose in p5.js finds key points on the user hand. The code maps these points to rotation and size values. The 3D asyq and yurt models load from external OBJ files. The models update their rotation and scale on every frame based on hand movement. The interaction is simple so anyone can explore the cultural objects.
The user brings their hand into the camera. Turning the index finger rotates the asyq. Bigger pinch between thumb and index distance makes the yurt grow. I want to implement moving the hand forward or backward changes the distance of the models. The idea is to mix physical movement with digital cultural storytelling.
The Arduino is optional but adds a second layer of interaction. A sensor like a potentiometer or flex sensor sends values to p5.js. The value can change things like background color or cultural patterns. But, currently, I’m thinking of using arcade buttons to change between different 3D models.
The Arduino sends sensor values through serial. p5.js reads the values using the Web Serial API. The values change simple visual settings. This lets the user control the cultural objects both by hand and by sensor.
I am proud that the project takes Kazakh cultural objects and makes them interactive. I like that people can learn culture through movement. I am happy that the 3D models work smoothly with hand tracking.
I created the asyq and yurt models with 3D tools. I used p5.js for graphics and Handpose for tracking. I used Arduino for extra input. I used generative AI to help fix errors, structure the code, and write the description. All cultural objects are based on real Kazakh designs. I found 3D models online on Sketchlab, and I will reference them.
I want to add more gestures for cultural actions like throwing the asyq. I want to add more objects like dombyra or shanyrak. I want to add sound from Kazakh music. I also want to create a cleaner guide so visitors can learn the meaning of each cultural object.