Concept
I will be creating an interactive game called “SignSprint”. It is a typing-like game that uses American Sign Language (ASL) as input instead of the regular keyboard input. The user is supposed to make signs corresponding ASL alphabets to match letters and words displayed on the screen. The user has a specified time to completely spell the words displayed using ASL and at the end of the specified time, the accuracy and speed of the user will be displayed. There will also be a leaderboard to display the fastest users.
The inspiration behind this game is a class I am currently taking on Disabilities Studies where I am required to learn some ASL. After learning about computer vision in class and playing with a few models, I decided to combine these two ideas and create a game out of it. I believe creating this game will be a great exercise to learn ASL and have fun while doing so.
Design
This game will be built using a machine learning computer vision model to identify the alphabets of ASL as input to play the game. The game will consist of 3 main screens:
Start screen: This will the first screen and the main menu . It will contain a guide on how to play the game. It will contain a button to that opens a page to show a full list of the ASL symbols, a button that open a trial mode for users to practice their ASL in a non competitive or game like environment and finally a button that opens the competitive game mode where the user will be timed and expected to make the appropriate symbols.
Game screen: There are two main game screen. The trial screen where a user practices their ASL with no time. The user can go on as long as they want to and exiting this screen takes them back to the start screen. The other form of the game screen is the test screen where a user has 60 mins to get as many letters correct as they can. The screen will automatically closed after the 60s and lead you to the end page.
End page: This will display the leaderboard of the fastest players and contain a button or instruction to restart the game.
Frightening part of the code
The frightening part of the code is getting the machine learning model to identify the signs and interpret it into input for the game. It model has to be quick and responsive to improve the user experience overall.
The reduce this risk, I am exploring the use of ml5js library to get the appropriate model get the user input.