Shuwa (手話) is Japanese for "Sign Language"

Shuwa Gesture Toolkit is a framework that detects and classifies arbitrary gestures in short videos. It is particularly useful for recognizing basic words in sign language. We collected thousands of example videos of people signing Japanese Sign Language (JSL) and Hong Kong Sign Language (HKSL) to train the baseline model for recognizing gestures and facial expressions.

The Shuwa Gesture Toolkit also allows you to train new gestures, so it can be trained to recognize any sign from any sign language in the world.

[Web Demo]

by combining the results of a Holistic model over multiple frames. We can create a reasonable set of requirements for interpreting sign language, which include body language, facial expression, and hand gestures.

The next step is to predict the sign features vector using a classifier model. Lastly, output the class prediction using K-Nearest Neighbor classification.

• Install python 3.9
• Install dependencies

  pip3 install -r requirements.txt 
  

python3 webcam_demo.py

• Use record mode to add more sign.
  

• Play mode.
  

1. Download hksl_jsl_samples.zip
2. Extract to root directory
3. run

   python3 webcam_demo.py
   

You can add a custom sign by using Record mode in the full demo program.
But if you want to train the classifier from scratch you can check out the process here