Enhancing Pipeline with Vietnamese Language Support about translate HOT 7 OPEN

Thank you for opening this issue.
To support Vietnamese sign language in the current approach of text->gloss->pose, please see https://github.com/sign-language-processing/spoken-to-signed-translation
It will allow you to load a dictionary (in a specific format) such that vietnamese is supported.

Then you will have two options:

1. Expose this tool using an API, and call it from a local version of sign/translate
2. Share the dictionary with me, and I will make it work in https://sign.mt alongside other languages.

from translate.

There seems to be a dictionary available for Vietnamese - https://tudienngonngukyhieu.com/tu-ngu-theo-bang-chu-cai

Your steps should be as follows:

1. Add support for this dictionary in sign-language-processing/datasets (Make a PR)
2. Run pose estimation on the data (video-to-pose -i video.mp4 -o video.pose)
3. Edit download_lexicon to support this dataset from the datasets library

Then, you will have a local demo running with this dictionary, and you will be able to apply some modifications if you want.

from translate.

Hi @AmitMY ,
Could you please provide an update on the status of this issue? Timely feedback would be greatly appreciated, thanks

from translate.

Hi @AmitMY ,

I have run the repository using the download_lexicon.py file, and as I understand, the output folder will be the dictionary that I can share with you. However, I encountered an issue as the dataset used in the demo repo is signsuisse, which only supports Swiss, French, German, and Italian languages. I have also searched for datasets supporting Vietnamese at sign-language-processing/datasets, but they do not exist.
What should be my next steps to proceed with adding Vietnamese language support?
Your timely feedback would be greatly appreciated as this is quite urgent.
Thank you very much!

from translate.

Hi @AmitMY,
I created PR at #pull/73
Please review to see if I did the job correctly, thank you very much

from translate.

Hi @AmitMY
I also reviewed possible flows in sign-language-processing/spoken-to-signed-translation such as: text_to_gloss_to_pose , text_to_gloss , pose_to_video but did not see any flows related to video_to_pose as you mentioned. I also reviewed sign-language-processing and saw a lot of work repo/work being created here. Does this project have any other project documentations besides the wiki and project board or am I missing something?
Right now I don't know where to start next for the betterment of Vietnamese. I think next I will improve the human liver while waiting for instructions. Thank you very much

from translate.

For sign-language-processing/spoken-to-signed-translation you need to extract poses - similar to the dummy lexicon - https://github.com/sign-language-processing/spoken-to-signed-translation/tree/main/assets/dummy_lexicon

the video_to_pose command exists in the https://github.com/sign-language-processing/pose package (pip install pose-format)

from translate.