SROIE dataset issues. about vibertgrid-pytorch HOT 5 CLOSED

About dataset re-labeling:

• We used an in-house labeling tool to re-label the data. First we generated the char-level OCR results (actually line-level with coordinates of each char) using an in-house OCR engine, then we marked the key-type category of each char comparing to the official SROIE gt labels.

About data label input format:

• Labes are in csv format. Each row should contain the left, top, right, bottom coordinates and the text string, key-type category of a word/text-line, see the example below
  

About args in pipeline/sroie_data_preprocessing.py

• dir_test_root refers to the root folder of the official raw SROIE test data, which should contain three folders: img with the images, box with the SROIE official OCR results, and key with the SROIE official key-type string labels. dir_test_processed is the dir to the processed label files.

Here's my advice on arranging the dataset, if you want to use the sroie_data_preprocessing.py:

1. Download the official SROIE dataset here(https://rrc.cvc.uab.es/?ch=13&com=downloads)
2. 0325updated.task1train(626p) contains images and OCR results of the traing set, put the images in img folder and txt files in box folder. txt files in 0325updatd.task2train(626p) are key-type labels, put them in the key folder. The arg dir_train_root is the dir to the root of the three folders mentioned above. The arg dir_train_processed is the dir to the processed csv labels generated by the sroie_data_preprocessing.py, named label. Put the img, key and label folders in the same root named train.
3. Download the raw data of test set from the link provided at the bottom of the official page, (https://rrc.cvc.uab.es/?com=downloads&action=download&ch=13&f=aHR0cHM6Ly9ycmMuY3ZjLnVhYi5lcy9kb3dubG9hZHMvU1JPSUVfdGVzdF9ndF90YXNrXzMuemlw and https://rrc.cvc.uab.es/?com=downloads&action=download&ch=13&f=aHR0cHM6Ly9ycmMuY3ZjLnVhYi5lcy9kb3dubG9hZHMvU1JPSUVfdGVzdF9pbWFnZXNfdGFza18zLnppcA==). Follow the instructions in 2, but name the folder as test. Put it in the same root with train folder and named the root as SROIE.
4. Change the arg data_root in the config yaml file to the SROIE folder, then the dataloader can recognize the the data and load them automatically when calling train scripts.

from vibertgrid-pytorch.

Thanks for the earlier reply just a few more concerns I have,

• The txt file in box folder should be converted to csv format with format as you showed in the snippet ?

Thanks for the reply and help.

from vibertgrid-pytorch.

No, just keep it as the txt format. (See line 147 in sroie_data_preprocessing.py, I used a readline() rather than pandas.read_csv()). The csv format showed in the snippet is the required format of the processed labels.

from vibertgrid-pytorch.

Thanks for the reply, since you mentioned that we can use .txt file. I think there is some error

• line 109, sroie_data_preprocessing.py : You have mentioned directory to bbox csv file I think that should be directory to bbox txt file.
• After defining the directory we are calling data_parser function, in which we are calling data_preprocessing_pipeline function which is expecting bounding box in csv file and key in json format (line 316 and 318 respectfully)

That why when I am running it with .txt files in bbox folder I am getting an error that "FileNotFoundError: [Errno 2] No such file or directory"

from vibertgrid-pytorch.

from vibertgrid-pytorch.