Problem of inference of Table Structure when tables very close to image corners about table-transformer HOT 4 CLOSED

I'm guessing if you made the change recently in the training it will need more epochs to adapt. However, you might have more success using or adapting another transform we include in the code but don't use for the paper and still haven't documented:

This one can be used to crop directly to the table bounding box by doing TightAnnotationCrop([0], 0, 0, 0, 0), or with some random padding around the table bounding box by doing TightAnnotationCrop([0], 10, 10, 10, 10).

Keep in mind that this goes beyond what we do in the paper so we aren't officially supporting/documenting it at this time. However I hope you find it helpful!

from table-transformer.

Hi, yes this is expected behavior given the training data and data augmentation used in the paper.

However, we designed the data with robustness to padding around the table in mind. With small changes to the training code it should be easy to achieve whatever robustness you would like to padding around the true table border.

(As an aside, note that the reason we include padding in the training images is that if we had cropped all of the table images tightly to the table border then the opposite problem would be encountered: there would be no robustness to table images that have padding around the table.)

To achieve more robustness to different amounts of padding, the simplest change would be to increase the amount of cropping done during training here:

You can also modify the transforms or make your own transform to achieve your desired result. The true border for every table is included in the labels so you can train a model with any amount of padding around the table you would like.

Hope that helps!

from table-transformer.

Hello,
Thanks for those clarifications.

I have made some tests modifying the RandomCrop values, and the problem seems to persist. I have only managed to complete a couple epochs however. Should I expect an improvement when training for more epochs or I should modify more lines of code than just these:

To achieve more robustness to different amounts of padding, the simplest change would be to increase the amount of cropping done during training here:

table-transformer/src/main.py

Line 75 in 3e1dd0c

RandomCrop(1, 10, 10, 10, 10),

I have modified both the maximum crop and also the minimum crop inside the RandomCrop transform.

Thanks again,

from table-transformer.

I believe I've addressed your question, so I will close this issue.

from table-transformer.