[Feature]: Allow screenshot archival to be optionally disabled about browsertrix HOT 3 OPEN

Hi @HeliosLHC, we could look into making this optional in the Helm chart, however screenshots and extracted text are both necessary for our QA features, so we'd have to find a way to make it clear to users that changing these settings would have an adverse effect on Quality Assurance.

from browsertrix.

The screenshots pngs are 100K-300K (if even that) in size, so the total size is generally negligible in the overall size of a crawl. The time it takes to take them is also very small. It will most likely not make much difference in storage or resource consumption, and will affect usability of other features. @HeliosLHC is there a particular issue you're trying to solve? How big are the screenshots compared to rest of the crawl data?

from browsertrix.

Hi @tw4l and @ikreymer, my primary use case in is to reduce the output size of crawls, especially for static text only sites.

In one of my crawls, the output WACZ contained 60 MB of crawl data + 30 MB extracted text data and 900 MB of screenshots (thumbnail + view) which is a 10x size increase. This was a barebones text-heavy site with little to no images.

For other more image/media heavy sites, I would expect this ratio to be lower (< 5x) as the actual crawl data becomes a much larger proportion of the output. As such, in such scenarios, the additional overhead of the screenshots are not as significant.

I don't mind generation of thumbnails/views as they are useful for monitoring crawls, but an option to disable writing them to WACZ files would be useful.

from browsertrix.