Crash trying to read/decode large Big TIFF File about libvips HOT 7 CLOSED

I made a PR to improve this (but not fix your problem). I now see:

$ cat Visium_HD_Human_Colon_Cancer_tissue_image.btf | vipsheader -a stdin
vipsheader: VipsImage: unable to load source

Not a great error message, but much better.

Thanks for reporting this!

from libvips.

Hi @snowzach, wow what a monster. I'll have a look.

from libvips.

It's working fine in git master libvips:

$ vipsheader Visium_HD_Human_Colon_Cancer_tissue_image.btf 
Visium_HD_Human_Colon_Cancer_tissue_image.btf: 75250x48740 uchar, 3 bands, srgb, tiffload

I'll try some older versions.

I can view it in the current vipsdisp binary:

See https://github.com/jcupitt/vipsdisp, you can install the current version with flatpak install flathub org.libvips.vipsdisp.

from libvips.

It seems to work okay when pointing it right to the file. I want to try to use thumbnail and header from an AWS Lambda which doesn't have enough disk space to store it. So I am trying to pipe it via stdin. It doesn't seem to like that. That's what's giving the error.

from libvips.

Ahhhh I see. I'll try again.

from libvips.

I think it's because you need to be able to seek() to read TIFF --- the format is made of a set of directories and data areas you can read out randomly. This means when you pipe a TIFF into vips, it's forced to buffer the entire file in memory.

We have an env var called VIPS_PIPE_READ_LIMIT which is supposed to limit memory buffering on pipes, but it doesn't seem to be working here, I'll have a look.

However this might not help you -- a lambda with 12gb of spare memory sounds expensive.

from libvips.

Yeah, that's what I was afraid of... There are ways to get it more disk but it's actually capped at 10g of memory. Appreciate you looking.

from libvips.