indexer C component needs to be rewritten to Go about soci-snapshotter HOT 7 OPEN

There is no existing Go library that exposes the zlib API at the level we want to. The task isn't to create our own Go zlib library, but build an abstraction layer on top of the zlib API, so it's more Go-ish.

from soci-snapshotter.

Copying this relevant comment here:

We are still implicitly leaking details about the underlying implementation. This should be addressed in future PRs before we attempt to define a real gzip-neutral interface (that is, before we add Zstd support).

1. NewGzipIndex and NewGzipIndexFromFile would both be taking Readers if the underlying implementation were in native Go.
2. HasBits is an implementation detail of Gzip, and probably this implementation of the gzip-ztoc technology. So we should not be exposing this concept to the caller.

from soci-snapshotter.

Cf. #60

from soci-snapshotter.

I've seen a couple references to this issue (e.g. #80 (comment)) that imply that this is scheduled work, but I'm not totally sold that this is something we want to do.

Specifically, my concern is whether there is an existing go library that exposes gzip/inflate APIs at the level we need. If doing this means writing and maintaining our own inflate library in go, then I'm not sure it's worth it.

from soci-snapshotter.

In that case, can we reword the issue? The current description sounds like the end result is a project with no c dependencies.

from soci-snapshotter.

In that case, can we reword the issue? The current description sounds like the end result is a project with no c dependencies.

Updated the wording.

from soci-snapshotter.

We are using the soci-snapshotter in an existing lambda which is used to copy images from our on premise container registry to ecr.

The reliance on the gzip_zinfo c code causes issues for us testing and debugging our lambda code on developer desktops (Mac and Windows). To test at the moment we have to commit the code and wait for the code to be compiled and tested on our Linux based build servers.

Rewriting the c code in golang would enable us to speed up development and testing of any changes to our lambdas.

from soci-snapshotter.