Docker keeps cached manifests and indicies forever somewhere. about moby HOT 6 OPEN

Sounds related to the caching added in this PR;

• #41607

Curious; was there a specific problem you ran into, for which you can't have the cached ones?

from moby.

Curious; was there a specific problem you ran into, for which you can't have the cached ones?

Hello. I'm making a mitm proxy that prevents image downloading when the critical vulnerability was found.

A fairly good way to cancel the download was to replace the layer body with a denied error from the registry after issuing the manifest and index to the client (since only client knows the desired platform). I could specifically change the hash to the random one in the response to the head request, and upon a subsequent unsuccessful attempt to access my random hash, it would invalidate the hash as I want, but also show the warning since its a deprecated fallback. I would like to avoid it and not remake all the hashes in the image with others (its probably the right way but too hard at the moment cause then I would have to insert some garbage into the contents of the layers).

The warning looks like this:
https://docs.docker.com/engine/deprecated/#pulling-images-from-non-compliant-image-registries

from moby.

Thanks for the extra info.

Hm wondering (but not at my computer right now); wouldn't it still do a HEAD request, and would that request be interceptible for your proxy to return an error?

from moby.

Wouldn't it still do a HEAD request, and would that request be interceptible for your proxy to return an error?

All I want ideally is to always force any client to invalidate the cache when it contacts me (the proxy side). In the future, caching will be handled by the proxy itself, if necessary.

Currently, I have implemented behavior that does not replace invalidation, but adds some atomicity. It makes it so that until layers are received, manifests will always be requested, and when layers are received, docker rmi <image> can be done.

It is a bit strange, it looks like this:

--> HEAD /v2/dockerhub/maven/manifests/latest
<-- Docker-Content-Digest: sha256:7c03439155 <- random hash
<-- 200 OK: 
{}

--> GET /v2/dockerhub/maven/manifests/sha256:7c03439155 
<-- 403 Forbidden: 
  "errors": [
    {
      "code": "DENIED",
      "message": "errorcode404:{
        \"errors\":[
          {
            \"code\":\"NOT_FOUND\",
            \"message\":\"artifact dockerhub/library/maven@sha256:7c03439155 not found\"
          }
        ]
      }"
    }
  ]
}

I guess, after the digests from local storage and head request do not match and docker gets error response on the new digest request, it sometimes (maybe I'm getting confused), tries to make a normal get request with the old digest, which turned out to be quite convenient for me as a proxy. Then I work as usual: I just wait for the manifest, download all the remaining parts and decide whether to return an error based on the analysis results.

Index:

GET /v2/dockerhub/maven/manifests/latest
<-- 200 OK: 
{"manifests": [ {"digest": "sha256:c1a446", "type": "v2", "platform": "linux/amd64"} ]}

Manifest:

GET /v2/dockerhub/maven/manifests/sha256:c1a446
<-- 200 OK: 
{"config": {"digest": "sha256:d5beea"}, "layers": [ {"digest": "sha256:6a299a"} ]}

Config:

GET /v2/dockerhub/maven/blobs/sha256:d5beea
<-- 200 OK: 
{"architecture": "amd64", "os": "linux"}

Layers:

GET /v2/dockerhub/maven/blobs/sha256:6a299a
<-- 200 OK: 
8f cd f9 0b a2 39 6b 45 9c ... 50

from moby.