For some reason, Github action does not have access to the .git directory:

Step 5/9 : RUN make conformance-binary && mv output/conformance.test /conformance.test
 ---> Running in 66d264dd2646
fatal: not a git repository (or any of the parent directories): .git
cd conformance && \
	CGO_ENABLED=0 go test -c -o /go/src/github.com/opencontainers/distribution-spec/output//conformance.test \
		--ldflags="-X github.com/opencontainers/distribution-spec/conformance.Version="
go: downloading github.com/opencontainers/distribution-spec v1.0.0-rc0.0.20200108182153-219f20cbcfa1

Perhaps we use build args instead of relying on git?

OCI Artifact Signing - Requirements

As containers become the common unit of deployment, users want to know the images in their private registries and their deployments are the same images that were initially published.

The OCI TOB has adopted OCI Artifacts, generalizing container images as one of many types of artifacts that may be stored in a registry. Other artifact types currently include:

• Singularity
• Helm Charts
• Car firmware updates, deployed from OCI Artifact registries

The Need for a Generalized Signing Solution

This document serves as the requirements and constraints of a generalized signing solution. It focused on the scenarios and needs, and very specifically avoids any reference to other projects or implementations. As our working group forms a consensus on the requirements, the group will then transition to a spec. Existing verification & signing solutions can be compared against the requirements and spec to identify best fits.

Key Stake Holders & Contributors

As we identify the requirements and constraints, a number of key contributors will be asked to represent their requirements and constraints.

Please add companies, projects, products that you believe should be included.

• Registry Cloud Operators
  • Azure Container Registry (acr) - Steve Lasker [email protected] (@SteveLasker)
  • AWS Container Registry (ecr)
  • Docker Hub
  • Google Container Registry (gcr)
  • GitHub Package Registry (gpr)
  • Quay
  • IBM Cloud Container Registry (icr)
• Registry Vendors, Projects & Products
  • Docker Trusted Registry
  • Harbor
  • JFrog Artifactory
• Artifact Types
  • Container Images - OCI Org
  • Helm Charts
  • Singularity

Requirements

As we begin a new design, we must ask what defines success.

Generic Artifact Support

While Docker Content Trust focused on container images, registries have evolved to now support any type of artifact. An image is just one of many artifacts. As a result, OCI has adopted OCI Artifacts

As we consider a new signing solution, we look to support any artifact to be signed, including images, Helm charts, OPA bundles, Singularity High Performance Computing images, to new, yet unknown artifacts.

Artifact Movement

1. As part of normal workflows, users move artifacts from one repository to another**. The movement of the artifact from dev-->staging-->production must not invalidate the signature.
2. As part of normal workflows, users move artifacts from one registry to another. The movement of the artifact must not invalidate the signature.
3. Users acquire artifacts from public registries, representing products. As they copy the artifact to their private registry, the signature must be verifiable as that of the source.

Signature Movement

1. Users place network rules on their deployments. These network rules restrict connections to trusted endpoints within the users control. The signature must be verifiable without having to reach untrusted endpoints. A user must be able to move signature information to services the user runs, and subsequently trusts.

Registry Operator Differentiated Value

1. Registry operators must be able to leverage other systems within their product and service lines. Specifically, the spec must allow for:

   1. External key management - leveraging other services they may provide
   2. Blob storage - leveraging unique cloud provider storage solutions
   3. Integrated authentication and authorization - while the spec must account for auth flows, each cloud provider must be able to provide flows that incorporate 2FA and service accounts

Open Topics

The following accounts for the list of open topics that should be considered.

Parking Log / Backlog

This section accounts for any topics the working group deems deferred for current conversation.

In the specification, layer upload de-duplication is explicitly listed as part of the spec.

Today, this is scoped to a given image, as opposed to the whole registry itself. The of de-duplication is to call the HEAD request on blobs, and see if the blob exists. Iff it doesn't, then upload it.

Unfortunately, in the case where you have two images that are built on top of a common image, this doesn't work. This is often a common pattern, a la:

                      +-----------+
                      |           |
                      |  image-1  |
+---------------+     |           |
|               |     +-----^-----+
| Ubuntu:latest +-----------|
|               |     +-----v-----+
+---------------+     |           |
                      |  image-2  |
                      |           |
                      +-----------+

See:

> HEAD /v2/image-1/blobs/sha256:4f1bb8b6572003ca3526ab23c80c385c5eadd1748e3996e444b66d47ce0ee7af HTTP/1.1
> Host: myregistry.com:7002
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 404 Not Found


> HEAD /v2/image-2/blobs/sha256:4f1bb8b6572003ca3526ab23c80c385c5eadd1748e3996e444b66d47ce0ee7af HTTP/1.1
> Host: myregistry.com:7002
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 307 Temporary Redirect
< Date: Mon, 25 Jun 2018 01:06:59 GMT
< Server: Apache
< Docker-Distribution-Api-Version: registry/2.0
< Content-Type: application/octet-stream
< Docker-Distribution-Api-Version: registry/2.0
< Location: http://<INSERT REDIRECT HERE>

On upload, the underlying registry may deduplicate these layers.

Although there exists a mechanism for image-1, and image-2 to have "cross-mounted" layers, this requires that the owners of image-1, and image-2 know of one another.

My proposal is to extend the Starting An Upload API. I suggest extending the API from:

POST /v2/<name>/blobs/uploads/

To accept the optional query parameter digest. If the optional query parameter digest exists, and we can find the layer from another image, we will return with status code 201, and link the layer in. Given that this field would be optional, this would a backwards compatible change, and could be implemented optionally.

Opinion?

The spec references "tags" quite a bit, but never defines them. I'm curious what everyone here (mostly registry operators) think a tag actually is.

My understanding of a tag is that it is effectively a git ref, i.e. it is a pointer to a manifest (or image index) digest that is namespaced to a repository.

I've been part of a conversation where users expected a repository to support having a tag for each platform so that they could push each platform's image individually and have the registry assemble a manifest list for them (cc @tianon please correct me if I'm wrong here).

Similarly, on this thread, some users expect the registry to support having a tag for each media type.

I wonder if it's worth clarifying somewhere (maybe not the spec, but just in this issue) so we can point to the answer of why we can't do this. It's clear that we can support pulling heterogeneous media types and platforms (that's the point of an image index), but we would require clients to assemble them rather than the registry (due to the trust model -- I expect the digest of what I pull to be what I pushed; you can't easily sign something the registry synthesizes).

cc @stevvooe @dmcgowan would love to hear your thoughts here.

PS: related, there is a TAG_INVALID error: During a manifest upload, if the tag in the manifest does not match the uri tag, this error will be returned.. cc @atlaskerr related to #47 this is a schema-1-ism.

I'm sure there's going to be a ton of really good discussion at KubeCon about the distribution-spec so in preparation for that, it would be nice have labels for discussions on different versions of the spec.

Perhaps we can start off small with just V1 and V2 labels.

After everyone is back from DockerCon, we should have a discussion on planning the first release of the distribution-spec to start gathering feedback and working on a "v1.0 FINAL"

cc: @opencontainers/distribution-spec-maintainers

The spec constantly refers to UUID that reference a given upload e.g. the Docker-Upload-UUID header.

However, there's no real reason this needs to be a UUID, it's just a reference number or ID (as noted in the spec itself "While the uuid parameter may be an actual UUID, this proposal imposes no constraints on the format and clients should never impose any"). Could it be changed to just id or refid or some such?

See #98 (review)

Add the content type header to blob upload endpoints with language about what the value should be. In the case of blobs, the content type should be ignored and this is best represented as using the application/octet-stream Content-Type. Any other types will be ignored by a registry and not be reflected when that blob is pulled.

I don't think a 415 return status is appropriate when application/octet-stream is not used. We should explicitly call that out since ignored != unsupported. Ping @rchincha

A few things leftover from schema

TAG_INVALID

The TAG_INVALID description says:

During a manifest upload, if the tag in the manifest does not match the uri tag, this error will be returned.

Manifests no longer contains tags, so we should update this. A couple options:

1. Remove the error entirely.
2. Use this error if the client tries to use a tag that doesn't match the expected format. Ref #52

Signatures

There are 8 uses of the word signature in the spec. AFAIK, manifest signatures aren't really a thing after docker v2 schema 1.

For example, the MANIFEST_UNVERIFIED description says:

During manifest upload, if the manifest fails signature verification, this error will be returned.

I assume we'd want to just remove that.

The rest of the instances of "signatures" refer to JWS payloads and such that don't make sense anymore, so we want to drop those descriptions and make sure the surrounding paragraphs still make sense.

I see Travis running CI tests on open pull requests, but don't see PullApprove doing anything. @caniszczyk, is there a bit you need to flip somewhere?

OCI Artifact Search Requirements

As registries support multiple artifact types, a search/catalog API that supports filtering on the artifact type will be needed.

The docker v1 registry spec supported Docker Search. While some vendors like Quay.io implemented the v1 search API, the majority of vendors require the v2 registry api which dropped search.

We believe revisiting the search api will support client CLIs that span registries, such as helm search, duffle search (CNAB), docker search, and other evolving artifact types.

By supporting a common search API across all registries, users could consistently use these new artifact CLIs across all registries.

This issue focuses on capturing the requirements for new Search and Eventing APIs. As the requirements are agreed upon, we'll move to a spec that captures the requirements.

KubeCon 2019 EU Notes: OCI Catalog Listing APIs

Use Cases

Search is a generic capability used across several different use cases.

• Artifact Tool Specific Searches
• Registry Specific Searches
• Registry Tools Searches

Tool Specific Searches

Helm, Singularity, Docker, OPA, CNAB and other tools will need to query their specific artifact types across various registries.
Example: The helm cli would need to query a registry for charts that match a specific name. The result should return helm only artifacts.

helm search demo42.azurecr.io hello-world

Results
--------------------------------------
samples/hello-world
marketing/products/hello-world-sample
dev/prototypes/sample-hello-world

Version specific searches:

helm search --versions demo42.azurecr.io samples/hello-world

Results
--------------------------------------
samples/hello-world   1.0
samples/hello-world   1.1
samples/hello-world   1.2

Registry Specific Search

Users want to query registries for the artifacts that match a specific name or list artifacts within a given path. In this case, the results contain multiple artifact types.

Today, registries have created unique client APIs and server APIs. Until we have a generic registry client, it's expected registries will have vendor specific APIs. However, having common registry server side APIs expands the possibility for common tooling across registries.

A registry search API would include

• repo listings
• tag/version listings
• limit by artifact type
• query by date range, such as what's changed/added since a given timestamp
• as results may be paged, sorting the results by name and/or version with ascending and descending options

Existing examples

ACR list repo example:

Without a common search/catalog API, cloud vendors have had to implement vendor specific experiences:

az acr repository list -n demo42

Name                         
-----------------------------
samples/demo42/queueworker   
samples/demo42/quotes-api    
samples/demo42/web           
samples/demo42/deploy/chart  
samples/demo42/deploy/cnab   
samples/demo42/deploy/arm    

ACR list tags example, w/ future type added:

az acr repository show-tags -n demo42 --repository samples/demo42/deploy/chart

Result  Type
-------------
1.0     helm-chart
1.1     helm-chart
1.1.1   helm-chart
2.0     helm-chart
3.0     helm-chart

A repo could contain multiple artifact types

az acr repository show-tags -n demo42 --repository samples/demo42/deploy

Result       Type
------------ ----------------
helm-1.0     helm-chart
helm-1.1     helm-chart
helm-1.1.1   helm-chart
cnab-1.0     cnab
arm-1.0      arm

Rather than each registry vendor having to offer unique APIs, the goal would be to offer a common API.

Registry Tool Search - Scanners

Vendors and the community have attempted to build tools atop registries.

• Image Layers
• Dive – A tool for exploring each layer in a Docker image | Hacker News
• Analyze And Explore The Contents Of Docker Images
• 10+ top open-source tools for Docker security | TechBeacon

Without a common search/catalog API, these tools must work with individual images.

One of the most common registry tools include image scanning tools like Aqua, Twistlock, Neuvector and Clair.
While the scanning tools protect runtime nodes, they all pre-scan registries to understand image vulnerabilities before they're run.

Scanners evaluate images in registries with a combination of a search/catalog API and events.

These vulnerability scanners need the following:

• list all repos and tags for the inital scan evaluation
• get paged results as they may contain thousands of images
• periodically list all new and update images and tags, to keep a registry up to date
• register for events to scan images as they arrive. Possibly using The Container Quarantine Pattern
• filter, or at least understand the different artifact types
• as new CVEs are found, re-scan the registry

Today, scanners assume all artifacts in a registry are a container image. As a registry stores new artifact types, scanners will either need to know how to scan these new artifacts, or at least filter the results to artifacts they support.

Artifact Types

A registry must know the types it hosts for it to provide meaningful search results.
Artifact types will be internally identified by an expanded set of OCI Media Types.

However, displaying application/vnd.cncf.helm.chart.v1+json does not make for a good user experience. To provide clean user experiences, a list of artifact types, a short description, and info on the artifact tooling will be maintained. Media Type Short Names

* most registry providers automatically convert oci.image manifests to the format requested by the client.

Registry Search Requirements

Listing repos

Listing artifacts

Listing versions

Filtering by artifacts

Filtering by date ranges

Search queries may specify date ranges, enabling the return of artifacts that have been created or changed since a given date:time

• date:time filters MUST be supported on manifests and tags.
• Registry operators MAY add additional value by parsing manifest.config objects. This allows a registry to option to add value, while not burdening all registries to parse all config objects of all artifact types.

Paging

Results may be paged, to provide a full list of artifacts.
A default page size of 100, with the ability to change the paging size.

Sorting

As results may be paged, being able to sort provides the ability to get the top n results, based on a given sort order. Sorting includes ascending and descending.

Role Based Access Control

Search results shall be limited to the artifacts the user has read access control. The user may be a person or service account. The spec shall not define specific rights or roles for how authorization should be implemented or managed, rather simply state the registry must be cognizant of security and support the security of it's product and/or platform. If a user has read access to repo1 and repo3, but not repo2, the repository listing should only return repo1, repo3. The spec will not define a role where read differentiates between management and data operations.

In https://github.com/opencontainers/distribution-spec/blob/master/spec.md#content-digests, the following statement fill a bit ambiguous to me:

While the algorithm does allow one to implement a wide variety of algorithms, compliant implementations SHOULD use sha256

The ambiguity is on the SHOULD word: if my registry uses sha512 for digesting contents (and thus for OCI descriptors), is it still compliant to the spec ?

More importantly, as a consumer of the registry API, can I rely on the fact that all OCI-compliant registries use sha256 for their descriptors ?
If it is not the case, it is a very serious limitation for interoperability: it means that I cant update a manifest with its dependencies in OCI Image layout from registry-a and be sure I can upload it as-is in registry-b. If the 2 registries use different digest algorithms for their descriptors, I need to modify all the descriptors in my manifests when doing a copy.
It also means that any digital signature I have for trusting a given manifest from registry-a can't be used to validate the same manifest once copied on registry-b (because my descriptors would have been rewritten).

certain language like "must" should be converted to "MUST" when describing an absolute requirement of the specification.

A repository name is broken up into path components. A component of a repository name MUST begin with one or more lowercase alpha-numeric characters. Subsequent lowercase alpha-numeric characters are OPTIONAL and MAY be separated by periods, dashes or underscores. More strictly, it MUST match the regular expression [a-z0-9]+(?:[._-][a-z0-9]+)*.

That the repository names are lowercase snake_case is a requirement we want to keep? And when camelCase, convert to camel_case for repository naming?

I think this test is a little over-zealous.

From the spec:

Delete the manifest identified by name and reference. Note that a manifest can only be deleted by digest.

This seems to me more a description of the registry's original behavior (really, a limitation IMO) and how clients should expect to interact with registries. It doesn't specifically require registries to fail if you delete by tag.

GCR will never be able to pass these conformance tests, because it has some semantics outside of this specification and treats tag references differently from digest references. (I think quay has the opposite behavior from GCR, which is fun 😄 )

To explain a bit further: GCR views tags as a separate entity (a la git tags) as just pointers into the dag. You can delete a tag separately from deleting a manifest by just doing DELELTE /v2/.../manifests/<tag>. In fact, GCR requires you to delete tags before deleting an image by digest in order to prevent you from accidentally deleting a tagged image.

Given that this behavior is a super-set of the registry spec, I think it's a reasonable thing for GCR to do, but the conformance tests treat this behavior as a failure.

WDYT? Am I just biased here?

https://groups.google.com/a/opencontainers.org/forum/#!topic/dev/jL5yhEIv3Ac

Rather than incrementing the distribution spec version as certain optional features are added, this API and mapped values of "feature flags" will be used by registries to indicate whether new features are implemented. (Like PEP or similar extensions)

The list of error codes contains some quite broad categories like DENIED, but also a lot of very specific errors. Are you all generally interested in extending this list as new needs arise in alternative implementations of the API? Or should implementations just choose the most appropriate pre-existing error code?

If there is interest, I propose the addition of an error code QUOTA_EXCEEDED which can be returned from any PUT, PATCH or POST operation. Right now, I'm using DENIED for this (example), but only because DENIED is the most generic of the existing error codes.

If there is interest in adding the new error code, I'll prepare a PR after the holidays.

Container registries primarily use port 80 and 443 for clean paths when pulling/pushing and signing content. It would be nice to standardize a default port to serve images from. I'm not sure if this is something we can tackle in v1.0 though.

We need to go through and review all the returned status codes for the various endpoints. This includes what is defined in the overview, what is defined in the detail, and whether it tested and which status code the test uses.

The result of this should be removing the disclaimer about discrepancies between header and detail.

Should show a summary of whether or not tests passed, how many passed/failed, and the configuration used to run the tests (which registry URL etc)

Examples:

All endpoints will be prefixed by the API version and the repository name:
One or more errors will be returned in the following format:
The code field will be a unique identifier, all caps with underscores by convention.
A 404 Not Found response will be returned if the image is unknown to the registry.

Examples like the last one have impeded progress on the conformance tests, since most of the error codes are defined in terms of "will be" language, and we're unsure how much is simply conventional vs required by the spec.

I would like to point out an element related to the efficiency of the registry protocol, which should be addressed.

In many deployment, the registry does not independently store any information, especially binary data, because it is a light, stateless service, which sends the heavy load to external storage services (object storage, swift, s3). However, the lightness is disturbed by a single element of the protocol - "Initiate Monolithic Blob Upload" does not create the possibility of redirection to services to handle heavy load out of main registry. This means that the main registry must act as a proxy server for heavy binary data, which are not efficient from network perspective.

In my opinion a monolithic blob upload request should be preceded by a request, which will indicate at which address the blob should be sent eg. via "Location" header similiar to "Initiate Resumable Blob Upload".

"Initiate Resumable Blob Upload" act as "pre-flight" request which allows via "Location" distribute load to multiple hosts without costly load-balancer in front of registry.

Similiar approach to reduce network cost & "pre-flight" request exists in Backblaze B2 storage API. Details: https://www.backblaze.com/blog/design-thinking-b2-apis-the-hidden-costs-of-s3-compatibility/

The current spec states that clients can create image manifests by posting JSON after all blobs have been uploaded to the registry server. Since manifests are uploaded to the registry as blobs, It seems like it would be cleaner to have clients simply specify which blobs should be recognized as manifests instead of re-uploading content that the registry already has.

Add Travis job to do things such as:

• Run golangci-lint
• Make sure that the test binary builds
• Run against local registry w/ known compliance (zot?)

See #82 (comment)

for more rich indexing and searching of container images in a registry.
There is the /v2/_catalog though it still seems not clear enough for implementers.

On failed request (4xx), check for JSON response body. In the body, the "code" key should be all-caps with underscores, taken from this list in the spec:

https://github.com/opencontainers/distribution-spec/blob/master/spec.md#errors-2

After import, I'll take an initial pass over the spec to make sure we use the "one sentence per line" guideline. There are definitely other formatting requirements but this one will be time consuming.

When we dropped catalog from the spec, we lost this line:

The implementation MAY impose a maximum limit and return a partial set with pagination links.

Which is under the non-paginated Catalog Fetch request, i.e. the request did not include an n parameter.

Some questions:

1. Should this have applied to tag listing as well as catalog?
2. Should this also apply to paginated requests?
3. If the answer is no for 1 or 2, what should registries do if they don't want to return the complete set?

Currently, ECR will return an error if you send a request where n > 1000. This seems somewhat reasonable, but I think it's easier for clients to just handle the partial set + Link header response transparently, as was expected for non-paginated catalog requests.

For a lot of registries, if you don't set n explicitly, they will default to some reasonable but low number. This can dramatically increase latency if you want to actually fetch the entire set of tags for a large repository.

There's currently no way to specify "give me everything you have, as fast as you are willing to give to me". For clients that means figuring out, empirically, what the maximum value for n should be -- maybe even on a per-registry basis, which seems like something the spec is trying to avoid.

Seems like we could do two things:

1. Clarify that registries MUST return the entire set for non-paginated requests.
2. Clarify that registries MAY return a partial set with pagination links for both non-paginated and paginated requests.

Any thoughts?

Since the only meaningful content in this repo is currently the license file, perhaps we should force-push project-template's master here? I haven't had much luck getting those docs landed in project repos after they're going (e.g. here), but it might be easier to just use project-template as a template ;).

[email protected] is no longer an active email address. @stevvooe needs to update their email address: https://github.com/opencontainers/distribution-spec/blob/master/MAINTAINERS

So it's my understanding that when doing resumable uploads, the server presents you with a unique ID and a location header pointing to the resource being uploaded. Since the digest is unique, is the upload ID necessary?

Under https://github.com/opencontainers/distribution-spec/blob/master/spec.md#pagination

The text reads "If the value of n is 2, v1 and v1 will be returned", presumably this should be "If the value of n is 2, v1 and v2 will be returned".

conformance.test should be able to be built with arguments like git commit etc, so that in the test report it is clear which version of the code built it

The spec defines the return type from a PUT Blob operation to be a 204 https://github.com/opencontainers/distribution-spec/blob/master/spec.md#put-blob-upload, but this disagrees with the conformance test https://github.com/opencontainers/distribution-spec/blob/master/conformance/03_blob_upload_monolithic_test.go#L29-L37.

As the conformance test agrees with existing implementations, and the 204 code seems inconsistent with other areas of the spec (e.g. PUT manifest returns a 201), I suggest the spec should be changed to show a 201 return.

There is an assumption to today that a server implementation of the distribution specification will either not care about the name used by the client or that all requests will have a known common namespace. An example of this is the Docker Hub assuming that all requests are prefixed with docker.io even though the registry hostname is registry-1.docker.io. However this has always caused difficulty when a client then wants to mirror content, localhost:5000/library/ubuntu could proxy to registry-1.docker.io/library/ubuntu, however localhost:5000 could never proxy to anything else. Complicated registry configurations have been proposed to remedy this as well as a backwards incompatible approach of requesting as localhost:5000/docker.io/library/ubuntu. However a goal of this specification should be simplicity and backwards compatibility. I believe that a solution does belong in the specification to unlock the mirroring use cases without complicated configuration or DNS setup.

My proposal is to add a way to pass up the name resolved by the client to the registry, (e.g. docker.io/library/ubuntu). So if a request is going to localhost:5000/library/ubuntu, it could mirror both docker.io/library/ubuntu and quay.io/library/ubuntu and switch based on request parameters. There are 2 possible ways to achieve this, one is by creating adding an HTTP request header (e.g. OCI-REF-NAME: docker.io/library/ubuntu) or by adding a query parameter ?oci-ref-name=docker.io/library/ubuntu). The first is clean but the second may be more useful for static mirroring. I am not suggesting one over the other yet, just stating the problem and solutions to discuss.

Add a GitHub action that others can use to easily run the tests and share output. Should exist in this repo, so that the source code is always up to date w/ the version of the action being requested (i.e. master, v1.0.0, etc)

As per spec, "Location" header is typically of the form:
Location: /v2//blobs/uploads/<session_id>

Also,

Though the URI format (/v2/<name>/blobs/uploads/<session_id>) for the Location header is specified, clients SHOULD treat it as an opaque url and SHOULD never try to assemble it.

However, docker distribution appears to return a fully qualified "Location" header value of the form:
https://<server_ip>:<server_port>/v2//blobs/uploads/<session_id>

This behavior needs further clarification. What should be a compliant client's expectation from the server? In one case, it needs to prepend the Location header with the (scheme,address,port) and in the other, not.

Download action repository 'opencontainers/distribution-spec@master'
##[error]'Dockerfile.conformance' should be either '[path]/Dockerfile' or 'docker://image[:tag]'.

Will take a look at actions syntax and see what can be done

The distribution spec currently has greater than 4,500 lines. Attempting to find or edit a section becomes a tedious task.

Issue is to break the spec up into logical units, similar to the image-spec.

On current master, under the heading "Initiate Monolithic Blob Upload", the spec says (here):

This upload will not be resumable unless a recoverable error is returned.

This is the only occurrence of the word "recoverable" in the spec, and it is not clear which types of errors should be considered recoverable.

In my own implementation of blob upload, I'm treating all errors during monolithic blob upload as non-recoverable: Either the blob gets uploaded and 201 is returned, or it's not uploaded and 4xx/5xx is returned (i.e. the client would have to resend the request entirely). There's no inbetween. This particular part of the spec should clarify which errors are considered recoverable, and what recoverability even means for a single-request workflow.

For context: On the PATCH endpoint that's used for streamed and chunked upload, the wording is clearer since for 400 and 404 responses, it says that the upload "MUST be restarted". So it appears that 400 and 404 would be non-recoverable, whereas 401, 403 and 429 are recoverable.

Currently there is language about the inclusion of V2 Client library implementation. This is somewhat confusing as a client library is not part of the scope, however such a library may be useful for conformance testing. We should update this language to mention conformance testing as a primary goal and within scope, and any library used for testing as a non-goal but potentially something we can support out of that work. Additionally there are many client libraries implementations in existence today, we may want to use one for the testing or create a new one, either way no opinion on a preferred library should be specified.

I understand that backwards compatibility is important but since the distribution spec's logic is based on the OCI's well defined and versioned media types, I think it should be removed.

Docker's current implementation even has a Docker-Distribution-Api-Version: registry/2.0 header which serves the same purpose and provides more meaningful information in a cleaner way.

Including the path prefix and the header into the spec is kind of redundant. I'd like to see us pick one or the other before the first RC.

Thoughts?

This is a task for auditing the spec to make sure that nothing in the distribution specification prevents unspecified media types from being used for descriptors.

Related: opencontainers/image-spec#759

(from the weekly discussion)

Regardless of whether a registry implements whitelist or blacklist.
With out divulging information, but giving the client an informed error code when the mediatype being pushed is rejected.
Like HTTP 415?

API versioning is common case in api design. i suggest give a practical way to design the distribution api group.

REF: https://apigee.com/about/blog/developer/common-misconceptions-about-api-versioning

The examples for pulling and pushing an image use docker v2 schema 1 images, e.g.

https://github.com/opencontainers/distribution-spec/blob/fb183ce74113c89a64b55ef2d877277268f9a8af/spec.md#pulling-an-image

https://github.com/opencontainers/distribution-spec/blob/fb183ce74113c89a64b55ef2d877277268f9a8af/spec.md#pushing-an-image-manifest

https://github.com/opencontainers/distribution-spec/blob/fb183ce74113c89a64b55ef2d877277268f9a8af/spec.md#on-success-ok-3

https://github.com/opencontainers/distribution-spec/blob/fb183ce74113c89a64b55ef2d877277268f9a8af/spec.md#put-manifest

Similarly we have a lot of references to signatures that aren't part of the OCI image format.

The current distribution-spec contains vendor-specific strings like Docker in HTTP headers. Though it's not clear to me what we should do about it, because simple string substitutions would be incompatible changes. So I'm simply listing them below, without creating a PR.

• Docker-Content-Digest

  • https://github.com/opencontainers/distribution-spec/blob/59d293a11d862b2aad0c984dc6516f0401830045/changes.md
  • https://github.com/opencontainers/distribution-spec/blob/59d293a11d862b2aad0c984dc6516f0401830045/spec.md#digest-header
  • https://github.com/opencontainers/distribution-spec/blob/59d293a11d862b2aad0c984dc6516f0401830045/spec.md#existing-manifests
  • https://github.com/opencontainers/distribution-spec/blob/59d293a11d862b2aad0c984dc6516f0401830045/spec.md#existing-layers
  • https://github.com/opencontainers/distribution-spec/blob/59d293a11d862b2aad0c984dc6516f0401830045/spec.md#completed-upload
  • https://github.com/opencontainers/distribution-spec/blob/59d293a11d862b2aad0c984dc6516f0401830045/spec.md#cross-repository-blob-mount
  • https://github.com/opencontainers/distribution-spec/blob/59d293a11d862b2aad0c984dc6516f0401830045/spec.md#on-success-ok-3
  • https://github.com/opencontainers/distribution-spec/blob/59d293a11d862b2aad0c984dc6516f0401830045/spec.md#on-success-created
  • https://github.com/opencontainers/distribution-spec/blob/59d293a11d862b2aad0c984dc6516f0401830045/spec.md#on-success-ok-4
  • https://github.com/opencontainers/distribution-spec/blob/59d293a11d862b2aad0c984dc6516f0401830045/spec.md#on-success-temporary-redirect
  • https://github.com/opencontainers/distribution-spec/blob/59d293a11d862b2aad0c984dc6516f0401830045/spec.md#on-success-accepted-1
  • https://github.com/opencontainers/distribution-spec/blob/59d293a11d862b2aad0c984dc6516f0401830045/spec.md#on-success-upload-complete

• Docker-Distribution-API-Version

  • https://github.com/opencontainers/distribution-spec/blob/59d293a11d862b2aad0c984dc6516f0401830045/spec.md#api-version-check

• Docker-Upload-UUID

  • https://github.com/opencontainers/distribution-spec/blob/59d293a11d862b2aad0c984dc6516f0401830045/spec.md#uploading-the-layer
  • https://github.com/opencontainers/distribution-spec/blob/59d293a11d862b2aad0c984dc6516f0401830045/spec.md#upload-progress
  • https://github.com/opencontainers/distribution-spec/blob/59d293a11d862b2aad0c984dc6516f0401830045/spec.md#chunked-upload
  • https://github.com/opencontainers/distribution-spec/blob/59d293a11d862b2aad0c984dc6516f0401830045/spec.md#on-success-created-1
  • https://github.com/opencontainers/distribution-spec/blob/59d293a11d862b2aad0c984dc6516f0401830045/spec.md#on-success-accepted-2
  • https://github.com/opencontainers/distribution-spec/blob/59d293a11d862b2aad0c984dc6516f0401830045/spec.md#on-success-created-2
  • https://github.com/opencontainers/distribution-spec/blob/59d293a11d862b2aad0c984dc6516f0401830045/spec.md#on-success-upload-progress
  • https://github.com/opencontainers/distribution-spec/blob/59d293a11d862b2aad0c984dc6516f0401830045/spec.md#on-success-data-accepted
  • https://github.com/opencontainers/distribution-spec/blob/59d293a11d862b2aad0c984dc6516f0401830045/spec.md#on-success-chunk-accepted