Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document Conditional Requests #251

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 75 additions & 0 deletions preconditions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
## Conditional Requests

The [Distribution Specification](spec.md) describes mutable tag identifiers, which can be used when pushing, pulling, and deleting manifests.
Given that HTTP is a stateless protocol for distributed systems, the mutability of tag references leads to many possible race conditions.
Fortunately, HTTP has a mechanism for dealing with race conditions: conditional requests.

[RFC 7232](https://tools.ietf.org/html/rfc7232) defines the semantics of Conditional Requests in great detail, so this document will simply highlight the relevant portions of the RFC as applied to the Distribution Specification.

### Safely Mutating Manifests
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this limited to only mutating a manifest for a given tag?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't really think of anywhere else this is needed, but I might be overlooking something. There are races when creating, moving, and deleting tags, but everything else should be safe in the face of concurrency... right?

I think it's fine for this to apply to e.g. a manifest PUT, by digest, but it seems not very useful. You could say "don't PUT this manifest if it already exists", but PUT should be idempotent regardless, and you either want the manifest to exist or not.

Copy link
Contributor Author

@jonjohnsonjr jonjohnsonjr Mar 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we could also use this when uploading blobs?

Instead of always doing a HEAD request to check for blob existence in a registry, you could just start with a POST that includes the If-None-Match header (if you know the blob digest). The registry returning a 412 would indicate that the blob already exists, and we could do both the existence check and the upload initiation in a single request.

Unfortunately, this would require us to have deterministic Etags, as @awakecoding is suggesting, which I like, but I haven't thought completely through.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that having deterministic etags would be great to have, but that is already something mostly managed by a registry themself, since any client would not particularly expect portability of a given etag across registries, would they?


The `If-Match` header from [section 3.1](https://tools.ietf.org/html/rfc7232#section-3.1):

> makes the request method conditional on
> the recipient origin server either having at least one current
> representation of the target resource, when the field-value is "\*",
> or having a current representation of the target resource that has an
> entity-tag matching a member of the list of entity-tags provided in
> the field-value.

Specifically interesting for mutating manifests:

> If-Match is most often used with state-changing methods (e.g., POST,
> PUT, DELETE) to prevent accidental overwrites when multiple user
> agents might be acting in parallel on the same resource (i.e., to
> prevent the "lost update" problem). It can also be used with safe
> methods to abort a request if the selected representation does not
> match one already stored (or partially stored) from a prior request.

A client wishing to safely mutate a manifest SHOULD include in the manifest PUT request the following header:

```
If-Match: "<ETag>"
```

Where `<ETag>` is the entity-tag that matches the representation of the manifest as expected by the client, i.e. the header returned by the registry, as described in [section 2.3](https://tools.ietf.org/html/rfc7232#section-2.3):

> The "ETag" header field in a response provides the current entity-tag
> for the selected representation, as determined at the conclusion of
> handling the request. An entity-tag is an opaque validator for
> differentiating between multiple representations of the same
> resource, regardless of whether those multiple representations are
> due to resource state changes over time, content negotiation
> resulting in multiple representations being valid at the same time,
> or both. An entity-tag consists of an opaque quoted string, possibly
> prefixed by a weakness indicator.

If the state of the manifest in the registry does not match the supplied ETag, the registry MUST return a `412 Precondition Failed` response.

### Avoiding Overwriting Tags

The `If-None-Match` header from [section 3.2](https://tools.ietf.org/html/rfc7232#section-3.2):

> makes the request method conditional
> on a recipient cache or origin server either not having any current
> representation of the target resource, when the field-value is "\*",
> or having a selected representation with an entity-tag that does not
> match any of those listed in the field-value

Specifically interesting for avoiding tag overwriting:

> If-None-Match can also be used with a value of "\*" to prevent an
> unsafe request method (e.g., PUT) from inadvertently modifying an
> existing representation of the target resource when the client
> believes that the resource does not have a current representation
> (Section 4.2.1 of [RFC7231]). This is a variation on the "lost
> update" problem that might arise if more than one client attempts to
> create an initial representation for the target resource.

A client wishing to avoid overwriting any existing tags SHOULD include in the manifest PUT request the following header:

```
If-None-Match: *
```

If there already exists a manifest in the registry with a matching tag identifier, the registry MUST return a `412 Precondition Failed` response.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a nitpick, but maybe we should include this in the spec and not have a bunch of files. I feel like the other files we've had have rotted over time because they were not being reviewed alongside everything else.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah yeah. That's a fair point.

5 changes: 5 additions & 0 deletions spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,11 @@ When process B attempts to upload the layer, the registry indicates that its not
If process A and B upload the same layer at the same time, both operations will proceed and the first to complete will be stored in the registry.
Even in the case where both uploads are accepted, the registry may securely only store one copy of the layer since the computed digests match.

### Conditional Requests

A container engine would like to safely push a new tag or modify an existing tag without racing against another container engine acting upon the registry at the same time.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*registry/repo:tag

Registries and clients MAY rely on [conditional request](./preconditions.md) semantics to avoid race conditions in the face of concurrent mutations.

## Conformance

For more information on testing for conformance, please see the [conformance README](./conformance/README.md)
Expand Down