Baggage value difinition

A value contains a URL encoded UTF-8 string. Leading and trailing whitespaces (OWS) are allowed but MUST be trimmed when converting the header into a data structure.

Note, value MAY contain any number of the equal sign (=) characters. Parsers MUST NOT assume that the equal sign is only used to separate key and value.

If value is utf-8 URL encoded, isn't it redundant to say that parsers must not assume there are more than one equal sign?

We should clarify (first among ourselves, then in the spec) what use cases are appropriate for correlation-context vs tracestate (relates to #10).

There are probably some more distinctions we should make :)

OpenTelemetry and Correlation Context

OpenTelemetry is proposing to propagate correlations according to the editor's draft of the correlation context spec, however, it is still in its early stages. As we discussed at today's meeting, we need to come up with a plan that sets this spec up for success and mitigates any potential challenges from early adoption. I'm creating this issue for us to collect ideas and discuss our options.

Two things we can consider are:

• Using a different header name for OpenTelemetry (temporarily)
• Versioning the header

Different header name

We could use a header such as otcorrelationcontext which would give us the freedom to make breaking changes to the spec without considerations for backwards compatibility. The upgrade path for OpenTelemtry would have to be intentional and would require introducing a suitable propagator for correlation context once the spec has been finalized. There is always the risk OpenTelemetry and the W3C spec could diverge and we would be left with multiple formats indefinitely. Even if the projects are aligned, we might be left with both formats regardless.

Versioning the header

If we can agree on a header name and a versioning scheme that will work with our future header representations, we could potentially implement and improve the spec incrementally and maintain backwards compatibility. The main challenges would be deciding on a versioning scheme that will work with the header defined in the editor's draft and how we intend to represent it in the future, such as using an IETF dictionary. Versioning is a challenge we are going to have to face as we finalize the spec either way, so maybe now is as good a time as any to discuss it?

Other options

I'm sure some exist. Please add them to this issue.

In our workshop we decided to go with the simple name1=value1[;properties1],name2=value2[;properties2] that is currently in the spec.

Still, the group is open to discussion. A PR will be made that represents the current state of our discussion.

This specification reached a state where it is safe to mark it as First Public Working Draft.

A FPWD is a preliminary version of a spec and there are no requirements for completeness.

Following the timeline the change would be made by October 13th, if we reach consensus.

Copy of w3c/trace-context#23

TTL property can be used by vendor to specify the short-living values. Examples may be component name that initiated the request. Such property should not be passed further than the first hop.

As part of our W3C TAG review for this, we've been talking about the name of the feature. Baggage is definitely a US/UK idiom, and may not be intuitive to anyone who hasn't been focusing on distributed tracing. Might it be useful to use something like Trace ID additional header? (Maybe best to do some user research on this — the important thing is that it's immediately understandable to as many of your users as possible.)

From https://github.com/w3c/distributed-tracing-wg/blob/master/meeting_notes/2019-04-23.md

We should make a survey who is actually going to implement it. We have to check if it’s worth being a standard.

I wrote a post listing a number of use cases: https://medium.com/jaegertracing/embracing-context-propagation-7100b9b6029a

In the overview, the spec states Libraries and platforms MAY propagate this header.. It also says This is a companion header for the traceparent.

In the spec for traceparent. At a minimum they MUST propagate the traceparent and tracestate headers.

So intuitively, it seems that if it's a companion to traceparent, and that MUST be propagated, this MUST be too and I'm confused why correlationcontext suggest it MAY propagate.

this is an alternative proposal to #13. During the workshop today there was a proposal to rename header to the Baggage. Creating issue to discuss this proposal

3.2.1.3 value
https://www.w3.org/TR/baggage/#value

A value contains UTF-8 encoded string. Any characters outside of the baggage-octet ranges of characters MUST be percent-encoded. Characters which are not required to be percent-encoded MAY be percent-encoded.

The value space appears to require that the UTF-8 character encoding form be used, but this is not explicitly stated. There is also no discussion of what happens with percent-encoded octets that do not match UTF-8. I would suggest the following edits:

A value contains a string whose character encoding MUST be UTF-8 [Encoding]. Any characters outside of the baggage-octet range of characters MUST be percent-encoded. Characters which are not required to be percent-encoded MAY be percent-encoded.

When decoding the value, percent-encoded octet sequences that do not match the UTF-8 encoding scheme MUST be replaced with the replacement character (U+FFFD).

Similar to what was done for trace context, we need to create a test to serve as a reference implementation (correctness is the key goal and not necessarily a performant implementation).

From the current syntax definition, values would be allowed to contain =:

This way, prop1=foo1=bar1;prop2=foo2=bar2,propX=fooX=barX, for example, would be valid content.
Is this intended and were implications considered? The grammar is still unambiguous since comma and semicolon are not allowed for values but I'd still call this out at least in the examples to make consumers aware of it. Implementers might otherwise end up writing something like string[] parsed = "prop1=foo1=bar1".split('='); key = parsed[0]; value = parsed[1].

As part of our W3C TAG review of this — in the privacy vein, we talked about whether it would be good for this to behave differently in private/incognito browsing mode. It struck us that there is nothing to stop the application provider from sending headers derived from information tied to the machine or usual user (their IP address, their user ID, etc.). We wondered if it might make sense to ask user agents to ignore the trace ID altogether in when in private/incognito mode, so that the user's experience isn't tied to the experience that same user (or whoever they share a browser with) gets when not in private/incognito mode. What are your thoughts on this?

This is a self-review based on https://w3c.github.io/apa/fast/checklist.html for the W3C Baggage specification:

1. Does the technology allow visual rendering of content?
   No. The baggage standard does not provide this.

2. Does the technology provide author control over color?
   No. The baggage standard does not provide this.

3. Does the technology provide features to accept user input?
   No. The baggage standard does not provide this.

4. Does the technology provide user interaction features?
   No. The baggage standard does not provide this.

5. Does the technology define document semantics?
   No. The baggage standard does not provide this.

6. Does the technology provide time-based visual media?
   No. The baggage standard does not provide this.

7. Does the technology provide audio?
   No. The baggage standard does not provide this.

8. Does the technology provide time limits?
   No. The baggage standard does not provide this.

9. Does the technology provide text content?
   No. The baggage standard does not provide this.

10. Does the technology create objects that don't have an inherent text representation?
    No. The baggage standard does not do this.

11. Does the technology provide content fallback mechanisms, whether text or other formats?
    No. The baggage standard does not provide this.

12. Does the technology provide visual graphics?
    No. The baggage standard does not provide this.

13. Does the technology provide internationalization support?
    No. The baggage standard does not provide this.

14. Does the technology define accessible alternative features?
    No. The baggage standard does not define this.

15. Does the technology provide content directly for end-users?
    No. The baggage standard does not provide this.

16. Does the technology define an API?
    No. The baggage standard does not define an API.

17. Does the technology define a transmission protocol?
    No. The baggage standard does not define a transmission protocol - it defines a new HTTP header.

https://www.w3.org/TR/baggage/

I think it should only say "distributed context".

Follow up from #73 (comment). Let's publish FPWD since the editors list update that was blocking us was updated. @plehegar can you help?

@danielkhan, @arminru, and others have been quite active in the drafting of this specification. Should they be listed as editors as well?

The release process is not blocked by this, so we don't need to complete this prior to the FPWD.

Currently the spec contains the following wording multiple times:

Leading and trailing whitespaces (OWS) are allowed but MUST be trimmed when converting the header into a data structure.

I believe this is beyond the scope of this specification. The specification should only refer to the expected wire-format of the data. How it is interpreted is up to the consumer and producer of the data.

https://tools.ietf.org/html/draft-ietf-httpbis-header-structure-08#section-3.1

Need to add information about reference implementations to README: describe support in OpenTelemetry and in .NET.

For details, see Renaming the master branches.

See
https://w3c.github.io/spec-prod/#deploy-to-w3c-using-echidna

• Working draft available for wide review: Latest Draft
• Send notifications to groups listed in Charter coordination section: Web Performance, IETF HTTP WG
• A11y review
• I18n review
• Privacy review
• Security review
• TAG review: w3ctag/design-reviews#650. completed on March 23, 2022.

See also How to do Wide Review.

Looking for a standard implementation spec for enforcing limitation standards. What can a client do to ensure that it is uniformly carrying out this list of concerns, so that every client touching these headers performs the same set of actions upstream.

Example.

Size limit reached. Should a client perform a FIFO, and append the new key=value to the header?

Text keeps referring to "trace", which is only tangentially related. It would be useful to introduce a term of "distributed request" or "distributed execution" at the beginning and use that, instead of "trace".

Originally posted by @yurishkuro in #27

I think it's not clear what the limits precisely mean

https://w3c.github.io/baggage/#limits

It uses the term name-value pair which I don't think is actually defined in the doc. Most importantly I'm wondering if the equals sign is taken into consideration in the limits. Also for the "total length" limit, it would help to clarify whether comma is taken into consideration too.

We need to define the header format using ABNF, as what we've done for tracecontext https://github.com/w3c/trace-context/blob/master/spec/20-http_header_format.md#field-value

The human readable requirement in this spec introduces security, privacy (PII etc.) considerations that should be called out.

For example, JWT provides an approach to communicate JSON based payload data usually via HTTP headers. And it also provides additional optional benefits such as signatures and encryption. Also, ID Token is a means of encapsulating user identity in JWT Tokens.

Some information conveyed through contexts may either be sensitive or may be susceptible to tampering. It would be great to either:

• add signature/encryption as standard offerings when conveying correlation context
• and/or provide guidance on how to deal with security or encryption (for example, values could be JWT tokens)
• and/or call it as out of scope for this specification

"Maximum number of name-value pairs - this limit was taken as a number of cookies allowed by Chrome."

this seems a very weak rationale as cookies were defined for a different purpose. 180 pairs.. to pass along every hop, seems excessive

Short i18n review checklist is here

1. If the spec (or its implementation) contains any natural language text that will be read by a human (this includes error messages or other UI text, JSON strings, etc, etc),

   The baggage spec doesn't contain any natural language text that will be read by a human.

2. If the spec (or its implementation) allows content authors to produce typographically appealing text, either in its own right, or in association with graphics.

   N/A for Baggage spec.

3. If the spec (or its implementation) allows the user to point into text, creates text fragments, concatenates text, allows the user to select or step through text (using a cursor or other methods), etc.

   N/A for Baggage spec.

4. If the spec (or its implementation) allows searching or matching of text, including syntax and identifiers

   N/A for Baggage spec.

5. If the spec (or its implementation) sorts text

   N/A for Baggage spec.

6. If the spec (or its implementation) captures user input

   N/A for Baggage spec.

7. If the spec (or its implementation) deals with time in any way that will be read by humans and/or crosses time zone boundaries

   N/A for Baggage spec.

8. If the spec (or its implementation) allows any character encoding other than UTF-8.

   N/A for Baggage spec.

9. If the spec (or its implementation) defines markup.

   N/A for Baggage spec.

10. If the spec (or its implementation) deals with names, addresses, time & date formats, etc

    N/A for Baggage spec.

11. If the spec (or its implementation) describes a format or data that is likely to need localization.

    N/A for Baggage spec.

12. If the spec (or its implementation) makes any reference to or relies on any cultural norms

    TODO: N/A for baggage spec. When we did the TAG review, there was a question about the name of the spec "Baggage" and whether it relies on any cultural norms, the TAG team was satisfied with the answer we gave: #83

The spec has several examples of escaping space (U+0020) but no examples of escaping a non-ASCII values. Adding an example (perhaps by replacing Alice with a name like Amélie (Am%C3%A9lie) would help international users without being obscure to English speakers.

considering trace instrumentation have to process and propagate these. much smaller limits are more reasonable, similar to the tracestate header. I'm not sure the rationale for those were fully inventoried, some looked a bit reverse engineered from hunches. I'll give a concrete example.

A number above 32 causes problems for those encoding pairwise as state changes cannot be tracked with a 64bit bitset (without tricks). Yes, we can use blocked bitsets and other techniques, but some care should be taken to not set limits hugely especially without relevant rationale to do so.

Originally posted by @codefromthecrypt in #22 (comment)

I tend to agree with @codefromthecrypt here. The original limits were "inspired" by the limits of the set-cookie header which we borrowed our format from, but cookies are only meant to be sent between the user-agent and whatever service it calls. Baggage will need to be propagated by many more components in the system so its costs will be significantly higher for the same limits.

The Limits Section in the Baggage Propagation document specifies the following:

1. A platform MUST propagate all list-members up to at least 64 list-members including any list-members added by the platform.
2. A platform MUST propagate all list-members including any list-members added by the platform if the resulting baggage-string would be 8192 bytes or less. If the resulting baggage-string would be greater than 8192 bytes, some list-members MAY be dropped until the resulting baggage-string is 8192 characters or less.

It isn't clear to me what is expected the following hypothetical scenario:

• Baggage has 50 list-members
• The baggage-string is 10KB

The first limit specification suggests the entire string must be propagated because it contains less than 64 list-members. OTOH, the second limit specification suggests some list-members MAY be dropped. However, doing so contradicts the first requirement, which is labelled as a MUST.

I don't see any size limit to a list-member the way it is defined, which implies that the suggested hypotetical scenario is possible.

Pracitcally speaking, it seems to me that if there is a limit on length is provided, the intent is to allow an implemenation to bound the resources it would need to process a baggage string, if necessary. So, this leads me to believe the intent is to allow the size limit to trump the number of list-members limit.

I'm looking for feedback to confirm my thinking on this. Thanks!

Currently, RFC 7230 (HTTP/1.1 Message Syntax and routing) is listed as an informative reference in the spec:

-- https://w3c.github.io/baggage/

This should be fine after the definition of the OWS rule is copied to this specification in #31.

I'm not really familiar with W3C lingo, but since the syntax definition of key in Baggage is directly based on the definition of token in RFC 2616, I would expect this one to be a normative reference which we will have to add.

@reyang changed the spec to allow 255 leading space characters and @SergeyKanzhelev merged it the next day. There was zero rationale for this, yet it is very unintuitive

w3c/trace-context#135

As the intuitive behavior was reversed, we need an explanation. This makes for very strange parsers especially odd when so much thrash happened in the name of making parsing easier in the past. This has caused a couple people to scratch their heads and it is unacceptable for a change like this to have been merged in the first place.

I raised a comment there. but it was ignored, so making this an issue instead. Here's the comment:

trying to understand why this is allowing 255 spaces before a non-space.

#188 has a lot of talk about impact of presumably having a trailing empty member. I saw no code references, just talk about test cases invisible to open source, yet used to justify that change. Regardless, this literally permits 255 empty spaces than a non-space. If what's said in 188 is true, and that libraries arbitrarily add white (please cite exact libraries and versions).. what does this help?

I think it just makes sure space is only allowed for left-padding a non-space. If correct, the description should be updated. If incorrect, I guess I am misreading the abnf and would appreciate knowing how.

In the ABNF value is defined as %x21 / %x23-2B / %x2D-3A / %x3C-5B / %x5D-7E, which only allows for a single octet in the value.

list        =  list-member 0*179( OWS "," OWS list-member )
list-member =  key OWS "=" OWS value *( OWS ";" OWS property )
property    =  key OWS "=" OWS value
property    =/ key OWS
key         =  token ; as defined in RFC 2616, Section 2.2
value       =  %x21 / %x23-2B / %x2D-3A / %x3C-5B / %x5D-7E
               ; US-ASCII characters excluding CTLs,
               ; whitespace, DQUOTE, comma, semicolon,
               ; and backslash
OWS         =  *( SP / HTAB ) ; optional white space, as defined in RFC 7230, Section 3.2.3

The ABNF should be extended to allow for an arbitrary number of octets.

I was exactly looking for this to find what is language client implementation expected do in the case of certain cases

• If number of baggage entries in the header exceed the maximum number of allowed entries, do we drop everything or do we drop extra entries and propagate the remaining entries?
• If a baggage entry is invalid as per the definition, do we drop entire baggage or just that one entry?

This is not clear from spec. I would really appreciate some clarification here.

Thanks

Originally posted by @lonewolf3739 in #74 (comment)

Reference the relevant RFC for URL encoding baggage values

Originally posted by @yurishkuro in #78 (comment)

Copy from w3c/trace-context#36

Having a single header for all of this data will be detrimental for several reasons. The first being that metrics are numbers, and we can do maths on them. We'll likely have systems which want these metrics for context. They can be sniffed or captured at various levels. Similarly, there are many use cases for Metadata, it could be the context of a transaction for other tools (security, forensics, topological systems), or instructions for orchestration. There are various reasons these should be split.
Metrics - measurename=number , processingtime=34.534
Metadata - metadataname=metadata , browserlocation=Boston,MA

Filing a placeholder for filling in the self-review questionnaire for Security and Privacy.

[Self-Review Questionnaire: Security and Privacy] (https://w3ctag.github.io/security-questionnaire/)

1. What information might this feature expose to Web sites or other parties,
   and for what purposes is that exposure necessary?
   "Distributed Context" (aka Baggage) is a mechanism that enables specific scenarios of information propagation across components of a distributed system. The specification does not solve the problem of "what" data will be propagated through the distributed application components - similar to how it happens with any custom header application authors can send today, there is no mechanism to control what will be sent via the baggage header. Its goal is to solve the problem of standardizing the way data is propagated. While many tools today allow propagation of such information using custom vendor-specific headers, users of those tools have less control over the data being transmitted. When tools will follow the spec, it will be easier for users to control what is transmitted.

2. Do features in your specification expose the minimum amount of information
   necessary to enable their intended uses?
   The specification defines a format and not the values. As a result of that, the specification doesn't require any information beyond what the application developers want to use it for. Application owners should either ensure that no proprietary or confidential information is stored in baggage, or they should ensure that baggage isn't present in requests that cross trust-boundaries.

3. How do the features in your specification deal with personal information,
   personally-identifiable information (PII), or information derived from
   them?
   Application owners should either ensure that no proprietary or confidential information is stored in baggage, or they should ensure that baggage isn't present in requests that cross trust-boundaries.

4. How do the features in your specification deal with sensitive information?
   Application owners should either ensure that no proprietary or confidential information is stored in baggage, or they should ensure that baggage isn't present in requests that cross trust-boundaries.

5. Do the features in your specification introduce new state for an origin
   that persists across browsing sessions?
   No.

6. Do the features in your specification expose information about the
   underlying platform to origins?
   No.

7. Does this specification allow an origin to send data to the underlying
   platform?
   No.

8. Do features in this specification enable access to device sensors?
   No - the features in the specification don't enable access to device sensors.

9. Do features in this specification enable new script execution/loading
   mechanisms?
   No.

10. Do features in this specification allow an origin to access other devices?
    No.

11. Do features in this specification allow an origin some measure of control over
    a user agent's native UI?
    No.

12. What temporary identifiers do the features in this specification create or
    expose to the web?
    N/A

13. How does this specification distinguish between behavior in first-party and
    third-party contexts?
    This specification doesn't distinguish between first-party and third-party contexts. We believe that defining the trust boundaries is not the part of specification, thus no specific details on how to implement CORS or configure trust boundaries in the web. Trust boundaries, especially involving many components of a distributed application, are typically very specific to the application and cannot be easily generalized as a specification.

14. How do the features in this specification work in the context of a browser’s
    Private Browsing or Incognito mode?
    The specification doesn't provide for any special treatment in the incognito mode. At the moment there are no plans to implement baggage as part of the user-agent. Additional discussion on this topic from the TAG review is at #84.

15. Does this specification have both "Security Considerations" and "Privacy
    Considerations" sections?
    Yes.

16. Do features in your specification enable origins to downgrade default
    security protections?
    No.

17. How does your feature handle non-"fully active" documents?
    N/A. At the moment there are no plans to implement baggage as part of the user-agent.

18. What should this questionnaire have asked?
    The above questions cover the relevant aspects from a security and privacy perspective.

3.2.1.2 key
https://www.w3.org/TR/baggage/#key

A token which identifies a value in the baggage. token is defined in RFC7230, Section 3.2.6. Leading and trailing whitespaces (OWS) are allowed and are not considered to be a part of the key.

RFC7230 (which is quoted as the definition) only allows the (usual) restricted subset of ASCII for keys. It's not clear why this restriction applies to keys here, since the keys are part of the value space of the header. Should the full range of Unicode be allowed?

Proposal

For trace-context we decided to use to use all lowercase alphabetic characters to make the header usable in non HTTP scenarios. Message queues such as JMS are more restrictive than HTTP in regards to header names. If we expect to propagate Correlation Context over protocols other than HTTP, we should consider using the same naming conventions as used for trace-context and name this header correlationcontext.

The rationale for this change is the same as the rationale for Trace Context headers. See: https://github.com/w3c/trace-context/blob/master/http_header_format_rationale.md#lowercase-concatenated-header-names

Some benefits of having an alphabetic lowercase header name:

• Maximizes the scenarios that this header name can be used in
  • HTTP headers,
  • key-value based systems of all kinds
• Reduces friction to adoption
  • There is only one canonical representation of the header that needs to looked up

Known Issues

Incompatible as a JMS Header

Correlation-Context is incompatible as a JMS header due to the - character.

Section 3.8.1.1 (Message Selector Syntax) of the JMS Specification states:

Identifiers:

• An identifier is an unlimited length character sequence that must begin with a Java identifier start character and all following characters must be Java identifier part characters. An identifier start character is any character for which the method Character.isJavaIdentifierStart returns true. This includes ‘_’ and ‘$’. An identifier part character is any character for which the method Character.isJavaIdentifierPart returns true.

Example:

import java.lang.*;

public class IdentifierDemo {

   public static void main(String[] args) {
      char dash = '-';
      boolean dashValid = Character.isJavaIdentifierPart(dash);
      System.out.println("Dash valid?: " + dashValid);    
   }
}

Output:

$javac IdentifierDemo.java
$java -Xmx128M -Xms16M IdentifierDemo
Dash valid?: false

In #20 @yurishkuro raised some concerns around the clarity/reasoning of some points in the rationale.
While we move ahead with the added normative wording that was the scope of the PR, this issue serves as tracking issue to review the rationale.

Copy from w3c/trace-context#22

Some tracing vendors support types for the context properties. We can add type property to allow an explicit type specification. Something like this:

Boolean

Binary flag. Supported values 1 for true and 0 for false.

Examples:

IsAuthenticated=1;type=bool
IsAuthenticated=0;type=bool

Number

Numeric value as described in IEEE 754-2008 binary64 (double precision) numbers IEEE754.

Examples:

ExposurePercentage=33.33;type=number
Step=10;type=number

See comment: #52 (review)

We declare limits in bytes and say ASCII when we say what symbols are allowed, but if somebody will read it into the Unicode, we may want to make sure the limits will be treated as characters limits. Similar to cookie spec note:

NOTE: Despite its name, the cookie-string is actually a sequence of
   octets, not a sequence of characters.  To convert the cookie-string
   (or components thereof) into a sequence of characters (e.g., for
   presentation to the user), the user agent might wish to try using the
   UTF-8 character encoding [RFC3629] to decode the octet sequence.
   This decoding might fail, however, because not every sequence of
   octets is valid UTF-8.

The link to https://www.w3.org/TR/correlation-context/ is broken, so perhaps remove the link to the "Latest published version" until there is one?

As discussed in the working group meeting on April 6, optional whitespace reduces clarity for things like determining the maximum length limits, and determining if a whitespace character is a part of the key or value.

In the meeting it was discussed that OWS would be removed

I remember we want to move the tracecontext test suite to a separate repo in order to keep the spec repo clean (e.g. we don't want to go through the spec updating process each time we improve the test cases).

Now we have a chance to do it in correlation context, where should we put the test? @bogdandrutu @SergeyKanzhelev

Copy from w3c/trace-context#24

Some vendors has sensitive context properties like user name that can be passed to the dependent service inside the organization, but should not be propagated to external services. PII flag of the property can indicate that the value is sensitive and should have a special treatment.