w3c / openscreenprotocol Goto Github PK

This work item covers the control protocol for the Presentation API, i.e. the syntax and meaning of the messages sent over the transport [1] to start and control the presentation.

It might make sense to pick an initial transport before undertaking this work item, since the format of the control messages may be tied to the transport. For example, we may be able to map control messages onto transport level mechanisms like QUIC resource streams.

[1] #8 Evaluate protocol transport mechanisms

uPnP is standardized outside of IETF/W3C and assets are owned by the Open Connectivity Foundation. Need to find if licensing terms prevent any use in Open Screen or implementers from adopting it.

FYI @schien

Creating a placeholder for datachannel.md

This issue covers authoring functional requirements to implement the Presentation API.

This meta-issue tracks the work to gather performance data on proposed discovery protocols.

Even though both SSDP and mDNS have been implemented in Chrome (and Mozilla?), for a variety of reasons, it can be tricky to use data from the field to compare performance of the discovery protocols themselves. Therefore I propose running some experiments to gather accurate data.

• Design experimental framework: what devices, what network, what software and what measurements.
• mDNS
  • Issue #37: [mDNS] Get reliability data
  • Issue #38: [mDNS] Get network and power efficiency data
• SSDP
  • Issue #68: [SSDP] Get reliability data
  • Issue #26: [SSDP] Get network and power efficiency data
• Collect data on latency of discovery (when devices enter/leave the LAN)

Separately from experiments, there may be some data we can share from Chrome internal metrics related to reliability. I will look into it and report back.

This is for data collection to compare discovery protocols.

BCP-47 states that language tags can be arbitrarily long in theory.

In #53 (comment) @tomoyukilabs wrote:

Okay. I just wanted to confirm the reason of 64 characters. Regarding the length, BCP-47 describes Length Considerations and indicates several examples and recommendations, e.g.:

• historically most language tags have consisted of language and region subtags with a combined total length of up to six characters
• RFC4646 recommended a minimum field size of 42 characters

This issue is to research real-world usage and/or ask for input from the i18n working group on language tag usage to inform if we want a fixed length string or variable length string.

The steps in the Presentation API spec that require communication between the controlling and presenting user agent should be cross referenced with messages in the Presentation API control protocol.

Related to this, there are implicit dependencies between messages: i.e. the Termination Request can't be sent before the Initiation Request for a given presentation. These should be made explicit through sequence diagrams showing how messages relate to the lifetime of a presentation or a presentation connection.

The control protocol uses ASCII art box diagrams to describe message contents. These are okay but results in redundancy for common values like URLs and presentation IDs. Also it takes extra effort to describe repeated values.

As the number of message types will only increase with work on Remote Playback API, I suggest adopting a message description syntax similar to (but much simpler than) TLS 1.3 "Presentation Language" [1]. Then we can simply describe a message with a C-like struct definition and re-use complex types across messages.

[1] https://tools.ietf.org/html/draft-ietf-tls-tls13-21#section-3

When working on spec documents I've been using the terms controller, controlling page, receiver, presentation display, presentation, etc. interchangeably.

This TODO is a reminder to settle on a set of terms (from the requirements document), edit the other documents to be consistent with those terms, and possibly add a short glossary for reference.

This issue tracks the work to evaluate QUIC [1] as an alternative transport for the Open Screen protocol.

See the meta issue #8 for the list of evaluation criteria.

[1] https://www.chromium.org/quic

This issue covers evaluation of presentation screen discovery mechanisms.

Per the charter scope, the focus will be on LAN discovery mechanisms. The following have been proposed thus far:

• Multicast DNS/DNS-SD #6
• SSDP/DIAL #7

(If there are any I have missed, please reply to this issue.)

The task for each is to evaluate them along the functional and non-functional requirements, and make an initial recommendation to explore further.

This task is to investigate to what extent:

• We can measure fine grained power consumption.
• We can attribute power consumption to a specific network activity.
• For background activity like discovery, does the degree of coalescing enabled by the platform affect either of the above.

We wish to collect a data curve showing network utilization and power consumption related to the following parameters (not exhaustive list):

• Number of controllers
• Number of receivers
• Response cache length (MAX-AGE)
• Number of retransmissions
• Frequency of devices entering and leaving network
• Continuous vs. intermittent queries

It may be possible to evaluate power consumption directly, or indirectly based on modeling WiFi behavior, which is likely to dominate power on mobile devices. Some research needed to see if there is any suitable model for this purpose.

This issue covers authoring functional requirements to implement the Remote Playback API.

Initially assigning to Remote Playback API editor @avayvod.

From https://github.com/webscreens/openscreenprotocol/pull/20/files#r108024631

mfoltzgoogle 3 days ago Collaborator
How is FRIENDLY-NAME encoded?
@louaybassbouss
louaybassbouss 2 days ago Collaborator
it is just a clear-text string field like the other SSDP headers.
@mfoltzgoogle
mfoltzgoogle 9 minutes ago Collaborator
Since it's displayed to humans, it should be able to represent Unicode code points. We can look at the SSDP spec to see what is supported by default.

This is an editorial task to ensure that either endpoint of a control channel can assume either role in the Presentation API (or Remote Playback API, or any other API going forward).

Example: Two controlling user agents are connected to a presentation receiver. One starts presentation of a video with closed captions, and uses its control channel to request that the display push the closed captions to the second controller.

Note that this also implies that the underlying discovery/transport protocols should not constrain which device is the controlling UA and which is the receiver. For example, the controlling UA could act as a QUIC server or mDNS responder for the purposes of establishing the control channel (even though current implementations typically go the other way). There might be practical reasons this isn't common, but it should be possible.

There some more API and protocol level work required to make this work. For example, when a control channel is established, part of the handshake should figure out whether each side can act as a controller, receiver or both.

This work item covers the control protocol for the Remote Playback API, i.e. the syntax and meaning of the messages sent over the transport [1] to start and control remote playback.

It might make sense to pick an initial transport before undertaking this work item, since the format of the control messages may be tied to the transport. For example, we may be able to map control messages onto transport level mechanisms like QUIC resource streams.

[1] #8 Evaluate protocol transport mechanisms

If possible to create a controlled environment for power usage, we could measure against mechanism parameters (i.e., MX, search frequency) and the number of controlling and receiving devices.

As discussed earlier during TPAC, I've noticed that the protocol is a hand-rolled binary protocol. I'm wondering if this is the best way forward, since the spec isn't in a stage whether or not this information should be exposed raw to the application. (I'm guessing in this context, it would be de-serialized in advance by the user agent)

Unfortunately JSON might not be the best option for this use case, since the commands are streamed from what I understand. But using a battle tested serialization format would probably be a better way forward.

RFC7049 http://cbor.io from IETF is one option (also mentioned earlier), but I remember reading something about a technical bug on this - I'll have to dig up the details, but IIRC it was related to unicode handling so this may not apply to your use case.

DIAL has incorporated Wake-On-Lan detection into its discovery protocol. Many presentation displays enter a low power state on sleep that don't respond to discovery requests until they are woken up. It would be useful for the display to be able advertise these capabilities, and to standardize how to leverage them to wake devices from sleep.

From TODO in quic.md

The uPNP stack has historically been a target for active attacks against network devices. This issue is to research the history of attacks against the portions of uPnP under consideration for Open Screen and address them with mitigations.

White paper with background: https://community.rapid7.com/docs/DOC-2150

Background: https://blog.cloudflare.com/ssdp-100gbps/

This issue covers authoring non-functional requirements for the open screen protocol, including, but not limited to:

• Security
• Efficiency (power, memory, CPU)
• Ease of implementation
• Usability
• Extensibility

From TODOs in quic.md

Before we can pursue apples-to-apples benchmarking of transport protocols, there needs to be a definition of the bootstrapping channel used to establish an RTCDataChannel between controller and receiver, so we can include its metrics in the measurements.

@schien is this something you can help with?

This meta-issue tracks the work to gather performance data on proposed trasnport protocols.

• Design experimental framework: what devices, what network, what software and what measurements.
• QUIC
  • Issue #63: [QUIC] Get reliability data
  • Issue #65: [QUIC] Get data on network and power efficiency
  • Issue #66: [QUIC] Get data on connection and message latency
• RTC DataChannel
  • Issue #30: [DataChannel] Get reliability data
  • Issue #33: [DataChannel] Obtain data on network and power efficiency
  • Issue #31: [DataChannel] Get data on connection and message latency

Placeholder for mDNS evaluation

FYI @schien

Creating a placeholder for datachannel.md

The draft protocol includes 64-bit values for message length and sequence ID. It would save space to use varints here.

See: https://en.wikipedia.org/wiki/Variable-length_quantity

As HbbTV 2.0 [1] and ATSC 3.0 [2] leverage WebSockets as part of their core presentation framwework, we should investigate to what extent the WebSocket protocol can be implemented with QUIC. This would provide part of a path to interop or backwards compatibility between a QUIC-based Open Screen protocol and these standards.

[1] https://www.hbbtv.org/
[2] https://www.atsc.org/atsc-30-standard/a3382017-companion-device/

Relevant issue from Second Screen WG: w3c/presentation-api#67

This work item tracks the evaluation of the Mozilla proposal [1] as a transport for the Open Screen protocol.

See the meta issue #8 for the list of criteria.

[1] https://wiki.mozilla.org/WebAPI/PresentationAPI:Protocol_Draft#Establish_Control_Channel

@schien Can you take stab at this when you are available?

See the meta-issue #5 for the list of evaluation criteria.

@mfoltzgoogle I will take this work item to prepare a report .

See the meta-issue #5 for the list of evaluation criteria.

From TODO in quic.md

FYI @schien

Creating a placeholder for datachannel.md

From https://www.w3.org/2017/11/07-webscreens-minutes.html#x11:

• ACTION: @louaybassbouss to complete the list of implementations for SSDP, add e.g. libupnp
• ACTION: @mfoltzgoogle to find out the MX value used in Chrome

Although specific mechanisms for WAN use are out of scope, it is useful to know if a transport can be extended in the future to support displays outside the controller's LAN, or is inherently tied to a specific LAN.

This work item covers the evaluation of alternatives for the protocol transport. Once a presentation display has been discovered, the transport provides a message-oriented channel for controlling the presentation receiver.

Two initial alternatives:

• QUIC #9
• TLS socket + RTC DataChannel (Mozilla proposal) #10

Additional possibilities are HTTP/2, WebSockets, or a custom protocol over TCP or UDP. These can additionally be evaluated once the initial alternatives are done.

This issue tracks proposed security mechanisms to authenticate the parties communicating via the open screen protocol.

Before proceeding, I would like to solicit consensus about what is in scope and out of scope regarding authentication. In particular, I proposed that the initial focus is on UA-to-UA authentication, versus authenticating users, applications, or the particular software running either UA.

These additional layers of authentication could potentially be written as protocol extensions to the base transport security, but would not be the initial focus of the proposals.

It would be helpful to have an upper bound for PresentationConnection message latency based on use cases like media synchronization or lip sync.

From https://www.w3.org/2017/11/07-webscreens-minutes.html#x08:

ACTION: @chrisn to propose latency ranges for media synchronization use cases (lip sync vs frame sync)

Forked from https://github.com/webscreens/openscreenprotocol/pull/20/files#r108024628.

Including presentation URLs in the SEARCH or response has a couple of issues:

• It's not great from a privacy perspective, as it exposes them to passive network listeners.
• These messages need to fit in a UDP packet, which limits them to about 1420 bytes.

@louaybassbouss

louaybassbouss 2 days ago Collaborator

I agree with you question is as I mentioned in other comments we need to make clear what is the input and output of the discovery process. Question is here do we want to have device filtering in discovery or later after handshake? filter devices during discovery reduce the number of unnecessary calls (to displays that cannot present a specific URL) but raises on the other hand couple of privacy issues as you mentioned. From my point of view, I prefer a solution where the presentation display exposes in a way more information about the content that can be presented. I was thinking about a regular expression that can be exposed in a SSDP header (in SEARCH response or alive messages) where the controller can detect if the display can present a specific URL or not. For example if a display allows only https then the regular expression will be something like https://* or if the receiver is cast device a accepts only cast applications then the regular expression could be https://cast.google.com#castAppId=*. In this case there is no need to share the presentation URLs in the discovery messages.

@mfoltzgoogle
mfoltzgoogle just now Collaborator
I think if there's a privacy preserving way to optimize away some connections to the receiver service that would be a benefit. If the primary benefit is to separate out generic endpoints (https://) from application-specific ones (e.g., casts://) a simple scheme advertisement approach might work.

This is a placeholder for the mDNS evaluation.

FYI @schien

Creating a placeholder for datachannel.md

From TODO in quic.md

Placeholder for mDNS evaluation

From https://www.w3.org/2017/11/07-webscreens-minutes.html#x05:

• ACTION: @mfoltzgoogle to clarify that device requirements are driven by rendering Web content and not the network stack
• ACTION: @mfoltzgoogle to consult internally to find out if smart TVs fit within these requirements
• ACTION: @mfoltzgoogle to add hardware cryptography capabilities to device specs

This issue will be used to track work items to author a consensus requirements document. This will help guide discussion whether protocol features are in scope or out of scope, and to evaluate the suitability of technology choices.

The requirements flow primarily from three sources:

• Requirements that come from the functions needed to implement the Presentation API [1] #2
• Requirements that come from the functions needed to implement the Remote Playback API [2] #3
• Non-functional requirements: security, efficiency, usability. #4

Starting points are the draft requirements for the community group recharter [3], along with the slide deck I presented at TPAC 2016. I can simplify these and put them into a PR to start this work item.

[1] https://w3c.github.io/presentation-api/
[2] https://w3c.github.io/remote-playback/
[3] https://github.com/webscreens/cg-charter/blob/gh-pages/requirements.md

Remote control of presentation displays via network commands is now possible with Chromecast and similar products.

A reasonable extension to the core Open Screen protocol would add standardized messages to implement these commands, for example:

• Change power/sleep state
• Volume control
• Mute
• Input switching

CEC defines a much broader list of capabilities for controlling TVs and attached devices; starting from a lowest common denominator and adding from there seems reasonable.