Re-decentralizing the Web
Solid is a proposed set of standards and tools for building decentralized Web applications based on Linked Data principles.
Read more on solidproject.org.
Solid Notifications Technical Reports
Home Page: https://solid.github.io/notifications/protocol
License: MIT License
Re-decentralizing the Web
Solid is a proposed set of standards and tools for building decentralized Web applications based on Linked Data principles.
Read more on solidproject.org.
I think we should serve that .js
form github so it is more reliable.
I had the same issue a while ago and mentioned it during one of the earlier meetings.
related: solid/notifications-panel#36
This should streamline the processing of the stream by the client and parsing notifications.
Depending on the application, clients might want to update subscriptions and/or create/destroy connections. This issue is meant to track these cases, since such updates inform the design of the protocol.
Let me start with cases which I care about:
Just a question on:
https://github.com/solid/solid-spec/blob/master/api-websockets.md
This says how a subscription is created, but it's not explicit about when the subscription expires. I'm guessing it's when the Websocket connection is closed. amirite?
To hide some of the complexity, would there be any advantage of having a universal notifications layer that will handle common elements of the different subscription types (similar in idea as solid/authorization-panel#253 for authorization)?
Originally posted by @kevin-inrupt in solid/notifications-panel#3 (comment)
The 0.8 Solid specification includes a WebSocket API, which defines a simple line-based interaction between client and server. For example, here is a subscription to a container followed by two notifications for child resources that are created:
> sub https://example.com/container
< pub https://example.com/container/1
< pub https://example.com/container/2
That design has the advantage of being simple to implement and simple to consume. That said, most WebSocket APIs that I have encountered use a JSON-based format, which provides a more structured format for consumption (the format could even be compact JSON-LD) along with easier points for extension (e.g. what if other keywords are to be added?).
In addition, the JavaScript API for WebSockets (WebSocket(url[, protocols])
) includes an optional protocols
parameter. If this interaction is formalized, would it make sense to define a solid
protocol that could be used for WebSocket interactions?
In any environment WebSocket (WS) connections could close unexpectedly for various reasons. Also, in some deployments connections are even closed if there is no recent traffic. For example; running a server behind and AWS Elastic Load Balancer will close idle connections (default is 60 seconds but it can be increased to a maximum of 4000 seconds). Ngnix (by default) will also close idle connections.
Clients would need to reconnect and resubscribe to resources its interested in. It would be great if this was in the specification about how a client should recover in situations like this.
For example; On a WS close event the client could attempt to reconnect to the WS endpoint a number of times with an exponential back-off time (1, 2, 4, 8, 16, 32 seconds) in between each attempt. If the connection is restored it will send the subscribe message to its interested resources.
Also, guidance on how to keep WS connections open would also be valuable. Making sure the Client and Server sends Heartbeat message would prevent idle connections. See:
https://tools.ietf.org/html/rfc6455#section-5.5.2
All of this would help improve notifications been missed by a client application.
In the bit about live update,
https://github.com/solid/solid-spec/blob/master/api-websockets.md
we need to specify that you can subscribe to changes in a resource which currently does not exist, and get a ping when it is created (and thereafter to any changes).
The headers Update-via
and `MS-Author-Via: are sent on 404 as well as 200.
NSS sends the headers now. i haven't tested the actual WS ping message on file creation.
See also
linkeddata/rdflib.js#392 in rdflib
which is depended on by
https://github.com/solid/solid-ui/issues/228 a issue with the chat UI needing this.
The term features
should be renamed to feature
, as is customary for RDF predicates.
The solid spec must define the whole protocol, including how the auth plugs into the notification. Propose that this document does define that. Otherwise we need another parallel document alongside this. The spec must completely define exactly ow to build a solid server.
Originally posted by @timbl in solid/notifications-panel#3 (comment)
Add Change Log section to Notifications Protocol that includes substantive changes or new features per significant publication, e.g., the next publication of Notifications Protocol will summarise changes since latest published version (which is https://solidproject.org/TR/2022/notifications-protocol-20220509 as of this writing).
It can be along the lines of https://www.w3.org/TR/ldn/#change-log (follows publication rules for W3C specifications).
I'm starting this issue as somewhere to capture comments on the spec that have come up whilst I was looking at building conformance tests.
spec:statement
at https://solid.github.io/notifications/protocol#server-subscription-access-controls does not contain the start of the requirement.Then I went on to look at the WebsocketSubscription2021 spec: https://solidproject.org/TR/websocket-subscription-2021. This spec says an API must conform to the notification protocol spec above, then lists some requirements but I think there are some issues:
client-subscription-feature
where I think it should be client-subscription-topic
https://solid.github.io/notifications/protocol#subscription
This section only describes a payload of Subscription Request. At the same time, various subscription types describe requirements on various Subscriptions Responses.
I think protocol should mention Subscription Response and shortly explain their role and mention that each subscription type will specify details of Subscription Response.
Is the "Solid Notifications Protocol" spec title too broad?
Within the scope of panel, we have a charter, we do stuff around notifications. Current protocol seems to be focused on subscriptions. Maybe we need to come back to use cases and see if any of them don't fit under the current spec.
Related to solid/specification#50, if WebSocket support becomes part of the Solid specification, it would be helpful to clarify whether the subscription mechanism is recursive. That is, if a client subscribes to the resource at /foo
, will the client receive notifications for all (recursively) contained resources?
Another way to ask this is, given a subscription for /foo
, which of the following actions will trigger a notification for that particular subscription?
/foo
container (e.g. /foo/bar
)/foo/bar
is updated/foo/bar/baz
/foo/bar/baz
is updatedThere is the Push API. Since the Solid Protocol has a section on Live Updates it would be useful to understand how these two complement each other, work together, what ideas can be taken from it, etc...
Some resources:
Any thoughts?
There current way the client subscribes to notifications of a document being changed has security issues in that the web socket URI could be guessed. Change this to be more secure.
Here are a number of review notes on the notificatiom protocol document.
At present, the Notification Protocol defines a gateway API that acts as a discovery endpoint for clients: agents submit POST
JSON-LD requests and receive JSON-LD response bodies. This is an unauthenticated endpoint.
Rather than providing a required request-response gateway API, it would arguably be simpler do just define the subscription types that are supported in a single RDF document. Clients can fetch that RDF document and, given the features supported by the various subscription types, the client can simply choose which path to follow.
One can argue that the current gateway API model already has such an RDF document implicitly and the HTTP API brokers access to that document. Removing that API has several advantages:
In the process of linking the vocabulary terms to relevant locations in the specification, I have found that a number of terms that are used in the protocol specification are left undefined. These include:
notify:subscription
notify:notificationChannel
notify:feature
There should be normative definitions for each of these.
We need to find a way for a server to be able to declare interest in a resource changing in some way.
This would require each resource to point to a resource where a client can subscribe to changes in the resource, perhaps using SPARQL, with a pointer to a container to send notifications to when those changes have been triggered. Probably requires some form of authentication of the user, to verify that the notification box is the right one.
The 0.8 Solid spec describes a WebSocket subscription protocol, but that protocol does not include any mention of authentication or authorization. If WebSockets are to be part of the 1.0 Specification, there should be some formalization of the authentication protocol as well as a description of how these notifications intersect with WebAC rules.
It should be noted that WebSockets are raw TCP sockets. HTTP headers are not relevant, and so reliance on an Authorization
header will be insufficient.
I think we should clarify the assumptions we make about how solid storage hosting resources (topics) relate to party hosting notification source or delivering the notification to the target.
I would like to know if at any point we assume that the same solid storage will host the subscription endpoint (or deliver notifications to the target), or it could be done by a distinct party.
I think details of how that party interacts with solid storage could be left up to implementations. It could be still worth assuming that those two could be two distinct parties.
Although it seems to me that subscription resource
is a long used and pretty set in stone term,
I would like point out that the naming may be confusing:
From the term alone, one could get the idea that a subscription resource
is a resource whose representation describes a subscription. Especially, when modelling the resource where requests are submitted to as a container (#36), or even more, when considering materializing requests for subscriptions in such a container.
In my mind, what is currently referred to as a subscription resource
is a resource whose representation describes a service that provides the subscription/notification functionalities. (independent of that resource being also a container or can be POSTed to)
Thus, I would like to propose the terms subscription service resource
which is described by its subscription service representation
.
Consider whether it is useful to define subscription service as a container. Originally mentioned in #29 (comment) :
(I love and hate this idea.) Change the subscription [service] from a non-container to a container. It will work with existing behaviour on POST to container in Solid Protocol. This will open up the possibility for a server to track/record subscription requests (as notifications) in and of itself. The immediate concern with subscription [service] as a container is that if the description is to be readable (200 to GET, HEAD as mentioned above), then the container would reveal containment statements - a "feature" we definitely don't want as is. To get around this, essentially there needs to be a requirement that would, if at all, only allow the owner of the storage to have access to these subscription requests. Alternatively, a requirement that would allow servers to omit containment statements. On the other hand, if GET, HEAD to subscription [service] is a 405, we don't have that problem.
Add a mention that existing Sold servers and clients implementing a notification protocol based on web sockets, and this being used for example for chat applications, collaborative editors, etc. It is disingenuous to write as though this is not currently already running. This protocol serves to provide a more secure and more modular version of the same functionality.
Originally posted by @timbl in solid/notifications-panel#3 (comment)
Suggest a level 2 protocol in which the triples which changed in an RDF resource are included in the notification.
Originally posted by @timbl in solid/notifications-panel#3 (comment)
How does one unsubscribe to watching a resource without terminating the websocket connection itself?
Introduce the notion of Notification Profile as an extension of the core Notification Data Model.
Specific Notification Profiles can then be independently published.
Add a Conformance Classes section listing the roles eg. Notification Server, Client, that the Solid Notifications Protocol specifies. The criteria for each of the roles can be described throughout the specification.
The Notifications Protocol currently does not describe the discovery of the Notification Subscription Metadata Resource.
This issue is a follow-up on the discussion at #54 . Some options are mentioned there in addition to our earlier consideration to use Server/Storage Metadata: solid/specification#355
As discussed after the meeting last week (7 July 2022), I would like to propose an alternate design for Solid Notifications.
Before I even begin, a few disclaimers are in order:
The current Solid Notification Protocol requires a number of discovery and initialization steps before we can actually start receiving any notifications:
These are clearly a lot of round-trips before we can actually start receiving notifications.
My other concern is with Internet outages (especially micro-outages), which are extremely common where I live (even on broadband, let alone mobile). Going through even an extra step to recover a disrupted WebSocket connection is going to lead to a poor user experience. See @acoburn comment below.
Further, it is not specified yet, if we shall have one channel per resource or one channel per client. Even in the latter case (one channel per client), if the client wants to add or remove resources and/or change notification features like extend subscription duration, it must re-negotiate a new endpoint first using HTTP. While this might be necessary for uni-directional protocols, it makes little sense to carry this over to a bi-directional protocol like WebSockets.
GET
to multiple resources on a Solid pod and with an endless response)Changes of features, addition or removal of resources can all be done in band through a client message, without the need to initiate new connections.
Even if we choose to go with one channel per resource, the operation is no more complex than a new connection & GET
like request.
@uvdsl already has a symmetry objection to this proposal. While I have hinted at it already, I shall defer to him for a full explanation.
I also invite other Subscription Type authors to consider how their proposals may be affected.
Index of Subscription Types that are published under https://solidproject.org/TR/
As only a single channel will be returned, I suggest that the request lists the types in preferred order just in case there is multiple overlap. In this way, the client gets their preferred type that is supported by the server.
Originally posted by @kevin-inrupt in solid/notifications-panel#3 (comment)
LinkedDataNotificationsSubscription2021 uses webid in the subscription response explaining:
All LDN notifications will be sent to the defined inbox using the agent identity specified by this WebID. A resource owner should ensure that the agent identified in this property is permitted to send authenticated HTTP requests to the defined inbox.
I think WebHookSubscription2021 and all target flow types, in general, could use this as a common approach.
@jaxoncreed what do you think about using this approach instead of defining custom AuthN?
https://github.com/solid/notifications/blob/main/webhook-subscription.md#10-webhook-request-with-token-signed-by-the-pod-key
Diagram should be clarified (content and visuals) to better distinguish between what's defined by the Notifications Protocol from the components/interactions defined elsewhere.
In the Notifications Protocol, the description of a notification channel includes its type, feature, and subscription. The value of the type is expressed as the same value as the subscription type. The question is whether the subscription type, e.g., WebSocketSubscription2021
is misleading or misused here as part of the notification channel type.
For an ecosystem that is based on capability/feature discovery, it is arguably better to think less about "endpoints" and more about "hypermedia APIs". As such, I would propose removing instances of 'endpoint' in this specification and replacing them with either 'API' or 'URI', as appropriate.
It may also be worth mentioning that a client should be able to trust that the notifications it receives aren't forged.
Originally posted by @justinwb in solid/notifications-panel#3 (comment)
Specs need both the version number, like 1.0 or 3.4.5, and the maturity like "draft, "Editor's Draft", "Recommendation".
Suggest publish this as say [Editor's] Draft of Version 1.0 of the Solid Notifications Protocol.
Making the version number < 1 messes up semver.
After 1.0, increment the major version for incompatible changes, minor for back-compatible enhancements, the patch level for bug fixes and wording improvements.
The version number is about the protocol. The maturity level is about the document and the community.
The Solid Editors came to this conclusion but maybe have not written it up.
As discussed in notifications-panel meeting on January 20, we need a consistent naming template for subscription types.
Currently, the name are opaque strings in the format:
<SubscriptionTypeName><Year>
This naming scheme is inadequate in my view, say, when there are multiple revisions to a given subscription type in the same year. A proposed naming scheme must make it simple for clients (actually client developers) to identify available subscription types.
Also, having a naming scheme provides clear guidance to implementers of new subscription types, as to how to name their implementations and revisions.
Current draft states:
Authentication and Authorization
The Subscription API requires authorization. This document does not define the specific technology used to authorize requests; rather, it defers to the Solid Protocol sections on Authentication and Authorization. It is out of scope for this document to describe how a client fetches an authorization token. Solid-OIDC is one example of an authentication mechanism that could be used with Solid Notifications.
Since it only mentions Subscription API, I assume that each subscription type will define details for how to handle AuthN / AuhZ?
Further in the document, I see:
Notification Subscription
The details of subscribing to a particular endpoint depend on the technology used, but all will follow a similar pattern. For WebSockets, this involves sending an authenticated subscription request to the endpoint retrieved from the gateway service.
It seem to reaffirm that each subscription type will define how to handle authentication and authorization
One of the advantages of defining subscription type based on streaming HTTP response (e.g. Fetch Response.body.getReader
) comes with using the same mechanism relying on HTTP headers.
I think we should also consider the option to negotiate usage of Capability URL for Subscription Endpoint. IMO this could save some complexity with WebSockets subscription type.
The WIP definition of notify:source
in the RDF vocab
The IRI of a resource used by a client to establish a connection to a notification channel.
is suitable towards general purpose use, however, it is tied to / isDefinedBy https://solidproject.org/TR/websocket-subscription-2021#websocketsubscription2021-source (noting that we have an understanding in the PR to use isDefinedBy/seeAlso that will change this eventually).
But the definition in WebSocketSubscription2021 is actually constrained to wss
.
Essentially, either notify:source
needs to be reserved for WebSocketSubscription - and if so, that's fine and we need to be more clear - or we leave it open for use by subscription types.
The proposal in the PR is that the term should be tied to the Notifications Protocol, which would be something like e.g.:
isDefinedBy http://www.w3.org/ns/solid/notifications#
seeAlso https://solidproject.org/TR/notifications-protocol#notify-source
#notify-source
will be described under Terminology (or possibly another section that may be more suitable later.)
Originally posted by @csarven in solid/vocab#62 (comment)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.