I think we should serve that .js form github so it is more reliable.

I had the same issue a while ago and mentioned it during one of the earlier meetings.

@csarven

related: solid/notifications-panel#36

This should streamline the processing of the stream by the client and parsing notifications.

Depending on the application, clients might want to update subscriptions and/or create/destroy connections. This issue is meant to track these cases, since such updates inform the design of the protocol.

Let me start with cases which I care about:

1. Client might want to change features. For example, client might want to slow down the rate of updates based on user activity.
2. Client might want to add or remove resources to watch (assuming we allow multiple resources on the same subscription) based on a top level container they are watching. These updates might even be recursive.

Just a question on:

https://github.com/solid/solid-spec/blob/master/api-websockets.md

This says how a subscription is created, but it's not explicit about when the subscription expires. I'm guessing it's when the Websocket connection is closed. amirite?

To hide some of the complexity, would there be any advantage of having a universal notifications layer that will handle common elements of the different subscription types (similar in idea as solid/authorization-panel#253 for authorization)?

Originally posted by @kevin-inrupt in solid/notifications-panel#3 (comment)

The 0.8 Solid specification includes a WebSocket API, which defines a simple line-based interaction between client and server. For example, here is a subscription to a container followed by two notifications for child resources that are created:

> sub https://example.com/container
< pub https://example.com/container/1
< pub https://example.com/container/2

That design has the advantage of being simple to implement and simple to consume. That said, most WebSocket APIs that I have encountered use a JSON-based format, which provides a more structured format for consumption (the format could even be compact JSON-LD) along with easier points for extension (e.g. what if other keywords are to be added?).

In addition, the JavaScript API for WebSockets (WebSocket(url[, protocols])) includes an optional protocols parameter. If this interaction is formalized, would it make sense to define a solid protocol that could be used for WebSocket interactions?

In any environment WebSocket (WS) connections could close unexpectedly for various reasons. Also, in some deployments connections are even closed if there is no recent traffic. For example; running a server behind and AWS Elastic Load Balancer will close idle connections (default is 60 seconds but it can be increased to a maximum of 4000 seconds). Ngnix (by default) will also close idle connections.

Clients would need to reconnect and resubscribe to resources its interested in. It would be great if this was in the specification about how a client should recover in situations like this.

For example; On a WS close event the client could attempt to reconnect to the WS endpoint a number of times with an exponential back-off time (1, 2, 4, 8, 16, 32 seconds) in between each attempt. If the connection is restored it will send the subscribe message to its interested resources.

Also, guidance on how to keep WS connections open would also be valuable. Making sure the Client and Server sends Heartbeat message would prevent idle connections. See:

https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API/Writing_WebSocket_servers#Pings_and_Pongs_The_Heartbeat_of_WebSockets

https://tools.ietf.org/html/rfc6455#section-5.5.2

All of this would help improve notifications been missed by a client application.

In the bit about live update,

https://github.com/solid/solid-spec/blob/master/api-websockets.md

we need to specify that you can subscribe to changes in a resource which currently does not exist, and get a ping when it is created (and thereafter to any changes).

The headers Update-via and `MS-Author-Via: are sent on 404 as well as 200.

NSS sends the headers now. i haven't tested the actual WS ping message on file creation.

See also

linkeddata/rdflib.js#392 in rdflib

which is depended on by

https://github.com/solid/solid-ui/issues/228 a issue with the chat UI needing this.

The term features should be renamed to feature, as is customary for RDF predicates.

The solid spec must define the whole protocol, including how the auth plugs into the notification. Propose that this document does define that. Otherwise we need another parallel document alongside this. The spec must completely define exactly ow to build a solid server.

Originally posted by @timbl in solid/notifications-panel#3 (comment)

Add Change Log section to Notifications Protocol that includes substantive changes or new features per significant publication, e.g., the next publication of Notifications Protocol will summarise changes since latest published version (which is https://solidproject.org/TR/2022/notifications-protocol-20220509 as of this writing).

It can be along the lines of https://www.w3.org/TR/ldn/#change-log (follows publication rules for W3C specifications).

I'm starting this issue as somewhere to capture comments on the spec that have come up whilst I was looking at building conformance tests.

1. Without a defined discovery mechanism we obviously can't test section 2.1 but also have to potentially support implementations' own versions of discovery in order to test any further in the spec. The sooner this can be added, the better for testing purposes.
2. The spec:statement at https://solid.github.io/notifications/protocol#server-subscription-access-controls does not contain the start of the requirement.
3. How does https://solid.github.io/notifications/protocol#server-subscription-access-read "A client MUST be permitted to read the resource to which it subscribes" add anything to the previous requirement? If it is simply saying you can only subscribe to a resource to which you have access, then I think the sentence doesn't parse well as it almost suggested that subscribing grants access. Perhaps "A client MUST have read access to a resource in order to subscribe to it"

Then I went on to look at the WebsocketSubscription2021 spec: https://solidproject.org/TR/websocket-subscription-2021. This spec says an API must conform to the notification protocol spec above, then lists some requirements but I think there are some issues:

1. https://solidproject.org/TR/websocket-subscription-2021#client-subscription-type duplicates the requirement in the notification protocol spec and is therefore redundant
2. https://solidproject.org/TR/websocket-subscription-2021#client-subscription-feature is also a duplication but additionally the IRI of the requirement is client-subscription-feature where I think it should be client-subscription-topic
3. I think the language of this requirement could be clearer as it is in the notification protocol spec but if it is removed as redundant that solves that concern.

https://solid.github.io/notifications/protocol#subscription

This section only describes a payload of Subscription Request. At the same time, various subscription types describe requirements on various Subscriptions Responses.

I think protocol should mention Subscription Response and shortly explain their role and mention that each subscription type will specify details of Subscription Response.

Action of https://github.com/solid/notifications-panel/blob/main/meetings/2022-03-17.md#is-the-solid-notifications-protocol-spec-title-too-broad

Is the "Solid Notifications Protocol" spec title too broad?
Within the scope of panel, we have a charter, we do stuff around notifications. Current protocol seems to be focused on subscriptions. Maybe we need to come back to use cases and see if any of them don't fit under the current spec.

Related to solid/specification#50, if WebSocket support becomes part of the Solid specification, it would be helpful to clarify whether the subscription mechanism is recursive. That is, if a client subscribes to the resource at /foo, will the client receive notifications for all (recursively) contained resources?

Another way to ask this is, given a subscription for /foo, which of the following actions will trigger a notification for that particular subscription?

• A new resource is created in the /foo container (e.g. /foo/bar)
• The child resource /foo/bar is updated
• A new resource is created at /foo/bar/baz
• The resource /foo/bar/baz is updated

There is the Push API. Since the Solid Protocol has a section on Live Updates it would be useful to understand how these two complement each other, work together, what ideas can be taken from it, etc...

Some resources:

• the blog post Push Notifications on the Open Web
• the IETF draft Generic Event Delivery Using HTTP Push (RFC 8030 Proposed Standard)

Any thoughts?

There current way the client subscribes to notifications of a document being changed has security issues in that the web socket URI could be guessed. Change this to be more secure.

Here are a number of review notes on the notificatiom protocol document.

At present, the Notification Protocol defines a gateway API that acts as a discovery endpoint for clients: agents submit POST JSON-LD requests and receive JSON-LD response bodies. This is an unauthenticated endpoint.

Rather than providing a required request-response gateway API, it would arguably be simpler do just define the subscription types that are supported in a single RDF document. Clients can fetch that RDF document and, given the features supported by the various subscription types, the client can simply choose which path to follow.

One can argue that the current gateway API model already has such an RDF document implicitly and the HTTP API brokers access to that document. Removing that API has several advantages:

• The server implementation provides what is, effectively, a static document, thereby making the implementation considerably simpler
• Any client code needs only to fetch and parse a single resource and can then decide on its own what subscription type(s) it uses, which arguably means fewer request/response cycles for the client

In the process of linking the vocabulary terms to relevant locations in the specification, I have found that a number of terms that are used in the protocol specification are left undefined. These include:

• notify:subscription
• notify:notificationChannel
• notify:feature

There should be normative definitions for each of these.

We need to find a way for a server to be able to declare interest in a resource changing in some way.
This would require each resource to point to a resource where a client can subscribe to changes in the resource, perhaps using SPARQL, with a pointer to a container to send notifications to when those changes have been triggered. Probably requires some form of authentication of the user, to verify that the notification box is the right one.

The 0.8 Solid spec describes a WebSocket subscription protocol, but that protocol does not include any mention of authentication or authorization. If WebSockets are to be part of the 1.0 Specification, there should be some formalization of the authentication protocol as well as a description of how these notifications intersect with WebAC rules.

It should be noted that WebSockets are raw TCP sockets. HTTP headers are not relevant, and so reliance on an Authorization header will be insufficient.

I think we should clarify the assumptions we make about how solid storage hosting resources (topics) relate to party hosting notification source or delivering the notification to the target.

I would like to know if at any point we assume that the same solid storage will host the subscription endpoint (or deliver notifications to the target), or it could be done by a distinct party.

I think details of how that party interacts with solid storage could be left up to implementations. It could be still worth assuming that those two could be two distinct parties.

Although it seems to me that subscription resource is a long used and pretty set in stone term,
I would like point out that the naming may be confusing:

From the term alone, one could get the idea that a subscription resource is a resource whose representation describes a subscription. Especially, when modelling the resource where requests are submitted to as a container (#36), or even more, when considering materializing requests for subscriptions in such a container.

In my mind, what is currently referred to as a subscription resource is a resource whose representation describes a service that provides the subscription/notification functionalities. (independent of that resource being also a container or can be POSTed to)

Thus, I would like to propose the terms subscription service resource which is described by its subscription service representation.

Consider whether it is useful to define subscription service as a container. Originally mentioned in #29 (comment) :

(I love and hate this idea.) Change the subscription [service] from a non-container to a container. It will work with existing behaviour on POST to container in Solid Protocol. This will open up the possibility for a server to track/record subscription requests (as notifications) in and of itself. The immediate concern with subscription [service] as a container is that if the description is to be readable (200 to GET, HEAD as mentioned above), then the container would reveal containment statements - a "feature" we definitely don't want as is. To get around this, essentially there needs to be a requirement that would, if at all, only allow the owner of the storage to have access to these subscription requests. Alternatively, a requirement that would allow servers to omit containment statements. On the other hand, if GET, HEAD to subscription [service] is a 405, we don't have that problem.

Add a mention that existing Sold servers and clients implementing a notification protocol based on web sockets, and this being used for example for chat applications, collaborative editors, etc. It is disingenuous to write as though this is not currently already running. This protocol serves to provide a more secure and more modular version of the same functionality.

Originally posted by @timbl in solid/notifications-panel#3 (comment)

Suggest a level 2 protocol in which the triples which changed in an RDF resource are included in the notification.

Originally posted by @timbl in solid/notifications-panel#3 (comment)

How does one unsubscribe to watching a resource without terminating the websocket connection itself?

Introduce the notion of Notification Profile as an extension of the core Notification Data Model.

Specific Notification Profiles can then be independently published.

Add a Conformance Classes section listing the roles eg. Notification Server, Client, that the Solid Notifications Protocol specifies. The criteria for each of the roles can be described throughout the specification.

The Notifications Protocol currently does not describe the discovery of the Notification Subscription Metadata Resource.

This issue is a follow-up on the discussion at #54 . Some options are mentioned there in addition to our earlier consideration to use Server/Storage Metadata: solid/specification#355

As discussed after the meeting last week (7 July 2022), I would like to propose an alternate design for Solid Notifications.

Before I even begin, a few disclaimers are in order:

• I am coming at this solely from a client's perspective.
• Apologies, in advance, for getting any nomenclature wrong (the language is in flux anyway).
• What little experience I have is with WebSockets and that is the perspective I take.

Background

The current Solid Notification Protocol requires a number of discovery and initialization steps before we can actually start receiving any notifications:

1. Obtain the link header pointing to the Storage Metadata Resource from a Solid resource.
2. Discover Notifications Channel and Capabilities for the Subscription/Channel Type from the Storage Metadata Resource.
3. Place a request for a Notifications (say, WebSockets) Endpoint with desired capabilities/features (and resource? resources?).
4. Initiate a Notifications (again WebSockets) connection on the received endpoint. (Are these Endpoints reusable? Can they be cached?)

These are clearly a lot of round-trips before we can actually start receiving notifications.

My other concern is with Internet outages (especially micro-outages), which are extremely common where I live (even on broadband, let alone mobile). Going through even an extra step to recover a disrupted WebSocket connection is going to lead to a poor user experience. See @acoburn comment below.

Further, it is not specified yet, if we shall have one channel per resource or one channel per client. Even in the latter case (one channel per client), if the client wants to add or remove resources and/or change notification features like extend subscription duration, it must re-negotiate a new endpoint first using HTTP. While this might be necessary for uni-directional protocols, it makes little sense to carry this over to a bi-directional protocol like WebSockets.

Proposal

1. Provide a generic endpoint for the most common way to connect, which I believe would be WebSockets, as a header on every resource (like we do now).
2. A client opens a WebSocket connection using the generic endpoint.
3. As soon as the connection is established, the client sends authentication data, features and the list of resources they wish to subscribe to, all in a single message. (Think of it like a GET to multiple resources on a Solid pod and with an endless response)
     a. The server may close the connection due to a failure to authenticate.
     b. The server may re-negotiate features.
     c. The server starts sending notifications (perhaps the first message contains an 'accept').

Changes of features, addition or removal of resources can all be done in band through a client message, without the need to initiate new connections.

Even if we choose to go with one channel per resource, the operation is no more complex than a new connection & GET like request.

Benefits

1. Faster discoverability + caching of endpoint.
2. Fewer round trips.
3. The WebSocket endpoint is not tied to connection properties and/or resources. Thus, we can renegotiate features, add or remove resources without stepping out of band.
4. Different properties can be specified for different resources within the same connection and perhaps even the same message (such as on start-up).
5. One connection per server/pod only; client has fewer connections to manage/recover in case of outage.

@uvdsl already has a symmetry objection to this proposal. While I have hinted at it already, I shall defer to him for a full explanation.

I also invite other Subscription Type authors to consider how their proposals may be affected.

Action of https://github.com/solid/notifications-panel/blob/main/meetings/2022-04-28.md#url-of-notifications-protocol-subscription-types

Index of Subscription Types that are published under https://solidproject.org/TR/

As only a single channel will be returned, I suggest that the request lists the types in preferred order just in case there is multiple overlap. In this way, the client gets their preferred type that is supported by the server.

Originally posted by @kevin-inrupt in solid/notifications-panel#3 (comment)

LinkedDataNotificationsSubscription2021 uses webid in the subscription response explaining:

All LDN notifications will be sent to the defined inbox using the agent identity specified by this WebID. A resource owner should ensure that the agent identified in this property is permitted to send authenticated HTTP requests to the defined inbox.

I think WebHookSubscription2021 and all target flow types, in general, could use this as a common approach.

@jaxoncreed what do you think about using this approach instead of defining custom AuthN?
https://github.com/solid/notifications/blob/main/webhook-subscription.md#10-webhook-request-with-token-signed-by-the-pod-key

Diagram should be clarified (content and visuals) to better distinguish between what's defined by the Notifications Protocol from the components/interactions defined elsewhere.

In the Notifications Protocol, the description of a notification channel includes its type, feature, and subscription. The value of the type is expressed as the same value as the subscription type. The question is whether the subscription type, e.g., WebSocketSubscription2021 is misleading or misused here as part of the notification channel type.

For an ecosystem that is based on capability/feature discovery, it is arguably better to think less about "endpoints" and more about "hypermedia APIs". As such, I would propose removing instances of 'endpoint' in this specification and replacing them with either 'API' or 'URI', as appropriate.

It may also be worth mentioning that a client should be able to trust that the notifications it receives aren't forged.

Originally posted by @justinwb in solid/notifications-panel#3 (comment)

Specs need both the version number, like 1.0 or 3.4.5, and the maturity like "draft, "Editor's Draft", "Recommendation".
Suggest publish this as say [Editor's] Draft of Version 1.0 of the Solid Notifications Protocol.

Making the version number < 1 messes up semver.

After 1.0, increment the major version for incompatible changes, minor for back-compatible enhancements, the patch level for bug fixes and wording improvements.

The version number is about the protocol. The maturity level is about the document and the community.
The Solid Editors came to this conclusion but maybe have not written it up.

As discussed in notifications-panel meeting on January 20, we need a consistent naming template for subscription types.

Currently, the name are opaque strings in the format:
<SubscriptionTypeName><Year>

This naming scheme is inadequate in my view, say, when there are multiple revisions to a given subscription type in the same year. A proposed naming scheme must make it simple for clients (actually client developers) to identify available subscription types.

Also, having a naming scheme provides clear guidance to implementers of new subscription types, as to how to name their implementations and revisions.

Current draft states:

Authentication and Authorization

The Subscription API requires authorization. This document does not define the specific technology used to authorize requests; rather, it defers to the Solid Protocol sections on Authentication and Authorization. It is out of scope for this document to describe how a client fetches an authorization token. Solid-OIDC is one example of an authentication mechanism that could be used with Solid Notifications.

Since it only mentions Subscription API, I assume that each subscription type will define details for how to handle AuthN / AuhZ?

Further in the document, I see:

Notification Subscription

The details of subscribing to a particular endpoint depend on the technology used, but all will follow a similar pattern. For WebSockets, this involves sending an authenticated subscription request to the endpoint retrieved from the gateway service.

It seem to reaffirm that each subscription type will define how to handle authentication and authorization

One of the advantages of defining subscription type based on streaming HTTP response (e.g. Fetch Response.body.getReader) comes with using the same mechanism relying on HTTP headers.

I think we should also consider the option to negotiate usage of Capability URL for Subscription Endpoint. IMO this could save some complexity with WebSockets subscription type.

The WIP definition of notify:source in the RDF vocab

The IRI of a resource used by a client to establish a connection to a notification channel.

is suitable towards general purpose use, however, it is tied to / isDefinedBy https://solidproject.org/TR/websocket-subscription-2021#websocketsubscription2021-source (noting that we have an understanding in the PR to use isDefinedBy/seeAlso that will change this eventually).

But the definition in WebSocketSubscription2021 is actually constrained to wss.

Essentially, either notify:source needs to be reserved for WebSocketSubscription - and if so, that's fine and we need to be more clear - or we leave it open for use by subscription types.

The proposal in the PR is that the term should be tied to the Notifications Protocol, which would be something like e.g.:

isDefinedBy http://www.w3.org/ns/solid/notifications#
seeAlso https://solidproject.org/TR/notifications-protocol#notify-source

#notify-source will be described under Terminology (or possibly another section that may be more suitable later.)

Originally posted by @csarven in solid/vocab#62 (comment)