The WebGL use cases, and interaction with the WEBGL_video_texture extension, should be filled out more in this explainer.

One still missing field in the VideoFrameMetadata dictionary is remote capture time (e.g. "int64 remoteCaptureTime".

The rationale is as follows: in WebRTC scenario the video originates at a separate machine, which has a separate clock. There might be clock offset or even drift.

WebRTC estimates capture time in both local and remote time-bases. It's logical, that capture time in local clock is already intended to be in captureTime field. This can be used to measure E2E delay.

But the same capture time in remote clock is very useful for synchronization purposes. Suppose the application wants to e.g. render emojis on top of the video or show chat messages, synchronized with the source video. The send side can easily obtain current local clock and attach that as a label to the chat message. Then receiving side could use remoteCaptureTime to decide if this is the frame suitable for showing the message, because they would be in the same timebase.

As this remoteCaptureTime is not in local clock it doesn't even have to be a timstamp type, just some monotonically increasing number, presumably in milliseconds at the send machine should work.

This field will duplicate rtpTimestamp a little, but it's quite different: it's not present for all the frames, unlike rtpTimestamp; It doesn't wrap and can be simply compared with the labels associated with a extra synchonizable content.

Currently the documentation for presentationTimestamp says:

The presentation timestamp in seconds of the frame presented. May not be known to the compositor or exist in all cases.

For a use case I have in mind, the presentationTimestamp is the field I care about the most (in fact, it's the only field I care about). I would like it to specified under which circumstances this field would not exist as it directly impacts the usefulness of this method to me.

If the function passed to video.requestAnimationFrame doesn't complete before expectedPresentationTime, will the frame still be presented, or would it be delayed until the function completes?