Comments (6)
It's also not clear to me what the exact behavior is when a large number of IDs are outstanding. There is only so much room to store references, eventually the encoder has to drop them. How much latency can there be before these schemes fall apart? How does this vary between encoder implementations?
[BA] The viability of LTR in conferencing scenarios relies on assumptions about how participants manage long-term references. They're called long-term references because the participants need to keep them long-term or else an encoder can't be assured that it can rely on an LTR to create a delta frame that all the conference participants will be able to decode. And of course, regardless of how long LTRs are kept, you can have a recent joiner who won't have received the LTR. So it is quite possible for an encoder to create a delta frame based on an LTR and then receive a PLI in response. Ooops!
from webcodecs.
Related: #285, aboba/hevc-webrtc#17, w3c/webrtc-nv-use-cases#118
from webcodecs.
We (Chrome implementers) discussed this recently. It's possible, but there are some design considerations that need to be solved:
- Not all encoders support it (eg. MediaCodec), so it needs to be feature-detectable.
- Different encoders use different signaling mechanisms, so we probably need to find a way to unify them all.
- Rather than creating serializable IDs, we could use opaque objects and leave mapping/serialization up to apps.
- Putting IDs on encoded chunks is ambiguous since there can be more than one frame in an encoded chunk. In practice this feature probably won't be used together with frame reordering, and if superframes do get involved it's probably fine to interpret a single ID as representing all frames in the chunk.
It's also not clear to me what the exact behavior is when a large number of IDs are outstanding. There is only so much room to store references, eventually the encoder has to drop them. How much latency can there be before these schemes fall apart? How does this vary between encoder implementations?
from webcodecs.
We (Chrome implementers) discussed this recently. It's possible, but there are some design considerations that need to be solved:
- Not all encoders support it (eg. MediaCodec), so it needs to be feature-detectable.
Are you suggesting to add something to be checked by MediaCapabilities API?
- Different encoders use different signaling mechanisms, so we probably need to find a way to unify them all.
- Rather than creating serializable IDs, we could use opaque objects and leave mapping/serialization up to apps.
- Putting IDs on encoded chunks is ambiguous since there can be more than one frame in an encoded chunk. In practice this feature probably won't be used together with frame reordering, and if superframes do get involved it's probably fine to interpret a single ID as representing all frames in the chunk.
It's also not clear to me what the exact behavior is when a large number of IDs are outstanding. There is only so much room to store references, eventually the encoder has to drop them. How much latency can there be before these schemes fall apart? How does this vary between encoder implementations?
I may create prototype CL in Chromium to examine the behavior on Windows at least.
from webcodecs.
Are you suggesting to add something to be checked by MediaCapabilities API?
More likely a flag in VideoEncoderConfig
so that VideoEncoder.isConfigSupported()
can be used to detect it.
from webcodecs.
sandersdan@ max number of outstanding long term ref picture indices(as well as max value of long_term_ref_frame_idx + 1 ) must not exceed maximum possible reference pictures (for AVC/HEVC, it is 16);
If application explicitly specifies maximum reference frame numbers, that will be further reduced to application specified limitations.
from webcodecs.
Related Issues (20)
- Support for AV1 switch frames HOT 5
- Fatal encoding/decoding error on Windows 10/11 HOT 2
- Why is isTypeSupported() promise-based? HOT 8
- Receiving Uncompressed Webcam Data without Browser Compression HOT 5
- After how many decode should the codec process the frames? HOT 10
- Clarify `reset()` behaviour when multiple things are being output HOT 3
- key-frame request handling when scalability mode is not L1T1 for encoder HOT 6
- Clarify the `Clone configuration` algorithm HOT 4
- Figure out what should happen to the unused bits in 10-bits and 12-bits pixel formats
- Rephrase non-normative uses of RFC2119 keywords HOT 1
- Web Audio API compatibility HOT 1
- Assign VideoFrame resource to [[resource reference]] in BufferSource constructor HOT 4
- VideoFrameBufferInit metadata field missing
- sourceWidthBytes from sampleWidth, not sampleHeight
- Vekil
- Sporadic build failures HOT 4
- Define scope for w3c candidate recommendation HOT 3
- Candidate Recommendation tracking issue
- VideoFrame copyTo() behavior with non-RGBA/RGBX/BGRA/BGRX formats HOT 1
- VideoPixelFormat enum values do not follow casing rule guidelines HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from webcodecs.