Send Picture Loss Indication (PLI) when a keyframe is needed about aiortc HOT 6 CLOSED

I want to know why we do not send FIR when a keyframe is needed.

from aiortc.

Regardless of whether we send FIR or PLI, the critical missing bit is detecting we need it - which I haven't had time to do yet. If you have time for a dive into the WebRTC code I'd be interested in a list of the criteria they use for this.

from aiortc.

I want to know why we do not send FIR when a keyframe is needed.

I noticed some browsers ignore the FIR and do respond with a keyframe to a PLI

from aiortc.

From reading the webrtc.org codebase, there seem to be multiple triggers for a PLI:

• if the NACK list grows beyond 1000 packets

https://webrtc.googlesource.com/src/+/05cb48599918d0089175a37d4ac716b4ceac8060/modules/video_coding/nack_module.cc#236

• if the video codec requests it:

https://webrtc.googlesource.com/src/+/05cb48599918d0089175a37d4ac716b4ceac8060/video/video_receive_stream.cc#473

• if no keyframe has been received for a long time:

https://webrtc.googlesource.com/src/+/05cb48599918d0089175a37d4ac716b4ceac8060/video/video_receive_stream.cc#479

• if decoding fails (I think):

https://webrtc.googlesource.com/src/+/05cb48599918d0089175a37d4ac716b4ceac8060/video/rtp_video_stream_receiver.cc#395

• some H.264-specific condition:

https://webrtc.googlesource.com/src/+/05cb48599918d0089175a37d4ac716b4ceac8060/video/rtp_video_stream_receiver.cc#267

from aiortc.

I had a deeper look at the code, and it looks like simply sending out a PLI is not going to be enough. In addition the jitter buffer code would need to be reworked to discard packets intelligently, otherwise it will keep waiting for the missing packets until it eventually overflows. In fact I'm not entirely convinced we can keep using a single jitter buffer implementation for audio and video. The webrtc.org codebase seems to use two distinct implementations.

from aiortc.

@Przem83 fixed this in 42f4e0f

from aiortc.