high packet loss reported when audio is silenced about jitsi-videobridge HOT 11 CLOSED

chatting with emcho that's probably not a bug but a result of the rtcp termination strategy.

from jitsi-videobridge.

An audio RTP packet sent by Chrome contains the audio level of the samples carried by the RTP packet. Videobridge looks at the audio level upon the receipt of the RTP packet and, if the audio level indicates that the RTP packet was generated by a muted audio source, Videobridge drops the RTP packet in order to optimize the overall performance (e.g. the RTP packet is not even decrypted). In such a case the audio RTP packet does not even reach the RTP stack of Videobridge. Could that be the cause of the "packet loss"?

from jitsi-videobridge.

lyubomir: yes. I suppose the behaviour is ok for the default termination strategy even (and makes alot of sense in general. I'll see if I can document that somewhere (possibly in the rtcp termination strategy doc).

from jitsi-videobridge.

IMO this should not be reported as lost, it sounds more like a discarded. Since the audio is muted, it wouldn't matter to the congestion control. But it may result in a slow start when the RTP resumes and what interval the RTCP RR covers.

My other concern is that something in the bridge pipeline is doing DPI (or skipping ahead) and dropping a packet before it arrives into the RTP stack, ergo messing with the RTCP RR.

from jitsi-videobridge.

We drop silence packets before decryption so that they don't waste resources. I am not sure if we took care of bubbling up an empty packet when we do that but we should. We'll check this next week.

from jitsi-videobridge.

Another clarification (nothing to do with RTCP or indicated losses): why is the muted endpoint sending audio? isn't there VAD and Comfort Noise packets sent instead? Or is the bridge interpreting from client audio level that there is no speech?

from jitsi-videobridge.

As I said earlier in this issue, audio RTP packets marked by the sender as generated from a muted audio source are dropped and do NOT reach the RTP stack (in any form). Our RTP stack will discard 'empty' RTP packets as bad early in its processing anyway.

from jitsi-videobridge.

@lyubomir yes, that's also what I was saying. What I was referring to is that I think we would report the gap in sequence number as packet loss in the SRs/RRs that we send (right?). I think that ideally we should be:

a) not reporting these packets as lost because they aren't
b) potentially not reporting them as sent either but I am not sure if that can work well without seq num overwrites.

@vr000m

why is the muted endpoint sending audio?
The endpoint being Chrome you may want to ask the question on discuss-webrtc. Personally I don't see a strong case for not sending it. Muting is about sending silence by definition so I don't see a problem with this. But again, this is a Chrome matter and not a JVB one.

Or is the bridge interpreting from client audio level that there is no speech?

Yes. We are heavily relying on RFC6464 ssrc-audio-level for that sort of stuff (as for dominant speaker detection, last n, adaptive last n and simulcast).

from jitsi-videobridge.

What does the JVB do to the associated video? Is the RTCP RR related to the video also reporting losses?

I am looking at this from a sender-driven congestion control perspective, i.e., what will the sender do in response to the losses reported in RTCP RR (of both audio and video).

from jitsi-videobridge.

@vr000m

What does the JVB do to the associated video?

Nothing. At least not as a direct consequence. However, when last N is enabled, silence on a specific audio stream may lead to the corresponding video stream not being re-sent to some participants

Is the RTCP RR related to the video also reporting losses?

Hm? No, why would it do that? The video stream is being received and processed.

I am looking at this from a sender-driven congestion control perspective, i.e., what will the sender
do in response to the losses reported in RTCP RR (of both audio and video).

It may try to adjust its audio sending bitrate ... obviously this wouldn't be a good thing so we should look at it at some point, but I don't believe it's too much of an issue right now. I don't think endpoints do this that much for audio.

... it may also trigger a circuit breaker ... but given how I generally consider them a bad idea, I am not particularly worried about any problems that we may cause with those. ;-)

from jitsi-videobridge.

All related issues have been fixed and if there's something left feel free to enter a new specific ticket indicating only the remaining problem.

from jitsi-videobridge.