traud / asterisk-opus Goto Github PK

Hi,

The documentation points to:
github.com/traud/asterisk-opus/archive/master.tar.gz

However, the master branch has not had a merge from 13.7 hence the install instructions do not work. So I suppose it needs a merge or the docs should be updated to download a different file instead.

Cheers!

Hi!

Just saw that the Asterisk team made a patch:

https://code.asterisk.org/code/viewrep/asterisk/res/res_format_attr_opus.c?r1&r2=2c031b67d3c7da83bab914bcdaad45d2a0bc9ff8

Is this patch applicable to this repo as well? Just asking since they seem to be using your code elsewhere.

Notes:
res_format_attr_opus: Fix fmtp generation.
res_format_attr_opus assumed that the string being passed into it was
empty. It tried to determine if the only thing it had written was

a=fmtp:

And if it had, it would reset the string. Its calculation was off when
working with chan_sip, though. chan_sip passes the entire built SDP
rather than an empty string. This resulted in always putting an empty
fmtp line in the SDP.

When applying enable_native_plc.patch , compilation of Asterisk 15 breaks with the following error:

   [CC] translate.c -> translate.o
translate.c: In function 'ast_translate':
translate.c:577:13: error: 'p' undeclared (first use in this function)
   for (tp = p; tp; tp = tp->next) {
             ^
translate.c:577:13: note: each undeclared identifier is reported only once for each function it appears in
make[1]: *** [translate.o] Error 1
make: *** [main] Error 2

It looks like the rewrite of the internal p variable in ast_translate() to step was not a good idea, because future versions for Asterisk may refer to this p in new code.

Hi, I'm using vp8 format to write frames into a file. It works great but I'm having issues to read this file or encode it into a playable format. Do you know an approach to do that?

for 100 calls it using 3 cores. can we optimize it for the same?

When Asterisk started the call but the remote party sends a re-INVITE on the SIP layer, the call might face one-way audio. One such a call scenario is call hold, for example. One-way audio happens because this codec here uses a dynamic RTP payload type in SDP negotiation, which Asterisk 13 LTS does not support correctly. Complete details are documented in ASTERISK-27056. Even if you are not affected, applying the patch provided there is recommended for any user. If you face an issues with that patch, report in ASTERISK-27056, please. If possible, consider upgrading to a newer Asterisk branch.

please I need your help, I am new with Ubuntu

You merged in a change to make this work on 14 but the readme doesn't tell you about it and the branch it was merged into was 13.7 ? Should all this be tidied up?

Several SIP ↔︎ DECT bases exist on the market, and the latest ones allow the Opus Codec:

• Grandstream DP750 (tested)
• Grandstream DP752
• Panasonic KX-TGP700 (tested)
• Poly Rove B2
• Poly Rove B4
• Snom M900 with DSP module A900 (tested)
• Yealink W60B
• Yealink W70B (tested)

If I am not misinformed, DECT is limited to only some audio codecs like G.722 for wideband and G.726-32 for narrowband, which means the DECT base always transcodes when it uses the Opus Codec. This has unexpected results:

Yealink
(tested 85.0.20) The decoder in the Yealink supported all my tested bandwidths. Incoming calls can trigger wideband or narrowband through their SDP offer; to be more exact: the SDP format parameter (fmtp) maxplaybackrate=8000. Outgoing calls/audio are always wideband (SiLK-only Wideband; ToC 0x48) as the SDP answer from the remote party gets ignored. This behavior is in line with the RFC. Therefore, I did not report it. However, I did not even need to measure the latency; a great difference thanks to the Opus Codec was hearable!

Panasonic
(tested 01.002) The decoder in the Panasonic supported all my tested bandwidths. Incoming calls can trigger wideband or narrowband through their SDP offer; to be more exact: the SDP format parameter (fmtp) maxplaybackrate=8000. Outgoing calls/audio were CELT-only; probably not OPUS_APPLICATION_VOIP but OPUS_APPLICATION_RESTRICTED_LOWDELAY is configured as application, which is wrong. Although Opus is visible as enabled in the Web interface, it is not offered. Instead, via a configuration file, you have to set OPUS_BAND_TYPE_x="0" (for narrowband) or WIDEBAND_AUDIO_ENABLE="Y" (for wideband): Maintenance → Provisioning → File URL. If configured for narrowband, every SDP offer with maxplaybackrate higher than 8000 gets rejected with SIP status 488, which is wrong. On default, the Opus encoder stops to work with the second received call. In other words, the other party does no hear anything. In my Panasonic, I went for the ‘multi-number mode’ and this software bug was not heard again.

Snom
(tested 05.30) The decoder and encoder in the RTX8663 work for me only with ToC 0x48 (SiLK-only Wideband). With ToC 0xb8 (CELT-only Wideband), I got a buzzing noise in the background. With other configurations like Medium- and Narrowband, I got no audio at all. This behavior was reported …

Grandstream
(tested 1.0.15.6) The decoder shows limitations …

Conclusions
The following patch always produces SiLK-only WB, ensuring an RTX DECT base handles the received Opus Codec stream: rtx.patch. Looking at the number of issues (and I did not test Poly yet), I recommend limiting the audio codecs of such a DECT base to its native audio codecs (G.722 and G.726-32) and instead let a media-plane back-to-back user agent (B2BUA) in your network do the transcoding.

fatal error: opus/opusfile.h: No such file or directory
The source from http://opus-codec.org/ doesn't have this file as well.

Snom added the Opus Codec for their Desk Phones in the year 2014. At that time RFC 7587 was not final and the Snom implementation based on a draft from 2012. Therefore, the rtpmap was not opus/48000/2 but without the channel: opus/48000. Furthermore in the year 2012, the fmtp parameter for the bandwidth was called maxcodedaudiobandwidth and not maxplaybackrate.

Recently, those issues got fixed. However, Snom Desk Phones still play choppy audio when the Opus Codec is negotiated and Asterisk transcodes from another audio format to Opus.

Snom uses opus-wb to send wideband audio (16000 kHz) and opus-nb to send narrowband audio (8000 kHz). RFC 7587 Table 2 states that for 20 ms the RTP timestamp should be increased by 960 samples, always. However, Snom increases the timestamp by

• 160 samples with opus-nb and
• 320 samples with opus-wb.

Snom does not only send but expects to receive such timestamps as well. Otherwise the audio is choppy. Asterisk increases by 960. In Asterisk, you can hot fix this by changing res/res_rtp_asterisk.c:ast_rtp_raw_write from

if (frame->frametype == AST_FRAME_VOICE) {
	pred = rtp->lastts + frame->samples;

	/* Re-calculate last TS */
	rtp->lastts = rtp->lastts + ms * rate;
	if (ast_tvzero(frame->delivery)) {
		…
	}
 }

to

if (frame->frametype == AST_FRAME_VOICE) {
	rtp->lastts = rtp->lastts + ms * 16;
}

for opus-wb. For opus-nb, instead of 16, you go for 8. That gives clean audio with Snom Desk Phones and proves the correctness of this approach. In the year 2014, a similar mistake was done by CSipSimple.

Currently, these software branches and head versions are affected:

• 8.7.5.35 is the current release version (Nov. 2015).
• 8.7.5.44 is the current beta version (Mar. 2016).
• 8.9.3.57 is the latest version of a new branch (Jan. 2017).

Because Snom has to solve this software bug and I see no way to solve it from within Asterisk, I reported this to Snom. Here, I leave this issue report open until this interoperability issue is fixed by Snom.

Hi @traud would you mind submitting asterisk.patch to gerrit? We talked about it on IRC, the configure script already checks for opus and since Asterisk itself won't link to opusfile it should be fine.

My goal is to remove this need for non-developers to run ./bootstrap.sh. I don't actually use opus (might in the future), this request is in response to a post on asterisk-dev mailing list.

Hi,
By default when I use the Opus codec the Packetization time of the RTP packets is 20 ms and If I try to change this time from 20 to 60 ms by adding the allow=opus:60 entry in the Asterisk's sip.conf then the SDP is getting updated but the actual RTP packets are still transmitting at 20 ms interval.
Also, the timestamp isn't getting updated from 960 (48 * 20) to 2880 (48 * 60).

so would you be able to tell me how can this be resolved?

Upon using Opus with Asterisk 14.7.X or Asterisk 15.5.0 an issue is exhibited, where the codec will simply stop transcoding in a certain direction. For example:

UA (A) ----- Opus -----> Asterisk ---- ulaw -----> PSTN Phone (B)

In the above case, after an unknown period of time the audio path from (A) to (B) is no longer
audioable, while the path from (B) to (A) is still available.

A debug of Asteris and 'rtp set debug on' produced the following output:

[Aug 20 19:56:10] DEBUG[13263][C-00000002]: translate.c:756 ast_translate: Sample size different 960 vs 160
Sent RTP packet to 167.99.150.244:14962 (type 08, seq 034158, ts 1722888, len 000160)
Got RTP packet from 167.99.150.244:14962 (type 08, seq 030122, ts 4209377896, len 000160)
[Aug 20 19:56:10] DEBUG[13263][C-00000002]: translate.c:756 ast_translate: Sample size different 160 vs 960
Sent RTP packet to 54.210.151.172:36190 (type 107, seq 029180, ts 10333536, len 000128)
[Aug 20 19:56:10] DEBUG[13263][C-00000002]: translate.c:756 ast_translate: Sample size different 960 vs 160
Sent RTP packet to 167.99.150.244:14962 (type 08, seq 034159, ts 1723048, len 000160)
Got RTP packet from 167.99.150.244:14962 (type 08, seq 030123, ts 4209378056, len 000160)
[Aug 20 19:56:10] DEBUG[13263][C-00000002]: translate.c:756 ast_translate: Sample size different 160 vs 960
Sent RTP packet to 54.210.151.172:36190 (type 107, seq 029181, ts 10334496, len 000099)
[Aug 20 19:56:10] DEBUG[13263][C-00000002]: translate.c:756 ast_translate: Sample size different 960 vs 160
Sent RTP packet to 167.99.150.244:14962 (type 08, seq 034160, ts 1723208, len 000160)
[Aug 20 19:56:10] DEBUG[13252][C-00000002]: res_rtp_asterisk.c:5457 ast_rtcp_interpret: Got RTCP report of 64 bytes from 54.210.151.172:36191
[Aug 20 19:56:10] NOTICE[13252][C-00000002]: translate.c:594 ast_translate: 10996 lost frame(s) 41118/30121 (slin@48000)->(opus@48000)

Now, after that happens, we shortly see the following:

Sent RTP packet to 54.210.151.172:36190 (type 107, seq 029219, ts 10370976, len 000128)
[Aug 20 19:56:11] NOTICE[13252][C-00000002]: translate.c:594 ast_translate: 24380 lost frame(s) 0/41154 (opus@48000)->(slin@48000)
Got RTP packet from 167.99.150.244:14962 (type 08, seq 030162, ts 4209384296, len 000160)

Which turns into:

[Aug 20 19:56:11] WARNING[13252][C-00000002]: channel.c:1136 __ast_queue_frame: Exceptionally long queue length queuing to SIP/subscribers.xxxxxxxxx.com-00000002
Sent RTP packet to 54.210.151.172:36190 (type 107, seq 029220, ts 10371936, len 000128)
Sent RTP packet to 54.210.151.172:36190 (type 107, seq 029221, ts 10372896, len 000128)
Sent RTP packet to 54.210.151.172:36190 (type 107, seq 029222, ts 10373856, len 000128)
Sent RTP packet to 54.210.151.172:36190 (type 107, seq 029223, ts 10374816, len 000128)
Sent RTP packet to 54.210.151.172:36190 (type 107, seq 029224, ts 10375776, len 000128)

Which at that point loses audio completely from (A) towards (B).

The issue is re-produceable and happens constantly, however, will happen at different intervals.

Building an external shared module (with the Makefile opposed to patching the Asterisk source) does not work anymore with Asterisk 16. It probably doesn't work starting with Asterisk 14, as this is apparently the version where the very very lightly documented requirement for AST_MODULE_SELF_SYM has been introduced.

https://issues.asterisk.org/jira/browse/ASTERISK-26278
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=919828

gcc -o codecs/codec_opus_open_source.so  -DAST_MODULE=\"codec_opus_open_source\" -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/<<BUILDDIR>>/asterisk-opus-13.7+20171009=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -g3 -O3 -lopus -shared -Wl,-z,relro codecs/codec_opus_open_source.c
In file included from codecs/codec_opus_open_source.c:39:
/usr/include/asterisk.h:232:2: error: #error "Externally compiled modules must declare AST_MODULE_SELF_SYM."
 #error "Externally compiled modules must declare AST_MODULE_SELF_SYM."
  ^~~~~
In file included from codecs/codec_opus_open_source.c:56:
codecs/codec_opus_open_source.c: In function 'load_module':
codecs/codec_opus_open_source.c:833:8: error: 'AST_MODULE_SELF' undeclared (first use in this function); did you mean 'AST_MODULE_INFO'?
  res = ast_register_translator(&opustolin);
        ^~~~~~~~~~~~~~~~~~~~~~~
codecs/codec_opus_open_source.c:833:8: note: each undeclared identifier is reported only once for each function it appears in
make[1]: *** [Makefile:50: codecs/codec_opus_open_source.so] Error 1
make[1]: Leaving directory '/<<BUILDDIR>>/asterisk-opus-13.7+20171009'

Hi @traud,

first of all, nice work! I'm the author of the original asterisk-opus patch, and I'm really impressed by the improvements you guys have made to the code, not only to support more recent versions of Asterisk, but also on advanced features like integration with SDP negotiation and, more importantly, FEC. I haven't been able to test the FEC support yet, but I plan to do that soon, so I hope I'll have feedback on any fix that might be needed.

I'm opening this issue for something partially related to that, actually, that is some further improvements that could be added to the management of an Opus transcoding setting in Asterisk. Since this patch is currently in a more advanced state than our original patch, it makes sense for me to contribute and integrate changes here and, if needed, backtrack it to 11 in there too.

One fix that you may integrate right away, and that we've been testing with success for quite some time already, is the use of the OPUS_SET_COMPLEXITY control. This know allows you to tune the complexity of the encoder: higher level means high quality but, at the same time, a lot of CPU; lower levels get you worse quality, but much less CPU consumption. Since it's a 0-10 value, playing with that value dynamically allows you to try and find the right trade-off between audio quality and CPU usage, which is important since by default, IIRC, the library sets the highest value, and so a very high CPU usage especially when you handle several calls at the same time. In an experimental branch we've been using 4 as a value, which seemed as the best trade-off in terms of quality/CPU, at least for our conferencing usages. Using a #define or a configurable value might provide users with a way to decide for themselves which one is the best for them. The right way to set the value is like this:

opus_encoder_ctl(opvt->opus, OPUS_SET_COMPLEXITY(complexity));

Another more important aspect I'd like to address, though, is a more dynamic reaction to what is happening in the network, something that can be seen as a complement to FEC itself. More specifically, the way I conceived the patch at the time (and the way it still works, I believe) was to check what we were going to transcode to/from in terms of sampling rate, and set a cap in the encoder settings with OPUS_SET_MAX_BANDWIDTH accordingly. So, if we're talking to the PSTN, we set the maximum value to OPUS_BANDWIDTH_NARROWBAND, if we're talking to, e.g, speex16 we set it to OPUS_BANDWIDTH_WIDEBAND, and so on. Anyway, in our experience it looks like that, more than a maximum sampling rate, the encoder seems to stick to that for the whole call (unless, say, silence is sent), which can be troublesome if you're using OPUS_BANDWIDTH_FULLBAND because you're talking to a 48kHz endpoint but there are network problems and a lot of losses. This can be probably ascribed to the lack of feedback from the Asterisk RTCP engine, which leaves the module/encoder clueless on what's really happening on the wire, and so with no way to try and adapt to the current conditions.

By looking at what the library provides, I believe there are a couple of things that could be done were there any way to get RTCP stats, e.g.:

1. quite simply dynamically change the OPUS_SET_MAX_BANDWIDTH dynamically, e.g., lowering it whenever there's excessive loss and raising it again when things get back to normal;
2. configure another control dynamically, OPUS_SET_PACKET_LOSS_PERC, which informs the encoder on the expected loss on the bitstream we're generating, which in turn, according to the docs, should make audio a bit worse when there's no loss, but in general better when there is loss (probably favouring redundant data in FEC usage over regular data).

Both approaches should help get a better, and more dynamic, behaviour of Opus in live calls, and I don't think one should exclude the other. Anyway, this all starts from the assumption that the module needs to have some way of receiving feedback from the RTCP engine on the call (e.g., RTCP RR/SR), and possibly also from the other leg that Asterisk is bridging in this case, if any (the transmission might be fine between us and Asterisk, and packets be lost between Asterisk and our peer), although that may not be really needed (Asterisk is still going to be transcoding for both anyway). Some indication on loss could also be partially evinced by the module itself, e.g., by looking at sequence numbers pretty much the way you already do to trigger a FEC-based recovery, but that would be an incomplete perspective, as it would only tell us what's happening on incoming packets we're decoding, but nothing on what happens to what we send.

What is your take on this? If you believe this is an interesting beast to tackle, would you be willing to work together on finding a way to make this all work?

Cheers!

I tried this with naf419's asterisk 15 repository for inter-operation with Google Voice - branch=gvsip and it worked fine.

I am building a repository - sundarnagarajan/googlevoice-appliance with scripts to automate building a 'Google Voice Appliance' using a Raspberry Pi.

My scripts currently work perfectly on Ubuntu 18.04 Bionic server image, but lack Opus support. I am currently integrating your patches and adding support for Raspbian Stretch minimal image. You are welcome to take a look, use and contribute!

Thanks a lot!
You can close this issue.