Comments (9)
Thanks for reporting this @j-schultz. This looks indeed like a bug though it's more likely related to neural PLC (DTX is handled by the PLC module and NoLACE is not active in this case). I tried a few files myself but could not reproduce the issue. Could you share an input file that triggers it? It would also be interesting to know whether the problem is present with dec_complexity = 5 (i.e. neural PLC active and enhancement inactive).
from opus.
It does happen with both decoding complexity 5 and 7. I'll see if I can get a minimal example put together - as we are streaming live audio with raw opus frames between clients, I'm not sure how comparable this is to using the file-based opus demo.
I also checked whether different encoding parameters could influence the result...
- Both application type
OPUS_APPLICATION_VOIP
andOPUS_APPLICATION_AUDIO
expose the issue - Encoding complexity: Tried two different values (5 and 8), no difference
- Does not matter if inband FEC is enabled or not
Apart from that, we force a frame duration: 20ms and obviously DTX is enabled.
from opus.
Thanks @j-schultz. In that case it's indeed rather neural PLC that's causing the issue (looping @jmvalin in). There could be many reasons for this to happen (DTX triggered during active speech, feature prediction going wrong in neural PLC, missing buffer update etc.) so it's crucial to find a file that triggers it.
Apart from this, we should probably revise DTX handling at the decoder in general. Handling it with neural PLC means that we run a relatively expensive neural vocoder to generate silence, which is quite wasteful. I will kick of this discussion in https://www.irccloud.com/irc/libera.chat/channel/opus
What you could try as a temporary fix is to set dec_complexity to 0 during DTX and back to 7 once the first active frame is received. That should solve the noise problem and would also save you some complexity.
from opus.
Thanks for the suggestion, I applied the temporary workaround and that does seem to do the trick for now.
from opus.
Actually I might have spoken too soon, while the (incorrect) work of the PLC can no longer be heard with this change, I still get some faint clicking sound every 400ms even though the source signal is 100% digital silence. So I think I'll wait for a proper fix before turning on NoLACE.
from opus.
When in DTX mode, the encoder will send a "refresh" (or keepalive) packet every 400 ms to update the decoder noise estimate. Maybe that's what causing the issue. Are you also setting dec_complexity to 0 on that one?
from opus.
For testing I set the complexity to 7 for every successfully received packet and to 0 for any missing packet. So the first packet of the DTX interval still has a complexity of 7. I will change this so that if the packet indicates the start of a DTX phase, it will already reduce the complexity to 0.
Edit: That did the trick.
from opus.
Is there a file and exact command line I can use to reproduce the problem?
from opus.
Here's a RAW sample file, together with the decoded result that I receive: sample.zip
Encoding command line: opus_demo.exe -e voip 48000 1 25000 -complexity 8 -dtx -framesize 20 withsilence.raw withsilence.opus
Decoding commandline: opus_demo.exe -d 48000 1 -dec_complexity 7 withsilence.opus withsilence.decoded.raw
Opus has been built with the following CMake configuration: cmake -DOPUS_BUILD_PROGRAMS=ON -DOPUS_DEEP_PLC=ON -DOPUS_DRED=ON -DOPUS_OSCE=ON -DOPUS_DNN=ON -DBUILD_SHARED_LIBS=OFF
from opus.
Related Issues (20)
- Limiting the stack allocation request HOT 2
- Please update opus-tools to support new features
- Deep PLC performance on Android HOT 16
- Compiling with CMake for Windows/MSVC only supports DLL runtime libraries HOT 9
- Added the OPUS_SET_INBAND_FEC(2) option HOT 1
- Please provide an option to not depend on downloading model data HOT 10
- v1.5.2 compile warnings
- Error: Range coder state mismatch between encoder and decoder in frame 1: 0x f3b1240 vs 0x40123b0f HOT 1
- Opus.lib: 1.3.1 version crashing in some windows machines in opus_encoder_create HOT 2
- Opus 1.5.x and main compilation failing with GCC 8.5.0 HOT 2
- Opus not building on Ubuntu 24.04
- No way to turn off the PLC/FEC?
- Floating Point libraries failing to build when targeting Fixed Point support only. MSVC 2017 v141 HOT 3
- Bug: The decoded dred_offset < 0 HOT 1
- How can I verify if FEC is working? I am not hearing any changes with fec enabled vs disabled with 30% packet loss HOT 2
- Cannot build because media.xiph.org is not responding HOT 4
- FTBFS with "fixed-point" and Deep PLC
- Libopus does not encode 5.1ch(side) / data loss when encoding from 5.1 ch -> 5.1(side) ch
- Need GN build system support HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from opus.