mozilla / cubeb Goto Github PK

We need to define the behaviour of what happens when users change devices, either via the audio configuration menu, or by plugging or unplugging a physical device.

Here is a proposal that allows implementing something flexible:

First, we need to implement the device change notification callback on all supported platforms.

Next, I propose that inside the audio backends, we call the user callback before changing the stream configuration. We also need to make sure that no audio callback will be called during the time where the device change callback is called.

Then, we can modify the signature of the callback, and make it like so:

typedef enum {
  CUBEB_INPUT_DEVICE_CHANGED = 1 << 0;
  CUBEB_OUTPUT_DEVICE_CHANGED = 1 << 1;
} cubeb_device_change_information;

typedef enum {
  CUBEB_USE_DEFAULT_INPUT = 1 << 0;
  CUBEB_USE_DEFAULT_OUTPUT = 1 << 1;
  CUBEB_STOP_INPUT = 1 << 2;
  CUBEB_STOP_OUTPUT = 1 << 3;
} cubeb_device_change_action;

typedef cubeb_device_change_action (* cubeb_device_changed_callback)(void * user_ptr, cubeb_device_change_information);

If it's ok to switch to the default devices, the user can return CUBEB_USE_DEFAULT_INPUT|CUBEB_USE_DEFAULT_OUTPUT, and the switch will occur. If the user prefers to kill one side of the stream, passing the appropriate enum member turns the stream into an half-duplex stream, or stops the stream if both CUBEB_STOP_INPUT and CUBEB_STOP_OUTPUT are or-ed and returned. Users can then react to the device change, for example by destroying this stream and creating a new one.

It sounds a bit dangerous to allow calling cubeb APIs from the device change notification callback.

For gecko, this allows implementing the following:

• Having a virtual default device in getUserMedia that uses the default device for the system, and continues across default device changes.
• Being able to keep playing audio in case of audio output device changes (for HTMLMediaElement or Web Audio API)
• Being able to send the "ended" event on the MediaStream associated with the getUserMedia call, as specced, and stopping the capture. This informs the user that something has happened and that getUserMedia should probably be called again.
• Being able to react to the case where the last audio device has been removed.

This issue tracks the big picture of multi-channel support. I'll update this later with a better summary.

The main items are:

• API design
• implementation for each backend
  • WASAPI
  • AudioUnit
  • PulseAudio
  • OpenSL
  • others (minimum changes to retain existing functionality, actual support will need to be contributed)
• deal with code duplicating between cubeb and Gecko's AudioConverter
• work out how this interacts with device switching (discussed in issue #167)

Right now, there's some discussion in pull #171 covering this that needs to be moved into this issue.

Using the wasapi backend assumes float sample format. As far as I understand it is possible to init a stream using signed 16bit samples without getting any errors. At least I can't find any error checking on cubeb_stream_params::format.

Running test/test_audio confirmed my suspicion after reading the wasapi backend code.

I've tried to compile cubeb on mingw, because I want to use it on Windows projects.

As far as I can see, it has a winmm backend, which should work, but the compile fails with this output:

https://gist.github.com/4131078

Here's what I ran:

dos2unix configure.ac
autoreconf --install
./configure
make

Any ideas?

We've changed recently the audiounit backend from c to cpp. Probably you will come in the situation to switch branches between a version that contain the c file from/to a version that contains the cpp file. If you try to make clean and build in that case you will get the following compile error:
(this is from a c version to cpp version)

make[2]: *** No rule to make target `src/cubeb_audiounit.c', needed by `src/cubeb_audiounit.lo'.  Stop.
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

If you git clean the repo, reconfigure and build the problem should be solved but for a quicker workaround do the following

vim src/.deps/cubeb_audiounit.Plo 

and change in first line the name of the audiounit file to the correct version. For example here I have:

src/cubeb_audiounit.lo: src/cubeb_audiounit.c \
  /usr/include/TargetConditionals.h /usr/include/assert.h \
<more lines>

will be changed to

src/cubeb_audiounit.lo: src/cubeb_audiounit.cpp \
  /usr/include/TargetConditionals.h /usr/include/assert.h \
<more lines>

Similarly the file test/test_ring_array.c has been changed to cpp so if you execute make check you will get the error:

make[2]: *** No rule to make target `test/test_ring_array.c', needed by `test/test_ring_array.o'.  Stop.
make[1]: *** [check-am] Error 2
make: *** [check-recursive] Error 1

you have to modify the following file in the same way:

test/.deps/test_ring_array.Po

I'm looking at using Cubeb for my own project, and one of the features I wish to support is explicit device selection. If a standard API can be decided upon, I can do the work for the Windows backends, but am not yet in the position to do so for Linux, mac, and Android. I'm seriously considering doing the work on Windows for my own project; consequently, I want to hash out what would be needed to get it upstream.

OpenBSD and DragonFly has similar defines. Not sure about NetBSD. The correct way is to not define _POSIX_SOURCE, e.g.

#if !defined(__DragonFly__) && !defined(__FreeBSD__)    \
  && !defined(__NetBSD__) && !defined(__OpenBSD__)
#define _POSIX_SOURCE
#endif

or just remove it unconditionally.

// sys/cdefs.h
#if defined(_POSIX_SOURCE) && !defined(_POSIX_C_SOURCE)
#define _POSIX_C_SOURCE     198808
#endif

// time.h
#if __POSIX_VISIBLE >= 199309
/*
 * New in POSIX 1003.1b-1993.
 */
#ifndef _CLOCKID_T_DECLARED
typedef __clockid_t clockid_t;
#define _CLOCKID_T_DECLARED
#endif

// sys/time.h
#if __XSI_VISIBLE
int getitimer(int, struct itimerval *);
int gettimeofday(struct timeval *, struct timezone *);
#endif



$ gmake
[...]
  CC       src/cubeb_alsa.lo
In file included from src/cubeb_alsa.c:12:
/usr/include/pthread.h:184:4: error: unknown type name 'clockid_t'
            clockid_t *);
            ^
/usr/include/pthread.h:203:39: error: unknown type name 'clockid_t'
int             pthread_getcpuclockid(pthread_t, clockid_t *);
                         ^
src/cubeb_alsa.c:133:3: warning: implicit declaration of function 'gettimeofday' is
  invalid in C99 [-Wimplicit-function-declaration]
  gettimeofday(&now, NULL);
  ^
1 warning and 2 errors generated.

Hi kinetiknz,

I am trying to build it in Mac OSX 10.10.3, and I get:

~/Projects/MyOwn$ git clone https://github.com/kinetiknz/cubeb.git
Cloning into 'cubeb'...
remote: Counting objects: 1745, done.
remote: Total 1745 (delta 0), reused 0 (delta 0), pack-reused 1745
Receiving objects: 100% (1745/1745), 683.86 KiB | 497.00 KiB/s, done.
Resolving deltas: 100% (1101/1101), done.
Checking connectivity... done.
~/Projects/MyOwn$ cd cubeb
/Projects/MyOwn/cubeb$ autoreconf --install
glibtoolize: putting auxiliary files in '.'.
glibtoolize: copying file './ltmain.sh'
glibtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'.
glibtoolize: copying file 'm4/libtool.m4'
glibtoolize: copying file 'm4/ltoptions.m4'
glibtoolize: copying file 'm4/ltsugar.m4'
glibtoolize: copying file 'm4/ltversion.m4'
glibtoolize: copying file 'm4/ltobsolete.m4'
configure.ac:40: installing './compile'
configure.ac:7: installing './config.guess'
configure.ac:7: installing './config.sub'
configure.ac:14: installing './install-sh'
configure.ac:14: installing './missing'
Makefile.am: installing './depcomp'
parallel-tests: installing './test-driver'
~/Projects/MyOwn/cubeb$ ./configure
checking build system type... x86_64-apple-darwin14.3.0
checking host system type... x86_64-apple-darwin14.3.0
checking target system type... x86_64-apple-darwin14.3.0
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... ./install-sh -c -d
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether make supports nested variables... (cached) yes
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking how to run the C preprocessor... gcc -E
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking dependency style of g++... gcc3
checking how to print strings... printf
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc... /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld
checking if the linker (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld) is GNU ld... no
checking for BSD- or MS-compatible name lister (nm)... /opt/local/bin/nm
checking the name lister (/opt/local/bin/nm) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 196608
checking how to convert x86_64-apple-darwin14.3.0 file names to x86_64-apple-darwin14.3.0 format... func_convert_file_noop
checking how to convert x86_64-apple-darwin14.3.0 file names to toolchain format... func_convert_file_noop
checking for /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... dlltool
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @file support... no
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /opt/local/bin/nm output from gcc object... ok
checking for sysroot... no
checking for a working dd... /bin/dd
checking how to truncate binary pipes... /bin/dd bs=4096 count=1
checking for mt... no
checking if : is a manifest tool... no
checking for dsymutil... dsymutil
checking for nmedit... nmedit
checking for lipo... lipo
checking for otool... otool
checking for otool64... no
checking for -single_module linker flag... yes
checking for -exported_symbols_list linker flag... yes
checking for -force_load linker flag... yes
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... yes
checking for gcc option to produce PIC... -fno-common -DPIC
checking if gcc PIC flag -fno-common -DPIC works... yes
checking if gcc static flag -static works... no
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... darwin14.3.0 dyld
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking how to run the C++ preprocessor... g++ -E
checking for ld used by g++... /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld
checking if the linker (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld) is GNU ld... no
checking whether the g++ linker (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld) supports shared libraries... yes
checking for g++ option to produce PIC... -fno-common -DPIC
checking if g++ PIC flag -fno-common -DPIC works... yes
checking if g++ static flag -static works... no
checking if g++ supports -c -o file.o... yes
checking if g++ supports -c -o file.o... (cached) yes
checking whether the g++ linker (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... darwin14.3.0 dyld
checking how to hardcode library paths into programs... immediate
checking for pkg-config... /opt/local/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking for PULSE... no
checking for JACK... no
checking for ALSA... no
checking AudioUnit/AudioUnit.h usability... yes
checking AudioUnit/AudioUnit.h presence... yes
checking for AudioUnit/AudioUnit.h... yes
checking AudioToolbox/AudioToolbox.h usability... yes
checking AudioToolbox/AudioToolbox.h presence... yes
checking for AudioToolbox/AudioToolbox.h... yes
checking sndio.h usability... no
checking sndio.h presence... no
checking for sndio.h... no
checking SLES/OpenSLES.h usability... no
checking SLES/OpenSLES.h presence... no
checking for SLES/OpenSLES.h... no
checking android/log.h usability... no
checking android/log.h presence... no
checking for android/log.h... no
checking for audioclient.h... no
checking for mmsystem.h... no
checking for doxygen... false
configure: WARNING: * doxygen not found, API documentation will not be built
checking for stdint types... stdint.h (shortcircuit)
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
make use of stdint.h in include/cubeb/cubeb-stdint.h (assuming C99 compatible system)
checking for ld used by gcc... (cached) /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld
checking if the linker (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld) is GNU ld... (cached) no
checking how to control symbol export... checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating docs/Makefile
config.status: creating docs/Doxyfile
config.status: creating cubeb.pc
config.status: creating cubeb-uninstalled.pc
config.status: creating config.h
config.status: executing depfiles commands
config.status: executing libtool commands
config.status: executing include/cubeb/cubeb-stdint.h commands
config.status: creating include/cubeb/cubeb-stdint.h : _LIBCUBEB_INCLUDE_CUBEB_CUBEB_STDINT_H
~/Projects/MyOwn/cubeb$ make
/Applications/Xcode.app/Contents/Developer/usr/bin/make all-recursive
Making all in docs
echo "* Warning: Doxygen not found; documentation will not be built."
*** Warning: Doxygen not found; documentation will not be built.
touch doxygen-build.stamp
CC src/cubeb.lo
CXX src/cubeb_panner.lo
CC src/cubeb_audiounit.lo
CC src/cubeb_audioqueue.lo
CXXLD src/libcubeb.la
Undefined symbols for architecture x86_64:
"_cubeb_set_coreaudio_notification_runloop", referenced from:
_audiounit_init in cubeb_audiounit.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [src/libcubeb.la] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

Any advice?
Thanks.

I've been having this glitch from the time cubeb got its first JACK code to the latest code that I've shoved into Firefox 49. It gives out xruns on Firefox's shutdown and rarely makes its HTML5 player stuck even before that. The longer Firefox session and more youtube videos played during it - the worse it is. With synchronous mode (-S) that allows lower jackd latency with risk of one client disrupting all sound output I get things like that:

20:01:37.209 redirection-to-jamin: jamin:a.master.out_L -> system:playback_1 checked.
20:01:37.218 redirection-to-jamin: jamin:a.master.out_R -> system:playback_2 checked.
20:01:37.229 redirection-to-jamin: CubebUtils:AudioStream_0_out_0 -> jamin:in_L checked.
20:01:37.239 redirection-to-jamin: CubebUtils:AudioStream_0_out_1 -> jamin:in_R checked.
20:01:37.249 redirection-to-jamin: PulseAudio JACK Sink:front-left -> jamin:in_L checked.
20:01:37.258 redirection-to-jamin: PulseAudio JACK Sink:front-right -> jamin:in_R checked.
JackPosixSemaphore::TimedWait err = Connection timed out
JackFreewheelDriver::ProcessSync: SuspendRefNum error
JackAudioDriver::ProcessGraphSync: ProcessWriteSlaves error, engine may now behave abnormally!!
JackPosixProcessSync::LockedTimedWait error usec = 5120000 err = Connection timed out
JackEngine::ClientDeactivate wait error ref = 6 name = CubebUtils
JackEngine::ClientKill ref = 6 cannot be removed from the graph !!
20:01:42.271 JACK connection graph change.
20:01:42.330 JACK active patchbay scan...
20:01:42.340 redirection-to-jamin: jamin:a.master.out_L -> system:playback_1 checked.
20:01:42.350 redirection-to-jamin: jamin:a.master.out_R -> system:playback_2 checked.
20:01:42.361 redirection-to-jamin: PulseAudio JACK Sink:front-left -> jamin:in_L checked.
20:01:42.371 redirection-to-jamin: PulseAudio JACK Sink:front-right -> jamin:in_R checked.
20:01:42.604 JACK connection change.
20:01:42.815 JACK active patchbay scan...
20:01:42.825 redirection-to-jamin: jamin:a.master.out_L -> system:playback_1 checked.
20:01:42.835 redirection-to-jamin: jamin:a.master.out_R -> system:playback_2 checked.
20:01:42.846 redirection-to-jamin: PulseAudio JACK Sink:front-left -> jamin:in_L checked.
20:01:42.856 redirection-to-jamin: PulseAudio JACK Sink:front-right -> jamin:in_R checked.
JackPosixProcessSync::LockedTimedWait error usec = 1024000 err = Connection timed out
JackEngine::ClientCloseAux wait error ref = 6
JackPosixSemaphore::TimedWait err = Connection timed out
JackAudioDriver::ProcessGraphSync: SuspendRefNum error, engine may now behave abnormally!!
20:01:47.272 XRUN callback (1).

That makes me seriously think about disabling cubeb's native JACK support for good and use PA as a compatibility shim. It seems it doesn't close its connections properly or something.

I've finally managed to get node-cubeb somewhat working (not segfaulting), but I noticed that my tests produce no audible output. I initially thought this is in my code but the test_tone example doesn't produce anything audible either.

I'm linking against the latest master of cubeb and have PA version 2.1 on a Arch Linux setup.

If we play a 48kHz stereo stream (which is what dmix defaults to using), every second time we wake up via poll snd_pcm_poll_descriptors_revents returns a zero revent. We used to treat this as an error, but due to it breaking playback we now ignore snd_pcm_poll_descriptors_revents completely (in favour of the raw poll revents) and use snd_pcm_avail_update to decide if we should refill. These workarounds were applied in kinetiknz@de49402 and kinetiknz@18efde6.

This has been discussed in https://bugzilla.mozilla.org/show_bug.cgi?id=1279125 and https://github.com/kinetiknz/cubeb/pull/127, but it's a bit scattered and I wanted a central place to track it.

snd_pcm_direct_poll_revents in snd/pcm/pcm_direct.c in alsa-lib checks avail against pcm->avail_min (which is usually 1 period, so 6000 frames for dmix) and treats a wakeup below this threshold as an "empty" wakeup where it clears the returned revent. I suspect this occurs because dmix uses a timer to drive poll wakeups and sometimes wakes up slightly early, resulting in avail being lower than expected. When treating this wakeup with zero revent as spurious per the ALSA docs (http://www.alsa-project.org/alsa-doc/alsa-lib/group___p_c_m.html#ga7e561f305702c6f52dab49b6c84f7df7 mentioning "null event"), the next wakeup we get will be for (nearly) 2 periods worth of data. Since we're running with only 2 periods, the chance of hitting an underrun and going into recovery is extremely high in this situation. The workaround in the two commits linked above avoids this by ignoring the converted revents and refilling whatever buffer space it can.

This behaviour doesn't seem to occur (except on the first 1-2 wakeups during startup) when playing a non-dmix-default rate, because in this situation there's a rate conversion plugin present in the chain and the avail < pcm->avail_min test in snd_pcm_direct_poll_revents almost always results in a "non empty' wakeup because avail always seems to have at least 1 full period of extra space available than in the non-rate-conversion case.

Per my recent digging, I think this is an ALSA/dmix issue and I don't have a better workaround that what we're currently doing -- might need to talk to upstream.

CC @i-rinat and @DanielFragaBR.

It could be useful for debugging to expose the backend that's being used by the cubeb context, perhaps simply:

char *cubeb_get_backend(cubeb *context);
and that would return a string like "alsa" or "pulseaudio".

Or maybe just have an enum for them.

I can see a race (at least on windows) where the following happens:

• play a duplex stream on an external device (I'm testing with a USB headset)
• make the device unavailable (for example by unplugging it)
• we get an event that says the output has been made unavailable (we haven't got yet an event that tells us the input has been made unavailable)
• we try to re-open the default input and output devices
• The default output is the new device, the default input is still the old device
• It fails

The concern is that device change callbacks could be coalesced (speculation), which would cause the count-based device tracking to miss some device changes.

See the discussion at https://github.com/kinetiknz/cubeb/pull/82#discussion_r56318044.

I think @achronop is already working on fixing this.

Hi/2.

Here is the patch for OS/2 KAI. Review, please...

From 384c76492de693e1281b7962fd60d47eeae34597 Mon Sep 17 00:00:00 2001
From: KO Myung-Hun <[email protected]>
Date: Mon, 15 Jun 2015 14:49:03 +0900
Subject: [PATCH] Add support for OS/2 KAI

---
 Makefile.am     |   3 +
 configure.ac    |  15 ++-
 src/cubeb.c     |   6 +
 src/cubeb_kai.c | 333 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 356 insertions(+), 1 deletion(-)
 create mode 100644 src/cubeb_kai.c

diff --git a/Makefile.am b/Makefile.am
index e114e35..cb280ec 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -56,6 +56,9 @@ endif
 if AUDIOTRACK
 src_libcubeb_la_SOURCES += src/cubeb_audiotrack.c
 endif
+if KAI
+src_libcubeb_la_SOURCES += src/cubeb_kai.c
+endif
 if SPEEX
 src_libcubeb_la_SOURCES += src/cubeb_resampler.cpp src/speex/resample.c
 endif
diff --git a/configure.ac b/configure.ac
index 84853a2..cc257af 100644
--- a/configure.ac
+++ b/configure.ac
@@ -182,6 +182,19 @@ AS_IF([test "x$with_winmm" != xno],
       ], [#include <windows.h>])])
 AM_CONDITIONAL([WINMM], [test $HAVE_WINMM -eq 1])

+AC_ARG_WITH([kai],
+            AS_HELP_STRING([--with-kai], [with KAI @<:@default=check@:>@]))
+AS_IF([test "x$with_kai" != xno],
+      [AC_CHECK_HEADERS([kai.h], [
+       HAVE_KAI=1
+       KAI_LIBS="-lkai"
+       AC_DEFINE([USE_KAI], [], [Use K Audio Interface])
+      ], [
+       HAVE_KAI=0
+       AS_IF([test "x$with_kai" = xyes], [AC_MSG_ERROR(KAI not detected)])
+      ])])
+AM_CONDITIONAL([KAI], [test $HAVE_KAI -eq 1])
+
 AM_CONDITIONAL([SPEEX], [test $NEED_SPEEX -eq 1])

 AS_IF([test $NEED_SPEEX -eq 1],
@@ -192,7 +205,7 @@ AS_IF([test $NEED_SPEEX -eq 1],
        AC_DEFINE([EXPORT], [], [Tell the resampler to export the symbols by default.])
       ])

-platform_lib="$PULSE_LIBS $JACK_LIBS $ALSA_LIBS $AUDIOUNIT_LIBS $AUDIOQUEUE_LIBS $SNDIO_LIBS $OPENSL_LIBS $DIRECTSOUND_LIBS $WINMM_LIBS $WASAPI_LIBS"
+platform_lib="$PULSE_LIBS $JACK_LIBS $ALSA_LIBS $AUDIOUNIT_LIBS $AUDIOQUEUE_LIBS $SNDIO_LIBS $OPENSL_LIBS $DIRECTSOUND_LIBS $WINMM_LIBS $WASAPI_LIBS $KAI_LIBS"

 dnl Check for doxygen
 AC_ARG_ENABLE([doc],
diff --git a/src/cubeb.c b/src/cubeb.c
index 8f3b040..0fa5381 100644
--- a/src/cubeb.c
+++ b/src/cubeb.c
@@ -56,6 +56,9 @@ int opensl_init(cubeb ** context, char const * context_name);
 #if defined(USE_AUDIOTRACK)
 int audiotrack_init(cubeb ** context, char const * context_name);
 #endif
+#if defined(USE_KAI)
+int kai_init(cubeb ** context, char const * context_name);
+#endif

 int
 validate_stream_params(cubeb_stream_params stream_params)
@@ -122,6 +125,9 @@ cubeb_init(cubeb ** context, char const * context_name)
 #if defined(USE_AUDIOTRACK)
     audiotrack_init,
 #endif
+#if defined(USE_KAI)
+    kai_init,
+#endif
   };
   int i;

diff --git a/src/cubeb_kai.c b/src/cubeb_kai.c
new file mode 100644
index 0000000..4bdbb92
--- /dev/null
+++ b/src/cubeb_kai.c
@@ -0,0 +1,333 @@
+/*
+ * Copyright 짤 2015 Mozilla Foundation
+ *
+ * This program is made available under an ISC-style license.  See the
+ * accompanying file LICENSE for details.
+ */
+#include <stdio.h>
+#include <string.h>
+#include <math.h>
+#include <sys/fmutex.h>
+
+#include <kai.h>
+
+#include "cubeb/cubeb.h"
+#include "cubeb-internal.h"
+
+/* We don't support more than 2 channels in KAI */
+#define MAX_CHANNELS 2
+
+#define NBUFS 2
+#define FRAME_SIZE 2048
+
+struct cubeb_stream_item {
+  cubeb_stream * stream;
+};
+
+static struct cubeb_ops const kai_ops;
+
+struct cubeb {
+  struct cubeb_ops const * ops;
+};
+
+struct cubeb_stream {
+  cubeb * context;
+  cubeb_stream_params params;
+  cubeb_data_callback data_callback;
+  cubeb_state_callback state_callback;
+  void * user_ptr;
+
+  HKAI hkai;
+  KAISPEC spec;
+  uint64_t total_frames;
+  float soft_volume;
+  _fmutex mutex;
+  float float_buffer[FRAME_SIZE * MAX_CHANNELS];
+};
+
+static inline long
+frames_to_bytes(long frames, cubeb_stream_params params)
+{
+  return frames * 2 * params.channels; /* 2 bytes per frame */
+}
+
+static inline long
+bytes_to_frames(long bytes, cubeb_stream_params params)
+{
+  return bytes / 2 / params.channels; /* 2 bytes per frame */
+}
+
+static void kai_destroy(cubeb * ctx);
+
+/*static*/ int
+kai_init(cubeb ** context, char const * context_name)
+{
+  cubeb * ctx;
+
+  XASSERT(context);
+  *context = NULL;
+
+  if (kaiInit(KAIM_AUTO))
+    return CUBEB_ERROR;
+
+  ctx = calloc(1, sizeof(*ctx));
+  XASSERT(ctx);
+
+  ctx->ops = &kai_ops;
+
+  *context = ctx;
+
+  return CUBEB_OK;
+}
+
+static char const *
+kai_get_backend_id(cubeb * ctx)
+{
+  return "kai";
+}
+
+static void
+kai_destroy(cubeb * ctx)
+{
+  kaiDone();
+
+  free(ctx);
+}
+
+static void
+float_to_s16ne(int16_t *dst, float *src, size_t n)
+{
+  long l;
+
+  while (n--) {
+    l = lrintf(*src++ * 0x8000);
+    if (l > 32767)
+      l = 32767;
+    if (l < -32768)
+      l = -32768;
+    *dst++ = (int16_t)l;
+  }
+}
+
+static ULONG APIENTRY
+kai_callback(PVOID cbdata, PVOID buffer, ULONG len)
+{
+  cubeb_stream * stm = cbdata;
+  void *p;
+  long wanted_frames;
+  long frames;
+
+  p = stm->params.format == CUBEB_SAMPLE_FLOAT32NE
+      ? stm->float_buffer : buffer;
+
+  wanted_frames = bytes_to_frames(len, stm->params);
+  frames = stm->data_callback(stm, stm->user_ptr, p, wanted_frames);
+
+  _fmutex_request(&stm->mutex, 0);
+  stm->total_frames += frames;
+  _fmutex_release(&stm->mutex);
+
+  if (frames < wanted_frames)
+    stm->state_callback(stm, stm->user_ptr, CUBEB_STATE_DRAINED);
+
+  if (stm->params.format == CUBEB_SAMPLE_FLOAT32NE)
+    float_to_s16ne(buffer, p, len / sizeof(int16_t));
+
+  return frames_to_bytes(frames, stm->params);
+}
+
+static void kai_stream_destroy(cubeb_stream * stm);
+
+static int
+kai_stream_init(cubeb * context, cubeb_stream ** stream,
+                char const * stream_name, cubeb_stream_params stream_params,
+                unsigned int latency, cubeb_data_callback data_callback,
+                cubeb_state_callback state_callback, void * user_ptr)
+{
+  cubeb_stream * stm;
+  KAISPEC wanted_spec;
+
+  if (stream_params.channels < 1 || stream_params.channels > MAX_CHANNELS)
+    return CUBEB_ERROR_INVALID_FORMAT;
+
+  XASSERT(context);
+  XASSERT(stream);
+
+  *stream = NULL;
+
+  stm = calloc(1, sizeof(*stm));
+  XASSERT(stm);
+
+  stm->context = context;
+  stm->params = stream_params;
+  stm->data_callback = data_callback;
+  stm->state_callback = state_callback;
+  stm->user_ptr = user_ptr;
+  stm->soft_volume = 1.0f;
+
+  if (_fmutex_create(&stm->mutex, 0)) {
+    free(stm);
+    return CUBEB_ERROR;
+  }
+
+  wanted_spec.usDeviceIndex   = 0;
+  wanted_spec.ulType          = KAIT_PLAY;
+  wanted_spec.ulBitsPerSample = BPS_16;
+  wanted_spec.ulSamplingRate  = stm->params.rate;
+  wanted_spec.ulDataFormat    = MCI_WAVE_FORMAT_PCM;
+  wanted_spec.ulChannels      = stm->params.channels;
+  wanted_spec.ulNumBuffers    = NBUFS;
+  wanted_spec.ulBufferSize    = frames_to_bytes(FRAME_SIZE, stm->params);
+  wanted_spec.fShareable      = TRUE;
+  wanted_spec.pfnCallBack     = kai_callback;
+  wanted_spec.pCallBackData   = stm;
+
+  if (kaiOpen(&wanted_spec, &stm->spec, &stm->hkai)) {
+    _fmutex_close(&stm->mutex);
+    free(stm);
+    return CUBEB_ERROR;
+  }
+
+  *stream = stm;
+
+  return CUBEB_OK;
+}
+
+static void
+kai_stream_destroy(cubeb_stream * stm)
+{
+  kaiClose(stm->hkai);
+  _fmutex_close(&stm->mutex);
+  free(stm);
+}
+
+static int
+kai_get_max_channel_count(cubeb * ctx, uint32_t * max_channels)
+{
+  XASSERT(ctx && max_channels);
+
+  *max_channels = MAX_CHANNELS;
+
+  return CUBEB_OK;
+}
+
+static int
+kai_get_min_latency(cubeb * ctx, cubeb_stream_params params, uint32_t * latency)
+{
+  /* We have at least two buffers. One is being played, the other one is being
+     filled. So there is as much latency as one buffer. */
+  *latency = FRAME_SIZE * 1000 / params.rate;
+
+  return CUBEB_OK;
+}
+
+static int
+kai_get_preferred_sample_rate(cubeb * ctx, uint32_t * rate)
+{
+  cubeb_stream_params params;
+  KAISPEC wanted_spec;
+  KAISPEC spec;
+  HKAI hkai;
+
+  params.format = CUBEB_SAMPLE_S16NE;
+  params.rate = 48000;
+  params.channels = 2;
+
+  wanted_spec.usDeviceIndex   = 0;
+  wanted_spec.ulType          = KAIT_PLAY;
+  wanted_spec.ulBitsPerSample = BPS_16;
+  wanted_spec.ulSamplingRate  = params.rate;
+  wanted_spec.ulDataFormat    = MCI_WAVE_FORMAT_PCM;
+  wanted_spec.ulChannels      = params.channels;
+  wanted_spec.ulNumBuffers    = NBUFS;
+  wanted_spec.ulBufferSize    = frames_to_bytes(FRAME_SIZE, params);
+  wanted_spec.fShareable      = TRUE;
+  wanted_spec.pfnCallBack     = kai_callback;
+  wanted_spec.pCallBackData   = NULL;
+
+  /* Test 48KHz */
+  if (kaiOpen(&wanted_spec, &spec, &hkai)) {
+    /* Not supported. Fall back to 44.1KHz */
+    params.rate = 44100;
+  } else {
+    /* Supported. Use 48KHz */
+    kaiClose(hkai);
+  }
+
+  *rate = params.rate;
+
+  return CUBEB_OK;
+}
+
+static int
+kai_stream_start(cubeb_stream * stm)
+{
+  if (kaiPlay(stm->hkai))
+    return CUBEB_ERROR;
+
+  stm->state_callback(stm, stm->user_ptr, CUBEB_STATE_STARTED);
+
+  return CUBEB_OK;
+}
+
+static int
+kai_stream_stop(cubeb_stream * stm)
+{
+  if (kaiStop(stm->hkai))
+    return CUBEB_ERROR;
+
+  stm->state_callback(stm, stm->user_ptr, CUBEB_STATE_STOPPED);
+
+  return CUBEB_OK;
+}
+
+static int
+kai_stream_get_position(cubeb_stream * stm, uint64_t * position)
+{
+  _fmutex_request(&stm->mutex, 0);
+  *position = stm->total_frames;
+  _fmutex_release(&stm->mutex);
+
+  return CUBEB_OK;
+}
+
+static int
+kai_stream_get_latency(cubeb_stream * stm, uint32_t * latency)
+{
+  /* Out of buffers, one is being played, the others are being filled.
+     So there is as much latency as total buffers - 1. */
+  *latency = bytes_to_frames(stm->spec.ulBufferSize, stm->params)
+             * (stm->spec.ulNumBuffers - 1);
+
+  return CUBEB_OK;
+}
+
+static int
+kai_stream_set_volume(cubeb_stream * stm, float volume)
+{
+  _fmutex_request(&stm->mutex, 0);
+  stm->soft_volume = volume;
+  _fmutex_release(&stm->mutex);
+
+  return CUBEB_OK;
+}
+
+static struct cubeb_ops const kai_ops = {
+  /*.init =*/ kai_init,
+  /*.get_backend_id =*/ kai_get_backend_id,
+  /*.get_max_channel_count=*/ kai_get_max_channel_count,
+  /*.get_min_latency=*/ kai_get_min_latency,
+  /*.get_preferred_sample_rate =*/ kai_get_preferred_sample_rate,
+  /*.destroy =*/ kai_destroy,
+  /*.stream_init =*/ kai_stream_init,
+  /*.stream_destroy =*/ kai_stream_destroy,
+  /*.stream_start =*/ kai_stream_start,
+  /*.stream_stop =*/ kai_stream_stop,
+  /*.stream_get_position =*/ kai_stream_get_position,
+  /*.stream_get_latency = */ kai_stream_get_latency,
+  /*.stream_set_volume =*/ kai_stream_set_volume,
+  /*.stream_set_panning =*/ NULL,
+  /*.stream_get_current_device =*/ NULL,
+  /*.stream_device_destroy =*/ NULL,
+  /*.stream_register_device_changed_callback=*/ NULL
+};
-- 
1.9.5

I'm working on this via making CMake scripts, which can be a nice alternative to autoconf on Windows (and possibly OS X). I'll submit a pull request when I have something that compiles; I expect few if any source changes will need to be made. The big one seems to be that we aren't running the ax_create_stdint_.h.m4, which can be gotten around via macros in the header that include stdint.h when VC++ is detected (I am going to try to detect C99 directly, but if that fails...).
Recent versions of Visual Studio implement most parts of C99 and basically all of C++11, so I can definitely get this going on VC++ 2013. I'm not sure how far back my changes will work due to not having older versions installed as multiple versions of VS never quite play nice in my experience.
if there are any must-nots, let me know.

Deleted, because #214, opened at the same time as this, is the same issue and a better report of it.

Method enumerate_devices for PulseAudio follows the pattern of calling async methods and waiting for the results in order to continue (like the rest of the code). What stranges me here is that it does not hold the mutex before async method(s) call and it does not signal from inside the callbacks like purposed in pulse documentations.

http://freedesktop.org/software/pulseaudio/doxygen/threaded_mainloop.html#basic_subsec

To be honest I use it a lot in my examples without any problem so I wonder what I am missing here and if we will be fine since this code is landed in firefox and will be overused.

Hi there, libsoundio developer here. I'm just curious about your project. It looks cool. What are some of the goals?

I'm looking forward to collaborating and cross-referencing each other's code in order to solve bugs.

It's currently spinning its own thread and sleeping, which is not optimal and provokes xruns.

It should be more like this: https://github.com/jackaudio/example-clients/blob/master/simple_client.c.

I realise this may not be any priority at all, but thought it might help raising an issue either way! I'll try to resolve this by myself as well.
Here's the error:

cubeb_wasapi.obj : error LNK2019: unresolved external symbol _InterlockedCompareExchange64 referenced in function "int __cdecl `anonymous namespace'::wasapi_stream_get_position(struct cubeb_stream *,unsigned __int64 *)" (?wasapi_stream_get_position@?A@@YAHPEAUcubeb_stream@@PEA_K@Z)

I've tried including winbase.h and even defining InterlockedExchangeAdd64, but still doesn't seem to do any good.

Forgot to mention, trying to build with Clang 3.7.0 64-Bit.

Hi there, I've written an Homebrew formula for cubeb: Homebrew/legacy-homebrew#16174

Unfortunately they won't accept a formula that's "HEAD-only", ie. that installs directly from the git repo. They want stable releases.

Do you plan to release stable versions of cubeb (basically, snapshots) at anytime?

If not, would you agree if I did upload snaphots to a repo (e.g. nddrylliog/cubeb-unofficial-releases) just for the purpose of distributing it through Homebrew?

Thanks in advance!

Mutex became recursive due to broken functionality on Nightly (Bug 1290625). The initial intention of the wrapper is to be non-recursive. This bug will tract the restore of non recursive functionality.

There are additional disagreements about the recursive behavior of the wrapper in #142 in comment https://github.com/kinetiknz/cubeb/pull/142#issuecomment-237148357

In order to restore a non recursive behavior audiounit backend needs to remove the recursive dependency when other cubeb methods (cubeb_stream_set_panning in this case) are used from inside user callback.
#143 add the recursive behavior and listed here for reference.

While re-reading some WASAPI docs, I found this:

The client must also specify the appropriate role for the endpoint in the
ERole attribute (eConsole or eCommunications). Do not use eMultimedia.

We're using eMultimedia, and we've never had issues, but maybe we could reconsider.

It's not rare to have to enable logging in a Firefox build, and to need logging from inside cubeb. Currently, this is not great, we need to make another build, and send it to the person, and hope to get results.

I propose the following:

• Add a new API on the cubeb context:

/** Level (verbosity) of logging for a particular cubeb context. */
enum {
  CUBEB_LOG_DISABLED = 0, /** < Logging disabled */
  CUBEB_LOG_NORMAL = 1, /**< Logging lifetime operation (creation/destruction). */
  CUBEB_LOG_VERBOSE = 1 << 1, /**< Verbose logging of callbacks, can have performance implications. */
} cubeb_device_fmt;

/**
 * User supplied callback called when a message needs logging.
 * @param context A pointer to the cubeb context.
 * @param msg A c-string that contains the message to log. */
typedef void (*cubeb_logging_callback)(cubeb * context, const char * msg);

/** Set a callback to be called with a message.
    @param ctx cubeb context with witch this callback is associated
    @param level CUBEB_LOG_VERBOSE, CUBEB_LOG_NORMAL, OR-able
    @param logging_callback A function called with a message when there is something to log. Passing stdout or stderr log to this fd instead. Passing NULL allows unregistering a function.
    @retval CUBEB_OK in case of success.
    @retval CUBEB_ERROR_INVALID_PARAMETER if either ctx or
            device_changed_callback are invalid pointers, or if level is not CUBEB_LOG_VERBOSE or CUBEB_LOG_NORMAL. */
int cubeb_set_logging_callback(cubeb * ctx, int level, cubeb_logging_callback logging_callback);

• When the function pointer is not null, enable logging internally, either at a VERBOSE level (including callback logging), or just NORMAL (audio stream setup, error cases, etc.)
• Harmonize logging for all backends to go through this infrastructure.

If we are clever, logging won't be a perf issue. In particular, I don't think we want to lock. The idea here is to enable logging before any streams is running, so that we can avoid locking on the memory that holds the log level and pointer.

Also we need to spec the invariants for the callback function, which basically boils down to "don't call a cubeb API, directly or indirectly".

@kinetiknz, @achronop, thoughts ?

It successfully compiled, but some tests fail.

Here's configure log and make log

test_audio (sound works)

Testing: volume
Volume: 0%
Volume: 25%
Volume: 50%
Volume: 75%
Volume: 100%
Testing: panning
Panning: -1.00%
lt-test_audio: src/cubeb_alsa.c:1115: alsa_stream_set_panning: Assertion `0 && "not implemented"' failed.

test_latency

latency_test start
cubeb_init ok
cubeb_get_max_channel_count ok
lt-test_latency: test/test_latency.cpp:30: int main(int, char**): Assertion `rv == CUBEB_OK && "Could not query the preferred sample rate."' failed.

test_sanity

START test_init_destroy_context
Backend: alsa
END test_init_destroy_context
START test_init_destroy_multiple_contexts
END test_init_destroy_multiple_contexts
START test_context_variables
lt-test_sanity: test/test_sanity.cpp:119: void test_context_variables(): Assertion `r == 0 && value > 0' failed.

I try to build my stand alone test and I get the following error:

In file included from cubeb_test.cpp:4:
../include/cubeb/cubeb.h:11:10: fatal error: 'cubeb_export.h' file not found
#include "cubeb_export.h"

When add the cubeb/cubeb-build/exports build is successful.

If the change is intentional you can ignore and close the issue.

To make libcubeb easier to build/test/debug on all platforms, a CMake-based build system that's able to generate Visual Studio projects and integrate with AndroidStudio and other build-systems/IDEs easily is desirable. This would avoid the need to install mingw to build on Windows, and allow easier integration with things like the AndroidStudio test project in https://github.com/kinetiknz/cubeb/pull/114.

Hi,

I've tried everything. Can't seem to install this on Mac OS X, no .so are created when building this.

Does someone have any pointers. I'm tyring to build sink.js which uses cubeb....

This is a problem reported during the landing of Bug 1286041. Check in is back out due to the following asan build error:
https://treeherder.mozilla.org/logviewer.html#?job_id=33402081&repo=mozilla-inbound#L1859

I tend to believe that the error is a race condition not immediately related to the patch. According to the asan logs the stream_drain_callback() method is called after the cubeb_stream_destroy(). I could not repro with firefox locally neither on try server (https://treeherder.mozilla.org/#/jobs?repo=try&revision=bd10d7af294d).

I believe the problem is that we wait an arbitrary amount of time in the case of drain, here:
https://github.com/kinetiknz/cubeb/blob/master/src/cubeb_pulse.c#L271

If happen to stop and destroy the stream before pulse thread clean up the pending request the error should reproduced (I played with the timer and wait a lot but did not happen).

Since I am not able to repro first I would like to ask if you agree with that conclusion.
In the case you do, the most appropriate solution I see is to check in stream_stop() if the stm->drain_timer is not NULL and somehow wait to drain first and then stop the stream.

If you have any other idea or approach please let me know.

In the cubeb root directory:

$ autoreconf --install
configure.ac:12: error: 'AM_CONFIG_HEADER': this macro is obsolete.
    You should use the 'AC_CONFIG_HEADERS' macro instead.
/usr/share/aclocal-1.13/obsolete-err.m4:12: AM_CONFIG_HEADER is expanded from...
configure.ac:12: the top level
autom4te: /usr/bin/m4 failed with exit status: 1
aclocal: error: echo failed with exit status: 1
autoreconf: aclocal failed with exit status: 1

$ autoreconf --version
autoreconf (GNU Autoconf) 2.69
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+/Autoconf: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>, <http://gnu.org/licenses/exceptions.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by David J. MacKenzie and Akim Demaille.

$ uname -r
3.7.8-1-ARCH

@tromey is trying to add -Wformat to the Gecko build, but there are a bunch of warnings produced by libcubeb. See https://treeherder.mozilla.org/logviewer.html#?job_id=33131814&repo=try#L47515

I fixed the WASAPI ones in 7073670 and b583da9, but we need to do a fix pass over the other backends.

Without the git submodule update --init --recursive command, no VC++ Project file will be produced.

1. git clone https://github.com/kinetiknz/cubeb.git
2. mkdir cubeb-build
3. cd cubeb && git submodule update --init --recursive
4. cd ../cubeb-build
5. cmake ../cubeb
6. cmake --build .
7. ctest

@padenot has for a long time been proposing adding capture support to cubeb for use in Firefox.

So how should this be done? Should we break API/ABI?
@padenot has an experimental patch and implementation for WASAPI in bugzilla: https://bugzilla.mozilla.org/show_bug.cgi?id=1142613
This patch (Part 1) doesn't currently apply cleanly on top of master, but more interestingly it changes cubeb_stream_init to take two params in the form of pointers now. This will absolutely break exiting applications out there. I'm curious to know if this is a problem or not. I'm guessing it is not, and that implementing capture/full duplex support is more important.
Anyway, changing cubeb_stream_init like this also makes the stream into a both input and output stream. Is that something that we would like, or should a stream be either input only or output only? I don't know.

The driver for adding capture support to cubeb is that @padenot wants full duplex support which makes it easier to both do drift resampling and to feed AEC used by WebRTC in Firefox more reliably.

Would be nice to get your input here @kinetiknz ! :)

This is possibly the same problem as 756944. The Pulse Audio thread dies of seemingly stuff blocking for too long in the main thread. This bug has come up in my (slow) attempt to create node-cubeb. I've attached a stacktrace for gdb node here just in case my assumptions are wrong:

(gdb) run tests/sine-tone.js 
Starting program: /usr/bin/node tests/sine-tone.js
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/libthread_db.so.1".
[New Thread 0x7ffff637a700 (LWP 6958)]
Creating a context...
[New Thread 0x7fffef461700 (LWP 6959)]
Asserting context properties...
Creating a stream...
[New Thread 0x7fffeec60700 (LWP 6960)]
Asserting stream properties...
Starting stream...
Stopping stream after 4000 ms
Data callback: 11069
State changed 0
Data callback: 5520
Data callback: 5520
Data callback: 5520
Data callback: 5520
Data callback: 5520
Data callback: 5520
[Thread 0x7fffef461700 (LWP 6959) exited]
Stopping stream...

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff50fa159 in pa_thread_is_running () from /usr/lib/libpulsecommon-1.1.so
(gdb) bac
#0  0x00007ffff50fa159 in pa_thread_is_running () from /usr/lib/libpulsecommon-1.1.so
#1  0x00007ffff5552873 in pa_threaded_mainloop_lock () from /usr/lib/libpulse.so.0
#2  0x00007ffff576bb96 in stream_cork (stm=0xc90840, state=3) at src/cubeb_pulse.c:175
#3  0x00007ffff576c221 in cubeb_stream_stop (stm=0xc90840) at src/cubeb_pulse.c:355
#4  0x00007ffff5974e78 in CubebStream::stop (this=0xc907c0) at ../src/stream.cpp:63
#5  0x00007ffff5975db5 in CubebStream::Stop (args=...) at ../src/stream.cpp:168
#6  0x000000000059b34d in ?? ()
#7  0x00001aa40daf414e in ?? ()
#8  0x00001aa40daf40c1 in ?? ()
#9  0x00007fffffffdf10 in ?? ()
#10 0x00007fffffffdf50 in ?? ()
#11 0x00001aa40db66668 in ?? ()
#12 0x00002ca629ab69d9 in ?? ()
#13 0x00002ca629ab5659 in ?? ()
#14 0x00000b75abed4b19 in ?? ()
#15 0x00000b75abf050f9 in ?? ()
#16 0x00007fffffffdfb8 in ?? ()
#17 0x00001aa40db95d2f in ?? ()
#18 0x00002ca629ac7171 in ?? ()
#19 0x00007fffffffe040 in ?? ()
#20 0x00002ca629ac5b51 in ?? ()
#21 0x00007fffffffdfb8 in ?? ()
#22 0x0000000000000001 in ?? ()
#23 0x00001aa40db95c38 in ?? ()
#24 0x00000fa000000000 in ?? ()
#25 0x00002fa8dc005251 in ?? ()
#26 0x00002ca629ac7171 in ?? ()
#27 0x00000b75abf03459 in ?? ()
#28 0x00002ca629ac5b51 in ?? ()
#29 0x00007fffffffdfe8 in ?? ()
#30 0x00001aa40daf540e in ?? ()
#31 0x00002ca629ac7291 in ?? ()
#32 0x0000000100000000 in ?? ()
#33 0x00000b75abf03459 in ?? ()
#34 0x0000000800000000 in ?? ()
#35 0x00007fffffffe028 in ?? ()
#36 0x00001aa40db0fea1 in ?? ()
#37 0x0000000000000000 in ?? ()

The test case is written in JS and whenever I do something blocking for a long time in the JS, the data callbacks stop occuring like this. I can give you the latest version of my test case if that's any help, but I doubt it as its merely blocking and not even communicating back to the cubeb instance for now.

Oh yeah, sorry for reporting it here, I wasn't sure where to report it.

Right now, there's no way to signal an underrun from the user's data_callback to cubeb. In Gecko, it fills the remaining space with silent samples and tries to adjust the audio clock based on that, but suffers from latency-related miscalculations (see BMO 1008057).

At this stage, it seems like the right thing to do is add explicit underrun handling to cubeb's API to allow the caller to signal underruns to cubeb for handling internally. In some cases, this allows cubeb to signal this fact to the OS (for instance, cubeb can mark buffers as containing silence in the AudioUnit backend).

nhq@NHQ:~/development/barn/cubeb$ ./configure
checking build system type... i686-pc-linux-gnu
checking host system type... i686-pc-linux-gnu
checking target system type... i686-pc-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking how to run the C preprocessor... gcc -E
checking whether gcc and cc understand -c and -o together... yes
./configure: line 4203: AC_LIBTOOL_WIN32_DLL: command not found
./configure: line 4204: AM_PROG_LIBTOOL: command not found
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking alsa/asoundlib.h usability... yes
checking alsa/asoundlib.h presence... yes
checking for alsa/asoundlib.h... yes
checking pulse/pulseaudio.h usability... no
checking pulse/pulseaudio.h presence... no
checking for pulse/pulseaudio.h... no
checking AudioToolbox/AudioToolbox.h usability... no
checking AudioToolbox/AudioToolbox.h presence... no
checking for AudioToolbox/AudioToolbox.h... no
checking windows.h usability... no
checking windows.h presence... no
checking for windows.h... no
checking dsound.h usability... no
checking dsound.h presence... no
checking for dsound.h... no
checking sndio.h usability... no
checking sndio.h presence... no
checking for sndio.h... no
checking SLES/OpenSLES.h usability... no
checking SLES/OpenSLES.h presence... no
checking for SLES/OpenSLES.h... no
checking for doxygen... false
configure: WARNING: *** doxygen not found, API documentation will not be built
checking for stdint types... stdint.h (shortcircuit)
make use of stdint.h in include/cubeb/cubeb-stdint.h (assuming C99 compatible system)
./configure: line 5095: AC_PROG_LD: command not found
./configure: line 5096: AC_PROG_LD_GNU: command not found
checking how to control symbol export... configure: creating ./config.status
config.status: error: cannot find input file: `Makefile.in'

nhq@NHQ:~/development/barn/cubeb$ make
make: *** No targets specified and no makefile found. Stop.

Noticed this while running tests on the CMake branch. It seems to take a number of runs of test_record to trigger, e.g. 1 in 20 or so, and then I hit:

assert(input_buffer && input_frames_count && *input_frames_count && !output_buffer);

With *input_frames_count == 0, which comes from an empty stm->linear_input_buffer in refill_callback_input.

This build is regularly failing with:

    git checkout -qf %APPVEYOR_REPO_COMMIT%
    fatal: reference is not a tree: 9c08e150b0dca336d40285a1ac9d75a304a39e69
    Command exited with code 128

I originally thought this was caused by rebasing PRs while the build was queued, causing the original build to try to checkout a cset that no longer existed. But PR #196 is (I think) a cleanly opened PR with no rebase and has failed the same way.

The unused frames from method cubeb_resampler_fill depending on the cubeb_resampler instance. The noop_resampler resampler does not buffer any of the frames internally so it is expected to have remaining frames. On the other hand the cubeb_resampler_speex resampler buffers internally all input frames and uses them in following iterations as needed. In that case it is not expected to have any left over frames after fill.

I raise this issue first to let you know about the case and second to find a way to make resampler behavior more uniform.

The beginthreadex calls at
https://github.com/kinetiknz/cubeb/blob/master/src/cubeb_directsound.cpp#L305
https://github.com/kinetiknz/cubeb/blob/master/src/cubeb_winmm.c#L268
https://github.com/kinetiknz/cubeb/blob/master/src/cubeb_wasapi.cpp#L913
request a stack size of 64kB, but beginthreadex just calls CreateThread internally. The CreateThread documentation describes the STACK_SIZE_PARAM_IS_A_RESERVATION flag [0] which gives the behavior these calls probably want. Without it, the stack size flag sets the initial commit size, but the reserved address space is rounded up to the nearest MB and the stack is still allowed to grow to that limit. 32-bit applications already have limited address space, so this isn't ideal!

To fix this, you can simply pass STACK_SIZE_PARAM_IS_A_RESERVATION into the initflag parameter of beginthreadex. I initially reported this bug in Mozilla's bug tracker ( https://bugzilla.mozilla.org/show_bug.cgi?id=958796 ), so check that bug for more information. In particular, note that chromium IPC code uses this flag for CreateThread, and this same bug was filed and fixed for the Java VM.

[0] http://msdn.microsoft.com/en-us/library/windows/desktop/ms682453.aspx

Edit: The leading underscore in the function name seems to make text cursive and it doesn't show up anyway, so I removed it.

This issue comes from an error during landing of Bug 1286041. The error is described here:
https://treeherder.mozilla.org/logviewer.html#?job_id=35361403&repo=mozilla-inbound#L7758
As temporary solution the assert deactivated for winmm:
https://bugzilla.mozilla.org/show_bug.cgi?id=1286041#c53

It seems that the call of the data callback during the winmm_stream_int is intentional and exists from the very first version of the file. I raise this issue to ask why we need it and to see if there is an easy way to change it.

@kinetiknz: I know it's been four years or more but if you remember anything about this specific part please let us know.

Using JACK for Firefox now is completely dependent on upstream cubeb, in-tree one is not even updated enough to expose any JACK support. This is the most xrun-heavy piece of code on my system. And i tried playing 1080p movies with DTS sound via PA with max resampling settings as JACK client for testing under heavy load which were not giving any, in contrast to cubeb.

There are two known major issues with JACK support in cubeb:

1. Initialization order with no selection options that puts PA over JACK even if PA runs as JACK client.
   Fixed here.
2. Supposedly xruns are the result of long sleep in cubeb here.
   I won't be surprised if there are more since that seems to be not tested at all.

While there are some tests in cubeb, it is not uncommon to find issues when
merging the code into gecko, that exercises the code much more. This is always
an issue. The edit-compile-test cycle of standalone cubeb compared to Gecko
being, for now, much faster, it seems a good thing to improve testing. In
addition, having a clear boundary between modules is a good thing, and testing
them in isolation leads to code that is less coupled, so it's a good thing as
well.

cubeb is supposed to be a cross-platform audio IO library, that exposes an API
that has the same behaviour accross platforms. It should perform more or less
the same regardless of the platform or device that is being used. Device
switching should be handled, and automatic rate/channel/clock domain conversion
should be handled. This is a (long) test plan that can help reaching this goal.

Here, we care mostly about Windows 7+, Linux/Pulse, OSX, and Android.

API check

The goal here is to test that all backends support the same external API

• Double cubeb_stream_start, cubeb_stream_stop, and other API invariants. We
  might need to make a little spec document or a user manual or something.
• Check that we're not pre-filling buffers (that no callback is called during
  cubeb_stream_init). I think that is still the case on Windows XP.

Output

This is the part where cubeb has the most tests. On Mozilla infrastructure, we
can tests output for all platforms. It mostly works well. One thing we can't do
is to inspect the audio that has been output. On Linux and Windows, it's doable
using a monitor (or loopback) stream. There are no easy-to-use CI services that
have android, and existing CI services that we use don't always have audio
devices.

Input

This has tests that work locally, but are quite racy. Locally, on OSX, Windows
and Linux, tests that assert that some noise has been seen in the input
buffers of the callback tend to not work the first time the test is being run,
because it takes some time to wake up the audio stack it seems. It consistently
works fine during subsequent runs of the tests (if there is actually noise to be
heard of course). I suppose that waiting more could work. When running in CI, we
need to have a solution that is completely in-the-box.

Device change

We need a way to change the audio device setup programmatically, without clicking
buttons. There are a number of hacks and proper solutions that can be done to
solve this. See the section References.

Device with particular channel layout or sample-rate

If we can create our own devices, we can test this easily. I could not find a
way to change the sample-rate of devices programmatically on Windows or OSX
(I have not searched very long on OSX).

Resampler/drift compensation testing in isolation

This is to test the resampler and drift compensation code (not written yet, but
necessary at least on Windows), with real callback sequences:

• First, take a device that has a weird behaviour (such as Plantronics 648
  USB headsets)
• Record a callback sequence until a problem happens (for example, making a call
  for more than 5 minutes is known to cause drifting on this pair of headsets).
  In particular, buffer size, input and output callback with the number of
  frames. I don't think timing information is necessary.
• Then, write a harness for the resampler, where some synthetic audio is fed
  in the input, and some synthetic audio is written to the output, and check
  that the resampling is ok (no discontinuities on the waveform) and the drift
  compensation worked.

This should hopefully be cross-platform.

References

Now, some technical solutions and some code that can help us reach a point where
we're confident that tests are solid. This is very platform specific.

• Windows:
  • https://github.com/Microsoft/Windows-driver-samples/tree/master/audio/sysvad,
    the source code for an audio driver we could install on test-slaves and
    modify to suit our needs (I'm not sure of the license).
  • DevCon
    https://msdn.microsoft.com/en-us/library/windows/hardware/ff544707(v=vs.85).aspx
    allows fiddling with drivers (enable/disable/install drivers, etc.). For
    now, removing the driver simply makes the callback stop being called.
    Surely we can register to more events and recover. I have some notes on
    how to operate this weird command-line tools.
  • A number of packages allow having virtual audio devices. Writing our own
    (if needed) is probably not very hard.
• Linux/PulseAudio
  • We can use special pulseaudio modules to generate a sine wave in the
    input. We do that for WebRTC already, it works well:
    http://searchfox.org/mozilla-central/source/testing/mochitest/runtests.py#716
  • We can use the loopback module, play a random file or synthetize a tone,
    and use the loopback module to turn it into a source stream.
• OSX
  • https://github.com/mattingalls/Soundflower is a well-known program that
    lets user route audio and dynamically change audio devices and such. We
    can maybe use it stock, or use it as a base to create a custom .kext that
    acts as a fake sound card
  • https://github.com/deweller/switchaudio-osx has some code to change the
    default device.
• Android
  • Device switching generally works well on Android. I don't know what to do
    there, but I don't think it is a priority.

Google is offering free fuzzing of widely-used open source projects with their OSS-Fuzz initiative.
https://github.com/google/oss-fuzz

Given that cubeb is used in Firefox, I think it would make sense to take advantage of that offer and see what they find? :)

test_duplex and test_record fail locally because my laptop has a viable-seeming input device, but it's muted, so the seen_noise bool never becomes true despite the rest of the test running.

It seems like has_available_input_devices needs to check the selected input device's mute/volume to know if it should expect to see sound.

CC @achronop @padenot

In test_resampler.cpp, we resample an input and an output stream that are latency compensated, we generate test vectors, and we compare the two.

We offset the test vector by the latency induced by the resampler, but for some resampling ration, it's slightly off

For example, resampling a 48000Hz output stream into a 44100Hz output stream, while resampling a 8000Hz input stream to 44100Hz, the input stream is put exactly at the right position, but the output stream is off by 8 or 9 frames (resampled stream on top, expected on the bottom):

I don't know yet how to fuzz for this, and it's not a very big deal, so I'm disabling it (after manually checking it), and I'll investigate after landing.

I have this configuration:

• "Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller" built-in audio
• "Creative SB Tactic(3D) Wrath Wireless" USB headphones
• Linux 4.0.2
• Firefox 38.0 (built w/o PulseAudio support)
• pulseaudio 6.0

I have alsa-pulse enabled as the default ALSA output. When I select headphones as the output for Firefox (in pavucontrol), HTML5 videos fast-forward as fast as they can be streamed with garbled sound. When using built-in audio, everything's okay. Firefox is the only application which behaves like this -- for example, DeaDBeeF (music player) with ALSA backend selected works fine with either setting (it plays though alsa-pulse, of course).

What I've tried:

• Building Firefox with pulseaudio support enabled -- this fixes the issue.
• Updating libcubeb in Firefox source from git and building it -- no change.

The sound is definitely played though libcubeb (from grepping source and from CubebUtils in pavucontrol when playing with PulseAudio support) and libcubeb update doesn't solve it, so I figured out the bug might be in libcubeb (sorry if I misunderstand something!) To be clear: using PulseAudio backend does solve the problem, but Firefox is the only application with it so there might be some bug in libcubeb's ALSA backend.

Cubeb has some logic in src/cubeb_pulse.c, functions pulse_get_state_from_sink_port() and pulse_get_state_from_source_port() that tries to filter out unplugged devices from enumeration. Unfortunately, it is too aggressive. Namely, it returns CUBEB_DEVICE_STATE_DISABLED for sinks and sources with no ports at all.

By accident, both ALSA and Bluetooth sinks created by module-alsa-card and module-bluez5-device contain ports, so common setups work. However, less common setups involve sources and sinks without ports:

• Manually-loaded module-alsa-source and module-alsa-sink (without module-alsa-card - this is useful e.g. to get SPDIF output in addition to the regular analog output)
• Module-tunnel-source, module-tunnel-sink
• Module-ladspa-sink
• Module-null-sink's monitor is useful for capturing the output of some other application without letting it play to real speakers

One of the important setups broken by upgrading Firefox to a version that contains cubeb with this filter is the headless xrdp setup where the only sinks and sources available are provided by xrdp for forwarding the client's speakers and microphone to the session.