So normally we can instantiate Arecorder and Aserver and boot them separately. But in

<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto

In the same above example, creating multiple objects also work: <div class="snippe

adapted minimal example. <div class="highlight highlight-source-python notranslate

Can't have Aserver() and Arecorder() booted separately when using the same device. about pya HOT 21 CLOSED

wiccy46 commented on June 15, 2024

Can't have Aserver() and Arecorder() booted separately when using the same device.

from pya.

Comments (21)

wiccy46 commented on June 15, 2024

I created a test object to have a play_stream and record_stream open separately, which worked fine. Both record and play callback are called repeatedly. So this suggests it is our implementation of Arecorder or Aserver which caused the issue. However, I have yet to figure out what caused this.

# This belongs to a test audio server class, no parent class.
    def boot_output_stream(self):
        """boot recorder. Once booted, 
        the pa stream is open, but no audio is coming out. """
        self.play_stream = self.pa.open(rate=self.sr, 
                                        channels=self.out_chns,
                                        frames_per_buffer=self.bs, 
                                        format=self.format,
                                        output=True, 
                                        input=False,
                                        output_device_index=self.device,
                                        stream_callback=self._play_callback)
        self.silence = np.zeros(self.bs * self.out_chns)
        self.play_stream.start_stream()
        _LOGGER.info("Play Server Booted")

    def boot_input_stream(self):
        """boot recorder. Once booted, 
        the pa stream is open, but no audio is coming out. """
        self.record_stream = self.pa.open(rate=self.sr, channels=self.in_chns,
                                          frames_per_buffer=self.bs, format=self.format,
                                          output=False, input=True,
                                          input_device_index=self.device,
                                          stream_callback=self._recorder_callback)
        self.record_buffer = []
        self._recording = False  # TODO is this still accessible?
        self.record_stream.start_stream()
        _LOGGER.info("Record Server Booted")

    def _play_callback(self, in_data, frame_count, time_info, flag):
        print("Play")
        return self.silence, pyaudio.paContinue

    def _recorder_callback(self, in_data, frame_count, time_info, flag):
        print("record")
        return None, pyaudio.paContinue

from pya.

aleneum commented on June 15, 2024

I created a test object to have a play_stream and record_stream open separately, which worked fine. Both record and play callback are called repeatedly.

did you close the streams after each call? I assume you cannot access a device with an open stream. That would explain why you have to pass in and output flags to the same function at the same time.

from pya.

aleneum commented on June 15, 2024

import pyaudio

p1 = pyaudio.PyAudio()
p2 = pyaudio.PyAudio()

p_args = dict(rate=44100, channels=2, frames_per_buffer=256, format=pyaudio.paFloat32,
              input_device_index=2, output_device_index=2,
              output=False, input=False, stream_callback=None)
p1_args = dict(p_args, output=True)
p2_args = dict(p_args, input=True)

p1s = p1.open(**p1_args)
p2s = p2.open(**p2_args)
>>> ||PaMacCore (AUHAL)|| Error on line 2490: err='-50', msg=Unknown Error

if i switch the last two lines (first open input and afterwards output)

p2s = p2.open(**p2_args)
p1s = p1.open(**p1_args)
>>> ||PaMacCore (AUHAL)|| Error on line 2490: err='-10863', msg=Audio Unit: cannot do in current context

I assume that somewhere open_stream will open a second thread which accesses the underlying coreaudio device and since this device is NOT threadsafe access fails.

from pya.

wiccy46 commented on June 15, 2024

Yes, and this will work, with only one PyAudio() object that returns multiple stream.

import pyaudio

p1 = pyaudio.PyAudio()

p_args = dict(rate=44100, channels=2, frames_per_buffer=256, format=pyaudio.paFloat32,
              input_device_index=3, output_device_index=3,
              output=False, input=False, stream_callback=None)
p1_args = dict(p_args, output=True)
p2_args = dict(p_args, input=True)

p1s = p1.open(**p1_args)
p2s = p1.open(**p2_args)

Do you think this is deep in PortAudio and we can't do anything about it?

from pya.

aleneum commented on June 15, 2024

Yes, and this will work, with only one PyAudio() object that returns multiple stream.

it seems that the number of PyAudio objects is not the decisive factor here. Even using one PyAudio object causes the same error. I checked that Friday and reassured it today.
I don't think that this a bug but just a hint about how streams should be used and handled. Usually you have one callback for both, input and output. If we need to handle streams differently we need to implement a proper management of streams and callbacks.

from pya.

wiccy46 commented on June 15, 2024

I think this is still pya related. This example instantiate one PyAudio and open two separated streams. Both streams worked fine without being blocked.

import numpy as np
import pyaudio
import time

class AudioServer():
    def __init__(self, sr=22050, bs=512, input_device=None, output_device=None, in_chns=None, out_chns=2, format=pyaudio.paFloat32):
        self.sr = sr
        self.bs = bs
        self.format = format
        self.in_chns = in_chns
        self.out_chns = out_chns
        self.pa = pyaudio.PyAudio()
        self.dtype = 'float32'
        self._input_device = 0  # init
        self._output_device = 1  # init
        self.input_device = self.pa.get_default_input_device_info(
        )['index'] if input_device is None else input_device
        self.output_device = self.pa.get_default_output_device_info(
        )['index'] if output_device is None else output_device

    @property
    def input_device(self):
        return self._input_device

    @input_device.setter
    def input_device(self, val):
        self._input_device = val
        self.input_device_dict = self.pa.get_device_info_by_index(
            self._input_device)
        self.max_in_chn = self.input_device_dict['maxInputChannels']
        self.in_chns = self.max_in_chn

    @property
    def output_device(self):
        return self._output_device

    @output_device.setter
    def output_device(self, val):
        self._output_device = val
        self.output_device_dict = self.pa.get_device_info_by_index(
            self._output_device)
        self.max_out_chn = self.output_device_dict['maxOutputChannels']
        if self.out_chns is None:
            self.out_chns = self.max_out_chn

    def boot_output_stream(self):
        self.play_stream = self.pa.open(rate=self.sr, channels=self.out_chns,
                                        frames_per_buffer=self.bs, format=self.format,
                                        output=True, input=False,
                                        output_device_index=self.output_device,
                                        stream_callback=self._play_callback)
        self.empty_buffer = np.zeros(self.bs * self.out_chns, dtype=self.dtype)
        self.play_stream.start_stream()

    def boot_input_stream(self):
        self.window = np.hanning(self.bs)  # Need a new window if bs changes
        self.record_stream = self.pa.open(rate=self.sr, channels=self.in_chns,
                                          frames_per_buffer=self.bs, format=self.format,
                                          output=False, input=True,
                                          input_device_index=self.input_device,
                                          stream_callback=self._recorder_callback)
        self.record_stream.start_stream()

    def _play_callback(self, in_data, frame_count, time_info, flag):
        print("player")
        return self.empty_buffer, pyaudio.paContinue

    def _recorder_callback(self, in_data, frame_count, time_info, flag):
        print("recorder")
        return None, pyaudio.paContinue
    
as1 = AudioServer(input_device=3, output_device=3)
as1.boot_output_stream()
as1.boot_input_stream()
time.sleep(0.5)

from pya.

wiccy46 commented on June 15, 2024

In the same above example, creating multiple objects also work:

as1 = AudioServer(output_device=3)
as1.boot_output_stream()

as2 = AudioServer(input_device=3)
as2.boot_input_stream()
time.sleep(0.5)

from pya.

aleneum commented on June 15, 2024

The problem seems to be related to the buffer size: the minimal code posted here does not return errors when the buffer size is 512 instead of the default 256. In your class you use 512 instead of 256 which makes it not return errors.

from pya.

aleneum commented on June 15, 2024

adapted minimal example.

import pyaudio
import numpy as np

bs = 512
channels = 2
buffer = np.zeros(bs * channels, dtype=np.float32)


def record(*args, **kwargs):
    print("recording")
    return None, pyaudio.paContinue


def play(*args, **kwargs):
    print("play")
    return buffer, pyaudio.paContinue


p_args = dict(rate=44100, channels=channels, frames_per_buffer=bs, format=pyaudio.paFloat32,
              input_device_index=2, output_device_index=2,
              output=False, input=False, stream_callback=None)
p1_args = dict(p_args, output=True, stream_callback=play)
p2_args = dict(p_args, input=True, stream_callback=record)
# # also works when bs=512
# p2_args = dict(p_args, input=True, stream_callback=record, frames_per_buffer=256) 

pa = pyaudio.PyAudio()
p1s = pa.open(**p1_args)
p2s = pa.open(**p2_args)

from pya.

wiccy46 commented on June 15, 2024

Interesting finding. For my example, I got the pamaccore error when buffer size is at <=256. >=512 seems to work just fine.

from pya.

aleneum commented on June 15, 2024

it's most likely a racing condition. I assume streams and/or their callbacks will be handled in threads and at some point during initialization there is a non-threadsafe ressource collision.
If you time.sleep(0.5) between opening streams, the error does not occur even with bs=256:

p1s = pa.open(**p1_args)
time.sleep(0.5)
p2s = pa.open(**p2_args)

from pya.

aleneum commented on June 15, 2024

with wait. order of stream opening does not matter.

import pyaudio
import numpy as np
import time

bs = 128
channels = 2
buffer = np.zeros(bs * channels, dtype=np.float32)


def record(*args, **kwargs):
    print("recording")
    return None, pyaudio.paContinue


def play(*args, **kwargs):
    print("play")
    return buffer, pyaudio.paContinue


p_args = dict(rate=44100, channels=channels, frames_per_buffer=bs, format=pyaudio.paFloat32,
              input_device_index=2, output_device_index=2,
              output=False, input=False, stream_callback=None)
p1_args = dict(p_args, output=True, stream_callback=play)
p2_args = dict(p_args, input=True, stream_callback=record)

pa = pyaudio.PyAudio()
p1s = pa.open(**p1_args)
time.sleep(0.05)
p2s = pa.open(**p2_args)

from pya.

wiccy46 commented on June 15, 2024

No. order does not matter. The problem seems to be in the thread indeed. When I put player and recorder in a thread. There is no issue, and I can even have different bs for each thread. :

import numpy as np
import pyaudio
import time
from threading import Thread

class AudioServer():
    def __init__(self, sr=44100, bs=16, input_device=None, output_device=None, in_chns=None, out_chns=2, format=pyaudio.paFloat32):
        self.sr = sr; self.bs = bs; self.format = format
        self.channels = 2
        self.pa = pyaudio.PyAudio()
        self.input_device = input_device; self.output_device = output_device

    def boot_output_stream(self):
        self.play_stream = self.pa.open(rate=self.sr, channels=self.channels,
                                        frames_per_buffer=self.bs, format=self.format,
                                        output=True, input=False,
                                        output_device_index=self.output_device,
                                        stream_callback=self._play_callback)
        self.empty_buffer = np.zeros(self.bs * self.channels, dtype='float32')
        self.play_stream.start_stream()

    def boot_input_stream(self):
        self.record_stream = self.pa.open(rate=self.sr, channels=self.channels,
                                          frames_per_buffer=self.bs, format=self.format,
                                          output=False, input=True,
                                          input_device_index=self.input_device,
                                          stream_callback=self._recorder_callback)
        self.record_stream.start_stream()

    def _play_callback(self, in_data, frame_count, time_info, flag):
        print("player")
        return self.empty_buffer, pyaudio.paContinue

    def _recorder_callback(self, in_data, frame_count, time_info, flag):
        print("recorder")
        return None, pyaudio.paContinue
    
    def quit_input(self):
        self.record_stream.stop_stream()
        self.record_stream.close()
            
    def quit_output(self):
        self.play_stream.stop_stream()
        self.play_stream.close()


class PlayerThread(Thread):
    def __init__(self, bs):
        Thread.__init__(self)
        self.as1 = AudioServer(bs=bs, input_device=3, output_device=3)
    
    def run(self):
        self.as1.boot_output_stream()
        
    def stop(self):
        self.as1.quit_output()
    

class RecorderThread(Thread):
    def __init__(self, bs):
        Thread.__init__(self)
        self.as1 = AudioServer(bs=bs, input_device=3, output_device=3)
        
    def run(self):
        self.as1.boot_input_stream()
        
    def stop(self):
        self.as1.quit_input()
       
bs = 128
recorder = RecorderThread(bs=bs)
recorder.start()
player = PlayerThread(bs=256)
player.start()
time.sleep(1)
player.stop()
recorder.stop()

So it seems the workaround maybe to have a class that has both player and recorder in pya. Then instantiate a different thread for each.

from pya.

aleneum commented on June 15, 2024

No. order does not matter. The problem seems to be in the thread indeed. When I put player and recorder in a thread. There is no issue:

that's not the proper solution because it induces a small delay by having to initialize threads.

from pya.

wiccy46 commented on June 15, 2024

A small delay for Aserver will be a problem. But with Arecorder this doesn't seem to be an issue though.

from pya.

aleneum commented on June 15, 2024

well, if the solution should be a time.sleep it needs to be in BOTH boot methods since we cannot assert which server is booted first.

from pya.

aleneum commented on June 15, 2024

A small delay for Aserver will be a problem.

its not certain but if the ressource collision is only a matter during opening streams, the time.sleep would be part of the boot procedure. Would a boot time which is 50ms longer actually be an issue?

from pya.

wiccy46 commented on June 15, 2024

If it is in boot() only there is no issue at all. Even a second or so. Originally, I thought you meant delay to the whole callback.

from pya.

aleneum commented on June 15, 2024

If it is in boot() only there is no issue at all. Even a second or so.

I guess we can start small, make it a class variable and increase the delay when needed.

If it is in boot() only there is no issue at all. Even a second or so.

as mentioned in my earlier post:

If you time.sleep(0.5) between opening streams, the error does not occur even with bs=256:

we maybe fine with only delaying opening operations. There might be issues during callbacks which we havent stumble upon yet. I'll add a callback delay and release 0.3.1 afterwards. unfortunately we cannot really test that corner case with CI.

from pya.

wiccy46 commented on June 15, 2024

I think we should start safe. I tried giving a sleep after start_stream() in both. I had some inconsistent results. My safe zone seems to be > 300ms.

from pya.

aleneum commented on June 15, 2024

#25 should solve this. Should be possible to do this in 0.3.1, shouldn't it @wiccy46?

from pya.

Can't have Aserver() and Arecorder() booted separately when using the same device. about pya HOT 21 CLOSED

Comments (21)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent