Dependabot couldn't fetch one or more of your project's path-based Rust dependencies. The affected dependencies were ../twang/Cargo.toml and ../fon/Cargo.toml.

To use path-based dependencies with Dependabot the paths must be relative and resolve to a directory in this project's source code.

View the update logs.

Hello @AldaronLau !

I am following your answer you made at the exact same question I am asking now but on reddit Cross platform desktop audio recording, do you know if it's already doable with your crate ? And if not do you know any other way to do it without configuring the audio output as input, maybe with https://github.com/RustAudio/rust-jack ?

I am asking this mainly because I can't run modprobe snd-aloop to create a loopback in my environment (bc its linux container on an arbitrary kernel that will not support module snd-aloop and netiher its host).

Thanks for your work !

I tried with no luck this adaptation of your test code:

use fon::{stereo::Stereo32, Sink, Audio};
use pasts::{exec, wait};
use wavy::{Speakers, SpeakersSink};

enum Event<'a> {
    Record(SpeakersSink<'a, Stereo32>),
}

struct State {
    buffer: Audio<Stereo32>,
}

impl State {
    fn event(&mut self, event: Event<'_>) {
        match event {
            Event::Record(microphone) => self.buffer.extend(microphone),
        }
    }
}

/// Program start.
fn main() {
    let mut state = State { buffer: Audio::with_silence(48_000, 0) };
    let mut speakers = Speakers::default();

    exec!(state.event(wait! {
        Event::Record(speakers.play().await),
    }));
}

Is your feature request related to a problem? Please describe.
It's really painful to work with e.g. tokio::tasks with non-send types.

Describe the solution you'd like
Implement send for the 4 relevant structs:

unsafe impl Send for Microphone {}
unsafe impl Send for MicrophoneStream {}
unsafe impl Send for Speakers {}
unsafe impl Send for SpeakersSink {}

Describe alternatives you've considered
Wrapping the structs in user code, very unergonomic.

Additional context
This should be safe without any other changes.

• Update to pasts 0.12.x #45
• Release fon 0.6.0
• Use upgraded fon 0.6.0
• Upgrade lookit crate to pasts 0.12.x
• Use lookit crate for quick asynchronous device discovery on Linux
• Figure out async discovery in JavaScript
• Make all types Send with whisk and smelling_salts 0.7.x #30

See #46 for next version roadmap.

New ticket.

uname -a
Linux archlinux 5.9.8-arch1-1 #1 SMP PREEMPT Tue, 10 Nov 2020 22:44:11 +0000 x86_64 GNU/Linux

aplay -l

**** List of PLAYBACK Hardware Devices ****
card 0: Device [USB PnP Sound Device], device 0: USB Audio [USB Audio]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: Intel [HDA Intel], device 0: CX20561 Analog [CX20561 Analog]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: Intel [HDA Intel], device 1: CX20561 Digital [CX20561 Digital]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

cat /etc/asound.conf

pcm.!default {
   type hw
   card 0
}
ctl.!default {
   type hw
   card 0
}

cargo run --example record

    Finished dev [unoptimized + debuginfo] target(s) in 1.07s
     Running `target/debug/examples/record`
thread '<unnamed>' panicked at 'Speaker configuration failed', src/ffi/linux/ffi.rs:93:29
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Any', /home/kolibrit/.cargo/registry/src/github.com-1ecc6299db9ec823/pasts-0.5.0/src/executor.rs:261:44

Dependabot couldn't fetch one or more of your project's path-based Rust dependencies. The affected dependencies were ../twang/Cargo.toml.

To use path-based dependencies with Dependabot the paths must be relative and resolve to a directory in this project's source code.

View the update logs.

Hello ! I've added wavy through cargo add wavy and effectively got wavy = "0.1.2" in my Cargo.toml, but I can't use it without getting a can't find crate rustc(E0463) error. Have I missed a step that would be specific to your crate ?

EDIT : I'm using rustc 1.40.0

When running the query example, I received no output.

When I checked, neither Speaker::query() nor Microphone::query() yielded any elements, although aplay -l and arecord -l do list multiple devices.
I would like to continuously stream audio from the mic for use in the application.

I'm on NixOS 23.11.20230626.6b3d1b1 (Tapir) with Linux-Kernel version 6.3.9 running pulseaudio

Output of arecord --dump-hw-params default:

Warning: Some sources (like microphones) may produce inaudible results
         with 8-bit sampling. Use '-f' argument to increase resolution
         e.g. '-f S16_LE'.
HW Params of device "default":
--------------------
ACCESS:  RW_INTERLEAVED
FORMAT:  U8 S16_LE S16_BE S24_LE S24_BE S32_LE S32_BE FLOAT_LE FLOAT_BE MU_LAW A_LAW S24_3LE S24_3BE
SUBFORMAT:  STD
SAMPLE_BITS: [8 32]
FRAME_BITS: [8 1024]
CHANNELS: [1 32]
RATE: [1 384000]
PERIOD_TIME: (2 4294967295)
PERIOD_SIZE: [1 1398102)
PERIOD_BYTES: [128 1398102)
PERIODS: [3 1024]
BUFFER_TIME: (7 4294967295]
BUFFER_SIZE: [3 4194304]
BUFFER_BYTES: [384 4194304]
TICK_TIME: ALL
--------------------

I am using wavy-version 0.9.1.
This issue is possibly/likely related to #55 which also occurs on NixOS

I recently tried building this on my Mac machine. It works fine on Linux, but failed on the Mac with the following build errors:

error[E0609]: no field `0` on type `&mut system::SpeakerSystem`
  --> /Users/bt/.cargo/registry/src/github.com-1ecc6299db9ec823/wavy-0.1.2/src/system.rs:71:14
   |
71 |         self.0.play(&mut || {
   |              ^

error[E0609]: no field `0` on type `&mut system::MicrophoneSystem`
  --> /Users/bt/.cargo/registry/src/github.com-1ecc6299db9ec823/wavy-0.1.2/src/system.rs:96:14
   |
96 |         self.0.record(generator);
   |              ^

The version of Rust I am running,

rustc --version
rustc 1.37.0 (eae3437df 2019-08-13)
cargo --version
cargo 1.37.0 (9edd08916 2019-08-02)

The Mac version is 10.14.6 (18G87)

The published library version is wavy = "0.1.2"

If I get time I'll look into this, but wanted to give you a head's up.

A compiled example panics when running let mut speakers = Speakers::default();

To reproduce the bug:

1. Go to the wavy documentation page.
2. Copy the dependencies to the Cargo.toml file
3. Replace the contents of the main.rs file with the example code.
4. Enter RUST_BACKTRACE=1 cargo run in the terminal.
5. See the error in the backtrace:

thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', /home/gabbie/.cargo/registry/src/github.com-1ecc6299db9ec823/wavy-0.8.1/src/ffi/linux/speakers.rs:87:18
stack backtrace:
   0: rust_begin_unwind
             at /rustc/fe5b13d681f25ee6474be29d748c65adcd91f69e/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/fe5b13d681f25ee6474be29d748c65adcd91f69e/library/core/src/panicking.rs:143:14
   2: core::panicking::panic
             at /rustc/fe5b13d681f25ee6474be29d748c65adcd91f69e/library/core/src/panicking.rs:48:5
   3: core::option::Option<T>::unwrap
             at /rustc/fe5b13d681f25ee6474be29d748c65adcd91f69e/library/core/src/option.rs:755:21
   4: <wavy::ffi::speakers::Speakers as core::default::Default>::default
             at /home/gabbie/.cargo/registry/src/github.com-1ecc6299db9ec823/wavy-0.8.1/src/ffi/linux/speakers.rs:86:13
   5: <wavy::speakers::Speakers as core::default::Default>::default
             at /home/gabbie/.cargo/registry/src/github.com-1ecc6299db9ec823/wavy-0.8.1/src/speakers.rs:66:21
   6: record_n_play::main
             at ./src/main.rs:32:24
   7: core::ops::function::FnOnce::call_once
             at /rustc/fe5b13d681f25ee6474be29d748c65adcd91f69e/library/core/src/ops/function.rs:227:5

My expectation was that audio would be recorded and played back in real time.

Laptop information:

• OS: NixOS 21.11.337975.eabc3821918 (Porcupine) with Linux kernel version 5.15.50
• Model: HP Laptop 14-dk0028wm

I look forward to learning from this.
Thanks!

• wavy v0.9.1
• OS Arch linux
• Confirmed default microphone to work in slack, discord, zoom, cpal crate

use anyhow::Result;
use std::thread::sleep;
use std::time::Duration;
// This example records audio for 5 seconds and writes to a raw PCM file.

use fon::{mono::Mono32, Audio, Frame};
use wavy::{Microphone, MicrophoneStream};

/// An event handled by the event loop.
enum Event<'a> {
  /// Microphone has recorded some audio.
  Record(MicrophoneStream<'a, Mono32>),
}

/// Shared state between tasks on the thread.
struct State {
  /// Temporary buffer for holding real-time audio samples.
  buffer: Audio<Mono32>,
}

impl State {
  /// Event loop.  Return false to stop program.
  fn event(&mut self, event: Event<'_>) {
    match event {
      Event::Record(microphone) => {
        println!("Recording");
        self.buffer.extend(microphone);
        if self.buffer.len() >= 48_000 * 10 {
          write_pcm(&self.buffer);
        }
      }
    }
  }
}

/// Save a Raw PCM File from an audio buffer.
fn write_pcm(buffer: &Audio<Mono32>) {
  let mut pcm: Vec<u8> = Vec::new();
  for frame in buffer.iter() {
    let sample: f32 = frame.channels()[0].into();
    pcm.extend(sample.to_le_bytes().iter());
  }

  dbg!(&pcm);
}

async fn wait() -> Result<()> {
  sleep(Duration::from_millis(2000));

  Ok(())
}

#[tokio::main]
async fn main() -> Result<()> {
  let mut state = State {
    buffer: Audio::with_silence(48_000, 0),
  };
  let mut microphone = Microphone::default();

  state.event(Event::Record(microphone.record().await));

  wait().await?;

  dbg!("finished");

  Ok(())
}

It logs "recording" once, but never reaches to the limit of the buffers to write.

Hello again)

in wavy/examples/record.rs:

struct State {
    buffer: Audio<Mono16>,
}

if change to:

struct State {
    buffer: Audio<Stereo16>,
}

problem:

state.buffer.extend(&mut stream);

the trait bound impl fon::Stream<Sample1<Ch16>>: fon::Stream<Sample2<Ch16>> is not satisfied
the trait fon::Stream<Sample2<Ch16>> is not implemented for impl fon::Stream<Sample1<Ch16>>

because:

let mut stream = mic.record().await;

pub async fn record(&mut self) -> impl Stream<Sample1> + '_

Stereo16 = Sample2<...>

Describe the bug
First of all, thanks for your great work on this library! While writing a program for reading an mp3 using rodio and playing it back using wavy, I noticed that samples seem to be consumed from Audio buffers with .drain() at a rate that does not correspond to the rate specified in the Audio buffer. See the program below for an example. This results in the audio being quite distorted and slowed down / stuttering in my case. I tried manually feeding chunks to the SpeakersSink at the appropriate sample rate. This does seem to fix the distorted sound issue (minus some audible clicks inbetween feeding the chunks, which are expected), but then the speakers.play().await call freezes randomly after a few seconds in the song. Perhaps I am using the API in a wrong way?

To Reproduce
Here is the example code to measure the approximate samples per second consumed from the buffer:

use fon::{stereo::Stereo32, Sink, Audio};
use pasts::{exec, wait};
use wavy::{Speakers, SpeakersSink};
use rodio::decoder::Decoder;
use std::io::BufReader;
use std::time::Instant;

enum Event<'a> {
    Play(SpeakersSink<'a, Stereo32>)
}

struct State {
    buffer: Audio<Stereo32>,
    last_time: Instant,
    last_len: usize
}

impl State {
    fn event(&mut self, event: Event<'_>) {
        match event {
            Event::Play(mut speakers) => { 
                speakers.stream(self.buffer.drain());
                let current_time = Instant::now();
                let current_len = self.buffer.len();
                let diff_time = current_time - self.last_time;
                let diff_len = self.last_len - current_len;
                //println!("Estimated samples per instant: {:?}/{:?}", diff_len, diff_time);
                let diff_time_ns = diff_time.subsec_nanos() as f64;
                let sps = (diff_len as f64) / diff_time_ns * 1000000000.0;
                println!("Estimated samples per second: {:?}", sps);

                self.last_time = current_time;
                self.last_len = current_len;
            }
        }
    }
}

fn main() {
    let mut speakers = Speakers::default();
    let example_decoder = Decoder::new(BufReader::new(std::fs::File::open("/path/to/song.mp3").unwrap())).unwrap();
    let vector: Vec<f32> = example_decoder.map(|x| (x as f32) / 32768.0).collect();
    let buffer = Audio::<Stereo32>::with_f32_buffer(48_000, vector);
    let mut state = State { last_len: buffer.len(), buffer: buffer, last_time: Instant::now() };

    exec!(state.event(wait! {
        Event::Play(speakers.play().await),
    }))
}

Expected behavior
I would expect samples to be consumed at 48,000 samples per second as specified in the Audio buffer (and on my soundcard), but they are consumed at about ~20,000 samples per second on my system.

Screenshots
N/A

Desktop (please complete the following information):

• OS: Linux 5.9.16

Additional context
The examples from the examples folder work fine (e.g. when recording using Microphone and immediately playing back like in monitor.rs, this issue does not occur).

Any idea what might be the cause?

Dependabot couldn't fetch one or more of your project's path-based Rust dependencies. The affected dependencies were ../../RedAldaron/pasts/Cargo.toml and ../../RedAldaron/smelling-salts/Cargo.toml.

To use path-based dependencies with Dependabot the paths must be relative and resolve to a directory in this project's source code.

View the update logs.

Can you provide any guidance on how to stream to something like AWS Transcribe instead of Speaker? I'd like to speak into a microphone and stream to AWS Transcribe for a real-time transcription.

Here's an example that reads from a wav file instead: https://github.com/awslabs/aws-sdk-rust/blob/main/examples/transcribestreaming/src/main.rs

In webaudio/record.html the latency is fine, but when the audio is sent to WASM and then back to JavaScript in examples/stdweb-record/ the latency is not good enough for real-time audio applications.

How to choose external USB-audio card for record?

Compiled the Record example for wasm and upon execution, it returns

panicked at 'called `Result::unwrap()` on an `Err` value: JsValue(IndexSizeError: BaseAudioContext.createScriptProcessor: 64 is not a valid bufferSize
init/imports.wbg.__wbg_createScriptProcessor_4b76206bc3949bda<@http://localhost:8000/gen/record.js:368:35
handleError/<@http://localhost:8000/gen/record.js:240:22
@http://localhost:8000/gen/record_bg.wasm:wasm-function[124]:0x80a3
@http://localhost:8000/gen/record_bg.wasm:wasm-function[59]:0x543f
@http://localhost:8000/gen/record_bg.wasm:wasm-function[54]:0x4bd8
@http://localhost:8000/gen/record_bg.wasm:wasm-function[125]:0x80f1
@http://localhost:8000/gen/record_bg.wasm:wasm-function[238]:0x90fc
wasm_start_main@http://localhost:8000/gen/record.js:234:10
@http://localhost:8000/:9:13
promise callback*@http://localhost:8000/:8:16
)', /home/frag/.cargo/registry/src/github.com-1ecc6299db9ec823/wavy-0.9.0/src/ffi/wasm/ffi.rs:75:18

Stack:

init/imports.wbg.__wbg_new_59cb74e423758ede@http://localhost:8000/gen/record.js:300:19
@http://localhost:8000/gen/record_bg.wasm:wasm-function[49]:0x433e
@http://localhost:8000/gen/record_bg.wasm:wasm-function[237]:0x90f7
@http://localhost:8000/gen/record_bg.wasm:wasm-function[79]:0x6b4e
@http://localhost:8000/gen/record_bg.wasm:wasm-function[96]:0x757a
@http://localhost:8000/gen/record_bg.wasm:wasm-function[162]:0x8a08
@http://localhost:8000/gen/record_bg.wasm:wasm-function[145]:0x8702
@http://localhost:8000/gen/record_bg.wasm:wasm-function[163]:0x8a3d
@http://localhost:8000/gen/record_bg.wasm:wasm-function[102]:0x789d
@http://localhost:8000/gen/record_bg.wasm:wasm-function[59]:0x5540
@http://localhost:8000/gen/record_bg.wasm:wasm-function[54]:0x4bd8
@http://localhost:8000/gen/record_bg.wasm:wasm-function[125]:0x80f1
@http://localhost:8000/gen/record_bg.wasm:wasm-function[238]:0x90fc
wasm_start_main@http://localhost:8000/gen/record.js:234:10
@http://localhost:8000/:9:13
promise callback*@http://localhost:8000/:8:16

Describe the bug
Running wavy in another async executor, such as the futures crate, results in an Invalid Instruction. I had a quick look into the pasts crate and what its block_on does, but I haven't figured out why this happens. It looks like undefined behaviour to me.

To Reproduce
Here's a modified version of monitor.rs, that I could trigger the bug with:

use fon::{stereo::Stereo32, Audio, Sink};
use futures::prelude::*;
use wavy::{Microphone, MicrophoneStream, Speakers, SpeakersSink};

/// An event handled by the event loop.
enum Event<'a> {
    /// Speaker is ready to play more audio.
    Play(SpeakersSink<'a, Stereo32>),
    /// Microphone has recorded some audio.
    Record(MicrophoneStream<'a, Stereo32>),
}

/// Shared state between tasks on the thread.
struct State {
    /// Temporary buffer for holding real-time audio samples.
    buffer: Audio<Stereo32>,
}

impl State {
    fn event(&mut self, event: Event<'_>) {
        match event {
            Event::Play(mut speakers) => speakers.stream(self.buffer.drain()),
            Event::Record(microphone) => self.buffer.extend(microphone),
        }
    }
}

async fn playback(mut state: State, mut speakers: Speakers, mut microphone: Microphone) {
    loop {
       let event = futures::select! {
            input = microphone.record().fuse() => Event::Record(input),
            output = speakers.play().fuse() => Event::Play(output)
       };
		
        state.event(event)
    }
}

/// Program start.
fn main() {
    let state = State {
        buffer: Audio::with_silence(48_000, 0),
    };
    let speakers = Speakers::default();
    let microphone = Microphone::default();

    futures::executor::block_on(playback(state, speakers, microphone));
    // This works
    // pasts::block_on(playback(state, speakers, microphone));
}

Expected behavior
Not an "invalid instruction" 😉

I expected it either to work the same as with the pasts crate or give me some form of compile time error.

Desktop (please complete the following information):

• OS: Debian buster
• No Browser, running as a native linux binary

Dependabot couldn't fetch one or more of your project's path-based Rust dependencies. The affected dependencies were ../pasts/Cargo.toml and ../smelling-salts/Cargo.toml.

To use path-based dependencies with Dependabot the paths must be relative and resolve to a directory in this project's source code.

View the update logs.

Dependabot couldn't fetch one or more of your project's path-based Rust dependencies. The affected dependencies were ../../lordaldaron/smelling-salts/Cargo.toml.

To use path-based dependencies with Dependabot the paths must be relative and resolve to a directory in this project's source code.

View the update logs.

Thanks for all your work in maintaining this awesome project!

Is your feature request related to a problem? Please describe.
I'm wondering if it'd be possible to have a working example of an API for splitting audio on silences. I tried to put something together using a concoction of the existing APIs but I couldn't really get it to work. :(

Describe the solution you'd like
The intended usage could be along the lines of:

use wavy::Microphone;
use fon::Audio;

#[tokio::main]
pub async fn main() {
    let mut microphone = Microphone::default();

    let stream = microphone.record().await;
    let threshold = ...;
    let noise = stream.split_on_silence(threshold);

    while let Ok(audio) = noise.next().await {
        // here `audio` is an Audio<...> instance.
        process_audio(audio)
    }
}

fn process_audio(audio: Audio<...>) {
    // do something with the samples.
}

(aside) Would it be possible to make these APIs compatible with tokio (and if it already is could we please add a couple of examples)?

• Finally finish groundwork for MacOS Support
• Windows support?
• Support microphones that don't work in interleaved f32le, (fallback to i24le, then i16le) - Convert to f32
• Support AudioWorklet as alternate DOM API (Maybe Push to 0.11.0)