Giter VIP home page Giter VIP logo

esp-skainet's Introduction

ESP-Skainet [中文]

ESP-Skainet is Espressif's intelligent voice assistant, which currently supports the Wake Word Engine and Speech Commands Recognition.

ESP32-S3 is recommend to run speech commands recognition, which supports AI instructions and high-speed octal SPI PSRAM. The Latest models will be deployed on ESP32-S3 first.

Overview

ESP-Skainet supports the development of wake word detection and speech commands recognition applications based around Espressif Systems' ESP32 series chip in the most convenient way. With ESP-Skainet, you can easily build up wake word detection and speech command recognition applications.

In general, the ESP-Skainet features will be supported, as shown below:

overview

Input Voice Stream

The input audio stream can come from any way of providing voice, such as MIC, wav/pcm files in flash/SD Card.

Wake Word Engine

Espressif wake word engine WakeNet is specially designed to provide a high performance and low memory footprint wake word detection algorithm for users, which enables devices always wait for wake words, such as "Alexa", “天猫精灵” (Tian Mao Jing Ling), and “小爱同学” (Xiao Ai Tong Xue).

Currently, Espressif has not only provided an official wake word "Hi, Lexin" to the public for free but also allows customized wake words. For details on how to customize your own wake words, please see Espressif Speech Wake Words Customization Process.

Speech Commands Recognition

Espressif's speech command recognition model MultiNet is specially designed to provide a flexible offline speech command recognition model. With this model, you can easily add your own speech commands, eliminating the need to train model again.

Currently, Espressif MultiNet supports up to 200 Chinese or English speech commands, such as “打开空调” (Turn on the air conditioner) and “打开卧室灯” (Turn on the bedroom light).

Audio Front End

Espressif Audio Front-End AFE integrates AEC (Acoustic Echo Cancellation), VAD (Voice Activity Detection),BSS (Blind Source Separation) and NS (Noise Suppression).

Our two-mic Audio Front-End (AFE) have been qualified as a “Software Audio Front-End Solution” for Amazon Alexa Built-in devices. afe

Quick Start with ESP-Skainet

Hardware Preparation

To run ESP-Skainet, you need to have an ESP32 or ESP32-S3 development board which integrates an audio input module . Development board Support:

Example Name Latest Models Supported Board
cn_speech_commands_recognition Multinet6 ESP32-Korvo, ESP32-S3-Korvo-1, ESP-BOX, ESP-S3-Korvo-2, ESP32-S3-EYE
en_speech_commands_recognition Multinet6 ESP32-S3-Korvo-1, ESP-BOX, ESP-S3-Korvo-2, ESP32-S3-EYE
wake_word_detection Wakenet9 ESP32-Korvo, ESP32-S3-Korvo-1, ESP-BOX, ESP-S3-Korvo-2, ESP32-S3-EYE
chinese_tts esp-tts-v1.7 ESP32-Korvo, ESP32-S3-Korvo-1, ESP-BOX, ESP-S3-Korvo-2
usb_mic_recorder ESP-BOX, ESP-S3-Korvo-2

On how to configure your applications, please refer to the README.md of each example.

Software Preparation

ESP-Skainet

Clone this project as follows:

git clone https://github.com/espressif/esp-skainet.git 

ESP-IDF

ESP-IDF v4.4 and ESP-IDF v5.0 are supported. If you had already configured ESP-IDF before, and do not want to change your existing one, you can configure the IDF_PATH environment variable to the path to ESP-IDF.

For details on how to set up the ESP-IDF, please refer to Getting Started Guide for ESP-IDF release/v4.4 branch

Examples

The folder of examples contains some applications demonstrating the API features of ESP-Skainet.

Please start with the wake_word_detection example.

  1. Navigate to one example folder `esp-skainet/examples/wake_word_detection).
cd esp-skainet/examples/wake_word_detection
  1. Compile and flash the project.
idf.py flash monitor
  1. Advanced users can add or modify speech commands by using the idf.py menuconfig command.

For details, please read the README file in each example.

Resources

esp-skainet's People

Contributors

aderusha avatar esp-doc avatar esp-momo avatar espwangning avatar feizi avatar heinzma avatar igrr avatar lijunru-hub avatar qianh-wan avatar shenxiaozheng avatar sun-xiangyu avatar tore-espressif avatar udoudou avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

esp-skainet's Issues

mase issue

Hellow.

  1. I found an error in the MASE algorithm. When MASE amplifies the signal, the INT16 value overflow is ignored and significant signal distortion is obtained. The signal should not overflow when it reaches the upper and lower limits of the value but should simply be cut off. Please fix this

bag_mase

top signal - after mase algortthm. bottom signal - simple example (the signal amplitude represented as INT16 without tracking overflow during the amplification (multiplication) operation results in this effect.)

  1. Also, the filter_strength parameter does not affect the result. Why do I get the same results when I select 0, 1,2,3,4,5,6?

When will it support arduino?

When will it support arduino? PlatformIO arduino?If it is supported, mqtt can be combined with speech recognition, which will be much more powerful!

使用Cmake编译报错

以将组件放置在工程的components目录下,但是编译还是报错,报如下错误

CMake Error at F:/esp/components/esp-sr/CMakeLists.txt:15 (target_link_libraries):
  Cannot specify link libraries for target "esp-sr" which is not built by
  this project.

距离较远时,语音命令的识别率较低.

在比较安静的环境下当距离麦克风60厘米左右时,经100次语音控制测试turn/off the light,唤醒词的识别率接近100%,但语音命令的识别率只有不到20%,不知这个问题有没有优化的方案呢?补充:20~40厘米的话识别率有50%~70%.

更换唤醒词,出现驱动解码芯片出错!

  • 我给客户演示了这个仓库的效果,还不错!就是想换个 唤醒词,刚好看到你们那有《你好小智》这个唤醒词,然后我改了下,竟然串口上电很大几率出现如下截图的红色字符,貌似是初始化解码芯片出错了,但是我换回来 唤醒词 “ 你好乐鑫” 又没事了!

  • 也许是这个原因,导致后面无法唤醒板子;

image

Esp32-Korvo. Wrong sampling rate? (AIS-1237)

Hi,

I'm streaming audio data over from the esp32-korvo to a server. All the examples returned very bad sound quality and the audio was distorted (could not hear my voice), so I reduced it to the absolute minimum to test (based on the getting started example from: https://github.com/espressif/esp-skainet/blob/master/examples/get_started/main/recsrc.c)

I only keep the code which reads form the i2s bus, do some basic transformation and then have a second task which takes the data from rec_rb and sends it to a server which writes out the wave file (Java: AudioFormat format = new AudioFormat(16000, 16, 1, false, false); )

Raw audio output:

#define I2S_CHANNEL_NUM 4

  {
        size_t bytes_read;
        int16_t *rsp_in = (int16_t *)malloc(AEC_FRAME_BYTES * I2S_CHANNEL_NUM);
        int16_t *aec_ref = (int16_t *)malloc(AEC_FRAME_BYTES);

        int nch = 1;
        int16_t *aec_rec = (int16_t *)malloc(AEC_FRAME_BYTES * nch);
        int16_t *aec_out = (int16_t *)malloc(AEC_FRAME_BYTES * nch);
        void *aec_handle = aec_create(16000, AEC_FRAME_LENGTH_MS, AEC_FILTER_LENGTH);

        while (1)
        {

            i2s_read(I2S_NUM_1, rsp_in, 2 * AEC_FRAME_BYTES, &bytes_read, portMAX_DELAY);
            for (int i = 0; i < AEC_FRAME_BYTES / 2; i++)
            {
                aec_out[i] = (rsp_in[2 * i] + rsp_in[2 * i + 1]) / 2;
            }
            rb_write(rec_rb, (uint8_t *)aec_out, AEC_FRAME_BYTES, portMAX_DELAY);
        }
    }

The audio is fine, however it is only half of normal speed, thus the sampling rate is wrong! If I change the sample rate, to 32000 in the AudioFormat constructor, the sound is very good (however no postprocessing for noise, etc has yet been applied).

As the constructor of AEC specifically mentions that the sampling rate must be 16000 and not 32000, I tried changing the bits per sample in MediaHal.c, as the sample rate seems though to be correctly set :
In #elif defined CONFIG_USE_ES7210
i2s_config.sample_rate = 16000;

i2s_config.bits_per_sample = 32; // for ES7210
to
i2s_config.bits_per_sample = 16; // for ES7210

For the raw audio output, I can now create the wave file with a frame size of 16.000, and the data output is cristal clear, but without any postprocessing done.

But none of the postprocessing filters of your example seem to work.

I now tried to apply the code to filter with both 2 and 3 mics (I have the 3 mic version), but the output gets very disturbed (e.g).


#define I2S_CHANNEL_NUM 4
  {

        size_t bytes_read;
        int16_t *rsp_in = (int16_t *)malloc(AEC_FRAME_BYTES * I2S_CHANNEL_NUM);
        int16_t *aec_ref = (int16_t *)malloc(AEC_FRAME_BYTES);

        int nch = 3;
    int16_t *aec_rec = (int16_t *) malloc(AEC_FRAME_BYTES * nch);
    int16_t *aec_out = (int16_t *) malloc(AEC_FRAME_BYTES * nch);
    void *aec_handle = aec_create_multimic(16000, AEC_FRAME_LENGTH_MS, AEC_FILTER_LENGTH, nch);
        while (1)
        {
        
           i2s_read(I2S_NUM_1, rsp_in, 4 *AEC_FRAME_BYTES, &bytes_read, portMAX_DELAY);
        for (int i = 0; i < AEC_FRAME_BYTES / 2; i++) {
            aec_ref[i] = rsp_in[4 * i + 2];
            aec_rec[i] = rsp_in[4 * i + 1];
            aec_rec[i + AEC_FRAME_BYTES / 2] = rsp_in[4 * i + 3];
            if (nch == 3)
            {
                aec_rec[i + AEC_FRAME_BYTES] = rsp_in[4 * i];
            }
        }
        aec_process(aec_handle, aec_rec, aec_ref, aec_out);
        rb_write(rec_rb,  (uint8_t *) aec_out, AEC_FRAME_BYTES * nch, portMAX_DELAY);
        }
    }

How do I make the examples work? It seems that the chipset ES7210 is initialised with the wrong sampling rate?

Thanks

can not be waked up after complie /esp-skainet/example/get_started (AIS-797)

burning the default factory firmware (ESP skainet / tools / default_firmware / esp32_korvo_v1_fw_v0_1_20200323. Bin), supporting voice wake-up and light color control.
I compiled and burned the sample project ESP skainet / examples / get_ Started, there are the following problems:

can not be waked up

The board can't be waken up

I have run the demo "garbage_classification" on the board ESP32-LyraT V4.3, but the board can't be waken up by the word "hilexin". So I tryed to debug did it read something in "ringbuf". Here is the log
I (29) boot: ESP-IDF v3.3 2nd stage bootloader
I (29) boot: compile time 09:29:28
I (29) boot: Enabling RNG early entropy source...
I (33) qio_mode: Enabling default flash chip QIO
I (38) boot: SPI Speed : 80MHz
I (42) boot: SPI Mode : QIO
I (46) boot: SPI Flash Size : 4MB
I (50) boot: Partition Table:
I (54) boot: ## Label Usage Type ST Offset Length
I (61) boot: 0 factory factory app 00 00 00010000 003c0000
I (69) boot: 1 nvs WiFi data 01 02 003d0000 00004000
I (76) boot: End of partition table
I (81) esp_image: segment 0: paddr=0x00010020 vaddr=0x3f400020 size=0x2c0394 (2884500) map
I (853) esp_image: segment 1: paddr=0x002d03bc vaddr=0x3ffb0000 size=0x02220 ( 8736) load
I (856) esp_image: segment 2: paddr=0x002d25e4 vaddr=0x40080000 size=0x00400 ( 1024) load
0x40080000: _WindowOverflow4 at /home/tfl/workspace/esp/esp-idf/components/freertos/xtensa_vectors.S:1779

I (860) esp_image: segment 3: paddr=0x002d29ec vaddr=0x40080400 size=0x0d624 ( 54820) load
I (887) esp_image: segment 4: paddr=0x002e0018 vaddr=0x400d0018 size=0x2b0e0 (176352) map
0x400d0018: _flash_cache_start at ??:?

I (933) esp_image: segment 5: paddr=0x0030b100 vaddr=0x4008da24 size=0x01834 ( 6196) load
0x4008da24: xTaskResumeAll at /home/tfl/workspace/esp/esp-idf/components/freertos/tasks.c:4560

I (945) boot: Loaded app from partition at offset 0x10000
I (945) boot: Disabling RNG early entropy source...
I (945) psram: This chip is ESP32-D0WD
I (950) spiram: Found 64MBit SPI RAM device
I (954) spiram: SPI RAM mode: flash 80m sram 80m
I (960) spiram: PSRAM initialized, cache is in low/high (2-core) mode.
I (967) cpu_start: Pro cpu up.
I (971) cpu_start: Application information:
I (975) cpu_start: Project name: garbage_classification
I (982) cpu_start: App version: v0.1.0-10-g74d715f-dirty
I (988) cpu_start: Compile time: Oct 24 2019 09:29:25
I (994) cpu_start: ELF file SHA256: 34e2ddf0cebbc776...
I (1000) cpu_start: ESP-IDF: v3.3
I (1005) cpu_start: Starting app cpu, entry point is 0x400815cc
0x400815cc: call_start_cpu1 at /home/tfl/workspace/esp/esp-idf/components/esp32/cpu_start.c:270

I (0) cpu_start: App cpu up.
I (1505) spiram: SPI SRAM memory test OK
I (1505) heap_init: Initializing. RAM available for dynamic allocation:
I (1505) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
I (1512) heap_init: At 3FFB33A0 len 0002CC60 (179 KiB): DRAM
I (1518) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (1524) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (1531) heap_init: At 4008F258 len 00010DA8 (67 KiB): IRAM
I (1537) cpu_start: Pro cpu start user code
I (1542) spiram: Adding pool of 4096K of external SPI memory to heap allocator
I (220) cpu_start: Chip Revision: 1
I (220) cpu_start: Starting scheduler on PRO CPU.
I (0) cpu_start: Starting scheduler on APP CPU.
I (225) spiram: Reserving pool of 32K of internal memory for DMA/internal allocations
I (235) I2S: DMA Malloc info, datalen=blocksize=1200, dma_buf_count=3
I (245) I2S: DMA Malloc info, datalen=blocksize=1200, dma_buf_count=3
I (245) I2S: APLL: Req RATE: 48000, real rate: 47999.961, BITS: 16, CLKM: 1, BCK_M: 8, MCLK: 12287990.000, SCLK: 1535998.750000, diva: 1, divb: 0
I (275) gpio: GPIO[21]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0
I (285) MEDIA_HAL: I2S_OUT_VOL_DEFAULT[60]
I (285) MSC_DSP: CONFIG_CODEC_CHIP_IS_ES8388
I (285) gpio: GPIO[22]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 1| Pulldown: 1| Intr:0
Quantized wakeNet5: wakeNet5_v1_hilexin_5_0.95_0.90, mode:0 (Oct 11 2019 20:08:50)
Quantized MN1 (Oct 14 2019 20:06:13)
I (305) MN: ---------------------SPEECH COMMANDS---------------------
I (315) MN: Command ID0, phrase 0: wei sheng zhi
I (315) MN: Command ID0, phrase 1: shi zhi jin
· · · · · ·
I (565) MN: ---------------------------------------------------------

chunk_num = 200
@count 1 read 960 bytes
@count 2 read 960 bytes
@count 3 read 960 bytes
@count 4 read 960 bytes
-----------awaits to be waken up-----------
@count 5 read 960 bytes
@count 6 read 960 bytes
· · · · · ·

From the log, the board frequently reads audio data, even if no one speaks. Is this data noise? Why can't the board recognize the wake-up command?

speech_commands_recognition demo的一些问题 (AIS-745)

  1. 你好,我使用的环境是cn_speech_commands_recognition(master) demo 程序,ESP32-S3-Korvo-v2.0开发板,IDF4.4版本,有以下问题望解答

  2. 在使用sdkconfig_esp32s3r8_8+4.defaults和修改开发板设置CONFIG_ESP32_S3_KORVO_V2_0_BOARD后,编译会报错,查看了一下发现 I2S0_CONFIG()里面宏定义结构体里面成员变量和有差异,修改后可以编译通过(同s3开发板宏定义结构体都一样,只有这个与实际i2s_config_t 不同,希望可以告知原因)

define I2S0_CONFIG(){ \
        .param_cfg = { \
            .mode = I2S_MODE_MASTER | I2S_MODE_TX, \
            .sample_rate = 16000, \
            .communication_format = I2S_COMM_FORMAT_STAND_I2S, \
            .slot_bits_cfg = (I2S_BITS_PER_SLOT_16BIT << SLOT_BIT_SHIFT) | I2S_BITS_PER_SAMPLE_16BIT, \
            .channel_format = I2S_CHANNEL_FMT_RIGHT_LEFT, \
            .slot_channel_cfg = (2 << SLOT_CH_SHIFT) | 2, \
            .active_slot_mask = I2S_TDM_ACTIVE_CH0 | I2S_TDM_ACTIVE_CH1, \
            .left_align_en = false, \
            .big_edin_en = false, \
            .bit_order_msb_en = false, \
        }, \
        .dma_buf_count = 6, \
        .dma_buf_len = 160, \
        .use_apll = false, \
        .intr_alloc_flags = ESP_INTR_FLAG_LEVEL1 \
}
  1. 另一个问题是在编译通过后会有下载程序到开发板,使用 idf.py -p COMx monitor 查看端口输出会出现一下问题
C:\esp-skainet\esp-idf\tools\idf.py' '-p' 'COM3'"...
�[0;33m--- WARNING: GDB cannot open serial ports accessed as COMx�[0m
�[0;33m--- Using \\.\COM3 instead...�[0m
�[0;33m--- idf_monitor on \\.\COM3 115200 ---�[0m
--- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
 header: 0xffffffff
inESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x1 (POWERON),boot:0x28 (SPI_FAST_FLASH_BOOT)
invalid header: 0xffffffff
invalid header: 0xffffffff
invalid header: 0xffffffff
invalid header: 0xffffffff
invalid header: 0xffffffff

Cannot compile any examples under ESP-IDF 4.3

There are several fixes and enhancements in 4.3, but none of the examples here compile with it.

Merged 1 ELF section
Generated /home/kevin/esp-skainet/examples/wake_word_detection/build/bootloader/bootloader.bin
[976/1001] Building C object esp-idf/hardware_driver/CMakeFiles/__idf_hardware_driver.dir/MediaHal/MediaHal.c.obj
FAILED: esp-idf/hardware_driver/CMakeFiles/__idf_hardware_driver.dir/MediaHal/MediaHal.c.obj 
/home/kevin/.espressif/tools/xtensa-esp32-elf/esp-2021r1-8.4.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc -DMBEDTLS_CONFIG_FILE=\"mbedtls/esp_config.h\" -Iconfig -I/home/kevin/esp-skainet/components/hardware_driver/MediaHal/Board -I/home/kevin/esp-skainet/components/hardware_driver/MediaHal/Codec -I/home/kevin/esp-skainet/components/hardware_driver/MediaHal -I/home/kevin/esp-skainet/components/hardware_driver/SDCardConfig -I/home/kevin/esp-skainet/components/hardware_driver/SystemSal -I/home/kevin/esp-skainet/components/hardware_driver/i2c_bus -I/home/kevin/esp-skainet/components/hardware_driver/userconfig -I/home/kevin/esp-skainet/components/hardware_driver/button/include -I/home/kevin/esp-skainet/components/hardware_driver/led/include -I/home/kevin/esp-idf/components/newlib/platform_include -I/home/kevin/esp-idf/components/freertos/include -I/home/kevin/esp-idf/components/freertos/port/xtensa/include -I/home/kevin/esp-idf/components/esp_hw_support/include -I/home/kevin/esp-idf/components/esp_hw_support/port/esp32/. -I/home/kevin/esp-idf/components/heap/include -I/home/kevin/esp-idf/components/log/include -I/home/kevin/esp-idf/components/lwip/include/apps -I/home/kevin/esp-idf/components/lwip/include/apps/sntp -I/home/kevin/esp-idf/components/lwip/lwip/src/include -I/home/kevin/esp-idf/components/lwip/port/esp32/include -I/home/kevin/esp-idf/components/lwip/port/esp32/include/arch -I/home/kevin/esp-idf/components/soc/include -I/home/kevin/esp-idf/components/soc/esp32/. -I/home/kevin/esp-idf/components/soc/esp32/include -I/home/kevin/esp-idf/components/hal/esp32/include -I/home/kevin/esp-idf/components/hal/include -I/home/kevin/esp-idf/components/esp_rom/include -I/home/kevin/esp-idf/components/esp_rom/esp32 -I/home/kevin/esp-idf/components/esp_rom/include/esp32 -I/home/kevin/esp-idf/components/esp_common/include -I/home/kevin/esp-idf/components/esp_system/include -I/home/kevin/esp-idf/components/esp32/include -I/home/kevin/esp-idf/components/driver/include -I/home/kevin/esp-idf/components/driver/esp32/include -I/home/kevin/esp-idf/components/esp_ringbuf/include -I/home/kevin/esp-idf/components/efuse/include -I/home/kevin/esp-idf/components/efuse/esp32/include -I/home/kevin/esp-idf/components/xtensa/include -I/home/kevin/esp-idf/components/xtensa/esp32/include -I/home/kevin/esp-idf/components/espcoredump/include -I/home/kevin/esp-idf/components/esp_timer/include -I/home/kevin/esp-idf/components/esp_ipc/include -I/home/kevin/esp-idf/components/esp_pm/include -I/home/kevin/esp-idf/components/vfs/include -I/home/kevin/esp-idf/components/esp_wifi/include -I/home/kevin/esp-idf/components/esp_wifi/esp32/include -I/home/kevin/esp-idf/components/esp_event/include -I/home/kevin/esp-idf/components/esp_netif/include -I/home/kevin/esp-idf/components/esp_eth/include -I/home/kevin/esp-idf/components/tcpip_adapter/include -I/home/kevin/esp-idf/components/app_trace/include -I/home/kevin/esp-idf/components/fatfs/diskio -I/home/kevin/esp-idf/components/fatfs/vfs -I/home/kevin/esp-idf/components/fatfs/src -I/home/kevin/esp-idf/components/wear_levelling/include -I/home/kevin/esp-idf/components/spi_flash/include -I/home/kevin/esp-idf/components/sdmmc/include -I/home/kevin/esp-idf/components/esp_adc_cal/include -I/home/kevin/esp-idf/components/nvs_flash/include -I/home/kevin/esp-idf/components/mbedtls/port/include -I/home/kevin/esp-idf/components/mbedtls/mbedtls/include -I/home/kevin/esp-idf/components/mbedtls/esp_crt_bundle/include -mlongcalls -Wno-frame-address   -ffunction-sections -fdata-sections -Wall -Werror=all -Wno-error=unused-function -Wno-error=unused-variable -Wno-error=deprecated-declarations -Wextra -Wno-unused-parameter -Wno-sign-compare -ggdb -mfix-esp32-psram-cache-issue -mfix-esp32-psram-cache-strategy=memw -Os -freorder-blocks -fstrict-volatile-bitfields -Wno-error=unused-but-set-variable -std=gnu99 -Wno-old-style-declaration -D_GNU_SOURCE -DIDF_VER=\"v4.3-194-g5b5e46971-dirty\" -DESP_PLATFORM -MD -MT esp-idf/hardware_driver/CMakeFiles/__idf_hardware_driver.dir/MediaHal/MediaHal.c.obj -MF esp-idf/hardware_driver/CMakeFiles/__idf_hardware_driver.dir/MediaHal/MediaHal.c.obj.d -o esp-idf/hardware_driver/CMakeFiles/__idf_hardware_driver.dir/MediaHal/MediaHal.c.obj   -c /home/kevin/esp-skainet/components/hardware_driver/MediaHal/MediaHal.c
In file included from /home/kevin/esp-idf/components/driver/include/driver/i2c.h:25,
                 from /home/kevin/esp-skainet/components/hardware_driver/MediaHal/Codec/ES8388_interface.h:30,
                 from /home/kevin/esp-skainet/components/hardware_driver/MediaHal/MediaHal.c:27:
/home/kevin/esp-idf/components/freertos/include/freertos/FreeRTOS.h:963:27: error: conflicting types for 'SemaphoreHandle_t'
  #define xSemaphoreHandle SemaphoreHandle_t
                           ^~~~~~~~~~~~~~~~~
/home/kevin/esp-skainet/components/hardware_driver/SystemSal/lock.h:7:16: note: in expansion of macro 'xSemaphoreHandle'
 typedef void * xSemaphoreHandle;
                ^~~~~~~~~~~~~~~~
In file included from /home/kevin/esp-idf/components/driver/include/driver/i2c.h:26,
                 from /home/kevin/esp-skainet/components/hardware_driver/MediaHal/Codec/ES8388_interface.h:30,
                 from /home/kevin/esp-skainet/components/hardware_driver/MediaHal/MediaHal.c:27:
/home/kevin/esp-idf/components/freertos/include/freertos/semphr.h:37:23: note: previous declaration of 'SemaphoreHandle_t' was here
 typedef QueueHandle_t SemaphoreHandle_t;
                       ^~~~~~~~~~~~~~~~~
/home/kevin/esp-skainet/components/hardware_driver/MediaHal/MediaHal.c:75:5: warning: 'I2S_COMM_FORMAT_I2S' is deprecated [-Wdeprecated-declarations]
     .communication_format = I2S_COMM_FORMAT_I2S,
     ^
In file included from /home/kevin/esp-idf/components/hal/esp32/include/hal/i2s_ll.h:30,
                 from /home/kevin/esp-idf/components/hal/include/hal/i2s_hal.h:28,
                 from /home/kevin/esp-idf/components/driver/include/driver/i2s.h:24,
                 from /home/kevin/esp-skainet/components/hardware_driver/MediaHal/MediaHal.c:35:
/home/kevin/esp-idf/components/hal/include/hal/i2s_types.h:70:5: note: declared here
     I2S_COMM_FORMAT_I2S       __attribute__((deprecated)) = 0x01, /*!< I2S communication format I2S, correspond to `I2S_COMM_FORMAT_STAND_I2S`*/
     ^~~~~~~~~~~~~~~~~~~
/home/kevin/esp-skainet/components/hardware_driver/MediaHal/MediaHal.c:178:19: warning: initialization of 'int (*)(void *)' from incompatible pointer type 'int (*)(Es8311Config *)' {aka 'int (*)(struct <anonymous> *)'} [-Wincompatible-pointer-types]
     .codec_init = Es8311Init,
                   ^~~~~~~~~~
/home/kevin/esp-skainet/components/hardware_driver/MediaHal/MediaHal.c:178:19: note: (near initialization for 'MediaHalConfig.codec_init')
In file included from /home/kevin/esp-skainet/components/hardware_driver/MediaHal/MediaHal.c:31:
/home/kevin/esp-skainet/components/hardware_driver/MediaHal/Codec/es7210.h:184:12: warning: 'es7210_mute' declared 'static' but never defined [-Wunused-function]
 static int es7210_mute(unsigned char mute);
            ^~~~~~~~~~~
[981/1001] Building C object esp-idf/spiffs/CMakeFiles/__idf_spiffs.dir/spiffs/src/spiffs_nucleus.c.obj
ninja: build stopped: subcommand failed.
ninja failed with exit code 1
kevin@ubuntu:~/esp-skainet/examples/wake_word_detection$ 

Not a bug just a question

Did you purposefully name this framework skainet = skai(greek for sky), ie skynet, the AI that destroyed the world in the Terminator Series?

关于文件缺失的问题 (AIS-817)

请问我在跑一个老例程时,出现了找不到MediaHal.h的问题,在github仓库内确实没有这个文件,请问是已经删除了吗。

../main/main.c: In function 'wakenetTask':
../main/main.c:64:9: error: implicit declaration of function 'rb_read'; did you mean 'wl_read'? [-Werror=implicit-function-declaration]
         rb_read(agc_rb, (uint8_t *)buffer, audio_chunksize * sizeof(int16_t), portMAX_DELAY);
         ^~~~~~~
         wl_read
../main/main.c:72:17: error: implicit declaration of function 'rb_reset'; did you mean 'fd_set'? [-Werror=implicit-function-declaration]
                 rb_reset(rec_rb);
                 ^~~~~~~~
                 fd_set
../main/main.c: In function 'app_main':
../main/main.c:272:5: error: implicit declaration of function 'codec_init'; did you mean 'gpio_init'? [-Werror=implicit-function-declaration]
     codec_init();
     ^~~~~~~~~~
     gpio_init
../main/main.c:276:13: error: implicit declaration of function 'rb_init'; did you mean 'gpio_init'? [-Werror=implicit-function-declaration]
  dump1_rb = rb_init(BUFFER_PROCESS, 8 * 1024, 1, NULL);
             ^~~~~~~
             gpio_init
../main/main.c:276:21: error: 'BUFFER_PROCESS' undeclared (first use in this function)
  dump1_rb = rb_init(BUFFER_PROCESS, 8 * 1024, 1, NULL);
                     ^~~~~~~~~~~~~~
../main/main.c:276:21: note: each undeclared identifier is reported only once for each function it appears in

降噪demo,耳机没有声音

我测试“noise_suppression”demo时,耳机没有声音,调试串口也没有输出。
使用的是手机耳机接"PHONEJACK"口。
唤醒词选择是nihaoxiaozhi (WakeNet5)。
这个demo需要先唤醒吗?

ESP32 Audio Kit可以跑这个吗

你好

我手上有一块ESP32 Audio Kit V2.2的开发板,请问可以跑这个吗,我看列出来的硬件只有Lyrat和Lyrat Mini,如果可以的话,应该怎么做,是不是要改硬件的配置

bootloader_common.c出现问题,请教大神如何解决

/home/user1/ESP/esp-skainet/esp-idf/components/bootloader_support/src/bootloader_common.c: In function 'bootloader_common_ota_select_invalid':
/home/user1/ESP/esp-skainet/esp-idf/components/bootloader_support/src/bootloader_common.c:50:43: error: 'esp_ota_select_entry_t' {aka 'const struct '} has no member named 'ota_state'; did you mean 'ota_seq'?
return s->ota_seq == UINT32_MAX || s->ota_state == ESP_OTA_IMG_INVALID || s->ota_state == ESP_OTA_IMG_ABORTED;
^~~~~~~~~
ota_seq
/home/user1/ESP/esp-skainet/esp-idf/components/bootloader_support/src/bootloader_common.c:50:56: error: 'ESP_OTA_IMG_INVALID' undeclared (first use in this function); did you mean 'ESP_ERR_IMAGE_INVALID'?
return s->ota_seq == UINT32_MAX || s->ota_state == ESP_OTA_IMG_INVALID || s->ota_state == ESP_OTA_IMG_ABORTED;
^~~~~~~~~~~~~~~~~~~
ESP_ERR_IMAGE_INVALID
/home/user1/ESP/esp-skainet/esp-idf/components/bootloader_support/src/bootloader_common.c:50:56: note: each undeclared identifier is reported only once for each function it appears in
/home/user1/ESP/esp-skainet/esp-idf/components/bootloader_support/src/bootloader_common.c:50:82: error: 'esp_ota_select_entry_t' {aka 'const struct '} has no member named 'ota_state'; did you mean 'ota_seq'?
return s->ota_seq == UINT32_MAX || s->ota_state == ESP_OTA_IMG_INVALID || s->ota_state == ESP_OTA_IMG_ABORTED;
^~~~~~~~~
ota_seq
/home/user1/ESP/esp-skainet/esp-idf/components/bootloader_support/src/bootloader_common.c:50:95: error: 'ESP_OTA_IMG_ABORTED' undeclared (first use in this function); did you mean 'ESP_ERR_NOT_SUPPORTED'?
return s->ota_seq == UINT32_MAX || s->ota_state == ESP_OTA_IMG_INVALID || s->ota_state == ESP_OTA_IMG_ABORTED;
^~~~~~~~~~~~~~~~~~~
ESP_ERR_NOT_SUPPORTED
/home/user1/ESP/esp-skainet/esp-idf/components/bootloader_support/src/bootloader_common.c: In function 'bootloader_common_get_sha256_of_partition':
/home/user1/ESP/esp-skainet/esp-idf/components/bootloader_support/src/bootloader_common.c:178:13: error: implicit declaration of function 'esp_image_verify'; did you mean 'esp_image_load'? [-Werror=implicit-function-declaration]
if (esp_image_verify(ESP_IMAGE_VERIFY_SILENT, &partition_pos, &data) != ESP_OK) {
^~~~~~~~~~~~~~~~
esp_image_load
/home/user1/ESP/esp-skainet/esp-idf/components/bootloader_support/src/bootloader_common.c:182:38: error: 'esp_image_metadata_t' {aka 'struct '} has no member named 'image_digest'; did you mean 'image_len'?
memcpy(out_sha_256, data.image_digest, ESP_PARTITION_HASH_LEN);
^~~~~~~~~~~~
image_len
/home/user1/ESP/esp-skainet/esp-idf/components/bootloader_support/src/bootloader_common.c: At top level:
/home/user1/ESP/esp-skainet/esp-idf/components/bootloader_support/src/bootloader_common.c:244:93: error: unknown type name 'esp_app_desc_t'; did you mean 'esp_chip_id_t'?
esp_err_t bootloader_common_get_partition_description(const esp_partition_pos_t *partition, esp_app_desc_t *app_desc)
^~~~~~~~~~~~~~
esp_chip_id_t
/home/user1/ESP/esp-skainet/esp-idf/components/bootloader_support/src/bootloader_common.c: In function 'bootloader_common_ota_select_invalid':
/home/user1/ESP/esp-skainet/esp-idf/components/bootloader_support/src/bootloader_common.c:51:1: error: control reaches end of non-void function [-Werror=return-type]
}
^
cc1: some warnings being treated as errors
/home/user1/ESP/esp-skainet/esp-idf/make/component_wrapper.mk:285: recipe for target 'src/bootloader_common.o' failed
make[2]: *** [src/bootloader_common.o] Error 1
/home/user1/ESP/esp-skainet/esp-idf/make/project.mk:493: recipe for target 'component-bootloader_support-build' failed
make[1]: *** [component-bootloader_support-build] Error 2
/home/user1/ESP/esp-skainet/esp-idf/components/bootloader/Makefile.projbuild:41: recipe for target '/home/user1/ESP/esp/esp-skainet/esp-idf/examples/get-started/hello_world/build/bootloader/bootloader.bin' failed
make: *** [/home/user1/ESP/esp/esp-skainet/esp-idf/examples/get-started/hello_world/build/bootloader/bootloader.bin] Error 2

关于唤醒时输出语音

你好:

我测试了下,在garbage_classification的例子中,通过唤醒后,增加输出音频(在的)后,再用说出各个垃圾后,识别非常困难,经常无法识别?

if (r) {
	// wakenet_open(); /* 输出语音 “在的” */
        float ms = (chunks * audio_chunksize * 1000.0) / frequency;
        printf("%.2f: %s DETECTED.\n", (float)ms / 1000.0, wakenet->get_word_name(model_data, r));
        detect_flag = 1;
        printf("-----------------LISTENING-----------------\n\n");
        led_on(LED_GPIO);
}

请问唤醒后,输出语音,让用户知道唤醒 了,这种情况该怎么解决?

关于编译wake_word_detection程序的问题 (AIS-801)

板子使用的esp32-korvo-v1.1
在编译wake_word_detection这个程序的时候出现了cc1.exe: some warnings being treated as errors的错误,在stackoverflow上找到了解决方法,在makefile中加入CFLAGS = "-Wno-error"
CXXFLAGS = "-Wno-error"来忽略警告引起的错误,但是仍然不起作用,请问有什么解决方案

C:/Users/MSI/Desktop/esp-skainet-master/components/hardware_driver/boards/esp32-korvo/bsp_board.c:395:64: note: each undeclared identifier is reported only once for each function it appears in
At top level:
C:/Users/MSI/Desktop/esp-skainet-master/components/hardware_driver/boards/esp32-korvo/bsp_board.c:262:18: warning: 'bsp_i2s_deinit' defined but not used [-Wunused-function]
static esp_err_t bsp_i2s_deinit(i2s_port_t i2s_num)
^~~~~~~~~~~~~~
cc1.exe: some warnings being treated as errors
[979/988] Building C object esp-idf/player/CMakeFiles/__idf_player.dir/esp_skainet_player.c.obj
In file included from C:/Users/MSI/esp/esp-idf/components/freertos/include/freertos/semphr.h:35,
from C:/Users/MSI/esp/esp-idf/components/driver/include/driver/i2s.h:20,
from C:/Users/MSI/Desktop/esp-skainet-master/components/player/esp_skainet_player.c:8:
C:/Users/MSI/Desktop/esp-skainet-master/components/player/esp_skainet_player.c: In function 'esp_skainet_stream_in_task':
C:/Users/MSI/Desktop/esp-skainet-master/components/player/esp_skainet_player.c:93:31: warning: passing argument 1 of 'xQueueGenericSend' from incompatible pointer type [-Wincompatible-pointer-types]
xQueueSend(player->player_queue, buffer, portMAX_DELAY);
C:/Users/MSI/esp/esp-idf/components/freertos/include/freertos/queue.h:421:80: note: in definition of macro 'xQueueSend'
#define xQueueSend( xQueue, pvItemToQueue, xTicksToWait ) xQueueGenericSend( ( xQueue ), ( pvItemToQueue ), ( xTicksToWait ), queueSEND_TO_BACK )
^~~~~~
C:/Users/MSI/esp/esp-idf/components/freertos/include/freertos/queue.h:572:45: note: expected 'QueueHandle_t' {aka 'struct QueueDefinition *'} but argument is of type 'struct RingBuf *' BaseType_t xQueueGenericSend( QueueHandle_t xQueue, const void * const pvItemToQueue, TickType_t xTicksToWait, const BaseType_t xCopyPosition ) PRIVILEGED_FUNCTION;
~~~~~~~~~~~~~~^~~~~~
C:/Users/MSI/Desktop/esp-skainet-master/components/player/esp_skainet_player.c:94:31: warning: passing argument 1 of 'xQueueGenericSend' from incompatible pointer type [-Wincompatible-pointer-types]
xQueueSend(player->player_queue, buffer+player->frame_size, portMAX_DELAY);
C:/Users/MSI/esp/esp-idf/components/freertos/include/freertos/queue.h:421:80: note: in definition of macro 'xQueueSend'
#define xQueueSend( xQueue, pvItemToQueue, xTicksToWait ) xQueueGenericSend( ( xQueue ), ( pvItemToQueue ), ( xTicksToWait ), queueSEND_TO_BACK )
^~~~~~
C:/Users/MSI/esp/esp-idf/components/freertos/include/freertos/queue.h:572:45: note: expected 'QueueHandle_t' {aka 'struct QueueDefinition *'} but argument is of type 'struct RingBuf *' BaseType_t xQueueGenericSend( QueueHandle_t xQueue, const void * const pvItemToQueue, TickType_t xTicksToWait, const BaseType_t xCopyPosition ) PRIVILEGED_FUNCTION;
~~~~~~~~~~~~~~^~~~~~
C:/Users/MSI/Desktop/esp-skainet-master/components/player/esp_skainet_player.c:98:30: warning: passing argument 1 of 'xQueueGenericSend' from incompatible pointer type [-Wincompatible-pointer-types]
xQueueSend(player->player_queue, buffer, portMAX_DELAY);
C:/Users/MSI/esp/esp-idf/components/freertos/include/freertos/queue.h:421:80: note: in definition of macro 'xQueueSend'

esp32-wrover-e multinet->create crash (AIS-879)

I am using ESP32-WROVER-E and ESP-IDF v4.2.3, multinet2.
Crash occurs when trying to call multinet->create().

static const esp_mn_iface_t *multinet = &MULTINET_MODEL;
printf("------------multinet create------------\n");
model_iface_data_t *model_data = multinet->create(&MULTINET_COEFF, 2048);

rst:0xc (SW_CPU_RESET),boot:0x1f (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:7052
ho 0 tail 12 room 4
load:0x40078000,len:13200
load:0x40080400,len:4584
entry 0x400806f4
I (29) boot: ESP-IDF v4.2.3 2nd stage bootloader
I (29) boot: compile time 16:21:01
I (29) boot: chip revision: 1
I (32) boot_comm: chip revision: 1, min. bootloader chip revision: 0
I (39) boot.esp32: SPI Speed : 40MHz
I (43) boot.esp32: SPI Mode : DIO
I (48) boot.esp32: SPI Flash Size : 16MB
I (53) boot: Enabling RNG early entropy source...
I (58) boot: Partition Table:
I (62) boot: ## Label Usage Type ST Offset Length
I (69) boot: 0 factory factory app 00 00 00010000 00500000
I (76) boot: End of partition table
I (80) boot_comm: chip revision: 1, min. application chip revision: 0
I (88) esp_image: segment 0: paddr=0x00010020 vaddr=0x3f400020 size=0x140c54 (1313876) map
I (597) esp_image: segment 1: paddr=0x00150c7c vaddr=0x3ffb0000 size=0x021b0 ( 8624) load
I (601) esp_image: segment 2: paddr=0x00152e34 vaddr=0x40080000 size=0x0aee4 ( 44772) load
I (623) esp_image: segment 3: paddr=0x0015dd20 vaddr=0x00000000 size=0x022f8 ( 8952)
I (627) esp_image: segment 4: paddr=0x00160020 vaddr=0x400d0020 size=0x1cba0 (117664) map
I (680) boot: Loaded app from partition at offset 0x10000
I (681) boot: Disabling RNG early entropy source...
I (681) cpu_start: Pro cpu up.
I (685) cpu_start: Application information:
I (689) cpu_start: Project name: wake_word_detection
I (695) cpu_start: App version: v0.3.0-105-g4847f52-dirty
I (702) cpu_start: Compile time: May 20 2022 16:20:40
I (708) cpu_start: ELF file SHA256: 35f293a763fba85a...
I (714) cpu_start: ESP-IDF: v4.2.3
I (719) cpu_start: Starting app cpu, entry point is 0x4008166c
I (711) cpu_start: App cpu up.
I (730) heap_init: Initializing. RAM available for dynamic allocation:
I (736) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
I (742) heap_init: At 3FFB2E28 len 0002D1D8 (180 KiB): DRAM
I (749) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (755) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (761) heap_init: At 4008AEE4 len 0001511C (84 KiB): IRAM
I (768) cpu_start: Pro cpu start user code
I (786) spi_flash: detected chip: generic
I (786) spi_flash: flash io: dio
I (787) cpu_start: Starting scheduler on PRO CPU.
I (0) cpu_start: Starting scheduler on APP CPU.
------------multinet create------------
Guru Meditation Error: Core 1 panic'ed (StoreProhibited). Exception was unhandled.

Core 1 register dump:
PC : 0x400d5018 PS : 0x00060130 A0 : 0x800d3c7c A1 : 0x3ffb79d0
A2 : 0xffffffff A3 : 0xffffffff A4 : 0x00000000 A5 : 0x0000004b
A6 : 0x00000000 A7 : 0x00000000 A8 : 0x800d5018 A9 : 0x3ffb79b0
A10 : 0x00000000 A11 : 0x3ffb7a20 A12 : 0x00000030 A13 : 0x3ffb6a44
A14 : 0x00000016 A15 : 0x00000000 SAR : 0x00000004 EXCCAUSE: 0x0000001d
EXCVADDR: 0x00000004 LBEG : 0x4000c2e0 LEND : 0x4000c2f6 LCOUNT : 0x00000000

Backtrace:0x400d5015:0x3ffb79d0 0x400d3c79:0x3ffb79f0 0x400d3dd0:0x3ffb7ad0 0x400d3235:0x3ffb7af0 0x400854b1:0x3ffb7be0

ELF file SHA256: 35f293a763fba85a

Rebooting...
ets Jun 8 2016 00:22:57

基于官方的2.0.3SDK,在Arduino上运行,能够唤醒,指令无法识别 (AIS-881)

问题如题,基于官方的2.0.3SDK在Arduino上运行,唤醒词能够识别,但是指令词无法识别。
烧录如下:

C:\Users\tangjie\AppData\Local\Arduino15\packages\esp32\tools\esptool_py\3.3.0/esptool.exe --chip esp32s3 --port COM9 --baud 921600 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size 8MB 0x0 C:\Users\tangjie\AppData\Local\Temp\arduino_build_885412/wakeWordDetection.ino.bootloader.bin 0x8000 C:\Users\tangjie\AppData\Local\Temp\arduino_build_885412/wakeWordDetection.ino.partitions.bin 0x10000 C:\Users\tangjie\AppData\Local\Temp\arduino_build_885412/wakeWordDetection.ino.bin 0x281000 C:\Users\tangjie\AppData\Local\Arduino15\packages\esp32\hardware\esp32\2.0.3/tools/sdk/esp32s3/bin/model.bin
esptool.py v3.3
Serial port COM9
Connecting...
Chip is ESP32-S3
Features: WiFi, BLE
Crystal is 40MHz
MAC: 68:b6:b3:22:13:20
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 921600
Changed.
Configuring flash size...
Flash will be erased from 0x00000000 to 0x00005fff...
Flash will be erased from 0x00008000 to 0x00008fff...
Flash will be erased from 0x00010000 to 0x000b3fff...
Flash will be erased from 0x00281000 to 0x0078cfff...
Compressed 22208 bytes to 13805...
Writing at 0x00000000... (100 %)
Wrote 22208 bytes (13805 compressed) at 0x00000000 in 0.5 seconds (effective 391.6 kbit/s)...
Hash of data verified.
Compressed 3072 bytes to 121...
Writing at 0x00008000... (100 %)
Wrote 3072 bytes (121 compressed) at 0x00008000 in 0.0 seconds (effective 519.9 kbit/s)...
Hash of data verified.
Compressed 668992 bytes to 357539...
Writing at 0x00010000... (4 %)
Writing at 0x0001a52a... (9 %)
Writing at 0x00026c93... (13 %)
Writing at 0x00039cb1... (18 %)
Writing at 0x00040fbf... (22 %)
Writing at 0x00046ca1... (27 %)
Writing at 0x0004c49b... (31 %)
Writing at 0x00051469... (36 %)
Writing at 0x00056617... (40 %)
Writing at 0x0005b57b... (45 %)
Writing at 0x00060be1... (50 %)
Writing at 0x00066322... (54 %)
Writing at 0x0006bf47... (59 %)
Writing at 0x00073afa... (63 %)
Writing at 0x0007bab3... (68 %)
Writing at 0x0008304f... (72 %)
Writing at 0x00088715... (77 %)
Writing at 0x0008e8be... (81 %)
Writing at 0x00096967... (86 %)
Writing at 0x0009d95b... (90 %)
Writing at 0x000a6a68... (95 %)
Writing at 0x000ae736... (100 %)
Wrote 668992 bytes (357539 compressed) at 0x00010000 in 7.4 seconds (effective 720.6 kbit/s)...
Hash of data verified.
Compressed 5292032 bytes to 2127378...
Writing at 0x00281000... (0 %)
Writing at 0x00285ee9... (1 %)
Writing at 0x0028acc2... (2 %)
Writing at 0x0028fa56... (3 %)
Writing at 0x002948c0... (3 %)
Writing at 0x002996e4... (4 %)
Writing at 0x0029df59... (5 %)
Writing at 0x002a2802... (6 %)
Writing at 0x002a6f9e... (6 %)
Writing at 0x002ab833... (7 %)
Writing at 0x002b0133... (8 %)
Writing at 0x002b4a18... (9 %)
Writing at 0x002b93e4... (10 %)
Writing at 0x002bdce9... (10 %)
Writing at 0x002c26e8... (11 %)
Writing at 0x002c71f4... (12 %)
Writing at 0x002cbc3e... (13 %)
Writing at 0x002d07a5... (13 %)
Writing at 0x002d530a... (14 %)
Writing at 0x002d9d54... (15 %)
Writing at 0x002de9a2... (16 %)
Writing at 0x002e36a6... (16 %)
Writing at 0x002e83c0... (17 %)
Writing at 0x002ecfda... (18 %)
Writing at 0x002f1ced... (19 %)
Writing at 0x002f6993... (20 %)
Writing at 0x002fb5a2... (20 %)
Writing at 0x003001a1... (21 %)
Writing at 0x00304c9f... (22 %)
Writing at 0x00309871... (23 %)
Writing at 0x0030e4ab... (23 %)
Writing at 0x003131c5... (24 %)
Writing at 0x00317eef... (25 %)
Writing at 0x0031ccbe... (26 %)
Writing at 0x00321b31... (26 %)
Writing at 0x003269a6... (27 %)
Writing at 0x0032b7ed... (28 %)
Writing at 0x0033038a... (29 %)
Writing at 0x00334de4... (30 %)
Writing at 0x0033994b... (30 %)
Writing at 0x0033e4b0... (31 %)
Writing at 0x00342f06... (32 %)
Writing at 0x00347b03... (33 %)
Writing at 0x0034c882... (33 %)
Writing at 0x003515aa... (34 %)
Writing at 0x0035634c... (35 %)
Writing at 0x0035afcb... (36 %)
Writing at 0x0035fcab... (36 %)
Writing at 0x00364b72... (37 %)
Writing at 0x00369a50... (38 %)
Writing at 0x0036e970... (39 %)
Writing at 0x0037388e... (40 %)
Writing at 0x003787de... (40 %)
Writing at 0x0037d8b4... (41 %)
Writing at 0x00382aa4... (42 %)
Writing at 0x00387d64... (43 %)
Writing at 0x0038cfbc... (43 %)
Writing at 0x0039230c... (44 %)
Writing at 0x003972b2... (45 %)
Writing at 0x0039be32... (46 %)
Writing at 0x003a0b55... (46 %)
Writing at 0x003a588d... (47 %)
Writing at 0x003aa5a2... (48 %)
Writing at 0x003af295... (49 %)
Writing at 0x003b42eb... (50 %)
Writing at 0x003b9442... (50 %)
Writing at 0x003be4f3... (51 %)
Writing at 0x003c35e4... (52 %)
Writing at 0x003c86d2... (53 %)
Writing at 0x003cceef... (53 %)
Writing at 0x003d1805... (54 %)
Writing at 0x003d6366... (55 %)
Writing at 0x003daec8... (56 %)
Writing at 0x003dfb96... (56 %)
Writing at 0x003e46ef... (57 %)
Writing at 0x003e9202... (58 %)
Writing at 0x003edbc4... (59 %)
Writing at 0x003f264e... (60 %)
Writing at 0x003f6fd2... (60 %)
Writing at 0x003fbad7... (61 %)
Writing at 0x0040077d... (62 %)
找到无效库在 H:\Arduino\libraries\esp32_SoundRecorder: no headers files (.h) found in H:\Arduino\libraries\esp32_SoundRecorder
Writing at 0x00405341... (63 %)
Writing at 0x00409c1c... (63 %)
Writing at 0x0040e6a5... (64 %)
Writing at 0x00413152... (65 %)
Writing at 0x00417cbc... (66 %)
Writing at 0x0041cb08... (66 %)
Writing at 0x00421b4a... (67 %)
Writing at 0x00426cd2... (68 %)
Writing at 0x0042bb2d... (69 %)
Writing at 0x004309a2... (70 %)
Writing at 0x00435777... (70 %)
Writing at 0x0043a4b8... (71 %)
Writing at 0x0043efbd... (72 %)
Writing at 0x00443d21... (73 %)
Writing at 0x00448c70... (73 %)
Writing at 0x0044d905... (74 %)
Writing at 0x004524d9... (75 %)
Writing at 0x00456f97... (76 %)
Writing at 0x0045ba90... (76 %)
Writing at 0x00461f6d... (77 %)
Writing at 0x00467e94... (78 %)
Writing at 0x0046ddf7... (79 %)
Writing at 0x00473e4d... (80 %)
Writing at 0x0047a103... (80 %)
Writing at 0x004804b3... (81 %)
Writing at 0x004866de... (82 %)
Writing at 0x0048c974... (83 %)
Writing at 0x00492a06... (83 %)
Writing at 0x004989ad... (84 %)
Writing at 0x0049e9a5... (85 %)
Writing at 0x004a4b81... (86 %)
Writing at 0x004aad2a... (86 %)
Writing at 0x004b0e78... (87 %)
Writing at 0x004b7114... (88 %)
Writing at 0x004bd2b2... (89 %)
Writing at 0x004c33e2... (90 %)
Writing at 0x004c95d7... (90 %)
Writing at 0x004cf6f7... (91 %)
Writing at 0x004d5a62... (92 %)
Writing at 0x004dbef0... (93 %)
Writing at 0x004e22b9... (93 %)
Writing at 0x004e92b2... (94 %)
Writing at 0x004efc70... (95 %)
Writing at 0x004f6719... (96 %)
Writing at 0x004ff64b... (96 %)
Writing at 0x0050778b... (97 %)
Writing at 0x00510bfe... (98 %)
Writing at 0x005227e6... (99 %)
Writing at 0x0053225c... (100 %)
Wrote 5292032 bytes (2127378 compressed) at 0x00281000 in 46.6 seconds (effective 908.3 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...
运行

Build:Mar 27 2021
rst:0x15 (USB_UART_CHIP_RESET),boot:0x2b (SPI_FAST_FLASH_BOOT)
Saved PC:0x40378b31
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fcd0108,len:0x178c
load:0x403b6000,len:0xe28
load:0x403ba000,len:0x30b8
entry 0x403b6274
�[0;32mI (28) boot: ESP-IDF v4.4 2nd stage bootloader�[0m
�[0;32mI (28) boot: compile time 16:32:36�[0m
�[0;32mI (28) boot: chip revision: 0�[0m
�[0;32mI (29) qio_mode: Enabling default flash chip QIO�[0m
�[0;32mI (35) boot.esp32s3: Boot SPI Speed : 80MHz�[0m
�[0;32mI (39) boot.esp32s3: SPI Mode : QIO�[0m
�[0;32mI (44) boot.esp32s3: SPI Flash Size : 8MB�[0m
�[0;32mI (49) boot: Enabling RNG early entropy source...�[0m
�[0;32mI (54) boot: Partition Table:�[0m
�[0;32mI (58) boot: ## Label Usage Type ST Offset Length�[0m
�[0;32mI (65) boot: 0 nvs WiFi data 01 02 00009000 00005000�[0m
�[0;32mI (72) boot: 1 otadata OTA data 01 00 0000e000 00002000�[0m
�[0;32mI (80) boot: 2 factory factory app 00 00 00010000 00271000�[0m
�[0;32mI (87) boot: 3 model Unknown data 01 82 00281000 0050c000�[0m
�[0;32mI (95) boot: End of partition table�[0m
�[0;32mI (99) boot: Defaulting to factory image�[0m
�[0;32mI (104) esp_image: segment 0: paddr=00010020 vaddr=3c070020 size=26dd0h (159184) map�[0m
�[0;32mI (136) esp_image: segment 1: paddr=00036df8 vaddr=3fc97290 size=02a44h ( 10820) load�[0m
�[0;32mI (139) esp_image: segment 2: paddr=00039844 vaddr=40374000 size=067d4h ( 26580) load�[0m
�[0;32mI (147) esp_image: segment 3: paddr=00040020 vaddr=42000020 size=66a2ch (420396) map�[0m

[16:43:00.246]收←◆�[0;32mI (214) esp_image: segment 4: paddr=000a6a54 vaddr=4037a7d4 size=0cab0h ( 51888) load�[0m
�[0;32mI (225) esp_image: segment 5: paddr=000b350c vaddr=50000000 size=00010h ( 16) load�[0m
�[0;32mI (232) boot: Loaded app from partition at offset 0x10000�[0m
�[0;32mI (232) boot: Disabling RNG early entropy source...�[0m
[ 252][I][esp32-hal-psram.c:96] psramInit(): PSRAM enabled
Initializing SPIFFS

[16:43:00.633]收←◆Partition size: total: 4857101, used: 2629225

[16:43:00.679]收←◆model_name: xiaoaitongxue7q8 model_data: /srmodel/xiaoaitongxue7q8/wn7q8_data

[16:43:01.513]收←◆MC Quantized-8 wakeNet7: wakeNet7Q8_v1_xiaoaitongxue_5_0.97_0.90, mode:2, p:3, (Jan 5 2022 11:31:01)
Initial ONE-MIC auido front-end for speech recognition, mode:0, (Dec 10 2021 11:08:01)

[16:43:04.129]收←◆------------detect start------------

[16:44:06.602]收←◆wakeword detected
-----------LISTENING-----------

[16:44:17.908]收←◆
-----------awaits to be waken up-----------
请问这会是什么问题导致的呢?

WakeNet 8-bit Quantized: "Item psram alloc failed" on ESP32-S3-Wroom-1-N8R2 (AIS-864)

I am using a custom board with 2 I2S microphones. Havn't set up microphones yet, compiling the WakeWord example with ESP32-S3-EYE set as board in menuconfig. I have Disable MultiNet since I have only 2MB of RAM available, but this should be ample for WakeNet, correct? Then I don't underestand why I get "Item psram alloc failed" from model. Am I running out of PSRAM?? I adjusted many settings in menuconfig, but still get same error.

ELF file SHA256: 90725b566f0ff608

Rebooting...
xESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0xc (RTC_SW_CPU_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
Saved PC:0x403799e0
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fcd0108,len:0x17b4
load:0x403b6000,len:0xe28
load:0x403ba000,len:0x30d0
entry 0x403b6274
I (29) boot: ESP-IDF v4.4-348-gc45dee0a7a 2nd stage bootloader
I (29) boot: chip revision: 0
I (32) qio_mode: Enabling default flash chip QIO
I (37) boot.esp32s3: Boot SPI Speed : 80MHz
I (42) boot.esp32s3: SPI Mode : QIO
I (47) boot.esp32s3: SPI Flash Size : 8MB
I (52) boot: Enabling RNG early entropy source...
I (57) boot: Partition Table:
I (61) boot: ## Label Usage Type ST Offset Length
I (68) boot: 0 factory factory app 00 00 00010000 00271000
I (75) boot: 1 model Unknown data 01 82 00281000 0050c000
I (83) boot: End of partition table
I (87) esp_image: segment 0: paddr=00010020 vaddr=3c060020 size=1f420h (128032) map
I (115) esp_image: segment 1: paddr=0002f448 vaddr=3fc9a800 size=00bd0h ( 3024) load
I (116) esp_image: segment 2: paddr=00030020 vaddr=42000020 size=5e6a8h (386728) map
I (179) esp_image: segment 3: paddr=0008e6d0 vaddr=3fc9b3d0 size=01c4ch ( 7244) load
I (181) esp_image: segment 4: paddr=00090324 vaddr=40378000 size=127f4h ( 75764) load
I (199) esp_image: segment 5: paddr=000a2b20 vaddr=50000000 size=00010h ( 16) load
I (207) boot: Loaded app from partition at offset 0x10000
I (207) boot: Disabling RNG early entropy source...
I (220) spiram: Found 16MBit SPI RAM device
I (220) spiram: SPI RAM mode: sram 80m
I (220) spiram: PSRAM initialized, cache is in normal (1-core) mode.
I (225) cpu_start: Pro cpu up.
I (228) cpu_start: Starting app cpu, entry point is 0x4037932c
I (213) cpu_start: App cpu up.
I (464) spiram: SPI SRAM memory test OK
I (465) spiram: Instructions copied and mapped to SPIRAM
I (510) spiram: Read only data copied and mapped to SPIRAM
I (525) cache: SPIRAM wrap enabled, size = 32.
I (534) cpu_start: Pro cpu start user code
I (534) cpu_start: cpu freq: 160000000
I (534) cpu_start: Application information:
I (537) cpu_start: Project name: wake_word_detection
I (543) cpu_start: App version: v0.3.0-105-g4847f52-dirty
I (555) cpu_start: ELF file SHA256: 90725b566f0ff608...
I (561) cpu_start: ESP-IDF: v4.4-348-gc45dee0a7a
I (568) heap_init: Initializing. RAM available for dynamic allocation:
I (575) heap_init: At 3FC9F058 len 00040FA8 (259 KiB): D/IRAM
I (581) heap_init: At 3FCE0000 len 0000EE34 (59 KiB): STACK/DRAM
I (588) heap_init: At 3FCF0000 len 00008000 (32 KiB): DRAM
I (594) heap_init: At 600FE000 len 00002000 (8 KiB): RTCRAM
I (600) spiram: Adding pool of 1536K of external SPI memory to heap allocator
I (608) spi_flash: detected chip: gd
I (612) spi_flash: flash io: qio
I (617) sleep: Configure to isolate all GPIO pins in sleep state
I (623) sleep: Enable automatic switching of GPIO sleep configuration
I (630) cpu_start: Starting scheduler on PRO CPU.
I (0) cpu_start: Starting scheduler on APP CPU.
I (641) spiram: Reserving pool of 32K of internal memory for DMA/internal allocations
Initializing SPIFFS
Partition size: total: 4857101, used: 874233
I (1121) I2S: DMA Malloc info, datalen=blocksize=640, dma_buf_count=6
model_name: hiesp8q8 model_data: /srmodel/hiesp8q8/wn8q8_data
Item psram alloc failed. Size: 163860 = 81920 x 2 + 16 + 4
item buff malloc fail
Guru Meditation Error: Core 0 panic'ed (StoreProhibited). Exception was unhandled.

Core 0 register dump:
PC : 0x40056fcc PS : 0x00060f30 A0 : 0x82008934 A1 : 0x3fcf3f10
A2 : 0x00000000 A3 : 0x3dede104 A4 : 0x00028000 A5 : 0x00000000
A6 : 0x0101fb0f A7 : 0x0a000101 A8 : 0xf90a060a A9 : 0x3fcf3ec0
A10 : 0x0000000a A11 : 0x00000002 A12 : 0x00014000 A13 : 0x00000002
A14 : 0x00000010 A15 : 0x00000004 SAR : 0x00000010 EXCCAUSE: 0x0000001d
EXCVADDR: 0x00000000 LBEG : 0x40056fc5 LEND : 0x40056fe7 LCOUNT : 0x000027ff

Backtrace:0x40056fc9:0x3fcf3f10 |<-CORRUPTED

undefined references to `xQueueGenericReceive' (AIS-743)

/home/winter/.espressif/tools/xtensa-esp32-elf/esp-2021r1-8.4.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: /home/winter/item/esp-skainet/components/esp-sr/lib/esp32/libdl_lib.a(sr_ringbuf.c.obj):(.literal.sr_rb_read+0x0): undefined reference to xQueueGenericReceive' /home/winter/.espressif/tools/xtensa-esp32-elf/esp-2021r1-8.4.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: /home/winter/item/esp-skainet/components/esp-sr/lib/esp32/libdl_lib.a(sr_ringbuf.c.obj): in function sr_rb_read':
/home/wangwangwang/workspace/ai_vnc/728_support/esp_sr_lib_tmp3/build/../components/dl_lib/sr_ringbuf.c:101: undefined reference to xQueueGenericReceive' /home/winter/.espressif/tools/xtensa-esp32-elf/esp-2021r1-8.4.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: /home/wangwangwang/workspace/ai_vnc/728_support/esp_sr_lib_tmp3/build/../components/dl_lib/sr_ringbuf.c:136: undefined reference to xQueueGenericReceive'
/home/winter/.espressif/tools/xtensa-esp32-elf/esp-2021r1-8.4.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: /home/wangwangwang/workspace/ai_vnc/728_support/esp_sr_lib_tmp3/build/../components/dl_lib/sr_ringbuf.c:148: undefined reference to xQueueGenericReceive' /home/winter/.espressif/tools/xtensa-esp32-elf/esp-2021r1-8.4.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: /home/winter/item/esp-skainet/components/esp-sr/lib/esp32/libdl_lib.a(sr_ringbuf.c.obj): in function sr_rb_write':
/home/wangwangwang/workspace/ai_vnc/728_support/esp_sr_lib_tmp3/build/../components/dl_lib/sr_ringbuf.c:167: undefined reference to xQueueGenericReceive' /home/winter/.espressif/tools/xtensa-esp32-elf/esp-2021r1-8.4.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld: /home/winter/item/esp-skainet/components/esp-sr/lib/esp32/libdl_lib.a(sr_ringbuf.c.obj):/home/wangwangwang/workspace/ai_vnc/728_support/esp_sr_lib_tmp3/build/../components/dl_lib/sr_ringbuf.c:202: more undefined references to xQueueGenericReceive' follow

命令词识别准确性有待提高!

  • 在 《get_start》 我添加了四个命令词,打开灯( ID0 ) 、关闭灯( ID1 ) 、 打开空调 ( ID2) 、关闭空调 ( ID3 ) ,但是我说 打开空调 都是识别 ID0 , 关闭空调 都是识别为 ID1 ;

  • 这个识别有点啥啊!

image

image

Request to run on other ESP32 boards

Is it possible to create an arduino library from skainet and run it on other esp32 boards? I currently have a esp-wroom-32 with an SD card connected to it, is there any way to run skainet on it and turn on / off GPIO's ?

cannot compile (AIS-798)

-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Project is not inside a git repository, or git repository has no commits; will not use 'git describe' to determine PROJECT_VER.
-- Building ESP-IDF components for target esp32
CMake Error at C:/Users/HP/esp/esp-idf/tools/cmake/build.cmake:189 (message):
Failed to resolve component 'esp-sr'.
Call Stack (most recent call first):
C:/Users/HP/esp/esp-idf/tools/cmake/build.cmake:215 (__build_resolve_and_add_req)
C:/Users/HP/esp/esp-idf/tools/cmake/build.cmake:435 (__build_expand_requirements)
C:/Users/HP/esp/esp-idf/tools/cmake/project.cmake:396 (idf_build_process)
CMakeLists.txt:6 (project)

-- Configuring incomplete, errors occurred!
See also "C:/Users/HP/esp/voice

ADF和skainet的AEC效果不一致问题 (AIS-875)

你好,我这边有测试在skainet上测试AEC的效果,发现并不是会消除的很彻底,我也做了一系列实验,利用相同的esp-sr的lib,adf的NS单声道信号,发现ADF的参考信号采集的方式AEC效果会好一点,请问这两种方式对AEC的效果有很大影响吗,还是esp-sr需要严格的音频相位同步呢,请问有具体的要求吗

espressif/esp-adf#799

Wake word demo SPIFFS error on custom board (AIS-863)

I am trying to run the demo on a custom board containing ESP32-S3_Wroom-N8R2 and 2 I2S microphones, but no SD card. I was able to compile and flash the Wake Word demo, but I get the following error via serial monitor:

I (815) spiram: Reserving pool of 32K of internal memory for DMA/internal allocations
Initializing SPIFFS
E (825) SPIFFS: spiffs partition could not be found
Failed to find SPIFFS partition
I (835) I2S: DMA Malloc info, datalen=blocksize=1280, dma_buf_count=6

What Should I do to use internal flash instead of SD card to fix this error?
Also, what else do I need to change in demo code to make it compatible with my custom board?

Thanks

get_started例程不能再lyrat-mini-v1.2上正常运行

Quantized wakeNet5: wakeNet5_v1_hilexin_5_0.95_0.90, mode:0 (Oct 14 2020 16:26:17)
CONTINUOUS_RECOGNITION: 2_0 MN1_4; core: 0; (May 15 2020 14:50:28)
SHIFT: 8, 12, 17, 17, 19, 17, 6, 16, 15, 14,
Guru Meditation Error: Core 0 panic'ed (Interrupt wdt timeout on CPU0)
Core 0 register dump:
PC : 0x40081a6e PS : 0x00060c35 A0 : 0x800864c2 A1 : 0x3ffb4d40
A2 : 0x00000000 A3 : 0x00060c20 A4 : 0x3ff00058 A5 : 0x3ffb4d68
A6 : 0x00060c20 A7 : 0x00000000 A8 : 0x00000000 A9 : 0x00000001
A10 : 0x00000000 A11 : 0x3ffb50cc A12 : 0x00000001 A13 : 0x00000001
A14 : 0x00060c23 A15 : 0x00000000 SAR : 0x00000004 EXCCAUSE: 0x00000005
EXCVADDR: 0x00000000 LBEG : 0x4008d691 LEND : 0x4008d6a1 LCOUNT : 0xfffffffd

ELF file SHA256:

Backtrace: 0x40081a6e:0x3ffb4d40 0x400864bf:0x3ffb4d60 0x4008655f:0x3ffb4db0 0x400f67c1:0x3ffb4de0 0x400f6982:0x3ffb4e10 0x400f69d3:0x3ffb4e30 0x400ee8c8:0x3ffb4e50 0x400edbaa:0x3ffb4e70 0x400eddfa:0x3ffb4fc0 0x400d65b0:0x3ffb4ff0 0x400d1958:0x3ffb5020 0x40089cbd:0x3ffb5040

输出语音如何制作成数组?

你好,

请问一下,如何将语音通过喇叭输出,我在之前的Issues中看到这个WAV,请问是这些步骤是怎么操作?

请尽快回复,谢谢!

自定义语音命令

在README里:
用户可方便根据需求自定义语音命令,无需重新训练模型。
目前模型支持类似“打开空调”,“打开卧室灯”等中文命令词识别。
类似“打开空调”,“打开卧室灯”等命令词具体是怎样的?

Several links in project README.md are broken (AIS-784)

While browsing the documentation, I found the following links are broken in README.md:

Wake Word Engine paragraph:
WakeNet
Espressif Speech Wake Words Customization Process

Speech Commands Recognition paragraph:
MultiNet

ESP-IDF in the Software Preparation section:
Getting Started Guide for the stable ESP-IDF version

esp32-s3-korvo-2加密的模组multinet有问题 (AIS-854)

环境
硬件:esp32-s3-korvo-2  
esp-skainet: dev2.0 Commit: 5aec971862b
esp-sr:   dev2.0  Commit: c873a359e7d
我现在的问题是在同样的工程代码(都是使用的cn_speech_commands_recognition的例程),在未加密的模组可以正常进行语音唤醒和识别,但是在另外一块已经加密的模组上,运行multinet->create(&MULTINET_COEFF, 5760);后就无法继续运行了,可以帮忙查看一下是什么问题导致的吗,万分感谢!下面是运行的一些log信息

未加密的模组可以正常运行

I (25) boot: ESP-IDF v4.4-beta1-50-g5e13980e99-dirty 2nd stage bootloader
I (25) boot: compile time 10:03:07
I (25) boot: chip revision: 0
I (30) boot.esp32s3: Boot SPI Speed : 80MHz
I (33) boot.esp32s3: SPI Mode       : DIO
I (38) boot.esp32s3: SPI Flash Size : 8MB
I (43) boot: Enabling RNG early entropy source...
W (48) bootloader_random: RNG for ESP32-S3 not currently supported
I (55) boot: Partition Table:
I (59) boot: ## Label            Usage          Type ST Offset   Length
I (66) boot:  0 factory          factory app      00 00 00010000 00200000
I (73) boot:  1 model            Unknown data     01 82 00210000 0050c000
I (81) boot: End of partition table
I (85) esp_image: segment 0: paddr=00010020 vaddr=3c080020 size=c3310h (799504) map
I (237) esp_image: segment 1: paddr=000d3338 vaddr=3fc9a2a0 size=028dch ( 10460) load
I (239) esp_image: segment 2: paddr=000d5c1c vaddr=40374000 size=0a3fch ( 41980) load
I (252) esp_image: segment 3: paddr=000e0020 vaddr=42000020 size=706ech (460524) map
I (335) esp_image: segment 4: paddr=00150714 vaddr=4037e3fc size=0be98h ( 48792) load
I (346) esp_image: segment 5: paddr=0015c5b4 vaddr=50000000 size=00010h (    16) load
I (354) boot: Loaded app from partition at offset 0x10000
I (355) boot: Disabling RNG early entropy source...
W (355) bootloader_random: RNG for ESP32-S3 not currently supported
Initializing SPIFFS
Partition size: total: 4857101, used: 3457023
model_name: xiaoaitongxue7 model_data: /srmodel/xiaoaitongxue7/wn7_data
MC Quantized wakeNet7: wakeNet7_v1_xiaoaitongxue_6_0.97_0.92, mode:2, p:3, (Feb  9 2022 16:22:38)
------------detect start------------
wakeword detected
-----------listening-----------
TOP 1, command_id: 0, phrase_id: 0, prob: 0.905920
TOP 2, command_id: 18, phrase_id: 18, prob: 0.375519
-----------listening-----------

-----------awaits to be waken up-----------
加密模组运行到75行multinet->create(&MULTINET_COEFF, 5760);就挂掉了
I (29) boot: ESP-IDF v4.4-beta1-50-g5e13980e99-dirty 2nd stage bootloader
I (30) boot: compile time 10:03:07
I (30) boot: chip revision: 0
I (33) boot.esp32s3: Boot SPI Speed : 80MHz
I (38) boot.esp32s3: SPI Mode       : DIO
I (43) boot.esp32s3: SPI Flash Size : 8MB
I (48) boot: Enabling RNG early entropy source...
W (53) bootloader_random: RNG for ESP32-S3 not currently supported
I (60) boot: Partition Table:
I (63) boot: ## Label            Usage          Type ST Offset   Length
I (71) boot:  0 factory          factory app      00 00 00010000 00200000
I (78) boot:  1 model            Unknown data     01 82 00210000 0050c000
I (86) boot: End of partition table
I (90) esp_image: segment 0: paddr=00010020 vaddr=3c080020 size=c3310h (799504) map
I (257) esp_image: segment 1: paddr=000d3338 vaddr=3fc9a2a0 size=028dch ( 10460) load
I (260) esp_image: segment 2: paddr=000d5c1c vaddr=40374000 size=0a3fch ( 41980) load
I (273) esp_image: segment 3: paddr=000e0020 vaddr=42000020 size=7067ch (460412) map
I (365) esp_image: segment 4: paddr=001506a4 vaddr=4037e3fc size=0be98h ( 48792) load
I (377) esp_image: segment 5: paddr=0015c544 vaddr=50000000 size=00010h (    16) load
I (386) boot: Loaded app from partition at offset 0x10000
I (386) boot: Disabling RNG early entropy source...
W (386) bootloader_random: RNG for ESP32-S3 not currently supported
Initializing SPIFFS
Partition size: total: 4857101, used: 3457023
model_name: xiaoaitongxue7 model_data: /srmodel/xiaoaitongxue7/wn7_data
MC Quantized wakeNet7: wakeNet7_v1_xiaoaitongxue_6_0.97_0.92, mode:2, p:3, (Feb  9 2022 16:22:38)

assert failed: spinlock_release spinlock.h:140 (core_id == lock->owner)


Backtrace:0x40375cea:0x3fcac9700x40381cf5:0x3fcac990 0x40388105:0x3fcac9b0 0x40384ec7:0x3fcacad0 0x40382120:0x3fcacaf0 0x40382d1f:0x3fcacb10 0x4202a595:0x3fcacb50 0x4202a65a:0x3fcacb70 0x4037bf61:0x3fcacb90 0x4037ce31:0x3fcacbb0 0x4037ce3d:0x3fcacbd0 0x4037fcd9:0x3fcacbf0 0x4037caf9:0x3fcacc10 0x4203087d:0x3fcacc50 0x42025a95:0x3fcacc80 0x42029537:0x3fcacca0 0x4202716f:0x3fcaccd0 0x42028946:0x3fcacd10 0x42026084:0x3fcacd50 0x42025865:0x3fcacd80 0x420039b9:0x3fcacda0 0x420519c6:0x3fcacdc0 0x42051a69:0x3fcacdf0 0x420082aa:0x3fcace10 0x42008452:0x3fcace30 0x42016614:0x3fcace50 0x42016977:0x3fcacfc0 0x42006fdf:0x3fcacff0 0x40384cd6:0x3fcad020
0x40375cea: panic_abort at C:/esp-idf/components/esp_system/panic.c:402

0x40381cf5: esp_system_abort at C:/esp-idf/components/esp_system/esp_system.c:121

0x40388105: __assert_func at C:/esp-idf/components/newlib/assert.c:85

0x40384ec7: spinlock_release at C:/esp-idf/components/esp_hw_support/include/soc/spinlock.h:140
 (inlined by) vPortCPUReleaseMutex at C:/esp-idf/components/freertos/port/xtensa/include/freertos/portmacro.h:568
 (inlined by) vPortExitCritical at C:/esp-idf/components/freertos/port/xtensa/port.c:298

0x40382120: prvIsQueueEmpty at C:/esp-idf/components/freertos/queue.c:2474

0x40382d1f: xQueueSemaphoreTake at C:/esp-idf/components/freertos/queue.c:1667

0x4202a595: esp_ipc_call_and_wait at C:/esp-idf/components/esp_ipc/src/esp_ipc.c:150

0x4202a65a: esp_ipc_call at C:/esp-idf/components/esp_ipc/src/esp_ipc.c:162

0x4037bf61: spi_flash_disable_interrupts_caches_and_other_cpu at C:/esp-idf/components/spi_flash/cache_utils.c:163

0x4037ce31: cache_disable at C:/esp-idf/components/spi_flash/spi_flash_os_func_app.c:70

0x4037ce3d: spi1_start at C:/esp-idf/components/spi_flash/spi_flash_os_func_app.c:99

0x4037fcd9: spiflash_start_default at C:/esp-idf/components/spi_flash/esp_flash_api.c:128

0x4037caf9: esp_flash_read at C:/esp-idf/components/spi_flash/esp_flash_api.c:821

0x4203087d: esp_partition_read at C:/esp-idf/components/spi_flash/partition.c:415

0x42025a95: spiffs_api_read at C:/esp-idf/components/spiffs/spiffs_api.c:36

0x42029537: spiffs_phys_rd at C:/esp-idf/components/spiffs/spiffs/src/spiffs_cache.c:161

0x4202716f: spiffs_obj_lu_find_entry_visitor at C:/esp-idf/components/spiffs/spiffs/src/spiffs_nucleus.c:166

0x42028946: spiffs_object_find_object_index_header_by_name at C:/esp-idf/components/spiffs/spiffs/src/spiffs_nucleus.c:1694

0x42026084: SPIFFS_open at C:/esp-idf/components/spiffs/spiffs/src/spiffs_hydrogen.c:230 (discriminator 2)

0x42025865: vfs_spiffs_open at C:/esp-idf/components/spiffs/esp_spiffs.c:479

0x420039b9: esp_vfs_open at C:/esp-idf/components/vfs/vfs.c:407 (discriminator 3)

0x420519c6: _fopen_r at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32s3-elf/src/newlib/newlib/libc/stdio/fopen.c:129

0x42051a69: fopen at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32s3-elf/src/newlib/newlib/libc/stdio/fopen.c:168

0x420082aa: get_file_length at /home/wangwangwang/workspace/ai_vnc/728_support/esp_sr_lib_vnc_1/build/../components/hufzip/hufzip_model.c:80

0x42008452: dl_matrixq_copy_to_psram_spiffs at /home/wangwangwang/workspace/ai_vnc/728_support/esp_sr_lib_vnc_1/build/../components/hufzip/hufzip_model.c:232

0x42016614: model_create at /home/wangwangwang/workspace/ai_vnc/728_support/esp_sr_lib_vnc_1/build/../components/multinet/multinet4_single_quantized_cn.c:795

0x42016977: model_init at /home/wangwangwang/workspace/ai_vnc/728_support/esp_sr_lib_vnc_1/build/../components/multinet/multinet4_single_quantized_cn.c:1826

0x42006fdf: detect_Task at c:\esp-skainet\examples\en_speech_commands_recognition\build/../main/main.c:75

0x40384cd6: vPortTaskWrapper at C:/esp-idf/components/freertos/port/xtensa/port.c:130

ESP-BOX ISSUE (AIS-783)

I (659) MEDIA_HAL: CONFIG_CODEC_CHIP_IS_ES8311
I (659) I2S: DMA Malloc info, datalen=blocksize=1280, dma_buf_count=6
I (669) I2S: DMA Malloc info, datalen=blocksize=1280, dma_buf_count=6
I (669) I2S: I2S1, MCLK output by GPIO0
I (679) DRV8311: Enter into es8311_Codec_Startup()

I (689) DRV8311: ES8311 in Slave mode

I (689) DRV8311: Enter into es8311_pcm_hw_params()

I (709) DRV8311: Exit es8311_Codec_Startup()

I (709) DRV8311: ES8311 in I2S Format
I (709) DRV8311: SET: volume:153

I (709) MEDIA_HAL: I2S_OUT_VOL_DEFAULT[60]
WYB:Slave mode:80 !
WYB:ES7210_MCLK_CTL_REG02 = 0xC3!
I (729) gpio: GPIO[46]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0
codec init done!!!!
I (729) DRV8311: SET: volume:165

ESP Chinese TTS v1.1 (Nov 12 2021 15:44:10)
欢迎使用乐鑫语音合成
I (739) tts_parser: unicode:0x6b22 -> huan1
I (749) tts_parser: unicode:0x8fce -> ying2
I (749) tts_parser: unicode:0x4f7f -> shi3
I (759) tts_parser: unicode:0x7528 -> yong4
I (759) tts_parser: unicode:0x4e50 -> le4
I (759) tts_parser: unicode:0x946b -> xin1
I (769) tts_parser: unicode:0x8bed -> yu3
I (769) tts_parser: unicode:0x97f3 -> yin1
I (779) tts_parser: unicode:0x5408 -> he2
I (779) tts_parser: unicode:0x6210 -> cheng2
Guru Meditation Error: Core 0 panic'ed (LoadProhibited). Exception was unhandled.

Core 0 register dump:
PC : 0x42005f12 PS : 0x00060c30 A0 : 0x82006258 A1 : 0x3fcf3f70
0x42005f12: esp_tts_stream_amr at /home/sunxiangyu/aishare/aigit2/esp_tts/build/../components/speech_synthesis/esp_tts_chinese/esp_tts.c:425

A2 : 0x00000002 A3 : 0x3c03bff0 A4 : 0x6e463b32 A5 : 0x3fce0c94
A6 : 0x3fcf3fc0 A7 : 0xb918ecc8 A8 : 0xf54970a4 A9 : 0x3fcf3f50
A10 : 0x00000001 A11 : 0x3c06503c A12 : 0x3fce0cdc A13 : 0x0000003c
A14 : 0x00000003 A15 : 0xff000000 SAR : 0x00000000 EXCCAUSE: 0x0000001c
EXCVADDR: 0xf54970a4 LBEG : 0x40056f5c LEND : 0x40056f72 LCOUNT : 0x00000000

Backtrace:0x42005f0f:0x3fcf3f700x42006255:0x3fcf3f90 0x420057d7:0x3fcf3fb0 0x42026fbc:0x3fcf43f0
0x42005f0f: esp_tts_stream_amr at /home/sunxiangyu/aishare/aigit2/esp_tts/build/../components/speech_synthesis/esp_tts_chinese/esp_tts.c:425

0x42006255: esp_tts_stream_play at /home/sunxiangyu/aishare/aigit2/esp_tts/build/../components/speech_synthesis/esp_tts_chinese/esp_tts.c:548

0x420057d7: app_main at /home/winter/item/esp-skainet/examples/chinese_tts/build/../main/main.c:85 (discriminator 1)

0x42026fbc: main_task at /home/winter/item/esp/esp-idf/components/freertos/port/port_common.c:129

lib文件报错,使用method1或者method2,也一样错误

Example build fails (AIS-858)

esp-skainet/components/hardware_driver/boards/esp32-korvo/bsp_board.c:226:107: error: unknown type name 'i2s_bits_per_chan_t';

questions about esp skainet cn_speech_command & chines_tts demo 关于skainet的中文命令字及中文tts例程问题 (AIS-794)

开发平台(platform)esp_s3_krovo1
1、中文命令字
demo可以运行的,在添加新命令字后,如果需要例程类似语音回复。请问可以用什么办法生成对应的.h文件?
2、中文tts
demo也是可以运行的,不过听起来由xiaole.a这个文件,inti voice set:template的标准模板设定,声音有点奇怪,像录音机没电的声音,我看过 sr里面有头文件esp_tts_voice_set应该有相应的设置,不过没有详细文档介绍?请问是否都有遇到类似情况。

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.