Use the yamnet TensorFlow model to classify live audio from a microphone and publish the predicted results to Home Assistant via MQTT

Open config.py and set the following variables:

• MQTT_HOST: MQTT server hostname
• MQTT_PORT: MQTT server port
• MQTT_USER: MQTT server username (optional)
• MQTT_PASS: MQTT server password (optional)
• MQTT_KEEPALIVE: MQTT server keepalive interval
• HA_SENSOR_NAME: Home Assistant sensor name
• HA_SENSOR_UUID: Home Assistant sensor UUID
• HA_SENSOR_EXPIRE_AFTER: How many seconds until Home Assistant considers the data to be stale
• TF_MODEL: TensorFlow model filename
• TF_NUM_THREADS: TensorFlow number of threads
• TF_SCORE_THRESHOLD: TensorFlow minimum score
• TF_MAX_RESULTS: TensorFlow maximum results
• TF_IGNORED_CATEGORIES: Categories to ignore
• LOG_LEVEL: Logging level
• LOG_FORMAT: Logging format
• LOG_COLORS: Logging level colors

Check your Python version and make sure version 3.9 or newer is installed on your system:

python3 --version

Install required python3 modules:

pip3 install -r requirements.txt

On Linux, install the PortAudio library:

sudo apt-get update && apt-get install libportaudio2

Run the listen.py script to start listening to the microphone and publishing the data to the MQTT server:

python3 ./listen.py

The sensor is now available in Home Assistant and can be used to trigger automations:

To run this script as a Docker container, modify the config.py file as decribed above and then build the image:

docker build -t audioclassifier-mqtt .

Now create a container with access to the system's audio device (/dev/dsp is the default) and host networking to communicate with the MQTT server:

docker run --name audioclassifier --restart=always -d --device /dev/dsp --network=host audioclassifier-mqtt

Copyright (C) 2022 Sam Steele. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.