BirdNET to MQTT publishing for Home Assistant consumption

Readme.md

BirdNET-Pi to MQTT Integration

This Python script birdnet_to_mqtt.py does monitoring on the BirdNET-Pi detections from the syslog file and publishes the bird species detection data to an MQTT broker, which can be used by Home Assistant or other systems.

Introduction

BirdNET-Pi is an tool for analyzing bird sounds using a Raspberry Pi. This project enhances its utility by integrating detected bird species data into Home Assistant through MQTT, enabling users to receive real-time alerts and perform automated actions based on bird detections.

Prerequisites

• A working installation of BirdNET-Pi that logs detections to the syslog.
• MQTT Broker (like Mosquitto) setup within your network.
• Python 3.x installed on your system.

Changes

• Error handling on mqtt connection
• Password and Username's
• Works with newer paho-mqtt versions

Setup

1. Clone the Repository

First, ensure you have the script on your local system by cloning the repository:

git clone https://gist.github.com/JuanMeeske/08b839246a62ff38778f701fc1da5554
cd your-folder-name

2. Install python Libaries

sudo pip3 install dateparser
sudo pip3 install paho-mqtt

3. Add your details

# MQTT Server Settings
mqtt_server = "192.168.xyz.xyz" #mqtt server address
mqtt_port = 1883  # Default port for MQTT
mqtt_keepalive = 60  # Keepalive interval in seconds
client_id = 'python-mqtt'

# MQTT Authentication Settings
mqtt_username = "mqtt_birdnet"  # Replace 'your_username' with your actual MQTT username
mqtt_password = "<yourpasswordhere>"  # Replace 'your_password' with your actual MQTT password

# mqtt topic where all heard birds will be published
mqtt_topic_all_birds = 'birdpi/all'

4. Add your details

python birdnet_to_mqtt.py

4. Run the script

python birdnet_to_mqtt.py

The script should output if connection was successful or not:

# python3 birdnet_to_mqtt.py
Connected to MQTT Broker!

4. Run the script as a service

Follow the steps in the gistfile1.txt

birdnet_to_mqtt.py

import time
import re
import dateparser
import datetime
import json
import sys
import logging
import paho.mqtt.client as mqtt

# Setup basic configuration for logging
logging.basicConfig(level=logging.INFO)

# Constants
SYSLOG_FILE_PATH = '/var/log/syslog'
MQTT_SERVER = "192.168.xyz.xyz"  # Replace with your MQTT server address or hostname
MQTT_PORT = 1883
MQTT_KEEPALIVE = 60
MQTT_TOPIC_ALL_BIRDS = 'birdpi/all'
CLIENT_ID = 'python-mqtt'
USERNAME = "mqtt_birdnet"  # Replace with your MQTT username
PASSWORD = "<passwordgoeshere>"  # Replace with your MQTT password

# Regular expression pattern to parse log entries
RE_BIRD_ENTRY = re.compile(
    r'(\d{4}-\d{2}-\d{2};\d{2}:\d{2}:\d{2};[^;]+;[^;]+;\d+\.\d+;\d+\.\d+;\d+\.\d+;\d+\.\d+;\d+;\d+\.\d+;\d+\.\d+;)([^ ]+\.mp3)'
)

def file_row_generator(file_path):
    """ Generator that yields new lines from a file continuously. """
    with open(file_path, 'r') as file:
        file.seek(0, 2)  # Move the pointer to the end of the file
        while True:
            line = file.readline()
            if not line:
                time.sleep(0.1)  # Sleep briefly to avoid busy waiting
                continue
            yield line

def on_connect(client, userdata, flags, rc, properties=None):
    """ Callback for when the client receives a CONNACK response from the server. """
    if rc == 0:
        logging.info("Connected to MQTT Broker!")
    else:
        logging.error(f"Failed to connect, return code {rc}\n")

def on_disconnect(client, userdata, rc, disconnect_flags, properties=None):
    """ Callback for when the client disconnects from the server. """
    logging.info(f"Disconnected from MQTT Broker with rc: {rc}")

# Setup MQTT client
mqtt_client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2, CLIENT_ID)
mqtt_client.username_pw_set(USERNAME, PASSWORD)
mqtt_client.on_connect = on_connect
mqtt_client.on_disconnect = on_disconnect

# Connect to MQTT Broker
mqtt_client.connect(MQTT_SERVER, MQTT_PORT, MQTT_KEEPALIVE)
mqtt_client.loop_start()

try:
    # Process each new line in the syslog file
    for row in file_row_generator(SYSLOG_FILE_PATH):
        try:
            match = RE_BIRD_ENTRY.search(row)
            if match:
                details, mp3_filename = match.groups()
                details_list = details.split(';')
                timestamp = details_list[0]
                species = details_list[2]
                common_name = details_list[3]
                confidence = float(details_list[4])

                # Convert timestamp to UNIX time
                parsed_time = datetime.datetime.timestamp(dateparser.parse(timestamp))

                bird_data = {
                    'timestamp': parsed_time,
                    'species': species,
                    'common_name': common_name,
                    'confidence': confidence,
                    'mp3_filename': mp3_filename
                }

                logging.info(f"Published bird data: {bird_data}")
                
                # Publishing data to MQTT
                mqtt_client.publish(MQTT_TOPIC_ALL_BIRDS, json.dumps(bird_data), qos=1)
            else:
                continue  # Skip this iteration if no matching log entry is found

        except Exception as e:
            logging.error(f"Error processing row: {e}")

finally:
    mqtt_client.loop_stop()
    mqtt_client.disconnect()

sys.exit(0)

Actually I have adapted the code just for fun :-) here as code that could work :

In constants :

# Extract the value of confidence and convert it to float
with open('/etc/birdnet/birdnet.conf', 'r') as file:
    lines = file.readlines()
    for line in lines:
        if line.startswith('CONFIDENCE='):
            CONFIDENCE = float(line.split('=')[1].strip())

then add this additional code (to get confidence, and only send bird_data if confidence is acceptable) just before the bird_data :

                confidence = float(details_list[4])
                if confidence > CONFIDENCE:
                    bird_data = {

hello impossible to copy the birdnet-to-mqtt.py folder in usr/local/bin how I can do thank you