ESPHome config - Raspiaudio Muse Luxe as a voice assistant satellite in Home Assistant

README.md

Introduction

The purpose of this ESPHome config is to be able to use the Raspiaudio Muse Luxe as a voice assistant satellite in Home Assistant.

Features

• wake word, push to talk and continuous conversation support
• response playback
• service exposed in HA to start and stop the voice assistant from another device/trigger
• visual feedback of the recording/success/error status via the Luxe's onboard LED

Pre-requisites

• Home Assistant 2023.10.0 or newer
• A voice assistant configured in HA with STT and TTS in a language of your choice
• ESPHome 2023.10.1 or newer

Known issues and limitations

• the media_player component in ESPHome does not play raw audio coming from Piper TTS. It works with cloud STT, though
• the battery percentage is calculated using info from this Reddit post, which either doesn't seem to be very accurate or it relies on inaccurate ADC values from the ESP.

Installation instructions

If the Muse Luxe has already been added to ESPHome

Edit your config and paste the one below. Double check Wifi connection details, API encryption key and device name/friendly name to make sure you use your own.

If the Muse Luxe is not already in ESPHome

Add a new ESP32 device in ESPHome, using the configuration below.

After the device has been added to ESPHome

Compile and install the firmware on the speaker. If auto discovery is turned on, the device should appear in Home Assistant automatically. Otherwise, check out this guide.

Credits

• Muse Luxe pinout from here https://github.com/RASPIAUDIO/Simple_Bluetooth_Speaker_ESP32
• Base ESPHome config from here https://github.com/esphome/media-players/blob/main/raspiaudio-muse-luxe.yaml
• A basic voice assistant config from here https://forum.raspiaudio.com/t/muse-luxe-voice-assistant-now-possible/726/14?u=tetele

or

muse-luxe.yaml

substitutions:
  name: "muse-luxe"
  friendly_name: "RaspiAudio Muse Luxe"
  wifi_ap_password: ""

esphome:
  name: ${name}
  friendly_name: ${friendly_name}
  name_add_mac_suffix: false
  min_version: 2023.10.1
  on_boot:
    then:
      - output.turn_on: dac_mute
      - light.turn_on:
          id: top_led
          effect: slow_pulse
          red: 100%
          green: 60%
          blue: 0%

esp32:
  board: esp-wrover-kit
  framework:
    type: arduino

logger:
api:
  services:
    - service: start_va
      then:
        - voice_assistant.start
    - service: stop_va
      then:
        - voice_assistant.stop
ota:

i2c:
  sda: GPIO18
  scl: GPIO23

wifi:
  ap:
    password: "${wifi_ap_password}"

captive_portal:

improv_serial:

external_components:
  - source: github://RASPIAUDIO/esphomeLuxe@main
    components: [es8388]
    refresh: 0s

es8388:

globals:
  - id: wifi_connected
    type: bool
    initial_value: "false"
    restore_value: false

interval:
  - interval: 1s
    then:
      - if:
          condition:
            and:
              - lambda: "return !id(wifi_connected);"
              - wifi.connected:
          then:
            - globals.set:
                id: wifi_connected
                value: "true"
            - light.turn_on:
                id: top_led
                effect: pulse
                red: 0%
                green: 100%
                blue: 0%
            - delay: 1s
            - light.turn_off: top_led

output:
  - platform: gpio
    id: dac_mute
    pin: GPIO21
    inverted: true

i2s_audio:
  - i2s_lrclk_pin: GPIO25
    i2s_bclk_pin: GPIO5

media_player:
  - platform: i2s_audio
    name: None
    id: luxe_out
    dac_type: external
    i2s_dout_pin: GPIO26
    mode: stereo
    on_state:
      if:
        condition:
          media_player.is_playing:
        then:
          output.turn_off: dac_mute
        else:
          output.turn_on: dac_mute

microphone:
  - platform: i2s_audio
    id: luxe_microphone
    i2s_din_pin: GPIO35
    adc_type: external
    pdm: false

voice_assistant:
  id: va
  microphone: luxe_microphone
  media_player: luxe_out
  use_wake_word: true
  on_listening:
    - light.turn_on:
        id: top_led
        blue: 100%
        red: 0%
        green: 0%
        brightness: 100%
        effect: pulse
  on_tts_start:
    - light.turn_on:
        id: top_led
        blue: 60%
        red: 20%
        green: 20%
        effect: none
  on_tts_end:
    - media_player.play_media: !lambda return x;
    - light.turn_on:
        id: top_led
        blue: 60%
        red: 20%
        green: 20%
        effect: pulse
    # This is useful when you want to stream the response on another media_player
    # - homeassistant.service:
    #     service: media_player.play_media
    #     data:
    #       entity_id: media_player.some_speaker
    #       media_content_id: !lambda 'return x;'
    #       media_content_type: music
    #       announce: "true"
  on_client_connected:
    - if:
        condition:
          - switch.is_on: use_wake_word
        then:
          - voice_assistant.start_continuous:
  on_client_disconnected:
    - if:
        condition:
          - switch.is_on: use_wake_word
        then:
          - voice_assistant.stop:
  on_end:
    - delay: 100ms
    - wait_until:
        not:
          media_player.is_playing: luxe_out
    - script.execute: reset_led
  on_error:
    - light.turn_on:
        id: top_led
        blue: 0%
        red: 100%
        green: 0%
        effect: none
    - delay: 1s
    - script.execute: reset_led
    - script.wait: reset_led
    - lambda: |-
        if (code == "wake-provider-missing" || code == "wake-engine-missing") {
          id(use_wake_word).turn_off();
        }
sensor:
  - platform: adc
    pin: GPIO33
    name: Battery voltage
    device_class: voltage
    unit_of_measurement: "V"
    accuracy_decimals: 2
    state_class: measurement
    entity_category: diagnostic
    update_interval: 15s
    attenuation: auto
    filters:
      - multiply: 2 # https://forum.raspiaudio.com/t/esp-muse-luxe-bluetooth-speaker/294/12
      - exponential_moving_average:
          alpha: 0.2
          send_every: 2
      - delta: 0.002
    on_value:
      then:
        - sensor.template.publish:
            id: battery_percent
            state: !lambda "return x;"

  - platform: template
    name: Battery
    id: battery_percent
    device_class: battery
    unit_of_measurement: "%"
    accuracy_decimals: 0
    state_class: measurement
    entity_category: diagnostic
    update_interval: 15s
    filters:
      - calibrate_polynomial:
          degree: 3
          datapoints:
            - 4.58 -> 100.0
            - 4.5 -> 97.1
            - 4.47 -> 94.2
            - 4.44 -> 88.4
            - 4.42 -> 82.7
            - 4.41 -> 76.9
            - 4.41 -> 71.1
            - 4.37 -> 65.3
            - 4.35 -> 59.5
            - 4.31 -> 53.8
            - 4.28 -> 48.0
            - 4.26 -> 42.2
            - 4.23 -> 36.4
            - 4.21 -> 30.6
            - 4.19 -> 24.9
            - 4.16 -> 19.1
            - 4.1 -> 13.3
            - 4.07 -> 10.4
            - 4.03 -> 7.5
            - 3.97 -> 4.6
            - 3.82 -> 1.7
            - 3.27 -> 0.0
      - lambda: return clamp(x, 0.0f, 100.0f);

binary_sensor:
  - platform: gpio
    pin:
      number: GPIO19
      inverted: true
      mode:
        input: true
        pullup: true
    name: Volume Up
    on_click:
      - media_player.volume_up: luxe_out
  - platform: gpio
    pin:
      number: GPIO32
      inverted: true
      mode:
        input: true
        pullup: true
    name: Volume Down
    on_click:
      - media_player.volume_down: luxe_out
  - platform: gpio
    pin:
      number: GPIO12
      inverted: true
      mode:
        input: true
        pullup: true
    name: Action
    on_click:
      - if:
          condition:
            switch.is_off: use_wake_word
          then:
            - if:
                condition: voice_assistant.is_running
                then:
                  - voice_assistant.stop:
                  - script.execute: reset_led
                else:
                  - voice_assistant.start:
          else:
            - voice_assistant.stop
            - delay: 1s
            - script.execute: reset_led
            - script.wait: reset_led
            - voice_assistant.start_continuous:

light:
  - platform: esp32_rmt_led_strip
    name: None
    id: top_led
    pin: GPIO22
    chipset: SK6812
    num_leds: 1
    rgb_order: grb
    rmt_channel: 0
    default_transition_length: 0s
    gamma_correct: 2.8
    effects:
      - pulse:
          name: pulse
          transition_length: 250ms
          update_interval: 250ms
      - pulse:
          name: slow_pulse
          transition_length: 1s
          update_interval: 2s

script:
  - id: reset_led
    then:
      - if:
          condition:
            switch.is_on: use_wake_word
          then:
            - light.turn_on:
                id: top_led
                blue: 100%
                red: 100%
                green: 0%
                brightness: 100%
                effect: none
          else:
            - light.turn_off: top_led

switch:
  - platform: template
    name: Use Wake Word
    id: use_wake_word
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    on_turn_on:
      - lambda: id(va).set_use_wake_word(true);
      - if:
          condition:
            not:
              - voice_assistant.is_running
          then:
            - voice_assistant.start_continuous
      - script.execute: reset_led
    on_turn_off:
      - voice_assistant.stop
      - lambda: id(va).set_use_wake_word(false);
      - script.execute: reset_led

I have both HA Cloud and Whisper. Tried both with the same result. HA is 2023.7.0

Is the ESPHome device allowed to make service calls in HA?

It was not. I just enabled but still seems to timeout waiting for speech. I will keep playing with it this weekend to see if I can figure it out. nice work !

It doesn't work for me.

When validating the yaml it gives me the following error:

Failed config
media_player.i2s_audio: [source /config/esphome/.esphome/packages/7b8574f6/raspiaudio-muse-luxe.yaml:51]
  
Too many candidates found for 'i2s_audio_id' type 'i2s_audio::I2SAudioComponent' You must assign an explicit ID to the parent component you want to use.

Do I have to change something in the configuration?

What version of ESPHome are you running @Zagur ?

@tetele The latest version available: 2023.7.1. Only works with 2013.5?

@Zagur sorry for the delay, I was out for a few days. I've tried reproducing your error and I can't. Are you sure you haven't defined multiple other I2S audio devices that are not present in the config in this gist? Could you share your full config and an updated error message?

@tetele I just tried again and this time it worked correctly. I don't know exactly what it was leaving me the other time, but it's finally working! Thank you very much!

First, I want to say thank you, this is awesome.

Second, what's going on with the voltages here for the battery? That Reddit post is correct, an 18650 100% is 4.2V and 0% is 3v.

Finally, is this able to stream audio? (Music, notifications, etc.) The Music Assistant add-on doesn't recognize it, and using the Media section in HA will not play anything.

@tetele A couple of questions for you.

Is this still required with my speaker running ESPHome 2023.11 and HA 2024.1?
If it is required, then I'm getting issues with the !lambda return x; in the yaml file. HA doesn't seem to know what that means.

You should be able to remove this from the readme now:

Known issues and limitations

the media_player component in ESPHome home-assistant/core#92969. It works with cloud STT, though

The link to the issue shows it has been resolved.

Since updating to 2024.5.x I can't send TTS cloud to my museluxe using this yaml. Speaker works with radio streams. TTS works to homepods. Any thoughts as to what broke it?

New to github but I think I fixed the issue @monsieurlatte . Looks like its something to do with how the current code is handling muting the dac:

output:
  - platform: gpio
    id: dac_mute
    pin: GPIO21
    inverted: true

I simply removed all references of dac_mute and used mute_pin from media-players/raspiaudio-muse-luxe.yaml:

media_player:
  - platform: i2s_audio
    name: None
    id: luxe_out
    dac_type: external
    i2s_dout_pin: GPIO26
    mode: stereo
    mute_pin:
      number: GPIO21
      inverted: true

This fixed my issue. dont forget to remove - output.turn_on: dac_mute from esphome: as well.

I am still receiving the following error BUT the audio from TTS is at least playing:

[01:54:45][W][component:237]: Component i2s_audio.media_player took a long time for an operation (542 ms).
[01:54:45][W][component:238]: Components should block for at most 30 ms.

EDIT: Guess I should include the whole thing...
EDIT 2: After sleeping on it and looking at some other examples, this is my latest yaml:

---
esphome:
  name: raspiaudio-muse-luxe
  friendly_name: RaspiAudio Muse Luxe
  min_version: 2024.6.0
  name_add_mac_suffix: true
  project:
    name: raspiaudio.muse-luxe-voice-assistant
    version: "24.7.4.1"

esp32:
  board: esp-wrover-kit
  framework:
    type: arduino

logger:
api:

ota:
  - platform: esphome
    id: ota_esphome

i2c:
  - id: i2c_bus
    sda: GPIO18
    scl: GPIO23

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  manual_ip:
    # Set this to the IP of the ESP
    static_ip: 10.0.0.100
    # Set this to the IP address of the router. Often ends with .1
    gateway: 10.0.0.1
    # The subnet of the network. 255.255.255.0 works for most home networks.
    subnet: 255.255.255.0
  ap:

improv_serial:

button:
  - platform: safe_mode
    id: button_safe_mode
    name: Safe Mode Boot

  - platform: factory_reset
    id: factory_reset_btn
    name: Factory reset

external_components:
  - source: github://pr#3552  # DAC support https://github.com/esphome/esphome/pull/3552
    components: [es8388]
    refresh: 0s

es8388:

i2s_audio:
  - id: i2s_audio_bus
    i2s_lrclk_pin: GPIO25
    i2s_bclk_pin: GPIO5

microphone:
  - platform: i2s_audio
    id: luxe_microphone
    i2s_din_pin: GPIO35
    adc_type: external
    pdm: false

media_player:
  - platform: i2s_audio
    name: None
    id: luxe_out
    dac_type: external
    i2s_dout_pin: GPIO26
    mode: stereo
    mute_pin:
      number: GPIO21
      inverted: true

voice_assistant:
  id: va
  microphone: luxe_microphone
  media_player: luxe_out
  use_wake_word: true
  noise_suppression_level: 2
  auto_gain: 31dBFS
  volume_multiplier: 2.0
  on_listening:
    - light.turn_on:
        id: led
        blue: 100%
        red: 0%
        green: 0%
        effect: "Slow Pulse"
  on_stt_vad_end:
    - light.turn_on:
        id: led
        blue: 100%
        red: 0%
        green: 0%
        effect: "Fast Pulse"
  on_tts_start:
    - light.turn_on:
        id: led
        blue: 100%
        red: 0%
        green: 0%
        brightness: 100%
        effect: none
  on_end:
    - delay: 100ms
    - wait_until:
        not:
          media_player.is_playing:
    - script.execute: reset_led
  on_error:
    - light.turn_on:
        id: led
        blue: 0%
        red: 100%
        green: 0%
        brightness: 100%
        effect: none
    - delay: 1s
    - script.execute: reset_led
  on_client_connected:
    - if:
        condition:
          switch.is_on: use_wake_word
        then:
          - voice_assistant.start_continuous:
          - script.execute: reset_led
  on_client_disconnected:
    - if:
        condition:
          switch.is_on: use_wake_word
        then:
          - voice_assistant.stop:
          - light.turn_off: led

sensor:
  - platform: adc
    pin: GPIO33
    name: Battery
    id: battery_sensor
    icon: "mdi:battery-outline"
    device_class: voltage
    state_class: measurement
    unit_of_measurement: V
    update_interval: 15s
    accuracy_decimals: 3
    attenuation: 11db
    raw: true
    filters:
      - multiply: 0.00173913  # 2300 -> 4, for attenuation 11db, based on Olivier's code
      - exponential_moving_average:
          alpha: 0.2
          send_every: 2
      - delta: 0.002

  - platform: template
    name: Battery
    id: battery_percent
    device_class: battery
    unit_of_measurement: "%"
    accuracy_decimals: 0
    state_class: measurement
    entity_category: diagnostic
    update_interval: 15s
    filters:
      - calibrate_polynomial:
          degree: 3
          datapoints:
            - 4.58 -> 100.0
            - 4.5 -> 97.1
            - 4.47 -> 94.2
            - 4.44 -> 88.4
            - 4.42 -> 82.7
            - 4.41 -> 76.9
            - 4.41 -> 71.1
            - 4.37 -> 65.3
            - 4.35 -> 59.5
            - 4.31 -> 53.8
            - 4.28 -> 48.0
            - 4.26 -> 42.2
            - 4.23 -> 36.4
            - 4.21 -> 30.6
            - 4.19 -> 24.9
            - 4.16 -> 19.1
            - 4.1 -> 13.3
            - 4.07 -> 10.4
            - 4.03 -> 7.5
            - 3.97 -> 4.6
            - 3.82 -> 1.7
            - 3.27 -> 0.0
      - lambda: return clamp(x, 0.0f, 100.0f);

binary_sensor:
  - platform: gpio
    pin:
      number: GPIO19
      inverted: true
      mode:
        input: true
        pullup: true
    name: Volume Up
    id: volume_up
    disabled_by_default: true
  - platform: gpio
    pin:
      number: GPIO32
      inverted: true
      mode:
        input: true
        pullup: true
    name: Volume Down
    id: volume_down
    disabled_by_default: true
  - platform: gpio
    pin:
      number: GPIO12
      inverted: true
      mode:
        input: true
        pullup: true
    name: Action
    id: action_button
    disabled_by_default: true
    on_multi_click:
      - timing:
          - ON for at least 250ms
          - OFF for at least 50ms
        then:
          - if:
              condition:
                switch.is_off: use_wake_word
              then:
                - if:
                    condition: voice_assistant.is_running
                    then:
                      - voice_assistant.stop:
                      - script.execute: reset_led
                    else:
                      - voice_assistant.start:
              else:
                - voice_assistant.stop
                - delay: 1s
                - script.execute: reset_led
                - script.wait: reset_led
                - voice_assistant.start_continuous:
      - timing:
          - ON for at least 10s
        then:
          - button.press: factory_reset_btn

light:
  - platform: esp32_rmt_led_strip
    name: None
    id: led
    pin: GPIO22
    chipset: SK6812
    num_leds: 1
    rgb_order: grb
    rmt_channel: 0
    default_transition_length: 0s
    gamma_correct: 2.8
    effects:
      - pulse:
          name: "Slow Pulse"
          transition_length: 250ms
          update_interval: 250ms
          min_brightness: 50%
          max_brightness: 100%
      - pulse:
          name: "Fast Pulse"
          transition_length: 100ms
          update_interval: 100ms
          min_brightness: 50%
          max_brightness: 100%

script:
  - id: reset_led
    then:
      - if:
          condition:
            switch.is_on: use_wake_word
          then:
            - light.turn_on:
                id: led
                red: 100%
                green: 89%
                blue: 71%
                brightness: 60%
                effect: none
          else:
            - light.turn_off: led

switch:
  - platform: template
    name: Use wake word
    id: use_wake_word
    optimistic: true
    restore_mode: RESTORE_DEFAULT_ON
    entity_category: config
    on_turn_on:
      - lambda: id(va).set_use_wake_word(true);
      - if:
          condition:
            not:
              - voice_assistant.is_running
          then:
            - voice_assistant.start_continuous
      - script.execute: reset_led
    on_turn_off:
      - voice_assistant.stop
      - lambda: id(va).set_use_wake_word(false);
      - script.execute: reset_led

Thank you @AnthonyJWinslow! Your fix works for me too - and you made my day!

Thanks @AnthonyJWinslow !
It actually solves the TTS which didn't work.
However it creates 3 new problems vs the original Yaml from @tetele :

1. For 30% of the cases, it just doesn't start at all, I've only this in the logs, no connection to wifi nor HA
   [17:33:34]Bets Jun 8 2016 00:22:57 [17:33:34] [17:33:34]rst:0x1 (POWERON_RESET),boot:0x0 (DOWNLOAD_BOOT(UART0/UART1/SDIO_FEI_FEO_V2)) [17:33:34]waiting for download
2. With wake word switch off, when I push the Play button for a few seconds to make the device start listening, the speaker makes a not cool sound during ~10 sec. It only happens for the first attempt after I switch on the speaker (it works well after that until the next reboot). See quick video https://photos.app.goo.gl/dv31Rze1cDyhh8MQ9
3. Volume buttons don't work anymore even if you remove #disabled_by_default: true. However this works well through the blue bar HA UI (but I'd prefer to control directly from the Muse Luxe)

And there is also still the same (and old) problem about the battery status that is not working properly (% does not match reality for instance it shows 5% whereas it's fully charged in reality)

Any idea ? Anyone seeing the same cc @MacrosorcH (I didn't change a word from the YAML) ?

Does anyone have issues with esphome 2024.10.1? Muse luxe keeps crashing. Going back to 2024.9.2 works fine (it also worked fine in 2024.10.0)

@luca-angemi 2024.10.1 appears to have broken more then muse, my everything presence lite sensor also locks up after booting with the new firmware. It's 6am here so I will look into this when I'm a little more awake 😴

Same here, I had to revert to a previous recovery/version of HA to make it work back!

ESPHome 2024.10.2 is out, with this fix esphome/esphome#7662

I can't try it atm, but please get back if it does fix the crash

It does fix it, thanks!

Hi all
after several attempts I manage to configure my muse as voice assist thanks to the efforts of @tetele. THANKS!
Now I just got the problem that I don't receive feedback from the speaker. Although it should work with cloud STT (as stated above mine isn't working now. Is that correct or probably there is something wrong with my setup?

if now it doesn't work even with cloud then If I understand right we have 2 options
to add a speaker

# This is useful when you want to stream the response on another media_player
# - homeassistant.service:
#     service: media_player.play_media
#     data:
#       entity_id: media_player.some_speaker
#       media_content_id: !lambda 'return x;'
#       media_content_type: music
#       announce: "true"

But when I try I get the following:
Can someone give me the right code for this option please?

`INFO ESPHome 2024.10.2
INFO Reading configuration /config/esphome/muse-voice-assist.yaml...
ERROR Error while reading config: Invalid YAML syntax:

while parsing a block collection
in "/config/esphome/muse-voice-assist.yaml", line 142, column 5
expected , but found ''
in "/config/esphome/muse-voice-assist.yaml", line 150, column 6`

---

or
2. change the code as per @AnthonyJWinslow above example.
if I just copy and paste this code from 'improv_serial and below will work?
`improv_serial:

button:

• platform: safe_mode
  id: button_safe_mode
  name: Safe Mode Boot`

---

Since I just manage to set up the assist after several hours of testing I am afraid to "touch" anything right now :)
So if possible someone advice if my above assumptions are correct and help me out with my questions in bold.
Thanks

any help for the above please?

any help for the above please?

Not with that unformatted code. YAML is super sensitive to whitespace and that's exactly the error you are getting. Paste the full config on something like dpaste.org and link it here if you want help

is this ok?
https://pastebin.com/y22mCFj9

edit: I am not sure what you are asking. I am just deleting the # in the above yaml like this
https://pastebin.com/xnKSaMJ6
and I get the error when I try to validate the code

Do you need something else? (sorry english is not my native language)

The errors above don't seem consistend with the second config you've sent.

First off, on line 142 replace !lambda return x; with !lambda 'return x;'

Then compile again and paste the errors you get. If yoy change anything else, also upload the config to pastebin again.

now that you told me that is the yaml indentation problem I deleted the # and one more "space" and the code is working!
Thanks!

2 more question if I may
I see that there are 2 more options in the below card

1. if I change the to "assist" it will use the local assist that I made? or do I need to make more changes? (this assist is in Greek)

2. Now I have it in Default. What "relaxed" and "aggresive" meaning?

1. Yes
2. Relaxed means it has to be pretty quiet for a while after you speak for the recording to stop. Aggressive means that as soon as you stop talking or lower your voice, the recording stops and it's sent to the STT

1. Yes
2. Relaxed means it has to be pretty quiet for a while after you speak for the recording to stop. Aggressive means that as soon as you stop talking or lower your voice, the recording stops and it's sent to the STT

Thanks!
(greek language is not working, probably I have to change the HA in Greek. Now is in English. I can see that it listens Greek and it reply in Greek but it "doesn't understand")

Excuse my ignorance, but does this implementation of voice assistant on the muse box include the timer feature? This is a deal breaker for my wife if it's to replace our google nests. I have messed around with the M5stack echo and this firmware which includes timer, but before I start adding code to your yaml, @tetele, I wanted to make sure this didn't already work. (My muse is on backorder so it will be a while before I can attempt to implement)