Muse Luxe voice assistant now possible?

ESPMUSE
#53

seems like this has only been added 1 week ago in

You do need that PR, but it should work if you add it. See my config for the Muse Luxe above.

Any idea how I could get the actual audio output from ESPHome to debug what comes out there ?

See these docs on how you could capture and stream the audio data. You can microphone.capture, then on_data: send that data to a small HTTP server that can collect and save it for debugging purposes.

#54

It seems that the Muse and I2S microphone will be added in the next release : feat(muse luxe): add voice assistant by bemble · Pull Request #37 · esphome/media-players · GitHub

#55

this is in the new update for esphome, not tried but sounds interesting :slight_smile:
https://github.com/esphome/esphome/pull/4743

#56

I can confirm that this does appear to work. it seems that I can’t change the playback volume but this does play the response tts. Here’s the yaml I’m using for my muse luxe, it could probably use a bit of cleaning up and a few more features but it works

substitutions:
  name: "raspiaudio-muse-luxe"
  friendly_name: "RaspiAudio Muse Luxe"
esphome:
  name: "${name}"
  name_add_mac_suffix: true
  project:
    name: raspiaudio.muse-luxe
    version: "2.0"

esp32:
  board: esp-wrover-kit
  framework:
    type: arduino

logger:
  level: DEBUG

api:
  encryption:
    key: U+3hBt3JMESTTn8OceUvwx/yPxWcUmQXRl8Iela5j7A=

ota:

wifi:
  networks:
  - ssid: !secret wifi_ssid
    password: !secret wifi_password

external_components:
  - source:
      type: git
      url: https://github.com/esphome/esphome
      ref: dev
    components: [adc, i2s_audio, microphone, media_player, speaker]
    refresh: 0s
  - source: github://pr#3552
    components: [es8388]
    refresh: 0s

i2c:
  sda: GPIO18
  scl: GPIO23

captive_portal:

improv_serial:

i2s_audio:
  - i2s_lrclk_pin: GPIO25
    i2s_bclk_pin: GPIO5

media_player:
  - platform: i2s_audio
    name: ${friendly_name}
    dac_type: external
    i2s_dout_pin: GPIO26
    mode: stereo
    mute_pin:
      number: GPIO21
      inverted: true

speaker:
  - platform: i2s_audio
    id: luxe_speaker
    dac_type: external
    i2s_dout_pin: GPIO26
    mode: stereo

es8388:

sensor:
 - platform: adc
   pin: GPIO33
   name: ${name} Battery
   icon: "mdi:battery-outline"
   device_class: voltage
   state_class: measurement
   unit_of_measurement: V
   update_interval: 15s
   accuracy_decimals: 3
   attenuation: 11db
   raw: true
   filters:
    - multiply: 0.00173913 # 2300 -> 4, for attenuation 11db, based on Olivier's code
    - exponential_moving_average:
        alpha: 0.2
        send_every: 2
    - delta: 0.002

light:
  - platform: fastled_clockless
    name: ${friendly_name}
    pin: GPIO22
    chipset: SK6812
    num_leds: 1
    rgb_order: grb

microphone:
  - platform: i2s_audio
    adc_type: external
    id: luxe_microphone
    pdm: false
    i2s_din_pin: GPIO35

voice_assistant:
  microphone: luxe_microphone
  speaker: luxe_speaker

binary_sensor:
  - platform: gpio
    pin:
      number: GPIO19
      inverted: true
      mode:
        input: true
        pullup: true
    name: ${friendly_name} Volume Up
    on_click:
      - media_player.volume_up:
  - platform: gpio
    pin:
      number: GPIO32
      inverted: true
      mode:
        input: true
        pullup: true
    name: ${friendly_name} Volume Down
    on_click:
      - media_player.volume_down:
  - platform: gpio
    pin:
      number: GPIO12
      inverted: true
      mode:
        input: true
        pullup: true
    name: ${friendly_name} Play Button
    on_press:
      - voice_assistant.start:
    on_release:
      - voice_assistant.stop:
#57

how did you get around the “Platform not found: ‘speaker.i2s_audio’” ?

#58

The speaker component in ESPHome seems to have some buffering issues, but it does work with Piper. @jesserockz (maintainer of ESPHome) said he would look into this next week.

https://imgur.com/a/9dc6AyJ

I’ve reported the issue, but it looks like we’re stuck with media_player for now, which doesn’t play raw audio.

#59

If your ESPHome is up to date and you’re not pulling in external_components that overwrite the speaker or i2s_audio components, just hard refresh the browser and it should work.

1 Like
#60

thanks that did it, i just copied the text from Zortan and it kept giving me red text :slight_smile:
on my esp32 it didn’t work because i had two different pin sets for i2s (mic and speaker) which the speaker compnent doesn’t seem to support. besides the buffering issue it has, volume also still seems to be uncontrollable. not perfect but a step forward to finally hear a voice talking back :smile:

#61

for the lazy here is the final working yaml for Muse Luxe with unneded stuff removed:
working config for Muse Luxe with new speaker component

link updated with edited code so LED turns off AFTER speaking. thanks to Tetele for the “NOT” syntax.

this can be copy pasted directly and will work, of course with the personal stuff above captive portal edited to your own. (API key, name ect.)

1 Like
#62

I don’t see the ES8388 component which I believe is needed in order to set up the audio codec for the Muse Luxe. Also thanks tetele and Deco, I hadn’t realized that all of these changes had been merged into the release version yet but it looks like they have been

#63

no it’s not needed, give it a go :slight_smile:

#64

Your understanding of “unneeded” seems to be very particular to what you think you need :slight_smile:

Why do you have both a speaker and a media_player? Why do you stop the media player on_end if you’ve never started it? Why have you included your native API key?

Speaking of unneeded, why do you even bother with the LED if all you want is to send audio back and forth? What exactly is needed and by whom?

#65

Oh, and regarding this…

Why don’t you give it a go? That PR and external_component initializes the DAC. Yes, taking out the component from the config and reinstalling the ESPHome image does work until the first power cycle because the DAC is kept under power and doesn’t need to be initialized again. However, the first time you turn the speaker off and on again, say goodbye to your home assistant :slight_smile:

Like I said before, please pay a bit more attention to what you do and don’t understand from the configs exposed here by you or others. You will only create frustration in people trying out your config if they find out it doesn’t work.

#66

yes i was going to say the led stuff is unneeded but left if there for those that find it cool, waiting for mediaplayer isn’t needed and should be “speaker.is_playing” but since there is no idle condition on speaker component i left it as it was until that part can be corrected. (until more conditions are added to speaker). - this only means the led will turn off right when it replies and not AFTER it’s done speaking.

#67

i’m not sure what your problem is with my posts, you seem to have a grudge against everything i post, and sorry i don not experience what you say, works just fine without, also after switching the power off/on.

#68 
condition:
  not:
    speaker.is_playing:

The media_player is not needed at all if you have the speaker component tied to the voice_assistant.

You said that some most of the config I proposed (check out the older versions) was unneeded, the major differences to what you’ve posted being:

  1. the fact that I’ve included a visual indicator of the speaker connecting to WiFi (LED blinking slowly in amber until connected, then a short burst of green flashes when it’s connected). I think that that’s also a “cool” feature. Not needed, just useful
  2. an external service for starting the voice_assistant from another device (from HA). Very useful, if you ask me
  3. the fact that I’ve disabled the button entity in HA (disabled_by_default: true), since there’s a “duplicate” indicator (“Assist in progress”, coming straight from the voice_assistant component). You said that disables the button, which it absolutely does not

There are a lot of people searching for info on getting their Luxes working as HA Assist satellites. This thread is one of the top results for such searches. Having wasted a lot of time in the past after being confronted with keyboard warriors who don’t know what they’re talking about (I’m not saying you are one, I’m just saying that’s what I’ve dealt with), I believe wrong posts should be corrected (by their original poster or by others) in high-traffic information sources in order to be helpful.

I don’t have a problem with your posts. I have a problem with the wrong information within them.

#69

i see that is is infact a grudge because i said you had unneeded code in your example, thats ok keep having a bad day, all code i post is tested and runs without errors, which is more than i can say for most of the other examples.

i did not know i could do a NOT condition, thanks that solves that minor problem.

#70

@Deco and @tetele please take this conversation off this post and into DMs. we don’t need this back and forth cluttering up the conversation for people looking for how to set up either their Muse Luxe or Muse Proto as a Home Assistant Voice Assistant

3 Likes
#71

I have yet to try it but for those that would rather not deal with configuring yaml and building code it looks like a basic version of voice assistant support has been added to the Muse Luxe here

I haven’t had time to test it but looking at the config for the Muse Luxe the tts responses from piper will most likely not work but everything else should

#72

Why do you say piper is unlikely to work?

The reason I’m asking is that – well – it isn’t working for me on my Muse Luxe. I’m getting a media_url sent over (sometimes raw and sometimes mp3) but nothing plays other than a “pop.”