Muse and HA as a Voicee Assistant deevice

[Gerhard56]

Hi,
and here another try with this .yaml

Muse and HA as a Voicee Assistant deevice

Hej, solved. I have to adopt the device and configure it, this time the installation was quite different and after fixing all this things, its talking again. But, I want to use Muse as a IO device for a Voice assistant, so listening is also a part of the game. I setted up a test button ‘testlamp’ and I am able to use the Assist (new icon in the right upper corner) to control it, by typing text. Works fine. So the last thing missing I guess is the speach-to-text part and the wake word detect…

Result

Sorry

[Gerhard56]

Any News?
I still can’t update any ESPHome in my sytem and so I can’t install the .yaml on my ESP Muse Luxe.
I think, the reason for this error isn’t in my universe …

Thanks for helping.

With best regards

Gerhard

[Raspiaudio]

Hi no worry we’ve all been there
Soon you will be a pro in yaml (i am not)

Could you give me your ESPhome version and home assistant version?

I will make a tutorial video on youtube next week as you are not the first one to ask this

[Gerhard56]

Hi,
thanks.

ESPHome version is 2024.11.1

Hardware is a HA Yellow with an large M2 SSD.

I have to move for the next days, but no problem another HA instance and another Muse luxe were waiting for me …

I run two ESPHome Wroom just as BT-Gateways, and I get the same error there, it looks like, this behavior starts after the last update for ESPHome Compiler.

With best regards

Gerhard

[Raspiaudio]

Hi I have made a video just for you and hoping it will help others!

How to compile and install a yaml on an ESPHOME device for Home assistant

[Gerhard56]

Ahhhhhhhhhhhhhhhhh, thats the trick.
Manuell compile, make a .bin and install the .bin via Com. I do it always ‘wirelessy’ … after copying the .yaml.

Ok, will try it within the next hour.

Thanks a lot.

With best regards

Gerhard

[Gerhard56]

Hi,
followed strictly your video instructions.
After copying the .yaml and choose install with option manual I got again_

Sorry, something wrong.

With best regards

Gerhard

[nbridge1977]

I discovered an issue after loading your updated yaml. Now when I try to play text-to-speech through the speaker it doesn’t play anything until I give it a voice prompt. Then it will play the text-to-speech I sent it previously. Thanks again for your assistance.

[Gerhard56]

Hi,
after solving the problem with the broken json file (see above) by removing ESP Home from my HA instance and reinstalling the actual version, I was able to start the compilation, but:

Sorry.

With best regards

Gerhard

[Gerhard56]

Ok, solved this issue.
Do everything like you show in the video.
Result:
Esp Muse generates some noise in random time intervals.
In the ESP Compiler Page I found the device under ‘detected’, but the dokumentation for ‘Take control’ told me, that I will loose everything, so didn’t use it.
I found some entities like ‘media player…’ as before, using this to try to use tts fails.
Any ideas?

With best regards

Gerhard

[Gerhard56]

Some news:
First of all, I switched ‘Use Wakeword’ off. And I changed the configuration of the device:

I repaired the actions because the entity names changes since the last attempt to install ESP Muse Luxe and now, the box is speeaking again BUT:
You have to switch the ‘Use wakeword’ option off.

This also stopps the ugly random noise.

Interesting:
When I switched the ‘Use wakeword’ switch to ‘off’ also the led switch is switched off and the selected color of the led changes …

How can I test, which version of software is running on the Muse, how can I check the audio channels, including the microphones and what can I do, that the random noise stopps and the wakeword will probably start working?

With best regards

Gerhard

[Gerhard56]

Just a little update.

ESPMuse luxe works fine for my announcements. I use now ChimeTTS which allows me to add some sounds to my messages, like ’ Ding Dong its 8 o’clock now’, but only when I switched of the ‘Wakeword’ option.
Switching this option on stopps all announcements.
I installed the software (.bin) as mentioned above.

With best regards

Gerhard

[Raspiaudio]

sorry I lost track what yaml are you using now?

[Gerhard56]

The last one you send me.
Sorry, find no way to upload the file here for reference.
How can I find out which version is running on my box now?

I am quite sure it’s this one:

RASPIAUDIO/esphomeLuxe/blob/main/raspiaudio-muse-luxe.yaml

With best regards

Gerhard

[Raspiaudio]

Ok so I’ve tried to recompile it today and I have an error with last HA/ESPHOME updates

Anyway one of our engineer is assigned on this until we found a solution, stay tuned

[Gerhard56]

Thanks for your work …

Maybe we can put sone info into the .yaml to see, which version is actually running, guess there will be somemore iterations.

[Raspiaudio]

Hello everyone,

We have updated the YAML file. You can either compile it yourself or use the pre-compiled version available here. Don’t forget to set up your Wi-Fi credentials in the web interface, as demonstrated in the previous video.

Features and Known Issues:

• Wake Word Functionality: You can activate the device using the wake word.
• Media Stream Playback: If you want to play a media stream in the web interface, you’ll need to start the stream first and then say the wake word to activate it (I know this is not ideal)
• Wake Word During Streaming: Unfortunately, the wake word cannot be used while a stream is playing.

We are continuously working to improve this!

Thank you for your support.

[wowann]

Hello everyone,

I’m working on a project using ESPHome on the RaspiAudio Muse Luxe module. The system compiles and initializes correctly. However, the microphone does not seem to capture any data or react to wake words.

Here are the relevant details:

• Hardware: ESP32, RaspiAudio Muse Luxe
• ESPHome Configuration: voice_assistant with use_wake_word: true
• Installed via https://raspiaudio.github.io/
• What works:
  • The system initializes successfully.
  • I2S audio output (media player) works fine.
  • The LED and wake word functionality seem to load without errors.
• What doesn’t work:
  • The microphone does not seem to pick up any audio.
  • Logs do not show any activity for the microphone or wake word detection.
  • I tried debugging the microphone using a template sensor with get_signal_level(), but it resulted in a compilation error:

error: 'class esphome::i2s_audio::I2SAudioMicrophone' has no member named 'get_signal_level'

What I’ve tried:

• Verified hardware connections and pins.
• Enabled logger: level: DEBUG but couldn’t see any microphone activity.
• Manually triggered the voice assistant to check its pipeline functionality.

Question: How can I verify if the microphone is functioning correctly? Is there a way to debug I2S microphone input in ESPHome, or is additional configuration required to make it work? Any suggestions or alternative testing methods are highly appreciated!

Thank you in advance!

Attached the Log from Booting
[19:41:57][I][logger:171]: Log initialized [19:41:57][C][safe_mode:079]: There have been 0 suspected unsuccessful boot attempts [19:41:57][D][esp32.preferences:114]: Saving 1 preferences to flash… [19:41:57][D][esp32.preferences:143]: Saving 1 preferences to flash: 0 cached, 1 written, 0 failed [19:41:57][I][app:029]: Running through setup()… [19:41:57][I][i2c.arduino:218]: Performing I2C bus recovery [19:41:57][D][binary_sensor:034]: ‘Volume Up’: Sending initial state OFF [19:41:57][D][binary_sensor:034]: ‘Volume Down’: Sending initial state OFF [19:41:57][C][esp32_rmt_led_strip:021]: Setting up ESP32 LED Strip… [19:41:57][D][binary_sensor:034]: ‘Action’: Sending initial state OFF [19:41:57][C][light:021]: Setting up light ‘RaspiAudio Muse Luxe’… [19:41:57][D][light:036]: ‘RaspiAudio Muse Luxe’ Setting: [19:41:57][D][light:041]: Color mode: RGB [19:41:57][D][template.switch:046]: Restored state ON [19:41:57][D][switch:012]: ‘Use Wake Word’ Turning ON. [19:41:57][D][switch:055]: ‘Use Wake Word’: Sending state ON [19:41:57][E][voice_assistant:575]: No API client connected [19:41:57][D][voice_assistant:512]: State changed from IDLE to IDLE [19:41:57][D][voice_assistant:518]: Desired state set to IDLE [19:41:57][D][light:036]: ‘RaspiAudio Muse Luxe’ Setting: [19:41:57][D][light:047]: State: ON [19:41:57][D][light:051]: Brightness: 100% [19:41:57][D][light:059]: Red: 100%, Green: 0%, Blue: 100% [19:41:57][D][light:036]: ‘RaspiAudio Muse Luxe’ Setting: [19:41:57][D][light:059]: Red: 100%, Green: 60%, Blue: 0% [19:41:57][D][light:109]: Effect: ‘slow_pulse’ [19:41:57][C][i2s_audio:028]: Setting up I2S Audio… [19:41:57][C][i2s_audio.microphone:018]: Setting up I2S Audio Microphone… [19:41:57][C][adc:047]: Setting up ADC ‘Battery voltage’… [19:41:57][C][adc:093]: ADC ‘Battery voltage’ setup finished! [19:41:57][C][wifi:048]: Setting up WiFi… [19:41:57][C][wifi:061]: Starting WiFi… [19:41:57][C][wifi:062]: Local MAC: D8:BC:38:E6:E8:F8 [19:41:57][D][wifi:074]: Loaded saved wifi settings: Olivia_SZ [19:41:57][D][wifi:482]: Starting scan… [19:41:57][D][esp32.preferences:114]: Saving 1 preferences to flash… [19:41:57][D][esp32.preferences:143]: Saving 1 preferences to flash: 1 cached, 0 written, 0 failed [19:41:57][D][sensor:094]: ‘Battery voltage’: Sending state 0.90305 V with 2 decimals of accuracy [19:41:57][D][sensor:094]: ‘Battery’: Sending state 100.00000 % with 0 decimals of accuracy [19:41:57][W][component:157]: Component wifi set Warning flag: scanning for networks [19:42:03][D][wifi:497]: Found networks: [19:42:03][I][wifi:541]: - ‘Olivia_SZ’ (DC:39:6F:34:77:2B) [redacted]▂▄▆█ [19:42:03][D][wifi:542]: Channel: 6 [19:42:03][D][wifi:543]: RSSI: -34 dB [19:42:03][D][wifi:546]: - ‘Olivia’[redacted] (04:B4:FE:7F:F0:22) [redacted]▂▄▆█ [19:42:03][D][wifi:546]: - ‘Olivia’[redacted] (0C:72:74:42:23:EA) [redacted]▂▄▆█ [19:42:03][D][wifi:546]: - ‘Olivia’[redacted] (D4:24:DD:89:4B:BD) [redacted]▂▄▆█ [19:42:03][D][wifi:546]: - ‘Jansen_1’[redacted] (BC:F2:AF:76:36:51) [redacted]▂▄▆█ [19:42:03][D][wifi:546]: - ‘devolo-f4068d86cf51’[redacted] (F4:06:8D:86:CF:51) [redacted]▂▄▆█ [19:42:03][D][wifi:546]: - ‘Jansen’[redacted] (48:5D:35:07:29:49) [redacted]▂▄▆█ [19:42:03][D][wifi:546]: - ‘devolo-f4068d86cf09’[redacted] (F4:06:8D:86:CF:09) [redacted]▂▄▆█ [19:42:03][I][wifi:313]: WiFi Connecting to ‘Olivia_SZ’… [19:42:04][W][wifi_esp32:545][arduino_events]: Event: Disconnected ssid=‘Olivia_SZ’ bssid=DC:39:6F:34:77:2B[redacted] reason=‘Authentication Failed’ [19:42:04][W][wifi:674]: WiFi Unknown connection status 0 [19:42:04][I][wifi:313]: WiFi Connecting to ‘Olivia_SZ’… [19:42:04][W][wifi_esp32:545][arduino_events]: Event: Disconnected ssid=‘Olivia_SZ’ bssid=DC:39:6F:34:77:2B[redacted] reason=‘Association Failed’ [19:42:04][D][wifi:697]: Retrying with hidden networks… [19:42:09][I][wifi:313]: WiFi Connecting to ‘Olivia_SZ’… [19:42:09][W][wifi:669]: Connecting to WiFi network failed. Are the credentials wrong? [19:42:09][I][wifi:313]: WiFi Connecting to ‘Olivia_SZ’… [19:42:09][W][wifi:669]: Connecting to WiFi network failed. Are the credentials wrong? [19:42:14][I][wifi:313]: WiFi Connecting to ‘Olivia_SZ’… [19:42:14][W][wifi_esp32:545][arduino_events]: Event: Disconnected ssid=‘Olivia_SZ’ bssid=DC:39:6F:34:77:2B[redacted] reason=‘Association Leave’ [19:42:14][W][wifi:674]: WiFi Unknown connection status 0 [19:42:14][D][wifi:697]: Retrying with hidden networks… [19:42:14][I][wifi:313]: WiFi Connecting to ‘Olivia_SZ’… [19:42:15][I][wifi:617]: WiFi Connected! [19:42:15][C][wifi:428]: Local MAC: D8:BC:38:E6:E8:F8 [19:42:15][C][wifi:433]: SSID: ‘Olivia_SZ’[redacted] [19:42:15][C][wifi:436]: IP Address: 192.168.178.50 [19:42:15][C][wifi:440]: BSSID: DC:39:6F:34:77:2B[redacted] [19:42:15][C][wifi:441]: Hostname: ‘muse-luxe’ [19:42:15][C][wifi:443]: Signal strength: -35 dB ▂▄▆█ [19:42:15][C][wifi:447]: Channel: 6 [19:42:15][C][wifi:448]: Subnet: 255.255.255.0 [19:42:15][C][wifi:449]: Gateway: 192.168.178.1 [19:42:15][C][wifi:450]: DNS1: 192.168.178.23 [19:42:15][C][wifi:451]: DNS2: 0.0.0.0 [19:42:15][D][wifi:626]: Disabling AP… [19:42:15][C][api:026]: Setting up Home Assistant API server… [19:42:15][C][audio:123]: Setting up Audio… [19:42:15][I][app:062]: setup() finished successfully! [19:42:15][D][light:036]: ‘RaspiAudio Muse Luxe’ Setting: [19:42:15][D][light:059]: Red: 0%, Green: 100%, Blue: 0% [19:42:15][D][light:109]: Effect: ‘pulse’ [19:42:15][W][component:170]: Component wifi cleared Warning flag [19:42:15][W][component:157]: Component api set Warning flag: unspecified [19:42:15][I][app:100]: ESPHome version 2024.11.3 compiled on Dec 14 2024, 19:03:32 [19:42:15][C][wifi:600]: WiFi: [19:42:15][C][wifi:428]: Local MAC: D8:BC:38:E6:E8:F8 [19:42:15][C][wifi:433]: SSID: ‘Olivia_SZ’[redacted] [19:42:15][C][wifi:436]: IP Address: 192.168.178.50 [19:42:15][C][wifi:440]: BSSID: DC:39:6F:34:77:2B[redacted] [19:42:15][C][wifi:441]: Hostname: ‘muse-luxe’ [19:42:15][C][wifi:443]: Signal strength: -35 dB ▂▄▆█ [19:42:15][C][wifi:447]: Channel: 6 [19:42:15][C][wifi:448]: Subnet: 255.255.255.0 [19:42:15][C][wifi:449]: Gateway: 192.168.178.1 [19:42:15][C][wifi:450]: DNS1: 192.168.178.23 [19:42:15][C][wifi:451]: DNS2: 0.0.0.0 [19:42:15][C][logger:185]: Logger: [19:42:15][C][logger:186]: Level: DEBUG [19:42:15][C][logger:188]: Log Baud Rate: 115200 [19:42:15][C][logger:189]: Hardware UART: UART0 [19:42:15][C][i2c.arduino:071]: I2C Bus: [19:42:15][C][i2c.arduino:072]: SDA Pin: GPIO18 [19:42:15][C][i2c.arduino:073]: SCL Pin: GPIO23 [19:42:15][C][i2c.arduino:074]: Frequency: 50000 Hz [19:42:15][C][i2c.arduino:086]: Recovery: bus successfully recovered [19:42:15][I][i2c.arduino:096]: Results from i2c bus scan: [19:42:15][I][i2c.arduino:102]: Found i2c device at address 0x10 [19:42:15][I][i2c.arduino:102]: Found i2c device at address 0x75 [19:42:15][C][template.sensor:022]: Template Sensor ‘Battery’ [19:42:15][C][template.sensor:022]: Device Class: ‘battery’ [19:42:15][C][template.sensor:022]: State Class: ‘measurement’ [19:42:15][C][template.sensor:022]: Unit of Measurement: ‘%’ [19:42:15][C][template.sensor:022]: Accuracy Decimals: 0 [19:42:15][C][template.sensor:023]: Update Interval: 15.0s [19:42:15][C][gpio.binary_sensor:015]: GPIO Binary Sensor ‘Volume Up’ [19:42:15][C][gpio.binary_sensor:016]: Pin: GPIO19 [19:42:15][C][gpio.binary_sensor:015]: GPIO Binary Sensor ‘Volume Down’ [19:42:15][C][gpio.binary_sensor:016]: Pin: GPIO32 [19:42:15][C][esp32_rmt_led_strip:187]: ESP32 RMT LED Strip: [19:42:15][C][esp32_rmt_led_strip:188]: Pin: 22 [19:42:15][C][esp32_rmt_led_strip:189]: Channel: 0 [19:42:15][C][esp32_rmt_led_strip:214]: RGB Order: GRB [19:42:15][C][esp32_rmt_led_strip:215]: Max refresh rate: 0 [19:42:15][C][esp32_rmt_led_strip:216]: Number of LEDs: 1 [19:42:15][C][gpio.binary_sensor:015]: GPIO Binary Sensor ‘Action’ [19:42:15][C][gpio.binary_sensor:016]: Pin: GPIO12 [19:42:15][C][light:092]: Light ‘RaspiAudio Muse Luxe’ [19:42:15][C][light:094]: Default Transition Length: 0.0s [19:42:15][C][light:095]: Gamma Correct: 2.80 [19:42:15][C][template.switch:068]: Template Switch ‘Use Wake Word’ [19:42:15][C][template.switch:091]: Restore Mode: restore defaults to ON [19:42:15][C][template.switch:057]: Optimistic: YES [19:42:15][C][adc:097]: ADC Sensor ‘Battery voltage’ [19:42:15][C][adc:097]: Device Class: ‘voltage’ [19:42:15][C][adc:097]: State Class: ‘measurement’ [19:42:15][C][adc:097]: Unit of Measurement: ‘V’ [19:42:15][C][adc:097]: Accuracy Decimals: 2 [19:42:15][C][adc:107]: Pin: GPIO33 [19:42:15][C][adc:109]: Attenuation: auto [19:42:15][C][adc:141]: Samples: 1 [19:42:15][C][adc:142]: Update Interval: 15.0s [19:42:15][C][captive_portal:089]: Captive Portal: [19:42:15][C][mdns:116]: mDNS: [19:42:15][C][mdns:117]: Hostname: muse-luxe [19:42:15][C][esphome.ota:073]: Over-The-Air updates: [19:42:15][C][esphome.ota:074]: Address: muse-luxe.local:3232 [19:42:15][C][esphome.ota:075]: Version: 2 [19:42:15][C][safe_mode:018]: Safe Mode: [19:42:15][C][safe_mode:020]: Boot considered successful after 60 seconds [19:42:15][C][safe_mode:021]: Invoke after 10 boot attempts [19:42:15][C][safe_mode:023]: Remain in safe mode for 300 seconds [19:42:15][C][api:140]: API Server: [19:42:15][C][api:141]: Address: muse-luxe.local:6053 [19:42:15][C][api:145]: Using noise encryption: NO [19:42:15][C][improv_serial:032]: Improv Serial: [19:42:15][C][audio:225]: Audio: [19:42:15][C][audio:247]: External DAC channels: 2 [19:42:15][C][audio:248]: I2S DOUT Pin: 26 [19:42:15][C][audio:249]: Mute Pin: GPIO21 [19:42:16][D][light:036]: ‘RaspiAudio Muse Luxe’ Setting: [19:42:16][D][light:047]: State: OFF [19:42:16][D][light:109]: Effect: ‘None’ [19:42:17][D][api:103]: Accepted 192.168.178.23 [19:42:17][W][component:170]: Component api cleared Warning flag [19:42:17][D][api.connection:1446]: Home Assistant 2024.12.3 (192.168.178.23): Connected successfully [19:42:18][D][voice_assistant:512]: State changed from IDLE to START_MICROPHONE [19:42:18][D][voice_assistant:518]: Desired state set to START_PIPELINE [19:42:18][D][voice_assistant:222]: Starting Microphone [19:42:18][D][ring_buffer:034]: Created ring buffer with size 16384 [19:42:18][D][voice_assistant:512]: State changed from START_MICROPHONE to STARTING_MICROPHONE [19:42:18][D][voice_assistant:512]: State changed from STARTING_MICROPHONE to START_PIPELINE [19:42:18][D][voice_assistant:276]: Requesting start… [19:42:18][D][voice_assistant:512]: State changed from START_PIPELINE to STARTING_PIPELINE [19:42:18][D][voice_assistant:533]: Client started, streaming microphone [19:42:18][D][voice_assistant:512]: State changed from STARTING_PIPELINE to STREAMING_MICROPHONE [19:42:18][D][voice_assistant:518]: Desired state set to STREAMING_MICROPHONE [19:42:18][D][voice_assistant:635]: Event Type: 1 [19:42:18][D][voice_assistant:638]: Assist Pipeline running [19:42:18][D][voice_assistant:635]: Event Type: 9

[Raspiaudio]

Hi,

Have you tried the latest precompiled binary available on apps.raspiaudio.com for the Luxe-HA? I used it again last week, and it worked successfully.

If you have, could you let me know your current Home Assistant (HA) and ESPHome versions?

To check the hardware microphone, you can load the Luxe-Bluetooth binary from apps.raspiaudio.com. Then, follow these steps to call the factory test:

1. Turn the device off.
2. Press and hold all three top round buttons simultaneously (this can be a bit tricky).
3. While holding the buttons, turn the device on.

The device will start to beep, sending a carrier signal recorded by the microphone. Based on the LED indicator:

• Red light: Something might be wrong with your microphone.
• Yellow light: The microphone is fine, but the SD card is missing.
• Green light: The microphone is working, and the SD card has been written successfully.

Alternatively, you can record using our library example, but this requires a bit more effort. In the Arduino IDE, search for the library “raspiaudio” and load the recording example.

[wowann]

Hi,

im Current using the newest HA Version 12.3

• Core2024.12.3
• Supervisor2024.11.4
• Operating System14.0

ESP Home Version: 2024.11.3

i installed now the Luxe Bluetooth Version,
when i press all 3 Buttons during Start first a Blue Light
it start to beep 5 times and the Light turn White.