I’ve been trying to get Home Assistant Voice Assistant wakeword to function for a week now. After fumbling around with ESP8266 and ESP32 trying to get an external mic/speaker to work I stumbled on this product with both onboard and thought my prayers were answered. I’m an experienced home networker/tech nerd, but relative novice with YAML/GPIO/etc. I have seen a lot of code on getting the MUSE LUXE to work, but nothing for the Muse Proto. I assumed the Luxe was just a Proto with some additional supporting peripherals, and thought the code should translate easily, but I cannot for the life of me get it to work. I have borrowed a few people’s Luxe code and tried to adapt it, but have so far failed. I’m hoping the kind strangers on the Internet can help a guy out.
Background:
- Muse Proto v1.7 successfully connects to network with static IP from DHCP mapping
- Home Assistant OS 11.1 running in VM
- ESPHome Add-On 2023.11.6
- I have verified the TTS and STT pipelines are set up correctly as they work in the mobile app and webpage with a mic connected to the computer
- I assume the correct board is the esp-wrover-kit, correct?
I successfully install the code onto the Proto, but which I click the mic button on the Home Assistant…uh…Assistant it immediately shows the … bubble as if the listening session never starts. One question I have is that the code I’ve tried to copy shows the arduino framework though one of the YT video tutorials says the esp-idf framework is required to use the wakeword features…is that not accurate?
Am I correct to assume that simply referencing GPIO35 invokes the on-board microphone?
I also don’t understand the difference between using speaker and media_player.
Below is the adopted code I’ve tried to use, mostly adapted from @tetele :
On boot I get the following LED feedback: a fading yellow light (boot), then a pulsing green (wifi connected), then it sits forever on a solid bright purple, which I think is the reset_led script? Then it sometimes randomly reboots and comes back with a flashing purple/red on the status LED. If I power cycle it goes back to the solid purple (at least for a while).
So my big question is: How can I modify this to work with JUST the Muse Proto onboard mic and speaker? Thanks in advance for the help!
############################################
# Board model ESP-WROVER-kit - raspiaudio - raspiaudio.com
# Board pinout diagram - https://forum.raspiaudio.com/t/muse-proto-board/231
# Code excerpted from https://gist.github.com/tetele/5cac735174527c3b373b10db8d9c8d77
# Pinout map (from backside)
# PIN LABEL FCN | LABEL FCN
# 1 GND Ground | GND Ground
# 2 23 GAIN_AMPLI | +3v3 VDD3V3
# 3 22 NEOPIXEL | +3v3 VDD3V3
# 4 TXD U0TXD | RST RST/EN
# 5 RXD U0RXD | VP SENSOR VP
# 6 21 ENABLE_AMPLI | VN SENSOR VN
# 7 19 GPIO19 | 34 SD_DETECT
# 8 18 GPIO18 | 35 I2S_CODEC_ASDOUT
# 9 5 I2S_CODEC_SCLK | 32 GPIO32
# 10 4 GPIO4 | 33 BAT_MONITOR
# 11 0 IO0 | 25 I2S_CODEC_LRCK
# 12 2 SD_D0 | 26 I2S_CODEC_DIN
# 13 15 SD_CMD | 27 AUX_DETECT_IO
# 14 SD1 SDI/SD1 | 14 SD_CLK
# 15 SD0 SDO/SD0 | 12 GPIO12
# 16 CLK SCK/CLK | 13 SD_D3
# 17 VBAT VBA^ | SD2 SHD/SD2
# 18 5V +5V | SD3 SWP/SD3
# 19 GND Ground | CMD SCS/CMD
# 20 GND Ground | GND Ground
#---------------
esphome:
name: muse-lux-clone-code
friendly_name: Muse-lux-clone-code
name_add_mac_suffix: false
min_version: 2023.10.1
on_boot:
then:
- output.turn_on: dac_mute
- light.turn_on:
id: top_led
effect: slow_pulse
red: 100%
green: 60%
blue: 0%
esp32:
board: esp-wrover-kit
framework:
type: arduino
logger:
api:
services:
- service: start_va
then:
- voice_assistant.start
- service: stop_va
then:
- voice_assistant.stop
encryption:
key: "7LHFlAioCBO90RvMnr3lkXzKHdpwUmYhkfXqb05+hyo="
ota:
password: "c2cc7d20564dd25d1faec06b3a51e00c"
i2c:
sda: GPIO18
scl: GPIO23
wifi:
ssid: !secret not_wifi_ssid
password: !secret not_wifi_password
captive_portal:
improv_serial:
external_components:
- source: github://pr#3552 # DAC support https://github.com/esphome/esphome/pull/3552
components: [es8388]
refresh: 0s
es8388:
globals:
- id: wifi_connected
type: bool
initial_value: "false"
restore_value: false
interval:
- interval: 1s
then:
- if:
condition:
and:
- lambda: "return !id(wifi_connected);"
- wifi.connected:
then:
- globals.set:
id: wifi_connected
value: "true"
- light.turn_on:
id: top_led
effect: pulse
red: 0%
green: 100%
blue: 0%
- delay: 1s
- light.turn_off: top_led
output:
- platform: gpio
id: dac_mute
pin: GPIO21
inverted: true
i2s_audio:
- i2s_lrclk_pin: GPIO25
i2s_bclk_pin: GPIO5
media_player:
- platform: i2s_audio
name: None
id: luxe_out
dac_type: external
i2s_dout_pin: GPIO26
mode: stereo
on_state:
if:
condition:
media_player.is_playing:
then:
output.turn_off: dac_mute
else:
output.turn_on: dac_mute
microphone:
- platform: i2s_audio
id: luxe_microphone
i2s_din_pin: GPIO35
adc_type: external
pdm: false
voice_assistant:
id: va
microphone: luxe_microphone
media_player: luxe_out
use_wake_word: true
on_listening:
- light.turn_on:
id: top_led
blue: 100%
red: 0%
green: 0%
brightness: 100%
effect: pulse
on_tts_start:
- light.turn_on:
id: top_led
blue: 60%
red: 20%
green: 20%
effect: none
on_tts_end:
- media_player.play_media: !lambda return x;
- light.turn_on:
id: top_led
blue: 60%
red: 20%
green: 20%
effect: pulse
# This is useful when you want to stream the response on another media_player
# - homeassistant.service:
# service: media_player.play_media
# data:
# entity_id: media_player.some_speaker
# media_content_id: !lambda 'return x;'
# media_content_type: music
# announce: "true"
on_client_connected:
- if:
condition:
- switch.is_on: use_wake_word
then:
- voice_assistant.start_continuous:
on_client_disconnected:
- if:
condition:
- switch.is_on: use_wake_word
then:
- voice_assistant.stop:
on_end:
- delay: 100ms
- wait_until:
not:
media_player.is_playing: luxe_out
- script.execute: reset_led
on_error:
- light.turn_on:
id: top_led
blue: 0%
red: 100%
green: 0%
effect: none
- delay: 1s
- script.execute: reset_led
- script.wait: reset_led
- lambda: |-
if (code == "wake-provider-missing" || code == "wake-engine-missing") {
id(use_wake_word).turn_off();
}
sensor:
- platform: adc
pin: GPIO33
name: Battery voltage
device_class: voltage
unit_of_measurement: "V"
accuracy_decimals: 2
state_class: measurement
entity_category: diagnostic
update_interval: 15s
attenuation: auto
filters:
- multiply: 2 # https://forum.raspiaudio.com/t/esp-muse-luxe-bluetooth-speaker/294/12
- exponential_moving_average:
alpha: 0.2
send_every: 2
- delta: 0.002
on_value:
then:
- sensor.template.publish:
id: battery_percent
state: !lambda "return x;"
- platform: template
name: Battery
id: battery_percent
device_class: battery
unit_of_measurement: "%"
accuracy_decimals: 0
state_class: measurement
entity_category: diagnostic
update_interval: 15s
filters:
- calibrate_polynomial:
degree: 3
datapoints:
- 4.58 -> 100.0
- 4.5 -> 97.1
- 4.47 -> 94.2
- 4.44 -> 88.4
- 4.42 -> 82.7
- 4.41 -> 76.9
- 4.41 -> 71.1
- 4.37 -> 65.3
- 4.35 -> 59.5
- 4.31 -> 53.8
- 4.28 -> 48.0
- 4.26 -> 42.2
- 4.23 -> 36.4
- 4.21 -> 30.6
- 4.19 -> 24.9
- 4.16 -> 19.1
- 4.1 -> 13.3
- 4.07 -> 10.4
- 4.03 -> 7.5
- 3.97 -> 4.6
- 3.82 -> 1.7
- 3.27 -> 0.0
- lambda: return clamp(x, 0.0f, 100.0f);
binary_sensor:
- platform: gpio
pin:
number: GPIO19
inverted: true
mode:
input: true
pullup: true
name: Volume Up
on_click:
- media_player.volume_up: luxe_out
- platform: gpio
pin:
number: GPIO32
inverted: true
mode:
input: true
pullup: true
name: Volume Down
on_click:
- media_player.volume_down: luxe_out
- platform: gpio
pin:
number: GPIO12
inverted: true
mode:
input: true
pullup: true
name: Action
on_click:
- if:
condition:
switch.is_off: use_wake_word
then:
- if:
condition: voice_assistant.is_running
then:
- voice_assistant.stop:
- script.execute: reset_led
else:
- voice_assistant.start:
else:
- voice_assistant.stop
- delay: 1s
- script.execute: reset_led
- script.wait: reset_led
- voice_assistant.start_continuous:
light:
- platform: esp32_rmt_led_strip
name: None
id: top_led
pin: GPIO22
chipset: SK6812
num_leds: 1
rgb_order: grb
rmt_channel: 0
default_transition_length: 0s
gamma_correct: 2.8
effects:
- pulse:
name: pulse
transition_length: 250ms
update_interval: 250ms
- pulse:
name: slow_pulse
transition_length: 1s
update_interval: 2s
script:
- id: reset_led
then:
- if:
condition:
switch.is_on: use_wake_word
then:
- light.turn_on:
id: top_led
blue: 100%
red: 100%
green: 0%
brightness: 100%
effect: none
else:
- light.turn_off: top_led
switch:
- platform: template
name: Use Wake Word
id: use_wake_word
optimistic: true
restore_mode: RESTORE_DEFAULT_ON
on_turn_on:
- lambda: id(va).set_use_wake_word(true);
- if:
condition:
not:
- voice_assistant.is_running
then:
- voice_assistant.start_continuous
- script.execute: reset_led
on_turn_off:
- voice_assistant.stop
- lambda: id(va).set_use_wake_word(false);
- script.execute: reset_led