Could we get an example of how to connect the Luxe’s microphone in ESPHome? The latest ESPHome and beta version of Home Assistant support creating voice assistants!
No wake words at this point but we could repurpose the play button as a Push to Talk activator. I’d love to experiment with the luxe on my sofa end table! Imagined the Luxe as a totally local smart home Voice Assistant!
UPDATE from the future 24/11/23
For new readers of that post we’ve made a tutorial video thanks to everything that was shared these last 6 months:
–
back to the original thread:
Thanks for asking,
Check this code that we are using in the function factory_test( of this code for the Luxe Speaker. It records with the microphone samples played on the left speakers left then the right speaker, then check if the frequency recorded is the same that the one expected. This way we can check that we have no distortion issue on the microphone nor speakers.
The difficulty is that you need to send the I2C commands for the codec on the LUXE, not sure how to do this in a yalm file
Another simpler approach could be to use the Muse Proto board that uses a simple I2S microphone (with also an incredibly good quality) thus no I2C command to send, so maybe it will be an easier start.
Let me know if you have any other questions.
Thanks to the code by Raspiaudio, I was able to figure out the yaml for esphome.
You want to add the following:
microphone:
- platform: i2s_audio
id: lux_microphone
i2s_din_pin: GPIO35
voice_assistant:
microphone: lux_microphone
and if you want to covert the play button into a push to talk button you want to replace
media_player.toggle with voice_assistant.start and stop
on_press:
- voice_assistant.start:
on_release:
- voice_assistant.stop:
# on_click:
# - media_player.toggle:
This should be all you need on the esphome side of things, however the actual voice part of Home Assistant doesn’t launch until 2023.05 so you’ll have to wait a few days to actually play with it, or use the beta versions.
Thanks for your reply in this… (just came across your post)…I’m also BETA testing the 2023.05 version of HA
Thanks!
Have a solve for the low audio volume?
Hello,
i’m trying to use my muse luxe as a local vocal assistant with HA beta
on my 2x muse luxe the provided code don’t work as expected 
when i press the play button, i can see in logs that the event is fired but i also get a terrible noisy loud sound that came from the speakers until i release the button, do you have a hint about that ?
Thanks in advance
[22:15:16][D][binary_sensor:036]: ‘muse-cuisine Play Button’: Sending state ON
[22:15:16][D][voice_assistant:065]: Requesting start…
[22:15:16][D][voice_assistant:045]: Starting…
EDIT = it is this exact sound :
but i get it only when i press the play button and continue to play until i release.
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
substitutions:
name: muse-cuisine
esphome:
name: ${name}
name_add_mac_suffix: false
esp32:
board: esp-wrover-kit
framework:
type: arduino
api:
encryption:
key: fffffffffffffffff
logger:
ota:
i2c:
sda: GPIO18
scl: GPIO23
dashboard_import:
package_import_url: github://esphome/media-players/raspiaudio-muse-luxe.yaml@main
web_server:
port: 80
auth:
username: ggg
password: ggg
captive_portal:
improv_serial:
external_components:
- source: github://pr#3552
components: [es8388]
refresh: 0s
i2s_audio:
- i2s_lrclk_pin: GPIO25
i2s_bclk_pin: GPIO5
media_player:
- platform: i2s_audio
name: ${name}
dac_type: external
i2s_dout_pin: GPIO26
mode: stereo
mute_pin:
number: GPIO21
inverted: true
es8388:
sensor:
- platform: adc
pin: GPIO33
name: ${name} Battery
icon: "mdi:battery-outline"
device_class: voltage
state_class: measurement
unit_of_measurement: V
update_interval: 15s
accuracy_decimals: 3
attenuation: 11db
raw: true
filters:
- multiply: 0.00173913 # 2300 -> 4, for attenuation 11db, based on Olivier's code
- exponential_moving_average:
alpha: 0.2
send_every: 2
- delta: 0.002
binary_sensor:
- platform: gpio
pin:
number: GPIO19
inverted: true
mode:
input: true
pullup: true
name: ${name} Volume Up
on_click:
- media_player.volume_up:
- platform: gpio
pin:
number: GPIO32
inverted: true
mode:
input: true
pullup: true
name: ${name} Volume Down
on_click:
- media_player.volume_down:
- platform: gpio
pin:
number: GPIO12
inverted: true
mode:
input: true
pullup: true
name: ${name} Play Button
on_press:
- voice_assistant.start:
on_release:
- voice_assistant.stop:
light:
- platform: fastled_clockless
name: ${name}
pin: GPIO22
chipset: SK6812
num_leds: 1
rgb_order: grb
microphone:
platform: i2s_audio
id: lux_microphone
i2s_din_pin: GPIO35
voice_assistant:
microphone: lux_microphonehave you tried muting the speaker when voice assistant is in use?
Like this:
....
on_press:
- voice_assistant.start:
on_release:
- voice_assistant.stop:
on_click:
- media_player.toggle: media_out
and media_out should be the id of your media_player
Thanks, sadly this does not fix the issue.
Also, this issue doesn’t appear immediatly after boot, it happen after about 10 tries.
To record anything on the MUSE LUXE’s microphone the codec (ES8388) needs to receive first an I2C command to enable the microphone. Not sure is the awful noise is related to this but we are going to try the PROTO so use a simple I2S microphone and post result here very soon.
Thanks for the informations,
I really hope the muse luxe will be “HA compliant” as a voice assistant
Should be, stay tuned here
I’m playing with the PROTO now and feel like I’m getting close(ish) can you confirm which pin is the i2s_din_pin: for the PROTO? Looks like it’s GPIO35 on the Luxe (according to DelusoinalAI’s comment above) but would like to know if it’s the same on the PROTO. Thanks.
YES the microphone is also on GPIO35 on the Proto
thanks
thank you!, it sort of works for me too besides whisper getting it wrong most of the time (“whats the temperature” becomes “let us me temperature”) i have managed to tell it to turn off livingroom lights and it actually did, but no audio response, no idea why not because the response url is there, so it works… partly
btw: tried to change whisper model to “base” but it still gets 90% of my sentenses wrong 
Yeah, Only cloud sorta works for me. I also have the atom echo and the luxe works a bit better than that… I think this level is expected at this point…
I guess these microphones are too crappy for Whisper to understand them correctly? Does anyone have a recorded snippet from the Muse Luxe so I get an idea what the quality is like? Considering buying one, but that only makes sense if the mic is reasonably good.
Nice! maybe for the quality it could be improved by changing the gain, there are many parameters to try on this codec.
We could help to find the right I2C commands to send but does anybody knows how I could send custom I2C commands? Could it be done simply in the YALM file?
There is also a jack ADC stereo input (line level) on the back of the LUXE that could be used to plug an external amplified microphone, but let’s try first to see what we could with the built-in microphone.
i got it to work with a esp32 + i2s microphone (inmp441) using the code here: https://github.com/esphome/esphome/pull/4775#issuecomment-1534999535
gotta say it picks up a lot better what i actually say but not perfect, also there’s still no sound output.
i do have a muse luxe and 3 muse protos, and got voice assistant working on both muse and esp32, though the results are a little different… i asked “whats the temperature in the kitchen”, the image to the left is Muse Luxe, and image to the right is ESP32, i do however have gain on the I2S microphone on esp32 so possible results with muse would be better with some gain. but overall this is still in baby stage, HA/VA often doesn’t know what i want, like in the example… google would have figured that out… kitchen… temperature… how hard can it be, but for HA/VA, very hard :-p
