To record anything on the MUSE LUXE’s microphone the codec (ES8388) needs to receive first an I2C command to enable the microphone. Not sure is the awful noise is related to this but we are going to try the PROTO so use a simple I2S microphone and post result here very soon.
2 Likes
Thanks for the informations,
I really hope the muse luxe will be “HA compliant” as a voice assistant
Should be, stay tuned here
1 Like
I’m playing with the PROTO now and feel like I’m getting close(ish) can you confirm which pin is the i2s_din_pin: for the PROTO? Looks like it’s GPIO35 on the Luxe (according to DelusoinalAI’s comment above) but would like to know if it’s the same on the PROTO. Thanks.
1 Like
YES the microphone is also on GPIO35 on the Proto
thanks
2 Likes
thank you!, it sort of works for me too besides whisper getting it wrong most of the time (“whats the temperature” becomes “let us me temperature”) i have managed to tell it to turn off livingroom lights and it actually did, but no audio response, no idea why not because the response url is there, so it works… partly 
btw: tried to change whisper model to “base” but it still gets 90% of my sentenses wrong 
Yeah, Only cloud sorta works for me. I also have the atom echo and the luxe works a bit better than that… I think this level is expected at this point…
I guess these microphones are too crappy for Whisper to understand them correctly? Does anyone have a recorded snippet from the Muse Luxe so I get an idea what the quality is like? Considering buying one, but that only makes sense if the mic is reasonably good.
Nice! maybe for the quality it could be improved by changing the gain, there are many parameters to try on this codec.
We could help to find the right I2C commands to send but does anybody knows how I could send custom I2C commands? Could it be done simply in the YALM file?
There is also a jack ADC stereo input (line level) on the back of the LUXE that could be used to plug an external amplified microphone, but let’s try first to see what we could with the built-in microphone.
1 Like
i got it to work with a esp32 + i2s microphone (inmp441) using the code here: https://github.com/esphome/esphome/pull/4775#issuecomment-1534999535
gotta say it picks up a lot better what i actually say but not perfect, also there’s still no sound output.
i do have a muse luxe and 3 muse protos, and got voice assistant working on both muse and esp32, though the results are a little different… i asked “whats the temperature in the kitchen”, the image to the left is Muse Luxe, and image to the right is ESP32, i do however have gain on the I2S microphone on esp32 so possible results with muse would be better with some gain. but overall this is still in baby stage, HA/VA often doesn’t know what i want, like in the example… google would have figured that out… kitchen… temperature… how hard can it be, but for HA/VA, very hard :-p
On the right what you refer as ‘ESP32’ is Muse Proto (yellow board), correct?
Could you it be an issue with the Whisper model that is used and the processing power of the Raspberry pi 4?
I have attached an example of the sound recorded on a Muse Proto I think that the quality is quite good here is a recording with an arduino app we wrote so it sets your expectations on the quality, talking at 30cm from the mic.
The Muse Proto sounds really good! So it seems the Luxe is significantly worse? I guess then I will rather use the Proto for voice assistant purposes.
i will also make a recording on the Luxe, but I think the quality is comparable to the Proto from what I remember. This is why I think it is more related to some settings tweaking. We have somebody of the team working on that subject this week, I will post result here.
3 Likes
no i mean a ESP32 D1 mini + INMP441 + MAX98357 (and 11 wires)
and yes it could very likely be a issue with whisper, there is lots of them (like not giving audio reply), i d believe it could be a matter of raising the gain, like i did with the ESP/INMP441. only did the 11db gain and have not tried on the muses.
it’s the microphone we discuss yes the sound is great on both luxe and proto, but for voice assistant to work it needs a good mic (gain), and even then whisper is horrible at “decoding” that to words, but it’s a work in progress, the year has many months left :))
Deco, would you please share your ESPhome config. It seems you figured out the audio part too.
1 Like
Thanks I have installed this yalm on the luxe without issue I can play TTS, but not sure how to record when pressing the play button it seems the be in “Assist in progress” but where should I see the speech to text result?
thanks
1 Like
i figured it out for the luxe, and for esp32, not proto, i just tried the same yalm from luxe on the proto and not sure what is different from luxe to proto, but lux finds a I2C device, proto does not:
Luxe:
[19:33:22][I][i2c.arduino:069]: Results from i2c bus scan:
[19:33:22][I][i2c.arduino:075]: Found i2c device at address 0x10
Proto:
[19:32:03][I][i2c.arduino:069]: Results from i2c bus scan:
[19:32:03][I][i2c.arduino:071]: Found no i2c devices!
what are you using? proto or luxe?, and sure i can send you the yaml i use in my Luxe if thats what you need :), or for ESP32+external I2S mic. 
1 Like