Request / discussion : New event after end of TTS (keep led on during TTS)

[llluis]

Hello @synesthesiam.
I'm using the event server to control the leds and would like to keep them on while aplay is being executed, i.e., while the TTS return is being played.

Right now TTS audio chunk is sent to the queue and there's no event tracking when the audio was played. The last event is the AudioStop but it's received when the server is finished sending chunks, not when the actual play ended.

One possible solution is the desconnection of the sound client here:

wyoming-satellite/wyoming_satellite/satellite.py

Line 530 in 4d52ef3

await _disconnect()

I made some tests and it works as expected is this scenario.
What would you say? Any other suggestion?

[synesthesiam]

A Wyoming audio output system is supposed to send a played message when the audio is finished playing, but I'm not sure of a good way to tell when aplay is done playing the audio its received.
This may be something that is only possible through the ALSA API itself.

[llluis]

Thanks for the feedback @synesthesiam.
I tested with the code in the PR and it works perfectly, but I don't know if it's respecting the standards.

[MrSco]

There were a couple gaps in feedback from the led status colors so I made a couple modifications to 2mic_service.py to handle Error event when no speech text detected (flash purple) and cyan while the assist is processing/playing audio back as a response...

IMG_4879.MOV

IMG_4880.MOV

...

from wyoming.tts import Synthesize
from wyoming.audio import AudioStop
from wyoming.snd import Played

...

...

class LEDsEventHandler(AsyncEventHandler):
    """Event handler for clients."""

    def __init__(
        self,
        cli_args: argparse.Namespace,
        leds: "APA102",
        *args,
        **kwargs,
    ) -> None:
        super().__init__(*args, **kwargs)

        self.cli_args = cli_args
        self.client_id = str(time.monotonic_ns())
        self.leds = leds
        self.previous_event = None 

        _LOGGER.debug("Client connected: %s", self.client_id)

    async def handle_event(self, event: Event) -> bool:
        _LOGGER.debug(event)

        if StreamingStarted.is_type(event.type):
            self.color(_YELLOW)
        elif Detection.is_type(event.type):
            self.color(_BLUE)
            await asyncio.sleep(1.0)  # show for 1 sec
        elif VoiceStarted.is_type(event.type):
            self.color(_YELLOW)
        elif Transcript.is_type(event.type):
            self.color(_GREEN)
            await asyncio.sleep(1.0)  # show for 1 sec
        elif StreamingStopped.is_type(event.type):
            self.color(_BLACK)
        elif RunSatellite.is_type(event.type):
            self.color(_BLACK)
        elif SatelliteConnected.is_type(event.type):
            # Flash
            for _ in range(3):
                self.color(_GREEN)
                await asyncio.sleep(0.3)
                self.color(_BLACK)
                await asyncio.sleep(0.3)
        elif SatelliteDisconnected.is_type(event.type):
            self.color(_RED)
        elif Error.is_type(event.type):            
            # Flash
            for _ in range(3):
                self.color(_PURPLE)
                await asyncio.sleep(0.3)
                self.color(_BLACK)
                await asyncio.sleep(0.3)
        # While the assist is responding with audio, the LEDs should be cyan
        elif Synthesize.is_type(event.type):
            self.color(_CYAN)
        # When the assist is done responding with audio, the LEDs should be black
        elif Played.is_type(event.type) and AudioStop.is_type(self.previous_event):
            self.color(_BLACK)
            
        self.previous_event = event.type 
        
        return True

...

[synesthesiam]

Merged #100

[sayam93]

There were a couple gaps in feedback from the led status colors so I made a couple modifications to 2mic_service.py to handle Error event when no speech text detected (flash purple) and cyan while the assist is processing/playing audio back as a response...

IMG_4879.MOV
IMG_4880.MOV
...

from wyoming.tts import Synthesize
from wyoming.audio import AudioStop
from wyoming.snd import Played

...

...

class LEDsEventHandler(AsyncEventHandler):
    """Event handler for clients."""

    def __init__(
        self,
        cli_args: argparse.Namespace,
        leds: "APA102",
        *args,
        **kwargs,
    ) -> None:
        super().__init__(*args, **kwargs)

        self.cli_args = cli_args
        self.client_id = str(time.monotonic_ns())
        self.leds = leds
        self.previous_event = None 

        _LOGGER.debug("Client connected: %s", self.client_id)

    async def handle_event(self, event: Event) -> bool:
        _LOGGER.debug(event)

        if StreamingStarted.is_type(event.type):
            self.color(_YELLOW)
        elif Detection.is_type(event.type):
            self.color(_BLUE)
            await asyncio.sleep(1.0)  # show for 1 sec
        elif VoiceStarted.is_type(event.type):
            self.color(_YELLOW)
        elif Transcript.is_type(event.type):
            self.color(_GREEN)
            await asyncio.sleep(1.0)  # show for 1 sec
        elif StreamingStopped.is_type(event.type):
            self.color(_BLACK)
        elif RunSatellite.is_type(event.type):
            self.color(_BLACK)
        elif SatelliteConnected.is_type(event.type):
            # Flash
            for _ in range(3):
                self.color(_GREEN)
                await asyncio.sleep(0.3)
                self.color(_BLACK)
                await asyncio.sleep(0.3)
        elif SatelliteDisconnected.is_type(event.type):
            self.color(_RED)
        elif Error.is_type(event.type):            
            # Flash
            for _ in range(3):
                self.color(_PURPLE)
                await asyncio.sleep(0.3)
                self.color(_BLACK)
                await asyncio.sleep(0.3)
        # While the assist is responding with audio, the LEDs should be cyan
        elif Synthesize.is_type(event.type):
            self.color(_CYAN)
        # When the assist is done responding with audio, the LEDs should be black
        elif Played.is_type(event.type) and AudioStop.is_type(self.previous_event):
            self.color(_BLACK)
            
        self.previous_event = event.type 
        
        return True

...

Hi there!

Need a little help.

Is this also required?

from wyoming.error import Error

Also, do these look fine?

_CYAN = (0, 255, 255)
_PURPLE = (128, 0, 128)

Lastly, is this working for you?

        elif SatelliteDisconnected.is_type(event.type):
            self.color(_RED)

Please let me know.

Thanks!