Skip to content

Home

Jump to bottom
ken-mycroft edited this page Jun 6, 2022 · 14 revisions

Welcome to the MiniMy wiki!

What is MiniMy

MiniMy is a Linux command line program which runs a few Python processes in the background. These processes provide the ability for a user to voice augment an application. The user inherits from the 'SimpleVoiceAssistant' Python base class which provides the following built-in methods

  • speak()
  • play_media()
  • converse()
  • get_user_input()
  • get_confirmation()
  • send_message()
  • register_intent()

MiniMy components communicate with each other using a 'targeted' WebSocket based message bus. Messages are sent using the send_message() method and received asynchronously by the message bus client.

Example Skill

Here is the example1 skill provided with the default version of MiniMy. It is located in the skills/user_skills directory. It is run by saying the wake word followed by 'run example one'. It simply asks the user where they are located and then it says the location back and asks the user to verify it is correct.

from skills.sva_base import SimpleVoiceAssistant
from threading import Event

class Example1Skill(SimpleVoiceAssistant):
    def __init__(self, bus=None, timeout=5):
        super().__init__(skill_id='example1_skill', skill_category='user')
        self.callback = None
        self.state = 'idle'

        # register intents
        self.register_intent('C', 'run example', 'one', self.start)
        self.register_intent('C', 'test example', 'one', self.start)
        self.register_intent('C', 'execute example', 'one', self.start)

    def handle_timeout(self):
        self.state = 'idle'
        self.play_media(self.skill_base_dir + '/assets/fail.mp3')

    def handle_user_confirmation_input(self, user_confirmation):
        self.state = 'idle'
        media_file = '/assets/fail.mp3'
        if user_confirmation == 'yes':
            media_file = '/assets/success.mp3'
        self.play_media(self.skill_base_dir + media_file)

    def handle_user_location_input(self, user_input):
        self.state = 'get_confirmation'
        self.get_user_confirmation(self.handle_user_confirmation_input,
                                   "You said %s, is that correct?" % (user_input,),
                                   self.handle_timeout)

    def start(self, message):
        self.state = 'get_location'
        self.get_user_input( self.handle_user_location_input,
                             "Please tell me where you are located",
                             self.handle_timeout )

    def stop(self):
        self.log.info("Do nothing stop method hit")

if __name__ == '__main__':
    ex1 = Example1Skill()
    Event().wait()  # Wait forever

Note that 'intents' are subject verb combinations further delineated by sentence type (C for command or Q for question).

What is NLU

NLU (short for natural language understanding) is a somewhat ambiguous term but for the purposes of this code repository it simply means the ability to take the spoken/written word and consistently convert it into an easy to use abstract representation.

Specifically, given a sentence, if we parse() that sentence we will get back a data Sentence Information object.

SentenceInfo
      Type
      Structure
      Insight

Where 'type' is one of

  • command
  • question
  • informational

A 'Structure' object has the following attributes.

Structure
      Original Sentence
      Normalized Sentence
      Tree
      Nodes
      Shallow Tokens

An 'Insight' object has the following attributes.

Insight
      Verb
      Subject
      Aux Verb
      Squal
      Question
      Question Type
      Value
      NP
      VP
      Dependent
      Proper Nouns
      Tense
      Plural
      Location
      Concept
      Context

Typically, an application uses the verb and subject to partition work but this is left up to the user. See the 'example1' skill for an introduction to building simple voice applications.

History

MiniMy was developed in response to the Mycroft AI open source project and shares many concepts but is a smaller version with many additional features.

  • Multiple Configurable Wake Words
  • System Level Supervisor
  • Targeted Messaging
  • Out of Band Verbs and Phrases
  • Local and Remote STT and TTS with Fallback
  • Subject/Verb Based Intents
  • Unlimited Skill Interrupts

When used in remote only mode MiniMy is extremely light weight basically just requiring http post/get support for the cloud based STT and/or TTS services.

Conversely, MiniMy can be configured to run in local only mode which requires limited or no network connectivity depending on the application's needs.

Clone this wiki locally