Merge pull request #139 from xmos/release/v0.21.0-beta.0

Release/v0.21.0 beta.0

CHANGELOG.rst

@@ -1,6 +1,14 @@
 XCORE-VOICE change log
 ======================
 
+0.21.0-beta.0
+-------------
+
+  * ADDED: Improved Interference Cancellation using Voice-to-Noise Ratio estimator.  
+  * ADDED: Low-power mode to far-field dictionary example design
+  * ADDED: USB DFU to far-field voice assistant example design
+  * ADDED: Optional AEC fixed-delay to far-field voice assistant example design
+
 0.20.0
 ------
 

doc/product_description/index.rst → doc/introduction/index.rst

@@ -1,16 +1,64 @@
 .. include:: <isonum.txt>
 .. include:: ../substitutions.rst
 
-.. _sln_voice_product_description:
+.. _sln_voice_introduction:
 
-###################
-PRODUCT DESCRIPTION
-###################
+############
+INTRODUCTION
+############
+
+*******************
+Product Description 
+*******************
 
 The XCORE-VOICE Solution consists of example designs and a C-based SDK for the development of audio front-end applications to support far-field voice use cases on the xcore.ai family of chips (XU316). The XCORE-VOICE design is currently based on FreeRTOS, leveraging the flexibility of the xcore.ai platform and providing designers with a familiar environment to customize and develop products.
 
 XCORE-VOICE example designs provide turn-key solutions to enable easier product development for smart home applications such as light switches, thermostats, and home appliances. xcore.ai’s unique architecture providing powerful signal processing and accelerated AI capabilities combined with the XCORE-VOICE framework allows designers to incorporate keyword, event detection, or advanced local dictionary support to create a complete voice interface solution. 
 
+************
+Key Features 
+************
+
+The XCORE-VOICE Solution takes advantage of the flexible software-defined xcore-ai architecture to support numerous far-field voice use cases through the available example designs and the ability to construct user-defined audio pipeline from the SW components and libraries in the C-based SDK. 
+
+These include: 
+
+**Voice Processing components**
+
+- Two PDM microphone interfaces 
+- Digital signal processing pipeline 
+- Full duplex, stereo, Acoustic Echo Cancellation (AEC) 
+- Reference audio via I2S with automatic bulk delay insertion 
+- Point noise suppression via interference canceller 
+- Switchable stationary noise suppressor 
+- Programmable Automatic Gain Control (AGC) 
+- Flexible audio output routing and filtering 
+- Independent audio paths for communications and Automatic Speech Recognition (ASR) 
+- Support for Wanson Speech Recognition or chooser-0defined 3rd party ASR 
+
+**Device Interface components**
+
+- Full speed USB2.0 compliant device supporting USB Audio Class (UAC) 2.0 
+- Flexible Peripheral Interfaces 
+- Programmable digital general-purpose inputs and outputs 
+
+**Example Designs utilising above components**
+
+- Far-Field Voice Local Control 
+- Far-Field Voice Assistance 
+
+**Firmware Management**
+
+- Boot from QSPI Flash 
+- Default firmware image for power-on operation 
+- Option to boot from a local host processor via SPI 
+- Device Firmware Update (DFU) via USB or other transport
+
+**Power Consumption**
+
+- Typical power consumption 300-350mW 
+- Low power modes down to 100mW 
+
 **********************
 Obtaining the Hardware
 **********************

doc/user_guide/ffd/ffd.rst

@@ -129,7 +129,7 @@ Run the following commands in the root folder to build the firmware:
 
     .. code-block:: console
 
-        cmake -G "NMake Makefiles" -B build -DCMAKE_TOOLCHAIN_FILE=xmos_cmake_toolchain/xs3a.cmake
+        cmake -G "NMake Makefiles" -B build -D CMAKE_TOOLCHAIN_FILE=xmos_cmake_toolchain/xs3a.cmake
         cd build
         nmake example_ffd
 

doc/user_guide/ffd/modifying_software.rst

@@ -167,7 +167,7 @@ The generic inference engine API only requires two functions be declared:
 
     /* Generic interface for inference engines */
     int32_t inference_engine_create(uint32_t priority, void *args);
-    int32_t inference_engine_sample_push(int32_t *buf, size_t bytes);
+    int32_t inference_engine_sample_push(int32_t *buf, size_t frames);
 
 Refer to the existing Wanson model implementation for details on how the output handler is set up, how the audio is conditioned to the expected model format, and how it receives frames from the audio pipeline.
 

doc/user_guide/ffd/software_desc/inference.rst

@@ -37,7 +37,7 @@ The inference module provides the application with two API functions:
     :caption: Inference API (inference_engine.h)
 
     int32_t inference_engine_create(uint32_t priority, void *args);
-    int32_t inference_engine_sample_push(int32_t *buf, size_t bytes);
+    int32_t inference_engine_sample_push(int32_t *buf, size_t frames);
 
 If replacing the existing model, these are the only two functions that are required to be populated.
 

doc/user_guide/ffd/software_desc/power.rst

@@ -0,0 +1,45 @@
+.. _sln_voice_ffd_power:
+
+#########
+power
+#########
+
+
+Overview
+========
+
+This folder contains modules for lower power control and state reporting in the
+FFD application.
+
+
+Configuration Notes
+===================
+
+The application monitors the VNR and produces low power events based on:
+
+- `appconfPOWER_VNR_THRESHOLD`
+- `appconfPOWER_LOW_ENERGY_THRESHOLD`
+- `appconfPOWER_HIGH_ENERGY_THRESHOLD`
+- `appconfPOWER_FULL_HOLD_DURATION`
+
+The first three configuration options above determine when to produce a
+`POWER_STATE_FULL` event. The last configuration option above determines when
+a timer period, on expiration, this timer will produce a `POWER_STATE_LOW`
+event. Additionally, when `POWER_STATE_FULL` events occur, this timer is reset
+thus keeping the device is `POWER_STATE_FULL` or at least another
+`appconfPOWER_FULL_HOLD_DURATION` milliseconds.
+
+`appconfPOWER_FULL_HOLD_DURATION` should be configured to take into account
+`appconfINFERENCE_RESET_DELAY_MS` and any time required to produce
+(audible/visual) feedback; in this application, the SLEEP_WAV tone should be
+considered.
+
+`appconfLOW_POWER_ENABLED` enables/disables use of this low power functionality.
+
+When `appconfLOW_POWER_SWITCH_CLK_DIV_ENABLE` is enabled,
+`appconfLOW_POWER_SWITCH_CLK_DIV` should be set appropriately. Clock divider
+values that result in frequencies greater than or equal to 20MHz have been
+observed to work.
+
+Values for `appconfLOW_POWER_CONTROL_TILE_CLK_DIV` that result in frequencies
+greater than or equal to 200MHz have been observed to work.

doc/user_guide/ffd/software_desc/src.rst

@@ -23,9 +23,11 @@ This folder contains the core application source.
    * - audio_pipeline directory
      - contains example XMOS audio pipeline
    * - gpio_ctrl directory
-     - contains general purpose input handling task and LED output heartbeat task
+     - contains general purpose input handling and LED handling tasks
    * - intent_handler directory
      - contains intent handling code
+   * - power directory
+     - contains low power state and control code
    * - rtos_conf directory
      - contains default FreeRTOS configuration header
    * - ssd1306
@@ -97,7 +99,12 @@ This function has the role of receiving the processed audio pipeline output.
 
 This function is weak so the application can override it if desired.
 
-In FFD, the output is sent to the inference engine.
+In FFD, the output is sent to the inference engine. If `appconfLOW_POWER_ENABLED`
+is set true, then the output will be dropped if the power state is not
+`POWER_STATE_FULL`. In certain conditions and environments, this behavior may
+cause the wake word to be missed. Further adjustments to the application
+configuration settings related to the VNR low power thresholds may mitigate such
+issues. See :ref:`sln_voice_ffd_power`.
 
 
 Main

doc/user_guide/ffd/software_description.rst

@@ -15,14 +15,24 @@ Software Description
    software_desc/filesystem_support
    software_desc/host
    software_desc/inference
+   software_desc/power
    software_desc/src
 
 Overview
 ========
 
-The estimated power usage of the example application varies from 100-141 mW.  This will vary based on component tolerances and any user added code and/or user added compile options.
+The estimated power usage of the example application, while in
+`POWER_STATE_FULL`, varies from 100-141 mW. This will vary based on component
+tolerances and any user added code and/or user added compile options.
 
-By default, the application will consume around 141 mW, with a system frequency of 600 MHz.  By changing the system frequency to 400 MHz, the application will consume around 110 mW.  By changing tile 0 to 400 MHz and tile 1 to 200 MHz, the application will consume 100 mW.  Tile frequencies lower than these may lead to application instability.
+By default, the application will startup using a system frequency of 600 MHz
+which will consume around 141 mW. After startup, `tile[1]` clock divider is
+enabled and set to 3 bringing the tile's frequency down to 200 MHz, where it
+will consumer around 113 mW. Tile frequencies lower than this may lead to
+application instability. When the application enters `POWER_STATE_LOW`,
+the `tile[0]` clock frequency will be divided by 600 and the switch clock
+frequency by 30 bringing the frequencies to 1 MHz and 20 MHz, respectively. This
+low power state consumes around 50 mW.
 
 .. list-table:: FFD Resources
    :widths: 30 10 30
@@ -67,6 +77,9 @@ The description of the software is split up by folder:
    * - inference
      - Inferencing engine integration
      - :ref:`sln_voice_ffd_inference`
+   * - power
+     - Low power state and control
+     - :ref:`sln_voice_ffd_power`
    * - src
      - Main application
      - :ref:`sln_voice_ffd_src`

doc/user_guide/stlp/stlp.rst

@@ -141,6 +141,83 @@ From the build folder run:
         nmake run_example_stlp_int_adec
         nmake run_example_stlp_ua_adec
 
+----------------------
+Upgrading the Firmware
+----------------------
+
+The UA variants of this application contain DFU over the USB DFU Class V1.1 transport method.
+
+To create an upgrade image from the build folder run:
+
+.. tab:: Linux and Mac
+
+    .. code-block:: console
+
+        make create_upgrade_img_example_stlp_ua_adec
+        make create_upgrade_img_example_stlp_ua_adec_altarch
+
+.. tab:: Windows
+
+    .. code-block:: console
+
+        nmake create_upgrade_img_example_stlp_ua_adec
+        nmake create_upgrade_img_example_stlp_ua_adec_altarch
+
+Once the application is running, a USB DFU v1.1 tool can be used to perform various actions.  This example will demonstrate with dfu-util commands.  Installation instructions for respective operating system can be found `here <https://dfu-util.sourceforge.net/>`__
+
+To verify the device is running run:
+
+.. code-block:: console
+
+    dfu-util -l
+
+This should result in an output containing:
+
+.. code-block:: console
+
+    Found DFU: [20b1:0020] ver=0001, devnum=100, cfg=1, intf=3, path="3-4.3", alt=2, name="DFU DATAPARTITION", serial="123456"
+    Found DFU: [20b1:0020] ver=0001, devnum=100, cfg=1, intf=3, path="3-4.3", alt=1, name="DFU UPGRADE", serial="123456"
+    Found DFU: [20b1:0020] ver=0001, devnum=100, cfg=1, intf=3, path="3-4.3", alt=0, name="DFU FACTORY", serial="123456"
+
+The DFU interprets the flash as 3 separate partitions, the read only factory image, the read/write upgrade image, and the read/write data partition containing the filesystem.
+
+The factory image can be read back by running:
+
+.. code-block:: console
+
+    dfu-util -e -d 0020 -a 0 -U readback_factory_img.bin
+
+The factory image can not be written to.
+
+From the build folder, the upgrade image can be written by running:
+
+.. code-block:: console
+
+    dfu-util -e -d 0020 -a 1 -D example_stlp_ua_adec_upgrade.bin
+    dfu-util -e -d 0020 -a 1 -D example_stlp_ua_adec_altarch_upgrade.bin
+
+The upgrade image can be read back by running:
+
+.. code-block:: console
+
+    dfu-util -e -d 0020 -a 1 -U readback_upgrade_img.bin
+
+On system reboot, the upgrade image will always be loaded if valid.  If the upgrade image is invalid, the factory image will be loaded.  To revert back to the factory image, you can upload an file containing the word 0xFFFFFFFF.
+
+The data partition image can be read back by running:
+
+.. code-block:: console
+
+    dfu-util -e -d 0020 -a 2 -U readback_data_partition_img.bin
+
+The data partition image can be written by running:
+
+.. code-block:: console
+
+    dfu-util -e -d 0020 -a 2 -D readback_data_partition_img.bin
+
+Note that the data partition will always be at the address specified in the initial flashing call.
+
 ----------------------
 Debugging the Firmware
 ----------------------

examples/audio_mux/src/app_conf.h

@@ -50,7 +50,7 @@
 #endif
 
 #ifndef appconfUSB_ENABLED
-#define appconfUSB_ENABLED         (appconfUSB_INPUT || appconfI2S_OUTPUT)
+#define appconfUSB_ENABLED         (appconfUSB_INPUT || appconfUSB_OUTPUT)
 #endif
 
 #ifndef appconfUSB_AUDIO_ENABLED