diff --git a/CHANGELOG.rst b/CHANGELOG.rst index cffac1ee..df355b6c 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -1,6 +1,14 @@ XCORE-VOICE change log ====================== +0.21.0-beta.0 +------------- + + * ADDED: Improved Interference Cancellation using Voice-to-Noise Ratio estimator. + * ADDED: Low-power mode to far-field dictionary example design + * ADDED: USB DFU to far-field voice assistant example design + * ADDED: Optional AEC fixed-delay to far-field voice assistant example design + 0.20.0 ------ diff --git a/doc/datasheet/index.rst b/doc/datasheet/index.rst deleted file mode 100644 index ef25d31c..00000000 --- a/doc/datasheet/index.rst +++ /dev/null @@ -1,80 +0,0 @@ -.. include:: -.. include:: ../substitutions.rst - -.. _sln_voice_datasheet: - -######### -DATASHEET -######### - -************ -Key Features -************ - -The XCORE-VOICE Solution takes advantage of the flexible software-defined xcore-ai architecture to support numerous far-field voice use cases through the available example designs and the ability to construct user-defined audio pipeline from the SW components and libraries in the C-based SDK. - -These include: - -**Voice Processing components** - -- Two PDM microphone interfaces -- Digital signal processing pipeline -- Full duplex, stereo, Acoustic Echo Cancellation (AEC) -- Reference audio via I2S with automatic bulk delay insertion -- Point noise suppression via interference canceller -- Switchable stationary noise suppressor -- Programmable Automatic Gain Control (AGC) -- Flexible audio output routing and filtering -- Independent audio paths for communications and Automatic Speech Recognition (ASR) -- Support for Wanson Speech Recognition or chooser-0defined 3rd party ASR - -**Device Interface components** - -- Full speed USB2.0 compliant device supporting USB Audio Class (UAC) 2.0 -- Flexible Peripheral Interfaces -- Programmable digital general-purpose inputs and outputs - -**Example Designs utilising above components** - -- Far-Field Voice Local Control -- Far-Field Voice Assistance - -**Firmware Management** - -- Boot from QSPI Flash -- Default firmware image for power-on operation -- Option to boot from a local host processor via SPI -- Device Firmware Update (DFU) via USB or other transport - -**Power Consumption** - -- Typical power consumption 300-350mW -- Low power modes down to 100mW - -**************** -Voice Processing -**************** - -TODO: Complete this section with an overview of the voice processing components. - -***************** -Device Interfaces -***************** - -TODO: Complete this section with the options for interfaces. - -******** -Hardware -******** - -TODO: Complete this section with more info on hardware and links to hardware datasheets. - -********************** -Copyright & Disclaimer -********************** - -Copyright © 2022 XMOS Ltd, All Rights Reserved. - -XMOS Ltd is the owner or licensee of this design, code, or Information (collectively, the “Information”) and is providing it to you “AS IS” with no warranty of any kind, express or implied and shall have no liability in relation to its use. XMOS Ltd makes no representation that the Information, or any particular implementation thereof, is or will be free from any claims of infringement and again, shall have no liability in relation to any such claims. - -XMOS, XCORE-VOICE and the XMOS logo are registered trademarks of XMOS Ltd in the United Kingdom and other countries and may not be used without written permission. Company and product names mentioned in this document are the trademarks or registered trademarks of their respective owners. diff --git a/doc/product_description/index.rst b/doc/introduction/index.rst similarity index 66% rename from doc/product_description/index.rst rename to doc/introduction/index.rst index 3a935031..43cb148c 100644 --- a/doc/product_description/index.rst +++ b/doc/introduction/index.rst @@ -1,16 +1,64 @@ .. include:: .. include:: ../substitutions.rst -.. _sln_voice_product_description: +.. _sln_voice_introduction: -################### -PRODUCT DESCRIPTION -################### +############ +INTRODUCTION +############ + +******************* +Product Description +******************* The XCORE-VOICE Solution consists of example designs and a C-based SDK for the development of audio front-end applications to support far-field voice use cases on the xcore.ai family of chips (XU316). The XCORE-VOICE design is currently based on FreeRTOS, leveraging the flexibility of the xcore.ai platform and providing designers with a familiar environment to customize and develop products. XCORE-VOICE example designs provide turn-key solutions to enable easier product development for smart home applications such as light switches, thermostats, and home appliances. xcore.ai’s unique architecture providing powerful signal processing and accelerated AI capabilities combined with the XCORE-VOICE framework allows designers to incorporate keyword, event detection, or advanced local dictionary support to create a complete voice interface solution. +************ +Key Features +************ + +The XCORE-VOICE Solution takes advantage of the flexible software-defined xcore-ai architecture to support numerous far-field voice use cases through the available example designs and the ability to construct user-defined audio pipeline from the SW components and libraries in the C-based SDK. + +These include: + +**Voice Processing components** + +- Two PDM microphone interfaces +- Digital signal processing pipeline +- Full duplex, stereo, Acoustic Echo Cancellation (AEC) +- Reference audio via I2S with automatic bulk delay insertion +- Point noise suppression via interference canceller +- Switchable stationary noise suppressor +- Programmable Automatic Gain Control (AGC) +- Flexible audio output routing and filtering +- Independent audio paths for communications and Automatic Speech Recognition (ASR) +- Support for Wanson Speech Recognition or chooser-0defined 3rd party ASR + +**Device Interface components** + +- Full speed USB2.0 compliant device supporting USB Audio Class (UAC) 2.0 +- Flexible Peripheral Interfaces +- Programmable digital general-purpose inputs and outputs + +**Example Designs utilising above components** + +- Far-Field Voice Local Control +- Far-Field Voice Assistance + +**Firmware Management** + +- Boot from QSPI Flash +- Default firmware image for power-on operation +- Option to boot from a local host processor via SPI +- Device Firmware Update (DFU) via USB or other transport + +**Power Consumption** + +- Typical power consumption 300-350mW +- Low power modes down to 100mW + ********************** Obtaining the Hardware ********************** diff --git a/doc/user_guide/ffd/ffd.rst b/doc/user_guide/ffd/ffd.rst index 79d5ed73..7af2170e 100644 --- a/doc/user_guide/ffd/ffd.rst +++ b/doc/user_guide/ffd/ffd.rst @@ -129,7 +129,7 @@ Run the following commands in the root folder to build the firmware: .. code-block:: console - cmake -G "NMake Makefiles" -B build -DCMAKE_TOOLCHAIN_FILE=xmos_cmake_toolchain/xs3a.cmake + cmake -G "NMake Makefiles" -B build -D CMAKE_TOOLCHAIN_FILE=xmos_cmake_toolchain/xs3a.cmake cd build nmake example_ffd diff --git a/doc/user_guide/ffd/modifying_software.rst b/doc/user_guide/ffd/modifying_software.rst index 05972c2a..613ef1d5 100644 --- a/doc/user_guide/ffd/modifying_software.rst +++ b/doc/user_guide/ffd/modifying_software.rst @@ -167,7 +167,7 @@ The generic inference engine API only requires two functions be declared: /* Generic interface for inference engines */ int32_t inference_engine_create(uint32_t priority, void *args); - int32_t inference_engine_sample_push(int32_t *buf, size_t bytes); + int32_t inference_engine_sample_push(int32_t *buf, size_t frames); Refer to the existing Wanson model implementation for details on how the output handler is set up, how the audio is conditioned to the expected model format, and how it receives frames from the audio pipeline. diff --git a/doc/user_guide/ffd/software_desc/inference.rst b/doc/user_guide/ffd/software_desc/inference.rst index 7b2d0225..837f1bc7 100644 --- a/doc/user_guide/ffd/software_desc/inference.rst +++ b/doc/user_guide/ffd/software_desc/inference.rst @@ -37,7 +37,7 @@ The inference module provides the application with two API functions: :caption: Inference API (inference_engine.h) int32_t inference_engine_create(uint32_t priority, void *args); - int32_t inference_engine_sample_push(int32_t *buf, size_t bytes); + int32_t inference_engine_sample_push(int32_t *buf, size_t frames); If replacing the existing model, these are the only two functions that are required to be populated. diff --git a/doc/user_guide/ffd/software_desc/power.rst b/doc/user_guide/ffd/software_desc/power.rst new file mode 100644 index 00000000..bcae2fb3 --- /dev/null +++ b/doc/user_guide/ffd/software_desc/power.rst @@ -0,0 +1,45 @@ +.. _sln_voice_ffd_power: + +######### +power +######### + + +Overview +======== + +This folder contains modules for lower power control and state reporting in the +FFD application. + + +Configuration Notes +=================== + +The application monitors the VNR and produces low power events based on: + +- `appconfPOWER_VNR_THRESHOLD` +- `appconfPOWER_LOW_ENERGY_THRESHOLD` +- `appconfPOWER_HIGH_ENERGY_THRESHOLD` +- `appconfPOWER_FULL_HOLD_DURATION` + +The first three configuration options above determine when to produce a +`POWER_STATE_FULL` event. The last configuration option above determines when +a timer period, on expiration, this timer will produce a `POWER_STATE_LOW` +event. Additionally, when `POWER_STATE_FULL` events occur, this timer is reset +thus keeping the device is `POWER_STATE_FULL` or at least another +`appconfPOWER_FULL_HOLD_DURATION` milliseconds. + +`appconfPOWER_FULL_HOLD_DURATION` should be configured to take into account +`appconfINFERENCE_RESET_DELAY_MS` and any time required to produce +(audible/visual) feedback; in this application, the SLEEP_WAV tone should be +considered. + +`appconfLOW_POWER_ENABLED` enables/disables use of this low power functionality. + +When `appconfLOW_POWER_SWITCH_CLK_DIV_ENABLE` is enabled, +`appconfLOW_POWER_SWITCH_CLK_DIV` should be set appropriately. Clock divider +values that result in frequencies greater than or equal to 20MHz have been +observed to work. + +Values for `appconfLOW_POWER_CONTROL_TILE_CLK_DIV` that result in frequencies +greater than or equal to 200MHz have been observed to work. \ No newline at end of file diff --git a/doc/user_guide/ffd/software_desc/src.rst b/doc/user_guide/ffd/software_desc/src.rst index 62e82347..b410ff6f 100644 --- a/doc/user_guide/ffd/software_desc/src.rst +++ b/doc/user_guide/ffd/software_desc/src.rst @@ -23,9 +23,11 @@ This folder contains the core application source. * - audio_pipeline directory - contains example XMOS audio pipeline * - gpio_ctrl directory - - contains general purpose input handling task and LED output heartbeat task + - contains general purpose input handling and LED handling tasks * - intent_handler directory - contains intent handling code + * - power directory + - contains low power state and control code * - rtos_conf directory - contains default FreeRTOS configuration header * - ssd1306 @@ -97,7 +99,12 @@ This function has the role of receiving the processed audio pipeline output. This function is weak so the application can override it if desired. -In FFD, the output is sent to the inference engine. +In FFD, the output is sent to the inference engine. If `appconfLOW_POWER_ENABLED` +is set true, then the output will be dropped if the power state is not +`POWER_STATE_FULL`. In certain conditions and environments, this behavior may +cause the wake word to be missed. Further adjustments to the application +configuration settings related to the VNR low power thresholds may mitigate such +issues. See :ref:`sln_voice_ffd_power`. Main diff --git a/doc/user_guide/ffd/software_description.rst b/doc/user_guide/ffd/software_description.rst index e9f1c4d0..f52090b2 100644 --- a/doc/user_guide/ffd/software_description.rst +++ b/doc/user_guide/ffd/software_description.rst @@ -15,14 +15,24 @@ Software Description software_desc/filesystem_support software_desc/host software_desc/inference + software_desc/power software_desc/src Overview ======== -The estimated power usage of the example application varies from 100-141 mW. This will vary based on component tolerances and any user added code and/or user added compile options. +The estimated power usage of the example application, while in +`POWER_STATE_FULL`, varies from 100-141 mW. This will vary based on component +tolerances and any user added code and/or user added compile options. -By default, the application will consume around 141 mW, with a system frequency of 600 MHz. By changing the system frequency to 400 MHz, the application will consume around 110 mW. By changing tile 0 to 400 MHz and tile 1 to 200 MHz, the application will consume 100 mW. Tile frequencies lower than these may lead to application instability. +By default, the application will startup using a system frequency of 600 MHz +which will consume around 141 mW. After startup, `tile[1]` clock divider is +enabled and set to 3 bringing the tile's frequency down to 200 MHz, where it +will consumer around 113 mW. Tile frequencies lower than this may lead to +application instability. When the application enters `POWER_STATE_LOW`, +the `tile[0]` clock frequency will be divided by 600 and the switch clock +frequency by 30 bringing the frequencies to 1 MHz and 20 MHz, respectively. This +low power state consumes around 50 mW. .. list-table:: FFD Resources :widths: 30 10 30 @@ -67,6 +77,9 @@ The description of the software is split up by folder: * - inference - Inferencing engine integration - :ref:`sln_voice_ffd_inference` + * - power + - Low power state and control + - :ref:`sln_voice_ffd_power` * - src - Main application - :ref:`sln_voice_ffd_src` diff --git a/doc/user_guide/stlp/stlp.rst b/doc/user_guide/stlp/stlp.rst index b1fcf208..a80a57b2 100644 --- a/doc/user_guide/stlp/stlp.rst +++ b/doc/user_guide/stlp/stlp.rst @@ -141,6 +141,83 @@ From the build folder run: nmake run_example_stlp_int_adec nmake run_example_stlp_ua_adec +---------------------- +Upgrading the Firmware +---------------------- + +The UA variants of this application contain DFU over the USB DFU Class V1.1 transport method. + +To create an upgrade image from the build folder run: + +.. tab:: Linux and Mac + + .. code-block:: console + + make create_upgrade_img_example_stlp_ua_adec + make create_upgrade_img_example_stlp_ua_adec_altarch + +.. tab:: Windows + + .. code-block:: console + + nmake create_upgrade_img_example_stlp_ua_adec + nmake create_upgrade_img_example_stlp_ua_adec_altarch + +Once the application is running, a USB DFU v1.1 tool can be used to perform various actions. This example will demonstrate with dfu-util commands. Installation instructions for respective operating system can be found `here `__ + +To verify the device is running run: + +.. code-block:: console + + dfu-util -l + +This should result in an output containing: + +.. code-block:: console + + Found DFU: [20b1:0020] ver=0001, devnum=100, cfg=1, intf=3, path="3-4.3", alt=2, name="DFU DATAPARTITION", serial="123456" + Found DFU: [20b1:0020] ver=0001, devnum=100, cfg=1, intf=3, path="3-4.3", alt=1, name="DFU UPGRADE", serial="123456" + Found DFU: [20b1:0020] ver=0001, devnum=100, cfg=1, intf=3, path="3-4.3", alt=0, name="DFU FACTORY", serial="123456" + +The DFU interprets the flash as 3 separate partitions, the read only factory image, the read/write upgrade image, and the read/write data partition containing the filesystem. + +The factory image can be read back by running: + +.. code-block:: console + + dfu-util -e -d 0020 -a 0 -U readback_factory_img.bin + +The factory image can not be written to. + +From the build folder, the upgrade image can be written by running: + +.. code-block:: console + + dfu-util -e -d 0020 -a 1 -D example_stlp_ua_adec_upgrade.bin + dfu-util -e -d 0020 -a 1 -D example_stlp_ua_adec_altarch_upgrade.bin + +The upgrade image can be read back by running: + +.. code-block:: console + + dfu-util -e -d 0020 -a 1 -U readback_upgrade_img.bin + +On system reboot, the upgrade image will always be loaded if valid. If the upgrade image is invalid, the factory image will be loaded. To revert back to the factory image, you can upload an file containing the word 0xFFFFFFFF. + +The data partition image can be read back by running: + +.. code-block:: console + + dfu-util -e -d 0020 -a 2 -U readback_data_partition_img.bin + +The data partition image can be written by running: + +.. code-block:: console + + dfu-util -e -d 0020 -a 2 -D readback_data_partition_img.bin + +Note that the data partition will always be at the address specified in the initial flashing call. + ---------------------- Debugging the Firmware ---------------------- diff --git a/examples/audio_mux/src/app_conf.h b/examples/audio_mux/src/app_conf.h index e320774b..3cd60ee2 100644 --- a/examples/audio_mux/src/app_conf.h +++ b/examples/audio_mux/src/app_conf.h @@ -50,7 +50,7 @@ #endif #ifndef appconfUSB_ENABLED -#define appconfUSB_ENABLED (appconfUSB_INPUT || appconfI2S_OUTPUT) +#define appconfUSB_ENABLED (appconfUSB_INPUT || appconfUSB_OUTPUT) #endif #ifndef appconfUSB_AUDIO_ENABLED diff --git a/examples/ffd/README.md b/examples/ffd/README.md index 1f56a443..bc6e5486 100644 --- a/examples/ffd/README.md +++ b/examples/ffd/README.md @@ -53,7 +53,7 @@ On Linux and Mac run: On Windows run: - cmake -G "NMake Makefiles" -B build -DCMAKE_TOOLCHAIN_FILE=xmos_cmake_toolchain/xs3a.cmake + cmake -G "NMake Makefiles" -B build -D CMAKE_TOOLCHAIN_FILE=xmos_cmake_toolchain/xs3a.cmake cd build nmake example_ffd diff --git a/examples/ffd/bsp_config/XK_VOICE_L71/platform/driver_instances.c b/examples/ffd/bsp_config/XK_VOICE_L71/platform/driver_instances.c index 6a4246bb..6ba3d85a 100644 --- a/examples/ffd/bsp_config/XK_VOICE_L71/platform/driver_instances.c +++ b/examples/ffd/bsp_config/XK_VOICE_L71/platform/driver_instances.c @@ -6,6 +6,9 @@ static rtos_intertile_t intertile_ctx_s; rtos_intertile_t *intertile_ctx = &intertile_ctx_s; +static rtos_clock_control_t cc_ctx_t0_s; +rtos_clock_control_t *cc_ctx_t0 = &cc_ctx_t0_s; + static rtos_qspi_flash_t qspi_flash_ctx_s; rtos_qspi_flash_t *qspi_flash_ctx = &qspi_flash_ctx_s; diff --git a/examples/ffd/bsp_config/XK_VOICE_L71/platform/driver_instances.h b/examples/ffd/bsp_config/XK_VOICE_L71/platform/driver_instances.h index 36984f57..72c1c384 100644 --- a/examples/ffd/bsp_config/XK_VOICE_L71/platform/driver_instances.h +++ b/examples/ffd/bsp_config/XK_VOICE_L71/platform/driver_instances.h @@ -9,6 +9,7 @@ extern "C" { #endif #include "rtos_intertile.h" +#include "rtos_clock_control.h" #include "rtos_qspi_flash.h" #include "rtos_gpio.h" #include "rtos_i2c_master.h" @@ -46,6 +47,7 @@ extern "C" { #define PORT_I2S_ADC_DATA I2S_MIC_DATA extern rtos_intertile_t *intertile_ctx; +extern rtos_clock_control_t *cc_ctx_t0; extern rtos_qspi_flash_t *qspi_flash_ctx; extern rtos_gpio_t *gpio_ctx_t0; extern rtos_gpio_t *gpio_ctx_t1; diff --git a/examples/ffd/bsp_config/XK_VOICE_L71/platform/platform_init.c b/examples/ffd/bsp_config/XK_VOICE_L71/platform/platform_init.c index 40f1a795..e2733f95 100644 --- a/examples/ffd/bsp_config/XK_VOICE_L71/platform/platform_init.c +++ b/examples/ffd/bsp_config/XK_VOICE_L71/platform/platform_init.c @@ -17,6 +17,28 @@ static void mclk_init(chanend_t other_tile_c) #endif } +static void clock_control_init(void) +{ + static rtos_driver_rpc_t clock_control_rpc_config_t0; + +#if ON_TILE(0) + rtos_intertile_t *client_intertile_ctx[] = {intertile_ctx}; + + rtos_clock_control_init(cc_ctx_t0); + + rtos_clock_control_rpc_host_init( + cc_ctx_t0, + &clock_control_rpc_config_t0, + client_intertile_ctx, + sizeof(client_intertile_ctx) / sizeof(rtos_intertile_t *)); +#else + rtos_clock_control_rpc_client_init( + cc_ctx_t0, + &clock_control_rpc_config_t0, + intertile_ctx); +#endif +} + static void flash_init(void) { #if ON_TILE(FLASH_TILE_NO) @@ -180,6 +202,7 @@ void platform_init(chanend_t other_tile_c) rtos_intertile_init(intertile_ctx, other_tile_c); mclk_init(other_tile_c); + clock_control_init(); gpio_init(); flash_init(); i2c_init(); diff --git a/examples/ffd/bsp_config/XK_VOICE_L71/platform/platform_start.c b/examples/ffd/bsp_config/XK_VOICE_L71/platform/platform_start.c index 3c643025..08c797b3 100644 --- a/examples/ffd/bsp_config/XK_VOICE_L71/platform/platform_start.c +++ b/examples/ffd/bsp_config/XK_VOICE_L71/platform/platform_start.c @@ -17,6 +17,18 @@ extern void i2s_rate_conversion_enable(void); +static void clock_control_start(void) +{ + rtos_clock_control_rpc_config( + cc_ctx_t0, + appconfCLOCK_CONTROL_PORT, + appconfCLOCK_CONTROL_RPC_HOST_PRIORITY); + +#if ON_TILE(0) + rtos_clock_control_start(cc_ctx_t0); +#endif +} + static void gpio_start(void) { rtos_gpio_rpc_config(gpio_ctx_t0, appconfGPIO_T0_RPC_PORT, appconfGPIO_RPC_PRIORITY); @@ -104,6 +116,7 @@ void platform_start(void) { rtos_intertile_start(intertile_ctx); + clock_control_start(); gpio_start(); flash_start(); i2c_master_start(); diff --git a/examples/ffd/bsp_config/XK_VOICE_L71_EXT/platform/driver_instances.c b/examples/ffd/bsp_config/XK_VOICE_L71_EXT/platform/driver_instances.c index 6a4246bb..6ba3d85a 100644 --- a/examples/ffd/bsp_config/XK_VOICE_L71_EXT/platform/driver_instances.c +++ b/examples/ffd/bsp_config/XK_VOICE_L71_EXT/platform/driver_instances.c @@ -6,6 +6,9 @@ static rtos_intertile_t intertile_ctx_s; rtos_intertile_t *intertile_ctx = &intertile_ctx_s; +static rtos_clock_control_t cc_ctx_t0_s; +rtos_clock_control_t *cc_ctx_t0 = &cc_ctx_t0_s; + static rtos_qspi_flash_t qspi_flash_ctx_s; rtos_qspi_flash_t *qspi_flash_ctx = &qspi_flash_ctx_s; diff --git a/examples/ffd/bsp_config/XK_VOICE_L71_EXT/platform/driver_instances.h b/examples/ffd/bsp_config/XK_VOICE_L71_EXT/platform/driver_instances.h index 67ebebc7..ec642892 100644 --- a/examples/ffd/bsp_config/XK_VOICE_L71_EXT/platform/driver_instances.h +++ b/examples/ffd/bsp_config/XK_VOICE_L71_EXT/platform/driver_instances.h @@ -9,6 +9,7 @@ extern "C" { #endif #include "rtos_intertile.h" +#include "rtos_clock_control.h" #include "rtos_qspi_flash.h" #include "rtos_gpio.h" #include "rtos_i2c_master.h" @@ -48,6 +49,7 @@ extern "C" { #define PORT_I2S_ADC_DATA I2S_MIC_DATA extern rtos_intertile_t *intertile_ctx; +extern rtos_clock_control_t *cc_ctx_t0; extern rtos_qspi_flash_t *qspi_flash_ctx; extern rtos_gpio_t *gpio_ctx_t0; extern rtos_gpio_t *gpio_ctx_t1; diff --git a/examples/ffd/bsp_config/XK_VOICE_L71_EXT/platform/platform_init.c b/examples/ffd/bsp_config/XK_VOICE_L71_EXT/platform/platform_init.c index f3922b4d..f41b4fd9 100644 --- a/examples/ffd/bsp_config/XK_VOICE_L71_EXT/platform/platform_init.c +++ b/examples/ffd/bsp_config/XK_VOICE_L71_EXT/platform/platform_init.c @@ -22,6 +22,28 @@ static void mclk_init(chanend_t other_tile_c) #endif } +static void clock_control_init(void) +{ + static rtos_driver_rpc_t clock_control_rpc_config_t0; + +#if ON_TILE(0) + rtos_intertile_t *client_intertile_ctx[] = {intertile_ctx}; + + rtos_clock_control_init(cc_ctx_t0); + + rtos_clock_control_rpc_host_init( + cc_ctx_t0, + &clock_control_rpc_config_t0, + client_intertile_ctx, + sizeof(client_intertile_ctx) / sizeof(rtos_intertile_t *)); +#else + rtos_clock_control_rpc_client_init( + cc_ctx_t0, + &clock_control_rpc_config_t0, + intertile_ctx); +#endif +} + static void flash_init(void) { #if ON_TILE(FLASH_TILE_NO) @@ -176,6 +198,7 @@ void platform_init(chanend_t other_tile_c) rtos_intertile_init(intertile_ctx, other_tile_c); mclk_init(other_tile_c); + clock_control_init(); gpio_init(); flash_init(); i2c_init(); diff --git a/examples/ffd/bsp_config/XK_VOICE_L71_EXT/platform/platform_start.c b/examples/ffd/bsp_config/XK_VOICE_L71_EXT/platform/platform_start.c index 00a113ea..6cb99c72 100644 --- a/examples/ffd/bsp_config/XK_VOICE_L71_EXT/platform/platform_start.c +++ b/examples/ffd/bsp_config/XK_VOICE_L71_EXT/platform/platform_start.c @@ -18,6 +18,18 @@ extern void i2s_rate_conversion_enable(void); +static void clock_control_start(void) +{ + rtos_clock_control_rpc_config( + cc_ctx_t0, + appconfCLOCK_CONTROL_PORT, + appconfCLOCK_CONTROL_RPC_HOST_PRIORITY); + +#if ON_TILE(0) + rtos_clock_control_start(cc_ctx_t0); +#endif +} + static void gpio_start(void) { rtos_gpio_rpc_config(gpio_ctx_t0, appconfGPIO_T0_RPC_PORT, appconfGPIO_RPC_PRIORITY); @@ -109,6 +121,7 @@ void platform_start(void) { rtos_intertile_start(intertile_ctx); + clock_control_start(); gpio_start(); flash_start(); i2c_master_start(); diff --git a/examples/ffd/ext/ffd_usb_audio_testing.cmake b/examples/ffd/ext/ffd_usb_audio_testing.cmake index 7e429722..be5d1b72 100644 --- a/examples/ffd/ext/ffd_usb_audio_testing.cmake +++ b/examples/ffd/ext/ffd_usb_audio_testing.cmake @@ -6,6 +6,7 @@ set(KEYWORD_USB_TESTING appconfINFERENCE_RAW_OUTPUT=1 appconfAUDIO_PLAYBACK_ENABLED=0 appconfINFERENCE_UART_OUTPUT_ENABLED=0 + appconfLOW_POWER_ENABLED=0 ) set(BYPASS_AUDIOPIPELINE_DEFINITIONS diff --git a/examples/ffd/ext/src/rtos_conf/FreeRTOSConfig.h b/examples/ffd/ext/src/rtos_conf/FreeRTOSConfig.h index b23c769a..9ca6b129 100644 --- a/examples/ffd/ext/src/rtos_conf/FreeRTOSConfig.h +++ b/examples/ffd/ext/src/rtos_conf/FreeRTOSConfig.h @@ -47,7 +47,7 @@ your application. */ #define configTOTAL_HEAP_SIZE 168*1024 #endif #if ON_TILE(1) -#define configTOTAL_HEAP_SIZE 128*1024 +#define configTOTAL_HEAP_SIZE 132*1024 #endif #define configAPPLICATION_ALLOCATED_HEAP 0 diff --git a/examples/ffd/ffd.cmake b/examples/ffd/ffd.cmake index aa4da4fe..01d16369 100644 --- a/examples/ffd/ffd.cmake +++ b/examples/ffd/ffd.cmake @@ -4,8 +4,12 @@ file(GLOB_RECURSE APP_SOURCES ${CMAKE_CURRENT_LIST_DIR}/src/*.c ) set(APP_INCLUDES ${CMAKE_CURRENT_LIST_DIR}/src - ${CMAKE_CURRENT_LIST_DIR}/src/ssd1306 + ${CMAKE_CURRENT_LIST_DIR}/src/audio_pipeline + ${CMAKE_CURRENT_LIST_DIR}/src/gpio_ctrl + ${CMAKE_CURRENT_LIST_DIR}/src/intent_handler ${CMAKE_CURRENT_LIST_DIR}/src/intent_handler/audio_response + ${CMAKE_CURRENT_LIST_DIR}/src/power + ${CMAKE_CURRENT_LIST_DIR}/src/ssd1306 ) set(RTOS_CONF_INCLUDES ${CMAKE_CURRENT_LIST_DIR}/src/rtos_conf @@ -28,7 +32,7 @@ set(APP_COMPILER_FLAGS ) set(APP_COMPILE_DEFINITIONS - DEBUG_PRINT_ENABLE=1 + configENABLE_DEBUG_PRINTF=1 PLATFORM_USES_TILE_0=1 PLATFORM_USES_TILE_1=1 ) @@ -45,6 +49,7 @@ set(APP_COMMON_LINK_LIBRARIES fwk_voice::ns fwk_voice::vnr::features fwk_voice::vnr::inference + rtos::drivers::clock_control ) #********************** diff --git a/examples/ffd/inference/api/inference_engine.h b/examples/ffd/inference/api/inference_engine.h index 828685ca..2e0a8b81 100644 --- a/examples/ffd/inference/api/inference_engine.h +++ b/examples/ffd/inference/api/inference_engine.h @@ -8,6 +8,6 @@ /* Generic interface for inference engines */ int32_t inference_engine_create(uint32_t priority, void *args); -int32_t inference_engine_sample_push(int32_t *buf, size_t bytes); +int32_t inference_engine_sample_push(int32_t *buf, size_t frames); #endif /* INFERENCE_ENGINE_H_ */ diff --git a/examples/ffd/inference/inference.cmake b/examples/ffd/inference/inference.cmake index 63334535..a99efea8 100644 --- a/examples/ffd/inference/inference.cmake +++ b/examples/ffd/inference/inference.cmake @@ -2,6 +2,7 @@ add_library(sln_voice_app_ffd_inference_engine_wanson INTERFACE) target_sources(sln_voice_app_ffd_inference_engine_wanson INTERFACE + ${CMAKE_CURRENT_LIST_DIR}/wanson/lib_xcore_math_compat.c ${CMAKE_CURRENT_LIST_DIR}/wanson/wanson_inf_eng.c ${CMAKE_CURRENT_LIST_DIR}/wanson/wanson_inf_eng_port.c ${CMAKE_CURRENT_LIST_DIR}/wanson/wanson_inf_eng_support.c diff --git a/examples/ffd/inference/wanson/lib_xcore_math_compat.c b/examples/ffd/inference/wanson/lib_xcore_math_compat.c new file mode 100644 index 00000000..8a025ec7 --- /dev/null +++ b/examples/ffd/inference/wanson/lib_xcore_math_compat.c @@ -0,0 +1,16 @@ +// Copyright (c) 2022 XMOS LIMITED. This Software is subject to the terms of the +// XMOS Public License: Version 1 + +#include + +#include "xmath/xmath.h" + +// The Wanson libasrengine static library is compiled against an older version of lib_xcore_math. +// This wrapper function is provided for compatibility +int64_t xs3_vect_s16_dot( + const int16_t b[], + const int16_t c[], + const unsigned length) +{ + return vect_s16_dot(b, c, length); +} \ No newline at end of file diff --git a/examples/ffd/inference/wanson/wanson_inf_eng.c b/examples/ffd/inference/wanson/wanson_inf_eng.c index e58e99b1..07e3f318 100644 --- a/examples/ffd/inference/wanson/wanson_inf_eng.c +++ b/examples/ffd/inference/wanson/wanson_inf_eng.c @@ -20,6 +20,7 @@ #include "wanson_inf_eng.h" #include "wanson_api.h" #include "xcore_device_memory.h" +#include "gpio_ctrl/leds.h" #define WANSON_SAMPLES_PER_INFERENCE (2 * appconfINFERENCE_SAMPLE_BLOCK_LENGTH) @@ -35,6 +36,9 @@ void vDisplayClearCallback(TimerHandle_t pxTimer) { if ((inference_state == STATE_EXPECTING_COMMAND) || (inference_state == STATE_PROCESSING_COMMAND)) { wanson_engine_proc_keyword_result(NULL, 50); /* 50 is a special id that will play the no longer listening for command sound */ +#if ON_TILE(0) + led_indicate_waiting(); +#endif } inference_state = STATE_EXPECTING_WAKEWORD; } @@ -122,6 +126,9 @@ void wanson_engine_task(void *args) wanson_engine_proc_keyword_result((const char **)&text_ptr, id); #else if (inference_state == STATE_EXPECTING_WAKEWORD && IS_WAKEWORD(id)) { +#if ON_TILE(0) + led_indicate_listening(); +#endif xTimerReset(display_clear_timer, 0); wanson_engine_proc_keyword_result((const char **)&text_ptr, id); inference_state = STATE_EXPECTING_COMMAND; diff --git a/examples/ffd/inference/wanson/wanson_inf_eng.h b/examples/ffd/inference/wanson/wanson_inf_eng.h index ea497261..18e4b260 100644 --- a/examples/ffd/inference/wanson/wanson_inf_eng.h +++ b/examples/ffd/inference/wanson/wanson_inf_eng.h @@ -10,6 +10,7 @@ void wanson_engine_task(void *args); void wanson_engine_task_create(unsigned priority); +void wanson_engine_stream_buf_reset(void); void wanson_engine_samples_send_local( size_t frame_count, int32_t *processed_audio_frame); diff --git a/examples/ffd/inference/wanson/wanson_inf_eng_port.c b/examples/ffd/inference/wanson/wanson_inf_eng_port.c index 337a6364..801a3e5a 100644 --- a/examples/ffd/inference/wanson/wanson_inf_eng_port.c +++ b/examples/ffd/inference/wanson/wanson_inf_eng_port.c @@ -96,42 +96,42 @@ void wanson_engine_proc_keyword_result(const char **text, int id) } } -#if appconfSSD1306_DISPLAY_ENABLED - // some temporary fixes to the strings returned - switch (id) { - case 50: - // Clear the display - ssd1306_display_ascii_to_bitmap("\0"); - break; - case 3: - // fix capital "On" - ssd1306_display_ascii_to_bitmap("Switch on the TV\0"); - break; - case 15: - // fix lower case "speed" - // fix word wrapping - ssd1306_display_ascii_to_bitmap("Speed up the fan\0"); - break; - case 16: - // fix lower case "slow" - ssd1306_display_ascii_to_bitmap("Slow down the fan\0"); - break; - case 17: - // fix lower case "set" - // fix word wrapping - ssd1306_display_ascii_to_bitmap("Set higher temperature\0"); - break; - case 18: - // fix lower case "set" - // fix word wrapping - ssd1306_display_ascii_to_bitmap("Set lower temperature\0"); - break; - default: - if(text != NULL) { - ssd1306_display_ascii_to_bitmap((char *)*text); - } - } -#endif +// #if appconfSSD1306_DISPLAY_ENABLED +// // some temporary fixes to the strings returned +// switch (id) { +// case 50: +// // Clear the display +// ssd1306_display_ascii_to_bitmap("\0"); +// break; +// case 3: +// // fix capital "On" +// ssd1306_display_ascii_to_bitmap("Switch on the TV\0"); +// break; +// case 15: +// // fix lower case "speed" +// // fix word wrapping +// ssd1306_display_ascii_to_bitmap("Speed up the fan\0"); +// break; +// case 16: +// // fix lower case "slow" +// ssd1306_display_ascii_to_bitmap("Slow down the fan\0"); +// break; +// case 17: +// // fix lower case "set" +// // fix word wrapping +// ssd1306_display_ascii_to_bitmap("Set higher temperature\0"); +// break; +// case 18: +// // fix lower case "set" +// // fix word wrapping +// ssd1306_display_ascii_to_bitmap("Set lower temperature\0"); +// break; +// default: +// if(text != NULL) { +// ssd1306_display_ascii_to_bitmap((char *)*text); +// } +// } +// #endif } int32_t inference_engine_create(uint32_t priority, void *args) diff --git a/examples/ffd/inference/wanson/wanson_inf_eng_support.c b/examples/ffd/inference/wanson/wanson_inf_eng_support.c index 099f1f45..99b95a7e 100644 --- a/examples/ffd/inference/wanson/wanson_inf_eng_support.c +++ b/examples/ffd/inference/wanson/wanson_inf_eng_support.c @@ -20,6 +20,12 @@ static StreamBufferHandle_t samples_to_engine_stream_buf = 0; +void wanson_engine_stream_buf_reset(void) +{ + if (samples_to_engine_stream_buf) + xStreamBufferReset(samples_to_engine_stream_buf); +} + void wanson_engine_samples_send_remote( rtos_intertile_t *intertile_ctx, size_t frame_count, diff --git a/examples/ffd/src/app_conf.h b/examples/ffd/src/app_conf.h index ef3a9fe4..cf12d594 100644 --- a/examples/ffd/src/app_conf.h +++ b/examples/ffd/src/app_conf.h @@ -10,7 +10,10 @@ #define appconfGPIO_T1_RPC_PORT 2 #define appconfINTENT_MODEL_RUNNER_SAMPLES_PORT 3 #define appconfI2C_MASTER_RPC_PORT 4 +#define appconfCLOCK_CONTROL_PORT 14 +#define appconfPOWER_CONTROL_PORT 15 +#define appconfPOWER_STATE_PORT 12 #define appconfWANSON_READY_SYNC_PORT 16 /* Application tile specifiers */ @@ -38,7 +41,7 @@ /* Maximum delay between a wake up phrase and command phrase */ #ifndef appconfINFERENCE_RESET_DELAY_MS -#define appconfINFERENCE_RESET_DELAY_MS 3000 +#define appconfINFERENCE_RESET_DELAY_MS 4000 #endif /* Output raw inferences, if set to 0, a state machine requires a wake up phrase @@ -57,6 +60,11 @@ #define appconfINTENT_QUEUE_LEN 10 #endif +/* Maximum number of detected intents to hold */ +#ifndef appconfWAKEUP_TRIGGER_LEN +#define appconfWAKEUP_TRIGGER_LEN 1 +#endif + /* External wakeup pin edge on intent found. 0 for rising edge, 1 for falling edge */ #ifndef appconfINTENT_WAKEUP_EDGE_TYPE #define appconfINTENT_WAKEUP_EDGE_TYPE 0 @@ -91,6 +99,37 @@ #define appconfI2S_ENABLED 1 #endif +#ifndef appconfLOW_POWER_ENABLED +#define appconfLOW_POWER_ENABLED 1 +#endif + +#ifndef appconfLOW_POWER_SWITCH_CLK_DIV_ENABLE +#define appconfLOW_POWER_SWITCH_CLK_DIV_ENABLE 1 +#endif + +#define appconfLOW_POWER_SWITCH_CLK_DIV 30 // Resulting clock freq >= 20MHz. +#define appconfLOW_POWER_OTHER_TILE_CLK_DIV 600 +#define appconfLOW_POWER_CONTROL_TILE_CLK_DIV 3 // Resulting clock freq >= 200MHz + +#define appconfPOWER_VNR_THRESHOLD (0.3f) +#define appconfPOWER_LOW_ENERGY_THRESHOLD (0.01f) +#define appconfPOWER_HIGH_ENERGY_THRESHOLD (4.0f) +#define appconfPOWER_FULL_HOLD_DURATION (appconfINFERENCE_RESET_DELAY_MS + 3000) // milliseconds + +/* Enable/disable the use of a ring buffer to hold onto pre-trigger audio + * samples while in low power mode. */ +#ifndef appconfAUDIO_PIPELINE_BUFFER_ENABLED +#define appconfAUDIO_PIPELINE_BUFFER_ENABLED 1 +#endif + +/* The number of frames (appconfAUDIO_PIPELINE_FRAME_ADVANCE) to store in a ring + * buffer while in low power mode. This may be tuned to ensure that unvoiced + * speech that is a pre-cursor to voiced speech in a wake-word such as "he" part + * of "hello" is captured and relayed to the inference engine. */ +#ifndef appconfAUDIO_PIPELINE_BUFFER_NUM_FRAMES +#define appconfAUDIO_PIPELINE_BUFFER_NUM_FRAMES 32 +#endif + #ifndef appconfAUDIO_PIPELINE_SKIP_IC_AND_VNR #define appconfAUDIO_PIPELINE_SKIP_IC_AND_VNR 0 #endif @@ -151,6 +190,8 @@ #define appconfINFERENCE_MODEL_RUNNER_TASK_PRIORITY (configMAX_PRIORITIES - 2) #define appconfINFERENCE_HMI_TASK_PRIORITY (configMAX_PRIORITIES / 2) #define appconfGPIO_RPC_PRIORITY (configMAX_PRIORITIES / 2) +#define appconfCLOCK_CONTROL_RPC_HOST_PRIORITY (configMAX_PRIORITIES / 2) +#define appconfPOWER_CONTROL_TASK_PRIORITY (configMAX_PRIORITIES / 2) #define appconfGPIO_TASK_PRIORITY (configMAX_PRIORITIES / 2 + 2) #define appconfI2C_TASK_PRIORITY (configMAX_PRIORITIES / 2 + 2) #define appconfI2C_MASTER_RPC_PRIORITY (configMAX_PRIORITIES / 2) @@ -159,7 +200,7 @@ #define appconfSPI_TASK_PRIORITY (configMAX_PRIORITIES / 2 + 1) #define appconfQSPI_FLASH_TASK_PRIORITY (configMAX_PRIORITIES - 1) #define appconfSSD1306_TASK_PRIORITY (configMAX_PRIORITIES / 2 - 1) -#define appconfLED_HEARTBEAT_TASK_PRIORITY (configMAX_PRIORITIES / 2 - 1) +#define appconfLED_TASK_PRIORITY (configMAX_PRIORITIES / 2 - 1) #include "app_conf_check.h" diff --git a/examples/ffd/src/audio_pipeline/audio_pipeline.c b/examples/ffd/src/audio_pipeline/audio_pipeline.c index 2b463435..062ddf96 100644 --- a/examples/ffd/src/audio_pipeline/audio_pipeline.c +++ b/examples/ffd/src/audio_pipeline/audio_pipeline.c @@ -23,6 +23,10 @@ /* App headers */ #include "app_conf.h" #include "audio_pipeline/audio_pipeline.h" +#include "power/power_state.h" + +#define VNR_AGC_THRESHOLD (0.5) +#define EMA_ENERGY_ALPHA (0.25) /* Note: Changing the order here will effect the channel order for * audio_pipeline_input() and audio_pipeline_output() @@ -30,15 +34,14 @@ typedef struct { int32_t samples[appconfAUDIO_PIPELINE_CHANNELS][appconfAUDIO_PIPELINE_FRAME_ADVANCE]; int32_t mic_samples_passthrough[appconfAUDIO_PIPELINE_CHANNELS][appconfAUDIO_PIPELINE_FRAME_ADVANCE]; - int32_t vnr_pred_flag; + float_s32_t vnr_pred; + float_s32_t ema_energy; } frame_data_t; #if appconfAUDIO_PIPELINE_FRAME_ADVANCE != 240 #error This pipeline is only configured for 240 frame advance #endif -#define VNR_AGC_THRESHOLD (0.5) - typedef struct ic_stage_ctx { ic_state_t state; } ic_stage_ctx_t; @@ -61,6 +64,10 @@ static vnr_pred_stage_ctx_t DWORD_ALIGNED vnr_pred_stage_state = {}; static ns_stage_ctx_t DWORD_ALIGNED ns_stage_state = {}; static agc_stage_ctx_t DWORD_ALIGNED agc_stage_state = {}; +static uq2_30 ema_energy_alpha_q30 = Q30(EMA_ENERGY_ALPHA); + +static power_data_t* power_app_data = 0; + static void *audio_pipeline_input_i(void *input_app_data) { frame_data_t *frame_data; @@ -71,7 +78,8 @@ static void *audio_pipeline_input_i(void *input_app_data) (int32_t **)frame_data->samples, 2, appconfAUDIO_PIPELINE_FRAME_ADVANCE); - frame_data->vnr_pred_flag = 0; + frame_data->vnr_pred = f32_to_float_s32(0.0); + frame_data->ema_energy = f32_to_float_s32(0.0); memcpy(frame_data->mic_samples_passthrough, frame_data->samples, sizeof(frame_data->mic_samples_passthrough)); @@ -81,7 +89,15 @@ static void *audio_pipeline_input_i(void *input_app_data) static int audio_pipeline_output_i(frame_data_t *frame_data, void *output_app_data) { - return audio_pipeline_output(output_app_data, + if (power_app_data) { + //NOTE: passing power_app_data to callback instead of output_app_data. + // They should be the same pointer. + assert(power_app_data == output_app_data); + power_app_data->vnr_pred = float_s32_to_float(frame_data->vnr_pred); + power_app_data->ema_energy = float_s32_to_float(frame_data->ema_energy); + } + + return audio_pipeline_output(power_app_data, (int32_t **)frame_data->samples, 4, appconfAUDIO_PIPELINE_FRAME_ADVANCE); @@ -115,13 +131,7 @@ static void stage_vnr_and_ic(frame_data_t *frame_data) vnr_inference(&ie_output, &feature_patch); vnr_pred_state->output_vnr_pred = float_s32_ema(vnr_pred_state->output_vnr_pred, ie_output, vnr_pred_state->pred_alpha_q30); -#if 0 - rtos_printf("VNR OUTPUT PRED: %ld %d\n", vnr_pred_stage_state.vnr_pred_state.output_vnr_pred.mant, vnr_pred_stage_state.vnr_pred_state.output_vnr_pred.exp); - rtos_printf("VNR INPUT PRED: %ld %d\n", vnr_pred_stage_state.vnr_pred_state.input_vnr_pred.mant, vnr_pred_stage_state.vnr_pred_state.input_vnr_pred.exp); -#endif - - float_s32_t agc_vnr_threshold = float_to_float_s32(VNR_AGC_THRESHOLD); - frame_data->vnr_pred_flag = float_s32_gt(vnr_pred_stage_state.vnr_pred_state.output_vnr_pred, agc_vnr_threshold); + frame_data->vnr_pred = vnr_pred_stage_state.vnr_pred_state.output_vnr_pred; ic_adapt(&ic_stage_state.state, vnr_pred_stage_state.vnr_pred_state.input_vnr_pred); @@ -152,7 +162,9 @@ static void stage_agc(frame_data_t *frame_data) int32_t DWORD_ALIGNED agc_output[appconfAUDIO_PIPELINE_FRAME_ADVANCE]; configASSERT(AGC_FRAME_ADVANCE == appconfAUDIO_PIPELINE_FRAME_ADVANCE); - agc_stage_state.md.vnr_flag = frame_data->vnr_pred_flag; + agc_stage_state.md.vnr_flag = float_s32_gt(frame_data->vnr_pred, f32_to_float_s32(VNR_AGC_THRESHOLD)); + + agc_process_frame( &agc_stage_state.state, @@ -161,6 +173,15 @@ static void stage_agc(frame_data_t *frame_data) &agc_stage_state.md); memcpy(frame_data->samples, agc_output, appconfAUDIO_PIPELINE_FRAME_ADVANCE * sizeof(int32_t)); #endif +#if appconfLOW_POWER_ENABLED + // Compute exponential moving average of frame energy + bfp_s32_t B; + bfp_s32_init(&B, &frame_data->samples[0][0], -31, appconfAUDIO_PIPELINE_FRAME_ADVANCE, 1); + float_s32_t energy = float_s64_to_float_s32(bfp_s32_energy(&B)); + frame_data->ema_energy = float_s32_ema(frame_data->ema_energy, energy, ema_energy_alpha_q30); +#else + frame_data->ema_energy = f32_to_float_s32(0.0); +#endif } static void initialize_pipeline_stages(void) { @@ -171,8 +192,8 @@ static void initialize_pipeline_stages(void) { vnr_feature_state_init(&vnr_pred_state->feature_state[1]); vnr_inference_init(); vnr_pred_state->pred_alpha_q30 = Q30(0.97); - vnr_pred_state->input_vnr_pred = float_to_float_s32(0.5); - vnr_pred_state->output_vnr_pred = float_to_float_s32(0.5); + vnr_pred_state->input_vnr_pred = f32_to_float_s32(0.5); + vnr_pred_state->output_vnr_pred = f32_to_float_s32(0.5); ns_init(&ns_stage_state.state); @@ -201,6 +222,7 @@ void audio_pipeline_init( initialize_pipeline_stages(); + power_app_data = (power_data_t *) output_app_data; generic_pipeline_init((pipeline_input_t)audio_pipeline_input_i, (pipeline_output_t)audio_pipeline_output_i, input_app_data, diff --git a/examples/ffd/src/gpio_ctrl/leds.c b/examples/ffd/src/gpio_ctrl/leds.c index 7ac4e635..16c72c52 100644 --- a/examples/ffd/src/gpio_ctrl/leds.c +++ b/examples/ffd/src/gpio_ctrl/leds.c @@ -14,77 +14,245 @@ #include "app_conf.h" #include "gpio_ctrl/leds.h" #include "platform/driver_instances.h" +#include "power/power_control.h" -#define LED_BLINK_DELAY (500 / portTICK_PERIOD_MS) -rtos_gpio_port_id_t gpo_port = 0; +#if ON_TILE(0) + +#define LED_BLINK_DELAY (500 / portTICK_PERIOD_MS) +#define TASK_NOTIF_MASK_HBEAT_TIMER 0x00000010 +#define TASK_NOTIF_MASK_ASLEEP 0x00000020 +#define TASK_NOTIF_MASK_AWAKE 0x00000040 +#define TASK_NOTIF_MASK_WAITING 0x00000080 +#define TASK_NOTIF_MASK_LISTEN 0x00000100 + +typedef enum led_state { + LED_OFF, + LED_ON, + LED_TOGGLE +} led_state_t; + +typedef enum led_color { + LED_GREEN, + LED_RED, + LED_YELLOW, /* RED + GREEN */ +} led_color_t; #if XK_VOICE_L71 -#define gpo_setup() { \ - gpo_port = rtos_gpio_port(PORT_GPO); \ - rtos_gpio_port_enable(gpio_ctx_t0, gpo_port); \ -} -#define green_led_on() { \ - uint32_t val = rtos_gpio_port_in(gpio_ctx_t0, gpo_port); \ - rtos_gpio_port_out(gpio_ctx_t0, gpo_port, val &= ~(1<<5)); \ -} -#define green_led_off() { \ - uint32_t val = rtos_gpio_port_in(gpio_ctx_t0, gpo_port); \ - rtos_gpio_port_out(gpio_ctx_t0, gpo_port, val |= (1<<5)); \ -} -#define red_led_on() { \ - uint32_t val = rtos_gpio_port_in(gpio_ctx_t0, gpo_port); \ - rtos_gpio_port_out(gpio_ctx_t0, gpo_port, val &= ~(1<<4)); \ -} -#define red_led_off() { \ - uint32_t val = rtos_gpio_port_in(gpio_ctx_t0, gpo_port); \ - rtos_gpio_port_out(gpio_ctx_t0, gpo_port, val |= (1<<4)); \ +#define LED_GREEN_MASK (1<<5) +#define LED_RED_MASK (1<<4) +#define LED_YELLOW_MASK (LED_GREEN_MASK | LED_RED_MASK) + +#define gpo_setup() { \ + gpo_port = rtos_gpio_port(PORT_GPO); \ + rtos_gpio_port_enable(gpio_ctx_t0, gpo_port); \ } #elif XCOREAI_EXPLORER /* LED 0 is "green" * LED 1 is "red" */ +#define LED_GREEN_MASK (1<<0) +#define LED_RED_MASK (1<<1) +#define LED_YELLOW_MASK (LED_GREEN_MASK | LED_RED_MASK) + #define gpo_setup() { \ gpo_port = rtos_gpio_port(PORT_LEDS); \ rtos_gpio_port_enable(gpio_ctx_t0, gpo_port); \ } -#define green_led_on() { \ - uint32_t val = rtos_gpio_port_in(gpio_ctx_t0, gpo_port); \ - rtos_gpio_port_out(gpio_ctx_t0, gpo_port, val |= (1<<0)); \ +#endif + +// Exclusively turn on the green LED. If the red LED is on, turn it off. +#define green_led_xon() { \ + uint32_t val = rtos_gpio_port_in(gpio_ctx_t0, gpo_port); \ + val = (val & ~LED_GREEN_MASK) | LED_RED_MASK; \ + rtos_gpio_port_out(gpio_ctx_t0, gpo_port, val); \ } -#define green_led_off() { \ - uint32_t val = rtos_gpio_port_in(gpio_ctx_t0, gpo_port); \ - rtos_gpio_port_out(gpio_ctx_t0, gpo_port, val &= ~(1<<0)); \ +#define green_led_on() { \ + uint32_t val = rtos_gpio_port_in(gpio_ctx_t0, gpo_port); \ + rtos_gpio_port_out(gpio_ctx_t0, gpo_port, val &= ~LED_GREEN_MASK); \ } -#define red_led_on() { \ - uint32_t val = rtos_gpio_port_in(gpio_ctx_t0, gpo_port); \ - rtos_gpio_port_out(gpio_ctx_t0, gpo_port, val |= (1<<1)); \ +#define green_led_off() { \ + uint32_t val = rtos_gpio_port_in(gpio_ctx_t0, gpo_port); \ + rtos_gpio_port_out(gpio_ctx_t0, gpo_port, val |= LED_GREEN_MASK); \ } -#define red_led_off() { \ - uint32_t val = rtos_gpio_port_in(gpio_ctx_t0, gpo_port); \ - rtos_gpio_port_out(gpio_ctx_t0, gpo_port, val &= ~(1<<1)); \ +#define green_led_toggle() { \ + uint32_t val = rtos_gpio_port_in(gpio_ctx_t0, gpo_port); \ + rtos_gpio_port_out(gpio_ctx_t0, gpo_port, val ^= LED_GREEN_MASK); \ } -#endif -void led_heartbeat_task(void *args) +// Exclusively toggle the green LED. If the red LED is on, turn it off. +#define green_led_xtoggle() { \ + uint32_t val = rtos_gpio_port_in(gpio_ctx_t0, gpo_port); \ + val = (val ^ LED_GREEN_MASK) | LED_RED_MASK; \ + rtos_gpio_port_out(gpio_ctx_t0, gpo_port, val); \ +} + +// Exclusively turn on the red LED. If the green LED is on, turn it off. +#define red_led_xon() { \ + uint32_t val = rtos_gpio_port_in(gpio_ctx_t0, gpo_port); \ + val = (val & ~LED_RED_MASK) | LED_GREEN_MASK; \ + rtos_gpio_port_out(gpio_ctx_t0, gpo_port, val); \ +} +#define red_led_on() { \ + uint32_t val = rtos_gpio_port_in(gpio_ctx_t0, gpo_port); \ + rtos_gpio_port_out(gpio_ctx_t0, gpo_port, val &= ~LED_RED_MASK); \ +} +#define red_led_off() { \ + uint32_t val = rtos_gpio_port_in(gpio_ctx_t0, gpo_port); \ + rtos_gpio_port_out(gpio_ctx_t0, gpo_port, val |= LED_RED_MASK); \ +} +#define red_led_toggle() { \ + uint32_t val = rtos_gpio_port_in(gpio_ctx_t0, gpo_port); \ + rtos_gpio_port_out(gpio_ctx_t0, gpo_port, val ^= LED_RED_MASK); \ +} + +// Exclusively toggle the red LED. If the green LED is on, turn it off. +#define red_led_xtoggle() { \ + uint32_t val = rtos_gpio_port_in(gpio_ctx_t0, gpo_port); \ + val = (val ^ LED_RED_MASK) | LED_GREEN_MASK; \ + rtos_gpio_port_out(gpio_ctx_t0, gpo_port, val); \ +} + +#define yellow_led_on() { \ + uint32_t val = rtos_gpio_port_in(gpio_ctx_t0, gpo_port); \ + rtos_gpio_port_out(gpio_ctx_t0, gpo_port, val &= ~LED_YELLOW_MASK); \ +} +#define yellow_led_off() { \ + uint32_t val = rtos_gpio_port_in(gpio_ctx_t0, gpo_port); \ + rtos_gpio_port_out(gpio_ctx_t0, gpo_port, val |= LED_YELLOW_MASK); \ +} +#define yellow_led_toggle() { \ + uint32_t val = rtos_gpio_port_in(gpio_ctx_t0, gpo_port); \ + val = (val & LED_YELLOW_MASK) ? (val & ~LED_YELLOW_MASK) : (val | LED_YELLOW_MASK); \ + rtos_gpio_port_out(gpio_ctx_t0, gpo_port, val); \ +} + +static TaskHandle_t ctx_led_task = NULL; + +static void hbeat_tmr_callback(TimerHandle_t pxTimer) +{ + xTaskNotify(ctx_led_task, TASK_NOTIF_MASK_HBEAT_TIMER, eSetBits); +} + +static void led_task(void *args) { + const uint32_t bits_to_clear_on_entry = 0x00000000UL; + const uint32_t bits_to_clear_on_exit = 0xFFFFFFFFUL; + rtos_gpio_port_id_t gpo_port = 0; + led_state_t state = LED_TOGGLE; + led_color_t color = LED_GREEN; + uint32_t notif_value; + TimerHandle_t tmr_hbeat = xTimerCreate("tmr_hbeat", + pdMS_TO_TICKS(LED_BLINK_DELAY), pdTRUE, NULL, + hbeat_tmr_callback); + gpo_setup(); + xTimerStart(tmr_hbeat, 0); while (1) { - green_led_on(); - vTaskDelay(LED_BLINK_DELAY); - green_led_off(); - vTaskDelay(LED_BLINK_DELAY); - } + xTaskNotifyWait(bits_to_clear_on_entry, bits_to_clear_on_exit, + ¬if_value, portMAX_DELAY); + + /* + * Process the notification event data. + */ + if (notif_value & TASK_NOTIF_MASK_ASLEEP) { + // Below VNR threshold; in low power mode. + xTimerStop(tmr_hbeat, 0); + color = LED_RED; + state = LED_ON; + } else if ((notif_value & TASK_NOTIF_MASK_AWAKE) || + (notif_value & TASK_NOTIF_MASK_WAITING)) { + // Above VNR threshold; normal operation (heartbeat). + color = LED_GREEN; + state = LED_TOGGLE; + xTimerStart(tmr_hbeat, 0); + } else if (notif_value & TASK_NOTIF_MASK_LISTEN) { + // Above VNR threshold; wake word detected, listening for command. + xTimerStop(tmr_hbeat, 0); + color = LED_YELLOW; + state = LED_ON; + } + + /* + * Update the LED indication. + */ + if (color == LED_GREEN) { + switch (state) { + case LED_ON: + green_led_xon(); + break; + case LED_TOGGLE: + green_led_xtoggle(); + break; + case LED_OFF: + default: + green_led_off(); + break; + } + } else if (color == LED_RED) { + switch (state) { + case LED_ON: + red_led_xon(); + break; + case LED_TOGGLE: + red_led_xtoggle(); + break; + case LED_OFF: + default: + red_led_off(); + break; + } + } else if (color == LED_YELLOW) { + switch (state) { + case LED_ON: + yellow_led_on(); + break; + case LED_TOGGLE: + yellow_led_toggle(); + break; + case LED_OFF: + default: + yellow_led_off(); + break; + } + } + if ((notif_value & TASK_NOTIF_MASK_AWAKE) || + (notif_value & TASK_NOTIF_MASK_ASLEEP)) { + power_control_req_complete(); + } + } } -void led_heartbeat_create(unsigned priority, void *args) +void led_task_create(unsigned priority, void *args) { - xTaskCreate((TaskFunction_t) led_heartbeat_task, - "inf_led_heartbeat", - RTOS_THREAD_STACK_SIZE(led_heartbeat_task), + xTaskCreate((TaskFunction_t) led_task, + "led_task", + RTOS_THREAD_STACK_SIZE(led_task), args, priority, - NULL); + &ctx_led_task); } + +void led_indicate_asleep(void) +{ + xTaskNotify(ctx_led_task, TASK_NOTIF_MASK_ASLEEP, eSetBits); +} + +void led_indicate_awake(void) +{ + xTaskNotify(ctx_led_task, TASK_NOTIF_MASK_AWAKE, eSetBits); +} + +void led_indicate_waiting(void) +{ + xTaskNotify(ctx_led_task, TASK_NOTIF_MASK_WAITING, eSetBits); +} + +void led_indicate_listening(void) +{ + xTaskNotify(ctx_led_task, TASK_NOTIF_MASK_LISTEN, eSetBits); +} + +#endif /* ON_TILE(0) */ \ No newline at end of file diff --git a/examples/ffd/src/gpio_ctrl/leds.h b/examples/ffd/src/gpio_ctrl/leds.h index c99f70b2..f3095389 100644 --- a/examples/ffd/src/gpio_ctrl/leds.h +++ b/examples/ffd/src/gpio_ctrl/leds.h @@ -4,7 +4,35 @@ #ifndef LEDS_H_ #define LEDS_H_ +/** + * @brief Create the LED task. + * + * @param priority The priority of the task. + * @param args The arguments to send to the task. + */ +void led_task_create(unsigned priority, void *args); -void led_heartbeat_create(unsigned priority, void *args); +/** + * @brief To be called only by the power control task. Indicates that the + * device is entering low power mode. + */ +void led_indicate_asleep(void); + +/** + * @brief To be called only by the power control task. Indicates that the device + * has exited low power mode. + */ +void led_indicate_awake(void); + +/** + * @brief To be called when waiting for the wake word or after a timeout period + * waiting for a spoken command. + */ +void led_indicate_waiting(void); + +/** + * @brief To be called when listening for a command. + */ +void led_indicate_listening(void); #endif /* LEDS_H_ */ \ No newline at end of file diff --git a/examples/ffd/src/main.c b/examples/ffd/src/main.c index 2913617b..6482598a 100644 --- a/examples/ffd/src/main.c +++ b/examples/ffd/src/main.c @@ -15,9 +15,11 @@ /* Library headers */ #include "rtos_printf.h" +#include "rtos_clock_control.h" /* App headers */ #include "app_conf.h" +#include "platform/platform_conf.h" #include "platform/platform_init.h" #include "platform/driver_instances.h" #include "audio_pipeline/audio_pipeline.h" @@ -29,10 +31,18 @@ #include "xcore_device_memory.h" #include "ssd1306_rtos_support.h" #include "intent_handler/intent_handler.h" +#include "power/power_state.h" +#include "power/power_status.h" +#include "power/power_control.h" +#include "power/low_power_audio_buffer.h" extern void startup_task(void *arg); extern void tile_common_init(chanend_t c); +#if ON_TILE(AUDIO_PIPELINE_TILE_NO) +static power_data_t wakeup_app_data; +#endif + //void uart_write(char data) {} //API for Wanson's Debug __attribute__((weak)) @@ -71,12 +81,38 @@ int audio_pipeline_output(void *output_app_data, size_t ch_count, size_t frame_count) { - (void) output_app_data; +#if ON_TILE(AUDIO_PIPELINE_TILE_NO) && (appconfLOW_POWER_ENABLED || appconfINFERENCE_ENABLED) + power_state_t power_state = POWER_STATE_FULL; +#endif -#if appconfINFERENCE_ENABLED - inference_engine_sample_push((int32_t *)output_audio_frames, frame_count); +#if ON_TILE(AUDIO_PIPELINE_TILE_NO) && appconfLOW_POWER_ENABLED + power_state = power_state_data_add((power_data_t *)output_app_data); #endif +#if ON_TILE(AUDIO_PIPELINE_TILE_NO) && appconfINFERENCE_ENABLED + if (power_state == POWER_STATE_FULL) { +#if LOW_POWER_AUDIO_BUFFER_ENABLED + const uint32_t max_dequeue_frames = 1; + const uint32_t max_dequeued_samples = (max_dequeue_frames * appconfAUDIO_PIPELINE_FRAME_ADVANCE); + + if (power_control_state_get() != POWER_STATE_FULL) { + low_power_audio_buffer_enqueue((int32_t *)output_audio_frames, frame_count); + } else if (low_power_audio_buffer_dequeue(max_dequeue_frames) == max_dequeued_samples) { + // Max data has been dequeued, enqueue the newest data. + low_power_audio_buffer_enqueue((int32_t *)output_audio_frames, frame_count); + } else // More data can be sent. +#endif // LOW_POWER_AUDIO_BUFFER_ENABLED + { + inference_engine_sample_push((int32_t *)output_audio_frames, frame_count); + } + } +#if LOW_POWER_AUDIO_BUFFER_ENABLED + else { + low_power_audio_buffer_enqueue((int32_t *)output_audio_frames, frame_count); + } +#endif // LOW_POWER_AUDIO_BUFFER_ENABLED +#endif // ON_TILE(AUDIO_PIPELINE_TILE_NO) && appconfINFERENCE_ENABLED + return AUDIO_PIPELINE_FREE_FRAME; } @@ -87,6 +123,14 @@ void vApplicationMallocFailedHook(void) for(;;); } +// static void mem_analysis(void) +// { +// for (;;) { +// rtos_printf("Tile[%d]:\n\tMinimum heap free: %d\n\tCurrent heap free: %d\n", THIS_XCORE_TILE, xPortGetMinimumEverFreeHeapSize(), xPortGetFreeHeapSize()); +// vTaskDelay(pdMS_TO_TICKS(5000)); +// } +// } + __attribute__((weak)) void startup_task(void *arg) { @@ -111,6 +155,14 @@ void startup_task(void *arg) inference_engine_create(appconfINFERENCE_MODEL_RUNNER_TASK_PRIORITY, q_intent); #endif +#if ON_TILE(0) + led_task_create(appconfLED_TASK_PRIORITY, NULL); +#endif + +#if appconfLOW_POWER_ENABLED + power_control_task_create(appconfPOWER_CONTROL_TASK_PRIORITY, NULL); +#endif + #if ON_TILE(AUDIO_PIPELINE_TILE_NO) #if appconfINFERENCE_ENABLED // Wait until the Wanson engine is initialized before we start the @@ -121,13 +173,19 @@ void startup_task(void *arg) rtos_intertile_rx_data(intertile_ctx, &ret, sizeof(ret)); } #endif - audio_pipeline_init(NULL, NULL); + audio_pipeline_init(NULL, &wakeup_app_data); +#if appconfLOW_POWER_ENABLED + power_state_init(); +#endif #endif -#if ON_TILE(0) - led_heartbeat_create(appconfLED_HEARTBEAT_TASK_PRIORITY, NULL); +#if appconfLOW_POWER_ENABLED && ON_TILE(AUDIO_PIPELINE_TILE_NO) + set_local_tile_processor_clk_div(1); + enable_local_tile_processor_clock_divider(); + set_local_tile_processor_clk_div(appconfLOW_POWER_CONTROL_TILE_CLK_DIV); #endif + //mem_analysis(); vTaskSuspend(NULL); while(1){;} /* Trap */ } diff --git a/examples/ffd/src/power/low_power_audio_buffer.c b/examples/ffd/src/power/low_power_audio_buffer.c new file mode 100644 index 00000000..eaae0fb3 --- /dev/null +++ b/examples/ffd/src/power/low_power_audio_buffer.c @@ -0,0 +1,140 @@ +// Copyright (c) 2022 XMOS LIMITED. This Software is subject to the terms of the +// XMOS Public License: Version 1 + +/* System headers */ +#include +#include +#include + +/* App headers */ +#include "app_conf.h" +#include "low_power_audio_buffer.h" +#include "inference_engine.h" + +#if LOW_POWER_AUDIO_BUFFER_ENABLED + +typedef struct ring_buffer +{ + int32_t * const buf; + const uint32_t size; + char *set_ptr; + char *get_ptr; + uint32_t count; + uint8_t full; + uint8_t empty; +} ring_buffer_t; + +int32_t sample_buf[appconfAUDIO_PIPELINE_BUFFER_NUM_FRAMES * appconfAUDIO_PIPELINE_FRAME_ADVANCE] = {0}; + +/* Ring buffer to hold onto the latest audio samples while in low power mode. + * This serves to capture the onset of speech that meet or exceed the trigger + * threshold to exit low power and to provide a more complete ASR payload to + * the inference engine. */ +ring_buffer_t ring_buf = { + sample_buf, + sizeof(sample_buf), + (char *)sample_buf, + (char *)sample_buf, + 0, + 0, + 1 +}; + +#endif // LOW_POWER_AUDIO_BUFFER_ENABLED + +void low_power_audio_buffer_enqueue(int32_t *frames, size_t num_frames) +{ +#if LOW_POWER_AUDIO_BUFFER_ENABLED + const uint32_t tail_addr = ((uint32_t)ring_buf.buf + (ring_buf.size * sizeof(int32_t))); + + size_t total_bytes = num_frames * sizeof(int32_t); + size_t tail_bytes = ring_buf.size - ((uint32_t)ring_buf.set_ptr - (uint32_t)ring_buf.buf); + + if (tail_bytes > total_bytes) + tail_bytes = total_bytes; + + uint32_t head_bytes = total_bytes - tail_bytes; + + memcpy(ring_buf.set_ptr, (char *)frames, tail_bytes); + memcpy(ring_buf.buf, (char *)frames + tail_bytes, head_bytes); + + if (head_bytes) + ring_buf.set_ptr = (char *)ring_buf.buf + head_bytes; + else + ring_buf.set_ptr += tail_bytes; + + if ((uint32_t)ring_buf.set_ptr >= tail_addr) + ring_buf.set_ptr = (char *)ring_buf.buf; + + ring_buf.full = ((ring_buf.count + num_frames) >= ring_buf.size); + + if (ring_buf.full) { + ring_buf.get_ptr = ring_buf.set_ptr; + ring_buf.count = ring_buf.size; + } else { + ring_buf.count += num_frames; + } + + ring_buf.empty = (ring_buf.count == 0); +#endif // LOW_POWER_AUDIO_BUFFER_ENABLED +} + +uint32_t low_power_audio_buffer_dequeue(uint32_t num_frames) +{ + uint32_t ret = 0; +#if LOW_POWER_AUDIO_BUFFER_ENABLED + if ((ring_buf.count == 0) || (num_frames == 0)) { + // No data to dequeue. + return ret; + } + + ring_buf.full = 0; + + uint32_t bufferred_audio_bytes = ring_buf.count * sizeof(int32_t); + int32_t tail_bytes = ring_buf.size - ((uint32_t)ring_buf.get_ptr - (uint32_t)ring_buf.buf); + + int32_t samples_to_dequeue = + ((num_frames * appconfAUDIO_PIPELINE_FRAME_ADVANCE) > ring_buf.count) ? + ring_buf.count : + (num_frames * appconfAUDIO_PIPELINE_FRAME_ADVANCE); + + ring_buf.empty = ((ring_buf.count - samples_to_dequeue) == 0); + ret = (uint32_t)samples_to_dequeue; + + if ((uint32_t)tail_bytes > bufferred_audio_bytes) + tail_bytes = bufferred_audio_bytes; + + int32_t head_bytes = bufferred_audio_bytes - tail_bytes; + + while (tail_bytes > 0) { + if (samples_to_dequeue <= 0) + break; + + inference_engine_sample_push((int32_t *)ring_buf.get_ptr, appconfAUDIO_PIPELINE_FRAME_ADVANCE); + ring_buf.get_ptr += (appconfAUDIO_PIPELINE_FRAME_ADVANCE * sizeof(int32_t)); + tail_bytes -= (appconfAUDIO_PIPELINE_FRAME_ADVANCE * sizeof(int32_t)); + samples_to_dequeue -= appconfAUDIO_PIPELINE_FRAME_ADVANCE; + ring_buf.count -= appconfAUDIO_PIPELINE_FRAME_ADVANCE; + } + + if (head_bytes == 0) { + // No data left to dequeue, return now. + return ret; + } + + ring_buf.get_ptr = (char *)ring_buf.buf; + + while (head_bytes > 0) { + if (samples_to_dequeue <= 0) + break; + + inference_engine_sample_push((int32_t *)ring_buf.get_ptr, appconfAUDIO_PIPELINE_FRAME_ADVANCE); + ring_buf.get_ptr += (appconfAUDIO_PIPELINE_FRAME_ADVANCE * sizeof(int32_t)); + head_bytes -= (appconfAUDIO_PIPELINE_FRAME_ADVANCE * sizeof(int32_t)); + samples_to_dequeue -= appconfAUDIO_PIPELINE_FRAME_ADVANCE; + ring_buf.count -= appconfAUDIO_PIPELINE_FRAME_ADVANCE; + } +#endif // LOW_POWER_AUDIO_BUFFER_ENABLED + + return ret; +} diff --git a/examples/ffd/src/power/low_power_audio_buffer.h b/examples/ffd/src/power/low_power_audio_buffer.h new file mode 100644 index 00000000..dc33dfd1 --- /dev/null +++ b/examples/ffd/src/power/low_power_audio_buffer.h @@ -0,0 +1,38 @@ +// Copyright (c) 2022 XMOS LIMITED. This Software is subject to the terms of the +// XMOS Public License: Version 1 + +#ifndef LOW_POWER_AUDIO_BUFFER_H_ +#define LOW_POWER_AUDIO_BUFFER_H_ + + +/* System headers */ +#include +#include +#include + +/* App headers */ +#include "app_conf.h" + +#define LOW_POWER_AUDIO_BUFFER_ENABLED ( \ + appconfLOW_POWER_ENABLED && \ + appconfAUDIO_PIPELINE_BUFFER_ENABLED && \ + ON_TILE(AUDIO_PIPELINE_TILE_NO) ) + +/** + * Enqueue audio frames into a ring buffer. Oldest data will be overwritten. + * + * \param frames The point to the frames to enqueue. + * \param num_frames The number of frames to enqueue. + */ +void low_power_audio_buffer_enqueue(int32_t *frames, size_t num_frames); + +/** + * Dequeue audio frames out of a ring buffer. These frames are sent onward to + * the inference engine. + * + * \param num_frames The requested number of frames to dequeue. + * \return The number of samples actually dequeued from the buffer. + */ +uint32_t low_power_audio_buffer_dequeue(uint32_t num_frames); + +#endif // LOW_POWER_AUDIO_BUFFER_H_ diff --git a/examples/ffd/src/power/power_control.c b/examples/ffd/src/power/power_control.c new file mode 100644 index 00000000..048142dc --- /dev/null +++ b/examples/ffd/src/power/power_control.c @@ -0,0 +1,209 @@ +// Copyright (c) 2022 XMOS LIMITED. This Software is subject to the terms of the +// XMOS Public License: Version 1 + +/* System headers */ +#include +#include + +/* FreeRTOS headers */ +#include "FreeRTOS.h" +#include "task.h" + +/* Library headers */ +#include "rtos_macros.h" +#include "rtos_clock_control.h" + +/* App headers */ +#include "app_conf.h" +#include "platform/driver_instances.h" +#include "gpio_ctrl/leds.h" +#include "power/power_state.h" +#include "power/power_control.h" +#include "wanson_inf_eng.h" + +#define TASK_NOTIF_MASK_LP_ENTER 1 // Used by tile: POWER_CONTROL_TILE_NO +#define TASK_NOTIF_MASK_LP_EXIT 2 // Used by tile: POWER_CONTROL_TILE_NO +#define TASK_NOTIF_MASK_LP_IND_COMPLETE 4 // Used by tile: !POWER_CONTROL_TILE_NO + +static TaskHandle_t ctx_power_control_task = NULL; + +#if ON_TILE(POWER_CONTROL_TILE_NO) + +static power_state_t power_state = POWER_STATE_FULL; +static unsigned tile0_div; +static unsigned switch_div; + +#endif + +static void driver_control_lock(void) +{ +#if ON_TILE(POWER_CONTROL_TILE_NO) + rtos_osal_mutex_get(&gpio_ctx_t0->lock, RTOS_OSAL_WAIT_FOREVER); +#else + rtos_osal_mutex_get(&qspi_flash_ctx->mutex, RTOS_OSAL_WAIT_FOREVER); + rtos_osal_mutex_get(&i2c_master_ctx->lock, RTOS_OSAL_WAIT_FOREVER); + rtos_osal_mutex_get(&uart_tx_ctx->lock, RTOS_OSAL_WAIT_FOREVER); +#endif +} + +static void driver_control_unlock(void) +{ +#if ON_TILE(POWER_CONTROL_TILE_NO) + rtos_osal_mutex_put(&gpio_ctx_t0->lock); +#else + rtos_osal_mutex_put(&uart_tx_ctx->lock); + rtos_osal_mutex_put(&i2c_master_ctx->lock); + rtos_osal_mutex_put(&qspi_flash_ctx->mutex); +#endif +} + +#if ON_TILE(POWER_CONTROL_TILE_NO) + +static void low_power_clocks_enable(void) +{ + // Save clock divider config before apply low power configuration. + tile0_div = rtos_clock_control_get_processor_clk_div(cc_ctx_t0); + rtos_clock_control_set_processor_clk_div(cc_ctx_t0, appconfLOW_POWER_OTHER_TILE_CLK_DIV); + +#if (appconfLOW_POWER_SWITCH_CLK_DIV_ENABLE) + switch_div = rtos_clock_control_get_switch_clk_div(cc_ctx_t0); + rtos_clock_control_set_switch_clk_div(cc_ctx_t0, appconfLOW_POWER_SWITCH_CLK_DIV); +#endif +} + +static void low_power_clocks_disable(void) +{ + // Restore the original clock divider state(s). +#if (appconfLOW_POWER_ENABLE_SWITCH_CONTROL) + set_node_switch_clk_div(TILE_ID(0), switch_div); +#endif + set_tile_processor_clk_div(TILE_ID(0), tile0_div); +} + +#endif /* ON_TILE(POWER_CONTROL_TILE_NO) */ + +static void power_control_task(void *arg) +{ + const uint32_t bits_to_clear_on_entry = 0x00000000UL; + const uint32_t bits_to_clear_on_exit = 0xFFFFFFFFUL; + power_state_t requested_power_state = POWER_STATE_FULL; + uint32_t notif_value; + + while (1) { +#if ON_TILE(POWER_CONTROL_TILE_NO) + xTaskNotifyWait(bits_to_clear_on_entry, + bits_to_clear_on_exit, + ¬if_value, + portMAX_DELAY); + + if (notif_value & TASK_NOTIF_MASK_LP_EXIT) { + // Ignore the event if already exited low power mode. + if (requested_power_state == POWER_STATE_FULL) + continue; + + requested_power_state = POWER_STATE_FULL; + debug_printf("Exiting low power...\n"); + low_power_clocks_disable(); + driver_control_unlock(); + debug_printf("Exited low power.\n"); + } else if (notif_value & TASK_NOTIF_MASK_LP_ENTER) { + // Ignore the event if already enterred low power mode. + if (requested_power_state == POWER_STATE_LOW) + continue; + + requested_power_state = POWER_STATE_LOW; + power_state = requested_power_state; + } + + // Send the requested power state to the other tile. + rtos_intertile_tx(intertile_ctx, + appconfPOWER_CONTROL_PORT, + &requested_power_state, + sizeof(requested_power_state)); + + // Wait for a response form other tile (the value is not used/important). + power_state_t power_state_response; + size_t len_rx = rtos_intertile_rx_len(intertile_ctx, appconfPOWER_CONTROL_PORT, RTOS_OSAL_WAIT_FOREVER); + configASSERT(len_rx == sizeof(power_state_response)); + rtos_intertile_rx_data(intertile_ctx, &power_state_response, sizeof(power_state_response)); + + if (requested_power_state == POWER_STATE_LOW) { + debug_printf("Entering low power...\n"); + driver_control_lock(); + low_power_clocks_enable(); + debug_printf("Entered low power.\n"); + } else if (requested_power_state == POWER_STATE_FULL) { + power_state = requested_power_state; + } +#else + // Wait for other tile to send the requested power state. + size_t len_rx = rtos_intertile_rx_len(intertile_ctx, appconfPOWER_CONTROL_PORT, RTOS_OSAL_WAIT_FOREVER); + configASSERT(len_rx == sizeof(requested_power_state)); + rtos_intertile_rx_data(intertile_ctx, &requested_power_state, sizeof(requested_power_state)); + + if (requested_power_state == POWER_STATE_FULL) { + driver_control_unlock(); + led_indicate_awake(); + + /* Wait for a notification, signaling that the LED indication has + * been applied. */ + xTaskNotifyWait(bits_to_clear_on_entry, + bits_to_clear_on_exit, + ¬if_value, + portMAX_DELAY); + } else { + led_indicate_asleep(); + + /* Wait for a notification, signaling that the LED indication has + * been applied and that the tile is ready to be set to low power + * mode by the other tile. */ + xTaskNotifyWait(bits_to_clear_on_entry, + bits_to_clear_on_exit, + ¬if_value, + portMAX_DELAY); + driver_control_lock(); + + wanson_engine_stream_buf_reset(); + } + + rtos_intertile_tx(intertile_ctx, + appconfPOWER_CONTROL_PORT, + &requested_power_state, + sizeof(requested_power_state)); +#endif /* ON_TILE(POWER_CONTROL_TILE_NO) */ + } +} + +void power_control_task_create(unsigned priority, void *args) +{ + xTaskCreate((TaskFunction_t)power_control_task, + RTOS_STRINGIFY(power_control_task), + RTOS_THREAD_STACK_SIZE(power_control_task), args, + priority, &ctx_power_control_task); +} + +#if ON_TILE(POWER_CONTROL_TILE_NO) + +void power_control_enter_low_power(void) +{ + xTaskNotify(ctx_power_control_task, TASK_NOTIF_MASK_LP_ENTER, eSetBits); +} + +void power_control_exit_low_power(void) +{ + xTaskNotify(ctx_power_control_task, TASK_NOTIF_MASK_LP_EXIT, eSetBits); +} + +power_state_t power_control_state_get(void) +{ + return power_state; +} + +#else + +void power_control_req_complete(void) +{ + xTaskNotify(ctx_power_control_task, TASK_NOTIF_MASK_LP_IND_COMPLETE, eSetBits); +} + +#endif /* ON_TILE(POWER_CONTROL_TILE_NO) */ diff --git a/examples/ffd/src/power/power_control.h b/examples/ffd/src/power/power_control.h new file mode 100644 index 00000000..5db9003e --- /dev/null +++ b/examples/ffd/src/power/power_control.h @@ -0,0 +1,51 @@ +// Copyright 2022 XMOS LIMITED. +// This Software is subject to the terms of the XMOS Public Licence: Version 1. + +#ifndef POWER_CONTROL_H_ +#define POWER_CONTROL_H_ + +#include "app_conf.h" +#include "power_state.h" + +// Specifies the tile that is controlling the low power mode. +#define POWER_CONTROL_TILE_NO AUDIO_PIPELINE_TILE_NO + +/** + * @brief Initialize the power control task. + * + * @param priority The priority of the task. + * @param args The arguments to send to the task. + */ +void power_control_task_create(unsigned priority, void *args); + +#if ON_TILE(POWER_CONTROL_TILE_NO) + +/** + * @brief Notify that the power control task should enter the low power state. + */ +void power_control_enter_low_power(void); + +/** + * @brief Notify that the power control task should exit the low power state. + */ +void power_control_exit_low_power(void); + +/** + * @brief Get the power control state. + * + * @returns The applied power state. + */ +power_state_t power_control_state_get(void); + +#else + +/** + * @brief Notify that the requested power control state on tile[0] has completed + * This serves control when tile[1] is allowed to commence with applying + * low power mode. + */ +void power_control_req_complete(void); + +#endif + +#endif /* POWER_CONTROL_H_ */ \ No newline at end of file diff --git a/examples/ffd/src/power/power_state.c b/examples/ffd/src/power/power_state.c new file mode 100644 index 00000000..a89d72eb --- /dev/null +++ b/examples/ffd/src/power/power_state.c @@ -0,0 +1,62 @@ +// Copyright (c) 2022 XMOS LIMITED. This Software is subject to the terms of the +// XMOS Public License: Version 1 + +/* STD headers */ +#include +#include + +/* FreeRTOS headers */ +#include "FreeRTOS.h" +#include "task.h" + +/* App headers */ +#include "app_conf.h" +#include "platform/driver_instances.h" +#include "power/power_state.h" +#include "power/power_control.h" + +#if ON_TILE(POWER_CONTROL_TILE_NO) + +static TimerHandle_t power_state_timer; +static power_state_t power_state = POWER_STATE_FULL; + +void vWakeupStateCallback(TimerHandle_t pxTimer) +{ + power_state_set(POWER_STATE_LOW); + xTimerStop(power_state_timer, 0); +} + +void power_state_set(power_state_t state) { + if (state != power_state) { + power_state = state; + + if (power_state == POWER_STATE_FULL) + power_control_exit_low_power(); + else + power_control_enter_low_power(); + } +} + +void power_state_init() { + power_state_timer = xTimerCreate( + "disp_reset", + pdMS_TO_TICKS(appconfPOWER_FULL_HOLD_DURATION), + pdFALSE, + NULL, + vWakeupStateCallback); + + xTimerStart(power_state_timer, 0); +} + +power_state_t power_state_data_add(power_data_t *data) { + if ((data->ema_energy >= appconfPOWER_HIGH_ENERGY_THRESHOLD) || + ((data->ema_energy >= appconfPOWER_LOW_ENERGY_THRESHOLD) && + (data->vnr_pred >= appconfPOWER_VNR_THRESHOLD))) { + power_state_set(POWER_STATE_FULL); + xTimerReset(power_state_timer, 0); + } + + return power_state; +} + +#endif // ON_TILE(POWER_CONTROL_TILE_NO) \ No newline at end of file diff --git a/examples/ffd/src/power/power_state.h b/examples/ffd/src/power/power_state.h new file mode 100644 index 00000000..789d09a4 --- /dev/null +++ b/examples/ffd/src/power/power_state.h @@ -0,0 +1,21 @@ +// Copyright (c) 2022 XMOS LIMITED. This Software is subject to the terms of the +// XMOS Public License: Version 1 + +#ifndef POWER_STATE_H_ +#define POWER_STATE_H_ + +typedef struct { + float vnr_pred; + float ema_energy; +} power_data_t; + +typedef enum power_state { + POWER_STATE_LOW, + POWER_STATE_FULL +} power_state_t; + +void power_state_init(); +void power_state_set(power_state_t state); +power_state_t power_state_data_add(power_data_t *data); + +#endif /* POWER_STATE_H_ */ diff --git a/examples/ffd/src/power/power_status.c b/examples/ffd/src/power/power_status.c new file mode 100644 index 00000000..17ffa01d --- /dev/null +++ b/examples/ffd/src/power/power_status.c @@ -0,0 +1,55 @@ +// Copyright (c) 2022 XMOS LIMITED. This Software is subject to the terms of the +// XMOS Public License: Version 1 + +/* STD headers */ +#include +#include +#include + +/* FreeRTOS headers */ +#include "FreeRTOS.h" +#include "task.h" +#include "queue.h" + +/* App headers */ +#include "app_conf.h" +#include "ssd1306_rtos_support.h" +#include "power/power_state.h" +#include "power/power_status.h" + +static void proc_power_status(void *args) { + + while (1) { + power_state_t power_state = 0; + size_t len_rx = 0; + + len_rx = rtos_intertile_rx_len(intertile_ctx, appconfPOWER_STATE_PORT, RTOS_OSAL_WAIT_FOREVER); + configASSERT(len_rx == sizeof(power_state)); + + rtos_intertile_rx_data(intertile_ctx, &power_state, sizeof(power_state)); + if (power_state == POWER_STATE_LOW) { +#if appconfSSD1306_DISPLAY_ENABLED + ssd1306_display_ascii_to_bitmap("Low power\0"); +#endif + rtos_printf("POWER_MODE: %d, Low\n", power_state); + + } else if (power_state == POWER_STATE_FULL) { +#if appconfSSD1306_DISPLAY_ENABLED + ssd1306_display_ascii_to_bitmap("Full power\0"); +#endif + rtos_printf("POWER_MODE: %d, Full\n", power_state); + } + } +} + +int32_t power_status_create(uint32_t priority, void *args) +{ + xTaskCreate((TaskFunction_t)proc_power_status, + "proc_power_status", + RTOS_THREAD_STACK_SIZE(proc_power_status), + args, + priority, + NULL); + + return 0; +} diff --git a/examples/ffd/src/power/power_status.h b/examples/ffd/src/power/power_status.h new file mode 100644 index 00000000..699c99f3 --- /dev/null +++ b/examples/ffd/src/power/power_status.h @@ -0,0 +1,9 @@ +// Copyright (c) 2022 XMOS LIMITED. This Software is subject to the terms of the +// XMOS Public License: Version 1 + +#ifndef POWER_STATUS_H_ +#define POWER_STATUS_H_ + +int32_t power_status_create(uint32_t priority, void *args); + +#endif /* POWER_STATUS_H_ */ diff --git a/examples/ffd/src/rtos_conf/FreeRTOSConfig.h b/examples/ffd/src/rtos_conf/FreeRTOSConfig.h index 6fb5f634..eaefdcaa 100644 --- a/examples/ffd/src/rtos_conf/FreeRTOSConfig.h +++ b/examples/ffd/src/rtos_conf/FreeRTOSConfig.h @@ -44,10 +44,10 @@ your application. */ #define configSUPPORT_STATIC_ALLOCATION 0 #define configSUPPORT_DYNAMIC_ALLOCATION 1 #if ON_TILE(0) -#define configTOTAL_HEAP_SIZE 64*1024 +#define configTOTAL_HEAP_SIZE (84 * 1024) #endif #if ON_TILE(1) -#define configTOTAL_HEAP_SIZE 64*1024 +#define configTOTAL_HEAP_SIZE (90 * 1024) #endif #define configAPPLICATION_ALLOCATED_HEAP 0 @@ -79,7 +79,9 @@ your application. */ #define configASSERT(x) xassert(x) /* Define to enable debug_printf() */ +#ifndef configENABLE_DEBUG_PRINTF #define configENABLE_DEBUG_PRINTF 1 +#endif /* Define to map sprintf and snprintf to the * lite versions in lib_rtos_support */ diff --git a/examples/stlp/audio_pipeline/audio_pipeline.cmake b/examples/stlp/audio_pipeline/audio_pipeline.cmake index c3710fe7..4d2e718d 100644 --- a/examples/stlp/audio_pipeline/audio_pipeline.cmake +++ b/examples/stlp/audio_pipeline/audio_pipeline.cmake @@ -1,3 +1,32 @@ +## Create custom stlp audiopipeline +add_library(xcore_sdk_app_stlp_audio_pipeline_fixed_delay_aec_2x_2y_no_comms INTERFACE) +target_sources(xcore_sdk_app_stlp_audio_pipeline_fixed_delay_aec_2x_2y_no_comms + INTERFACE + ${CMAKE_CURRENT_LIST_DIR}/src/fixed_delay/audio_pipeline_t0.c + ${CMAKE_CURRENT_LIST_DIR}/src/fixed_delay/audio_pipeline_t1.c + ${CMAKE_CURRENT_LIST_DIR}/src/fixed_delay/aec/aec_process_frame_1thread.c +) +target_include_directories(xcore_sdk_app_stlp_audio_pipeline_fixed_delay_aec_2x_2y_no_comms + INTERFACE + ${CMAKE_CURRENT_LIST_DIR}/api + ${CMAKE_CURRENT_LIST_DIR}/src/fixed_delay +) +target_link_libraries(xcore_sdk_app_stlp_audio_pipeline_fixed_delay_aec_2x_2y_no_comms + INTERFACE + core::general + rtos::freertos + rtos::sw_services::generic_pipeline + fwk_voice::aec + fwk_voice::agc + fwk_voice::ic + fwk_voice::ns + fwk_voice::vnr::features + fwk_voice::vnr::inference +) + +## Create an alias +add_library(sln_voice::app::stlp::ap::fixed_delay ALIAS xcore_sdk_app_stlp_audio_pipeline_fixed_delay_aec_2x_2y_no_comms) + ## Create custom stlp audiopipeline add_library(sln_voice_app_stlp_audio_pipeline_adec_aec_2x_2y_no_comms INTERFACE) @@ -33,9 +62,6 @@ target_link_libraries(sln_voice_app_stlp_audio_pipeline_adec_aec_2x_2y_no_comms ## Create an alias add_library(sln_voice::app::stlp::ap::adec ALIAS sln_voice_app_stlp_audio_pipeline_adec_aec_2x_2y_no_comms) - - - ## Create custom stlp audiopipeline add_library(sln_voice_app_stlp_audio_pipeline_adec_aec_2x_2y_no_comms_altarch INTERFACE) target_sources(sln_voice_app_stlp_audio_pipeline_adec_aec_2x_2y_no_comms_altarch diff --git a/examples/stlp/audio_pipeline/src/adec/aec/aec_memory_pool.h b/examples/stlp/audio_pipeline/src/adec/aec/aec_memory_pool.h index 87b1662c..f19e2c1f 100644 --- a/examples/stlp/audio_pipeline/src/adec/aec/aec_memory_pool.h +++ b/examples/stlp/audio_pipeline/src/adec/aec/aec_memory_pool.h @@ -5,7 +5,7 @@ #define AEC_MEMORY_POOL_H #include "audio_pipeline_dsp.h" -#include "xs3_math.h" +#include "xmath/xmath.h" //Memory pool definition typedef struct { diff --git a/examples/stlp/audio_pipeline/src/adec/audio_pipeline_t0.c b/examples/stlp/audio_pipeline/src/adec/audio_pipeline_t0.c index a77efa65..c0a3d0bb 100644 --- a/examples/stlp/audio_pipeline/src/adec/audio_pipeline_t0.c +++ b/examples/stlp/audio_pipeline/src/adec/audio_pipeline_t0.c @@ -98,7 +98,7 @@ static void stage_vnr_and_ic(frame_data_t *frame_data) vnr_inference(&ie_output, &feature_patch); vnr_pred_state->output_vnr_pred = float_s32_ema(vnr_pred_state->output_vnr_pred, ie_output, vnr_pred_state->pred_alpha_q30); - float_s32_t agc_vnr_threshold = float_to_float_s32(VNR_AGC_THRESHOLD); + float_s32_t agc_vnr_threshold = f32_to_float_s32(VNR_AGC_THRESHOLD); frame_data->vnr_pred_flag = float_s32_gt(vnr_pred_stage_state.vnr_pred_state.output_vnr_pred, agc_vnr_threshold); ic_adapt(&ic_stage_state.state, vnr_pred_stage_state.vnr_pred_state.input_vnr_pred); @@ -151,8 +151,8 @@ static void initialize_pipeline_stages(void) vnr_feature_state_init(&vnr_pred_state->feature_state[1]); vnr_inference_init(); vnr_pred_state->pred_alpha_q30 = Q30(0.97); - vnr_pred_state->input_vnr_pred = float_to_float_s32(0.5); - vnr_pred_state->output_vnr_pred = float_to_float_s32(0.5); + vnr_pred_state->input_vnr_pred = f32_to_float_s32(0.5); + vnr_pred_state->output_vnr_pred = f32_to_float_s32(0.5); ns_init(&ns_stage_state.state); diff --git a/examples/stlp/audio_pipeline/src/adec/stage1/stage_1.c b/examples/stlp/audio_pipeline/src/adec/stage1/stage_1.c index d97fc35b..d47784d9 100644 --- a/examples/stlp/audio_pipeline/src/adec/stage1/stage_1.c +++ b/examples/stlp/audio_pipeline/src/adec/stage1/stage_1.c @@ -45,7 +45,7 @@ static inline void get_delayed_frame( void stage_1_init(stage_1_state_t *state, aec_conf_t *de_conf, aec_conf_t *non_de_conf, adec_config_t *adec_config) { state->delay_estimator_enabled = 0; - state->ref_active_threshold = double_to_float_s32(pow(10, REF_ACTIVE_THRESHOLD_dB/20.0)); //-60dB + state->ref_active_threshold = f64_to_float_s32(pow(10, REF_ACTIVE_THRESHOLD_dB/20.0)); //-60dB state->hold_aec_count = 0; //No. of consecutive frames reference has been absent for state->hold_aec_limit = (16000*HOLD_AEC_LIMIT_SECONDS)/AP_FRAME_ADVANCE; //bypass AEC only when reference has been absent for atleast 3 seconds (200 frames) diff --git a/examples/stlp/audio_pipeline/src/adec_alt_arch/aec/aec_memory_pool.h b/examples/stlp/audio_pipeline/src/adec_alt_arch/aec/aec_memory_pool.h index 87b1662c..f19e2c1f 100644 --- a/examples/stlp/audio_pipeline/src/adec_alt_arch/aec/aec_memory_pool.h +++ b/examples/stlp/audio_pipeline/src/adec_alt_arch/aec/aec_memory_pool.h @@ -5,7 +5,7 @@ #define AEC_MEMORY_POOL_H #include "audio_pipeline_dsp.h" -#include "xs3_math.h" +#include "xmath/xmath.h" //Memory pool definition typedef struct { diff --git a/examples/stlp/audio_pipeline/src/adec_alt_arch/audio_pipeline_t0.c b/examples/stlp/audio_pipeline/src/adec_alt_arch/audio_pipeline_t0.c index 23c5cdb6..2e409abb 100644 --- a/examples/stlp/audio_pipeline/src/adec_alt_arch/audio_pipeline_t0.c +++ b/examples/stlp/audio_pipeline/src/adec_alt_arch/audio_pipeline_t0.c @@ -106,7 +106,7 @@ static void stage_vnr_and_ic(frame_data_t *frame_data) vnr_inference(&ie_output, &feature_patch); vnr_pred_state->output_vnr_pred = float_s32_ema(vnr_pred_state->output_vnr_pred, ie_output, vnr_pred_state->pred_alpha_q30); - float_s32_t agc_vnr_threshold = float_to_float_s32(VNR_AGC_THRESHOLD); + float_s32_t agc_vnr_threshold = f32_to_float_s32(VNR_AGC_THRESHOLD); frame_data->vnr_pred_flag = float_s32_gt(vnr_pred_stage_state.vnr_pred_state.output_vnr_pred, agc_vnr_threshold); ic_adapt(&ic_stage_state.state, vnr_pred_stage_state.vnr_pred_state.input_vnr_pred); @@ -159,8 +159,8 @@ static void initialize_pipeline_stages(void) vnr_feature_state_init(&vnr_pred_state->feature_state[1]); vnr_inference_init(); vnr_pred_state->pred_alpha_q30 = Q30(0.97); - vnr_pred_state->input_vnr_pred = float_to_float_s32(0.5); - vnr_pred_state->output_vnr_pred = float_to_float_s32(0.5); + vnr_pred_state->input_vnr_pred = f32_to_float_s32(0.5); + vnr_pred_state->output_vnr_pred = f32_to_float_s32(0.5); ns_init(&ns_stage_state.state); diff --git a/examples/stlp/audio_pipeline/src/adec_alt_arch/stage1/stage_1.c b/examples/stlp/audio_pipeline/src/adec_alt_arch/stage1/stage_1.c index be2b0410..51a18265 100644 --- a/examples/stlp/audio_pipeline/src/adec_alt_arch/stage1/stage_1.c +++ b/examples/stlp/audio_pipeline/src/adec_alt_arch/stage1/stage_1.c @@ -53,7 +53,7 @@ static inline void get_delayed_frame( void stage_1_init(stage_1_state_t *state, aec_conf_t *de_conf, aec_conf_t *non_de_conf, adec_config_t *adec_config) { state->delay_estimator_enabled = 0; - state->ref_active_threshold = double_to_float_s32(pow(10, REF_ACTIVE_THRESHOLD_dB/20.0)); //-60dB + state->ref_active_threshold = f64_to_float_s32(pow(10, REF_ACTIVE_THRESHOLD_dB/20.0)); //-60dB state->hold_aec_count = 0; //No. of consecutive frames reference has been absent for state->hold_aec_limit = (16000*HOLD_AEC_LIMIT_SECONDS)/AP_FRAME_ADVANCE; //bypass AEC only when reference has been absent for atleast 3 seconds (200 frames) diff --git a/examples/stlp/audio_pipeline/src/fixed_delay/aec/aec_memory_pool.h b/examples/stlp/audio_pipeline/src/fixed_delay/aec/aec_memory_pool.h new file mode 100644 index 00000000..f19e2c1f --- /dev/null +++ b/examples/stlp/audio_pipeline/src/fixed_delay/aec/aec_memory_pool.h @@ -0,0 +1,54 @@ +// Copyright 2022 XMOS LIMITED. +// This Software is subject to the terms of the XMOS Public Licence: Version 1. + +#ifndef AEC_MEMORY_POOL_H +#define AEC_MEMORY_POOL_H + +#include "audio_pipeline_dsp.h" +#include "xmath/xmath.h" + +//Memory pool definition +typedef struct { + /** Memory pointed to by aec_shared_state_t::y and aec_shared_state_t::Y*/ + int32_t mic_input_frame[AEC_MAX_Y_CHANNELS][AEC_PROC_FRAME_LENGTH + AEC_FFT_PADDING]; + /** Memory pointed to by aec_shared_state_t::x and aec_shared_state_t::X. Also reused for main filter + * aec_state_t::T*/ + int32_t ref_input_frame[AEC_MAX_X_CHANNELS][AEC_PROC_FRAME_LENGTH + AEC_FFT_PADDING]; + /** Memory pointed to by aec_shared_state_t::prev_y*/ + int32_t mic_prev_samples[AEC_MAX_Y_CHANNELS][AEC_PROC_FRAME_LENGTH - AEC_FRAME_ADVANCE]; + /** Memory pointed to by aec_shared_state_t::prev_x*/ + int32_t ref_prev_samples[AEC_MAX_X_CHANNELS][AEC_PROC_FRAME_LENGTH - AEC_FRAME_ADVANCE]; + /** Memory pointed to by main filter aec_state_t::H_hat, aec_shared_state_t::X_fifo, main filter + * aec_state_t::X_fifo_1d and shadow filter aec_state_t::X_fifo_1d*/ + complex_s32_t phase_pool_H_hat_X_fifo[((AEC_MAX_Y_CHANNELS*AEC_MAX_X_CHANNELS*AEC_MAIN_FILTER_PHASES) + (AEC_MAX_X_CHANNELS*AEC_MAIN_FILTER_PHASES)) * AEC_FD_FRAME_LENGTH]; + /** Memory pointed to by main filter aec_state_t::Error and aec_state_t::error*/ + complex_s32_t Error[AEC_MAX_Y_CHANNELS][AEC_FD_FRAME_LENGTH]; + /** Memory pointed to by main filter aec_state_t::Y_hat and aec_state_t::y_hat*/ + complex_s32_t Y_hat[AEC_MAX_Y_CHANNELS][AEC_FD_FRAME_LENGTH]; + /** Memory pointed to by main_filter aec_state_t::X_energy*/ + int32_t X_energy[AEC_MAX_X_CHANNELS][AEC_FD_FRAME_LENGTH]; + /** Memory pointed to by aec_shared_state_t::sigma_XX*/ + int32_t sigma_XX[AEC_MAX_X_CHANNELS][AEC_FD_FRAME_LENGTH]; + /** Memory pointed to by main filter aec_state_t::inv_X_energy*/ + int32_t inv_X_energy[AEC_MAX_X_CHANNELS][AEC_FD_FRAME_LENGTH]; + /** Memory pointed to by main filter aec_state_t::overlap*/ + int32_t overlap[AEC_MAX_Y_CHANNELS][AEC_UNUSED_TAPS_PER_PHASE*2]; +}aec_memory_pool_t; + +typedef struct { + /** Memory pointed to by shadow filter aec_state_t::H_hat*/ + complex_s32_t phase_pool_H_hat[AEC_MAX_Y_CHANNELS * AEC_MAX_X_CHANNELS * AEC_SHADOW_FILTER_PHASES * AEC_FD_FRAME_LENGTH]; + /** Memory pointed to by shadow filter aec_state_t::Error and aec_state_t::error*/ + complex_s32_t Error[AEC_MAX_Y_CHANNELS][AEC_FD_FRAME_LENGTH]; + /** Memory pointed to by shadow filter aec_state_t::Y_hat and aec_state_t::y_hat*/ + complex_s32_t Y_hat[AEC_MAX_Y_CHANNELS][AEC_FD_FRAME_LENGTH]; + /** Memory pointed to by shadow filter aec_state_t::T*/ + complex_s32_t T[AEC_MAX_X_CHANNELS][AEC_FD_FRAME_LENGTH]; + /** Memory pointed to by shadow_filter aec_state_t::X_energy*/ + int32_t X_energy[AEC_MAX_X_CHANNELS][AEC_FD_FRAME_LENGTH]; + /** Memory pointed to by shadow_filter aec_state_t::inv_X_energy*/ + int32_t inv_X_energy[AEC_MAX_X_CHANNELS][AEC_FD_FRAME_LENGTH]; + /** Memory pointed to by shadow filter aec_state_t::overlap*/ + int32_t overlap[AEC_MAX_Y_CHANNELS][AEC_UNUSED_TAPS_PER_PHASE*2]; +}aec_shadow_filt_memory_pool_t; +#endif diff --git a/examples/stlp/audio_pipeline/src/fixed_delay/aec/aec_process_frame_1thread.c b/examples/stlp/audio_pipeline/src/fixed_delay/aec/aec_process_frame_1thread.c new file mode 100644 index 00000000..240925de --- /dev/null +++ b/examples/stlp/audio_pipeline/src/fixed_delay/aec/aec_process_frame_1thread.c @@ -0,0 +1,221 @@ +// Copyright 2021-2022 XMOS LIMITED. +// This Software is subject to the terms of the XMOS Public Licence: Version 1. + +#include +#include +#include "aec_defines.h" +#include "aec_api.h" + +/* This is an example of processing one frame of data through the AEC pipeline stage. The example runs on 1 thread and + * can be compiled for both bare metal and x86. + */ +static unsigned X_energy_recalc_bin = 0; +void aec_process_frame_1thread( + aec_state_t *main_state, + aec_state_t *shadow_state, + int32_t (*output_main)[AEC_FRAME_ADVANCE], + int32_t (*output_shadow)[AEC_FRAME_ADVANCE], + const int32_t (*y_data)[AEC_FRAME_ADVANCE], + const int32_t (*x_data)[AEC_FRAME_ADVANCE]) +{ + // Read number of mic and reference channels. These are specified as part of the configuration when aec_init() is called. + int num_y_channels = main_state->shared_state->num_y_channels; //Number of mic channels + int num_x_channels = main_state->shared_state->num_x_channels; //Number of reference channels + + // Set up the input BFP structures main_state->shared_state->y and main_state->shared_state->x to point to the new frame. + // Initialise some other BFP structures that need to be initialised at the beginning of each frame + aec_frame_init(main_state, shadow_state, y_data, x_data); + + // Calculate Exponential moving average (EMA) energy of the mic and reference input. + for(int ch=0; chshared_state->y_ema_energy[ch], &main_state->shared_state->y[ch], + AEC_PROC_FRAME_LENGTH - AEC_FRAME_ADVANCE, AEC_FRAME_ADVANCE, &main_state->shared_state->config_params); + } + for(int ch=0; chshared_state->x_ema_energy[ch], &main_state->shared_state->x[ch], + AEC_PROC_FRAME_LENGTH - AEC_FRAME_ADVANCE, AEC_FRAME_ADVANCE, &main_state->shared_state->config_params); + } + + // Calculate mic input spectrum for all num_y_channels of mic input + /* The spectrum calculation is done in place. Taking mic input as example, after the aec_forward_fft() call + * main_state->shared_state->Y[ch].data and main_state->shared_state->y[ch].data point to the same memory address. + * The spectral representation of the input is used after this function. Time domain input + * BFP structure main_state->shared_state->y should not be used. + * main_state->shared_state->Y[ch].data points to AEC_PROC_FRAME_LENGTH/2 + 1 complex 32bit spectrum samples, + * which represent the spectrum samples from DC to Nyquist frequency. + * Same is true for reference spectrum samples pointed to by main_state->shared_state->X[ch].data + * as well. + */ + for(int ch=0; chshared_state->Y[ch], &main_state->shared_state->y[ch]); + } + // Calculate reference input spectrum for all num_x_channels of reference input + for(int ch=0; chshared_state->X[ch], &main_state->shared_state->x[ch]); + } + + // Calculate sum of X energy over X FIFO phases for all num_x_channels reference channels + /* AEC data structures store a single copy of the X FIFO that is shared between the main and shadow filter. + * Since main filter phases main_state->num_phases are more than the shadow filter phases shadow_state->num_phases, + * X FIFO holds main_state->num_phases most recent frames of reference input spectrum, where the frames are ordered + * from most recent to least recent. For shadow filter operation, out of this shared X FIFO, the first shadow_state->num_phases + * frames are considered. + */ + for(int ch=0; chX_energy[ch] points to AEC_PROC_FRAME_LENGTH/2 + 1 real 32bit values where value at index n is + * the nth X sample's energy summed over main_state->num_phases number of frames in the X FIFO. + */ + aec_calc_X_fifo_energy(main_state, ch, X_energy_recalc_bin); + + // Calculate sum of X energy for shadow filter + /* BFP struct shadow_state->X_energy[ch] points to AEC_PROC_FRAME_LENGTH/2 + 1 real 32bit values where value at index n is + * the nth X sample's energy summed over shadow_state->num_phases number of frames in the X FIFO. + */ + aec_calc_X_fifo_energy(shadow_state, ch, X_energy_recalc_bin); + } + + // Increment X_energy_recalc_bin to the next sample index. + /* Passing X_energy_recalc_bin to aec_calc_X_fifo_energy() ensures that energy of sample at index X_energy_recalc_bin + * is recalculated without the speed optimisations so that quantisation error can be kept in check + */ + X_energy_recalc_bin += 1; + if(X_energy_recalc_bin == (AEC_PROC_FRAME_LENGTH/2) + 1) { // Wrap around to 0 on completing one (AEC_PROC_FRAME_LENGTH/2) + 1 samples pass. + X_energy_recalc_bin = 0; + } + + // Update X-FIFO and calculate sigma_XX. + /* Add the current X frame to the X FIFO and remove the oldest X frame from the X FIFO. + * Also, calculate state->shared_state->sigma_XX. sigma_XX is the EMA of current X frame energy. + * It is later used to time smooth the X_energy while calculating the normalisation spectrum + */ + for(int ch=0; chshared_state->X_fifo BFP struct to main_state->X_fifo_1d and shadow_state->X_fifo_1d BFP structs + /* The updated state->shared_state->X_FIFO BFP structures are copied to an alternate set of BFP structs present in the + * main and shadow filter state structure, that are used to efficiently access the X FIFO in the Error computation and filter + * update steps. + */ + aec_update_X_fifo_1d(main_state); + aec_update_X_fifo_1d(shadow_state); + + // Calculate error spectrum and estimated mic spectrum for main and shadow adaptive filters + for(int ch=0; chError[ch] and main_state->Y_hat[ch] are updated + aec_calc_Error_and_Y_hat(main_state, ch); + + // shadow_state->Error[ch] and shadow_state->Y_hat[ch] are updated + aec_calc_Error_and_Y_hat(shadow_state, ch); + } + + // Calculate time domain error and time domain estimated mic input from their spectrums calculated in the previous step. + /* The time domain estimated mic_input (y_hat) is used to calculate the average coherence between y and y_hat in aec_calc_coherence. + * Only the estimated mic input calculated using the main filter is needed for coherence calculation, so the y_hat calculation is + * done only for main filter. + */ + for(int ch=0; cherror[ch], &main_state->Error[ch]); + aec_inverse_fft(&shadow_state->error[ch], &shadow_state->Error[ch]); + aec_inverse_fft(&main_state->y_hat[ch], &main_state->Y_hat[ch]); + } + + // Calculate average coherence and average slow moving coherence between mic and estimated mic time domain signals + for(int ch=0; chshared_state->coh_mu_state[ch].coh and main_state->shared_state->coh_mu_state[ch].coh_slow are updated + aec_calc_coherence(main_state, ch); + } + + // Calculate AEC filter time domain output. This is the output sent to downstream pipeline stages + for(int ch=0; cherror_ema_energy[ch], &temp, 0, AEC_FRAME_ADVANCE, &main_state->shared_state->config_params); + } + + // Convert shadow and main filters error back to frequency domain since subsequent AEC functions will use the error spectrum. + // The error spectrum is later used to compute T values which are then used while updating the adaptive filter. + for(int ch=0; chError[ch] is updated + aec_forward_fft(&main_state->Error[ch], &main_state->error[ch]); + + // shadow_state->Error[ch] is updated + aec_forward_fft(&shadow_state->Error[ch], &shadow_state->error[ch] + ); + } + + // Calculate energies of mic input and error spectrum of main and shadow filters. + // These energy values are later used in aec_compare_filters_and_calc_mu() to estimate how well the filters are performing. + for(int ch=0; choverall_Error[ch] is updated + aec_calc_freq_domain_energy(&main_state->overall_Error[ch], &main_state->Error[ch]); + + // shadow_state->overall_Error[ch] is updated + aec_calc_freq_domain_energy(&shadow_state->overall_Error[ch], &shadow_state->Error[ch]); + + // main_state->shared_state->overall_Y[ch] is updated + aec_calc_freq_domain_energy(&main_state->shared_state->overall_Y[ch], &main_state->shared_state->Y[ch]); + } + + // Compare and update filters. Calculate adaption step_size mu + /* At this point we're ready to check how well the filters are performing and update them if needed. + * + * main_state->shared_state->shadow_filter_params are updated to indicate the current state of filter comparison algorithm. + * main_state->H_hat, main_state->Error, shadow_state->H_hat, shadow_state->Error are optionally updated depending on the update needed. + * + * After the filter comparison and update step, the adaption step size mu is calculated for main and shadow filter. + * main_state->mu and shadow_state->mu are updated. + */ + aec_compare_filters_and_calc_mu( + main_state, + shadow_state); + + // Calculate smoothed reference FIFO energy that is later used to scale the X FIFO in the filter update step. + // This calculation is done differently for main and shadow filters, so a flag indicating filter type is specified as one of the input arguments. + for(int ch=0; chinv_X_energy[ch] is updated. + aec_calc_normalisation_spectrum(main_state, ch, 0); + + // shadow_state->inv_X_energy[ch] is updated. + aec_calc_normalisation_spectrum(shadow_state, ch, 1); + } + + for(int ych=0; ychmu, state->Error and state->inv_X_energy. + for(int xch=0; xchT[ch] is updated + aec_calc_T(main_state, ych, xch); + + // shadow_state->T[ch] is updated + aec_calc_T(shadow_state, ych, xch); + } + // Update filters + + // Update main_state->H_hat + aec_filter_adapt(main_state, ych); + + // Update shadow_state->H_hat + aec_filter_adapt(shadow_state, ych); + } +} diff --git a/examples/stlp/audio_pipeline/src/fixed_delay/audio_pipeline_dsp.h b/examples/stlp/audio_pipeline/src/fixed_delay/audio_pipeline_dsp.h new file mode 100644 index 00000000..7e012a2a --- /dev/null +++ b/examples/stlp/audio_pipeline/src/fixed_delay/audio_pipeline_dsp.h @@ -0,0 +1,103 @@ +// Copyright (c) 2022 XMOS LIMITED. This Software is subject to the terms of the +// XMOS Public License: Version 1 + +#ifndef AUDIO_PIPELINE_DSP_H_ +#define AUDIO_PIPELINE_DSP_H_ + +#include +#include "FreeRTOS.h" +#include "stream_buffer.h" +#include "app_conf.h" +#include + +/* Pipeline config */ +#define AP_MAX_Y_CHANNELS (2) +#define AP_MAX_X_CHANNELS (2) +#define AP_FRAME_ADVANCE (240) + +/* AEC config */ +#define AEC_MAX_Y_CHANNELS (AP_MAX_Y_CHANNELS) +#define AEC_MAX_X_CHANNELS (AP_MAX_X_CHANNELS) +#define AEC_MAIN_FILTER_PHASES (10) +#define AEC_SHADOW_FILTER_PHASES (5) + +/* Delay buffer config */ +#define MAX_DELAY_BUF_CHANNELS (2) +#define DELAY_BUF_MAX_DELAY_MS ( 150 ) +#define DELAY_BUF_MAX_DELAY_SAMPLES ( 16000*DELAY_BUF_MAX_DELAY_MS/1000 ) + +#include "aec_api.h" +#include "aec/aec_memory_pool.h" +#include "agc_api.h" +#include "ic_api.h" +#include "ns_api.h" +#include "vnr_features_api.h" +#include "vnr_inference_api.h" + + +/* Note: Changing the order here will effect the channel order for + * audio_pipeline_input() and audio_pipeline_output() + */ +typedef struct { + int32_t samples[appconfAUDIO_PIPELINE_CHANNELS][appconfAUDIO_PIPELINE_FRAME_ADVANCE]; + int32_t aec_reference_audio_samples[appconfAUDIO_PIPELINE_CHANNELS][appconfAUDIO_PIPELINE_FRAME_ADVANCE]; + int32_t mic_samples_passthrough[appconfAUDIO_PIPELINE_CHANNELS][appconfAUDIO_PIPELINE_FRAME_ADVANCE]; + + /* Below is additional context needed by other stages on a per frame basis */ + int32_t vnr_pred_flag; + float_s32_t max_ref_energy; + float_s32_t aec_corr_factor; + int32_t ref_active_flag; +} frame_data_t; + +typedef struct stage_delay_ctx { + StreamBufferHandle_t delay_buf; +} stage_delay_ctx_t; + +typedef struct aec_ctx { + aec_state_t DWORD_ALIGNED aec_main_state; + aec_state_t DWORD_ALIGNED aec_shadow_state; + aec_shared_state_t DWORD_ALIGNED aec_shared_state; + uint8_t DWORD_ALIGNED aec_main_memory_pool[sizeof(aec_memory_pool_t)]; + uint8_t DWORD_ALIGNED aec_shadow_memory_pool[sizeof(aec_shadow_filt_memory_pool_t)]; +} aec_ctx_t; + +typedef struct ic_stage_ctx { + ic_state_t DWORD_ALIGNED state; +} ic_stage_ctx_t; + +typedef struct vnr_pred_stage_ctx { + vnr_pred_state_t vnr_pred_state; +} vnr_pred_stage_ctx_t; + +typedef struct ns_stage_ctx { + ns_state_t DWORD_ALIGNED state; +} ns_stage_ctx_t; + +typedef struct agc_stage_ctx { + agc_meta_data_t md; + agc_state_t DWORD_ALIGNED state; +} agc_stage_ctx_t; + +/** + * Delay buffer contains the configured buffer delay size + 1 frame worth of audio + * When appconfINPUT_SAMPLES_MIC_DELAY_MS is positive, the mic is delayed + * When appconfINPUT_SAMPLES_MIC_DELAY_MS is negative, the mic is "advanced" by delaying ref + */ +#define ABS(A) ((A >= 0) ? A : -A) +#define AP_INPUT_SAMPLES_MIC_DELAY_SIZE_PER_CHAN ( 16000*ABS(appconfINPUT_SAMPLES_MIC_DELAY_MS)/1000 ) +#define AP_INPUT_SAMPLES_MIC_DELAY_CHAN_CNT ( (appconfINPUT_SAMPLES_MIC_DELAY_MS > 0) ? AEC_MAX_Y_CHANNELS : AEC_MAX_X_CHANNELS ) +#define AP_INPUT_SAMPLES_MIC_DELAY_SIZE_CHAN ( AP_INPUT_SAMPLES_MIC_DELAY_SIZE_PER_CHAN * AP_INPUT_SAMPLES_MIC_DELAY_CHAN_CNT ) +#define AP_INPUT_SAMPLES_MIC_DELAY_SIZE_CUR_FRAME_WORDS ( AP_INPUT_SAMPLES_MIC_DELAY_CHAN_CNT * appconfAUDIO_PIPELINE_FRAME_ADVANCE) +#define AP_INPUT_SAMPLES_MIC_DELAY_CUR_FRAME_BYTES ( AP_INPUT_SAMPLES_MIC_DELAY_SIZE_CUR_FRAME_WORDS * sizeof(int32_t)) +#define AP_INPUT_SAMPLES_MIC_DELAY_BUF_SIZE_BYTES ( AP_INPUT_SAMPLES_MIC_DELAY_SIZE_CHAN * sizeof(int32_t) ) + +void aec_process_frame_1thread( + aec_state_t *main_state, + aec_state_t *shadow_state, + int32_t (*output_main)[AEC_FRAME_ADVANCE], + int32_t (*output_shadow)[AEC_FRAME_ADVANCE], + const int32_t (*y_data)[AEC_FRAME_ADVANCE], + const int32_t (*x_data)[AEC_FRAME_ADVANCE]); + +#endif /* AUDIO_PIPELINE_DSP_H_ */ diff --git a/examples/stlp/audio_pipeline/src/fixed_delay/audio_pipeline_t0.c b/examples/stlp/audio_pipeline/src/fixed_delay/audio_pipeline_t0.c new file mode 100644 index 00000000..53c77806 --- /dev/null +++ b/examples/stlp/audio_pipeline/src/fixed_delay/audio_pipeline_t0.c @@ -0,0 +1,193 @@ +// Copyright (c) 2022 XMOS LIMITED. This Software is subject to the terms of the +// XMOS Public License: Version 1 + +/* STD headers */ +#include +#include +#include + +/* FreeRTOS headers */ +#include "FreeRTOS.h" +#include "task.h" +#include "timers.h" +#include "queue.h" +#include "stream_buffer.h" + +/* Library headers */ +#include "generic_pipeline.h" +#include "aec_api.h" +#include "agc_api.h" +#include "ic_api.h" +#include "ns_api.h" +#include "vnr_features_api.h" +#include "vnr_inference_api.h" + +/* App headers */ +#include "app_conf.h" +#include "audio_pipeline.h" +#include "audio_pipeline_dsp.h" + +#if appconfAUDIO_PIPELINE_FRAME_ADVANCE != 240 +#error This pipeline is only configured for 240 frame advance +#endif + +#define VNR_AGC_THRESHOLD (0.5) + +#if ON_TILE(0) +static ic_stage_ctx_t DWORD_ALIGNED ic_stage_state = {}; +static vnr_pred_stage_ctx_t DWORD_ALIGNED vnr_pred_stage_state = {}; +static ns_stage_ctx_t DWORD_ALIGNED ns_stage_state = {}; +static agc_stage_ctx_t DWORD_ALIGNED agc_stage_state = {}; + +static void *audio_pipeline_input_i(void *input_app_data) +{ + frame_data_t *frame_data; + + frame_data = pvPortMalloc(sizeof(frame_data_t)); + memset(frame_data, 0x00, sizeof(frame_data_t)); + + size_t bytes_received = 0; + bytes_received = rtos_intertile_rx_len( + intertile_ctx, + appconfAUDIOPIPELINE_PORT, + portMAX_DELAY); + + xassert(bytes_received == sizeof(frame_data_t)); + + rtos_intertile_rx_data( + intertile_ctx, + frame_data, + bytes_received); + + return frame_data; +} + +static int audio_pipeline_output_i(frame_data_t *frame_data, + void *output_app_data) +{ + return audio_pipeline_output(output_app_data, + (int32_t **)frame_data->samples, + 6, + appconfAUDIO_PIPELINE_FRAME_ADVANCE); +} + +static void stage_vnr_and_ic(frame_data_t *frame_data) +{ +#if appconfAUDIO_PIPELINE_SKIP_IC_AND_VAD +#else + int32_t DWORD_ALIGNED ic_output[appconfAUDIO_PIPELINE_FRAME_ADVANCE]; + ic_filter(&ic_stage_state.state, + frame_data->samples[0], + frame_data->samples[1], + ic_output); + + // VNR + bfp_s32_t feature_patch; + int32_t feature_patch_data[VNR_PATCH_WIDTH * VNR_MEL_FILTERS]; + float_s32_t ie_output; + vnr_pred_state_t *vnr_pred_state = &vnr_pred_stage_state.vnr_pred_state; + + vnr_extract_features(&vnr_pred_state->feature_state[0], &feature_patch, + feature_patch_data, &ic_stage_state.state.Y_bfp[0]); + vnr_inference(&ie_output, &feature_patch); + vnr_pred_state->input_vnr_pred = float_s32_ema(vnr_pred_state->input_vnr_pred, ie_output, vnr_pred_state->pred_alpha_q30); + + vnr_extract_features(&vnr_pred_state->feature_state[1], &feature_patch, + feature_patch_data, &ic_stage_state.state.Error_bfp[0]); + vnr_inference(&ie_output, &feature_patch); + vnr_pred_state->output_vnr_pred = float_s32_ema(vnr_pred_state->output_vnr_pred, ie_output, vnr_pred_state->pred_alpha_q30); + + float_s32_t agc_vnr_threshold = f32_to_float_s32(VNR_AGC_THRESHOLD); + frame_data->vnr_pred_flag = float_s32_gt(vnr_pred_stage_state.vnr_pred_state.output_vnr_pred, agc_vnr_threshold); + + ic_adapt(&ic_stage_state.state, vnr_pred_stage_state.vnr_pred_state.input_vnr_pred); + + /* Intentionally ignoring comms ch from here on out */ + memcpy(frame_data->samples, ic_output, appconfAUDIO_PIPELINE_FRAME_ADVANCE * sizeof(int32_t)); +#endif +} + +static void stage_ns(frame_data_t *frame_data) +{ +#if appconfAUDIO_PIPELINE_SKIP_NS +#else + int32_t DWORD_ALIGNED ns_output[appconfAUDIO_PIPELINE_FRAME_ADVANCE]; + configASSERT(NS_FRAME_ADVANCE == appconfAUDIO_PIPELINE_FRAME_ADVANCE); + ns_process_frame( + &ns_stage_state.state, + ns_output, + frame_data->samples[0]); + memcpy(frame_data->samples, ns_output, appconfAUDIO_PIPELINE_FRAME_ADVANCE * sizeof(int32_t)); +#endif +} + +static void stage_agc(frame_data_t *frame_data) +{ +#if appconfAUDIO_PIPELINE_SKIP_AGC +#else + int32_t DWORD_ALIGNED agc_output[appconfAUDIO_PIPELINE_FRAME_ADVANCE]; + configASSERT(AGC_FRAME_ADVANCE == appconfAUDIO_PIPELINE_FRAME_ADVANCE); + + agc_stage_state.md.vnr_flag = frame_data->vnr_pred_flag; + agc_stage_state.md.aec_ref_power = frame_data->max_ref_energy; + agc_stage_state.md.aec_corr_factor = frame_data->aec_corr_factor; + + agc_process_frame( + &agc_stage_state.state, + agc_output, + frame_data->samples[0], + &agc_stage_state.md); + memcpy(frame_data->samples, agc_output, appconfAUDIO_PIPELINE_FRAME_ADVANCE * sizeof(int32_t)); +#endif +} + +static void initialize_pipeline_stages(void) +{ + ic_init(&ic_stage_state.state); + + vnr_pred_state_t *vnr_pred_state = &vnr_pred_stage_state.vnr_pred_state; + vnr_feature_state_init(&vnr_pred_state->feature_state[0]); + vnr_feature_state_init(&vnr_pred_state->feature_state[1]); + vnr_inference_init(); + vnr_pred_state->pred_alpha_q30 = Q30(0.97); + vnr_pred_state->input_vnr_pred = f32_to_float_s32(0.5); + vnr_pred_state->output_vnr_pred = f32_to_float_s32(0.5); + + ns_init(&ns_stage_state.state); + + agc_init(&agc_stage_state.state, &AGC_PROFILE_ASR); + agc_stage_state.md.aec_ref_power = AGC_META_DATA_NO_AEC; + agc_stage_state.md.aec_corr_factor = AGC_META_DATA_NO_AEC; +} + +void audio_pipeline_init( + void *input_app_data, + void *output_app_data) +{ + const int stage_count = 3; + + const pipeline_stage_t stages[] = { + (pipeline_stage_t)stage_vnr_and_ic, + (pipeline_stage_t)stage_ns, + (pipeline_stage_t)stage_agc, + }; + + const configSTACK_DEPTH_TYPE stage_stack_sizes[] = { + configMINIMAL_STACK_SIZE + RTOS_THREAD_STACK_SIZE(stage_vnr_and_ic) + RTOS_THREAD_STACK_SIZE(audio_pipeline_input_i), + configMINIMAL_STACK_SIZE + RTOS_THREAD_STACK_SIZE(stage_ns), + configMINIMAL_STACK_SIZE + RTOS_THREAD_STACK_SIZE(stage_agc) + RTOS_THREAD_STACK_SIZE(audio_pipeline_output_i), + }; + + initialize_pipeline_stages(); + + generic_pipeline_init((pipeline_input_t)audio_pipeline_input_i, + (pipeline_output_t)audio_pipeline_output_i, + input_app_data, + output_app_data, + stages, + (const size_t*) stage_stack_sizes, + appconfAUDIO_PIPELINE_TASK_PRIORITY, + stage_count); +} + +#endif /* ON_TILE(0)*/ diff --git a/examples/stlp/audio_pipeline/src/fixed_delay/audio_pipeline_t1.c b/examples/stlp/audio_pipeline/src/fixed_delay/audio_pipeline_t1.c new file mode 100644 index 00000000..272e98fa --- /dev/null +++ b/examples/stlp/audio_pipeline/src/fixed_delay/audio_pipeline_t1.c @@ -0,0 +1,177 @@ +// Copyright (c) 2022 XMOS LIMITED. This Software is subject to the terms of the +// XMOS Public License: Version 1 + +/* STD headers */ +#include +#include +#include + +/* FreeRTOS headers */ +#include "FreeRTOS.h" +#include "task.h" +#include "timers.h" +#include "queue.h" +#include "stream_buffer.h" + +/* Library headers */ +#include "generic_pipeline.h" + +/* App headers */ +#include "app_conf.h" +#include "audio_pipeline.h" +#include "audio_pipeline_dsp.h" + +#if appconfAUDIO_PIPELINE_FRAME_ADVANCE != 240 +#error This pipeline is only configured for 240 frame advance +#endif + +#if ON_TILE(1) +#if appconfINPUT_SAMPLES_MIC_DELAY_MS != 0 +static stage_delay_ctx_t DWORD_ALIGNED delay_buf_state = {}; +#endif +static aec_ctx_t DWORD_ALIGNED aec_state = {}; + + +static void *audio_pipeline_input_i(void *input_app_data) +{ + frame_data_t *frame_data; + + frame_data = pvPortMalloc(sizeof(frame_data_t)); + memset(frame_data, 0x00, sizeof(frame_data_t)); + + audio_pipeline_input(input_app_data, + (int32_t **)frame_data->aec_reference_audio_samples, + 4, + appconfAUDIO_PIPELINE_FRAME_ADVANCE); + + frame_data->vnr_pred_flag = 0; + + memcpy(frame_data->samples, frame_data->mic_samples_passthrough, sizeof(frame_data->samples)); + + return frame_data; +} + +static int audio_pipeline_output_i(frame_data_t *frame_data, + void *output_app_data) +{ + rtos_intertile_tx(intertile_ctx, + appconfAUDIOPIPELINE_PORT, + frame_data, + sizeof(frame_data_t)); + return AUDIO_PIPELINE_FREE_FRAME; +} + +static void stage_delay(frame_data_t *frame_data) +{ +#if appconfAUDIO_PIPELINE_SKIP_STATIC_DELAY +#else +#if (appconfINPUT_SAMPLES_MIC_DELAY_MS > 0) /* Delay mics */ + size_t bytes_sent = xStreamBufferSend( + delay_buf_state.delay_buf, + &frame_data->samples, + AP_INPUT_SAMPLES_MIC_DELAY_CUR_FRAME_BYTES, + 0); + + configASSERT(bytes_sent == AP_INPUT_SAMPLES_MIC_DELAY_CUR_FRAME_BYTES); + + if (xStreamBufferBytesAvailable(delay_buf_state.delay_buf) > AP_INPUT_SAMPLES_MIC_DELAY_BUF_SIZE_BYTES) { + size_t bytes_rx = xStreamBufferReceive( + delay_buf_state.delay_buf, + &frame_data->samples, + AP_INPUT_SAMPLES_MIC_DELAY_CUR_FRAME_BYTES, + 0); + + configASSERT(bytes_rx == AP_INPUT_SAMPLES_MIC_DELAY_CUR_FRAME_BYTES); + } +#elif (appconfINPUT_SAMPLES_MIC_DELAY_MS < 0) /* Delay Ref*/ + size_t bytes_sent = xStreamBufferSend( + delay_buf_state.delay_buf, + &frame_data->aec_reference_audio_samples, + AP_INPUT_SAMPLES_MIC_DELAY_CUR_FRAME_BYTES, + 0); + + configASSERT(bytes_sent == AP_INPUT_SAMPLES_MIC_DELAY_CUR_FRAME_BYTES); + + if (xStreamBufferBytesAvailable(delay_buf_state.delay_buf) > AP_INPUT_SAMPLES_MIC_DELAY_BUF_SIZE_BYTES) { + size_t bytes_rx = xStreamBufferReceive( + delay_buf_state.delay_buf, + &frame_data->aec_reference_audio_samples, + AP_INPUT_SAMPLES_MIC_DELAY_CUR_FRAME_BYTES, + 0); + + configASSERT(bytes_rx == AP_INPUT_SAMPLES_MIC_DELAY_CUR_FRAME_BYTES); + } +#else /* Delay None */ +#endif +#endif /* appconfAUDIO_PIPELINE_SKIP_DELAY */ +} + +static void stage_aec(frame_data_t *frame_data) +{ +#if appconfAUDIO_PIPELINE_SKIP_AEC +#else + int32_t DWORD_ALIGNED stage1_output[AEC_MAX_Y_CHANNELS][appconfAUDIO_PIPELINE_FRAME_ADVANCE]; + + aec_process_frame_1thread( + &aec_state.aec_main_state, + &aec_state.aec_shadow_state, + stage1_output, + NULL, + frame_data->samples, + frame_data->aec_reference_audio_samples); + + frame_data->max_ref_energy = aec_calc_max_input_energy( + frame_data->aec_reference_audio_samples, + aec_state.aec_main_state.shared_state->num_x_channels); + frame_data->aec_corr_factor = aec_calc_corr_factor(&aec_state.aec_main_state, 0); + memcpy(frame_data->samples, stage1_output, AEC_MAX_Y_CHANNELS * appconfAUDIO_PIPELINE_FRAME_ADVANCE * sizeof(int32_t)); +#endif +} + +static void initialize_pipeline_stages(void) +{ +#if (appconfINPUT_SAMPLES_MIC_DELAY_MS != 0) + configASSERT(AP_INPUT_SAMPLES_MIC_DELAY_BUF_SIZE_BYTES > 0); + delay_buf_state.delay_buf = xStreamBufferCreate((size_t)AP_INPUT_SAMPLES_MIC_DELAY_BUF_SIZE_BYTES + AP_INPUT_SAMPLES_MIC_DELAY_CUR_FRAME_BYTES, 0); + configASSERT(delay_buf_state.delay_buf); +#endif + + aec_init(&aec_state.aec_main_state, + &aec_state.aec_shadow_state, + &aec_state.aec_shared_state, + &aec_state.aec_main_memory_pool[0], + &aec_state.aec_shadow_memory_pool[0], + AEC_MAX_Y_CHANNELS, + AEC_MAX_X_CHANNELS, + AEC_MAIN_FILTER_PHASES, + AEC_SHADOW_FILTER_PHASES); +} + +void audio_pipeline_init( + void *input_app_data, + void *output_app_data) +{ + const int stage_count = 2; + + const pipeline_stage_t stages[] = { + (pipeline_stage_t)stage_delay, + (pipeline_stage_t)stage_aec, + }; + + const configSTACK_DEPTH_TYPE stage_stack_sizes[] = { + configMINIMAL_STACK_SIZE + RTOS_THREAD_STACK_SIZE(stage_delay) + RTOS_THREAD_STACK_SIZE(audio_pipeline_input_i), + configMINIMAL_STACK_SIZE + RTOS_THREAD_STACK_SIZE(stage_aec) + RTOS_THREAD_STACK_SIZE(audio_pipeline_output_i), + }; + + initialize_pipeline_stages(); + + generic_pipeline_init((pipeline_input_t)audio_pipeline_input_i, + (pipeline_output_t)audio_pipeline_output_i, + input_app_data, + output_app_data, + stages, + (const size_t*) stage_stack_sizes, + appconfAUDIO_PIPELINE_TASK_PRIORITY, + stage_count); +} +#endif /* ON_TILE(1) */ diff --git a/examples/stlp/bsp_config/XK_VOICE_L71/XK_VOICE_L71.cmake b/examples/stlp/bsp_config/XK_VOICE_L71/XK_VOICE_L71.cmake index 7d567974..2e5661d4 100644 --- a/examples/stlp/bsp_config/XK_VOICE_L71/XK_VOICE_L71.cmake +++ b/examples/stlp/bsp_config/XK_VOICE_L71/XK_VOICE_L71.cmake @@ -20,6 +20,7 @@ target_link_libraries(sln_voice_app_stlp_board_support_xk_voice_l71 rtos::drivers::general rtos::drivers::audio rtos::drivers::usb + rtos::drivers::dfu_image sln_voice::app::stlp::dac::dac3101 ) target_compile_options(sln_voice_app_stlp_board_support_xk_voice_l71 @@ -39,8 +40,6 @@ target_compile_definitions(sln_voice_app_stlp_board_support_xk_voice_l71 PLATFORM_SUPPORTS_TILE_3=0 USB_TILE_NO=0 USB_TILE=tile[USB_TILE_NO] - # MIC_ARRAY_CONFIG_MCLK_FREQ=24576000 - # MIC_ARRAY_CONFIG_MCLK_FREQ=12288000 MIC_ARRAY_CONFIG_PDM_FREQ=3072000 MIC_ARRAY_CONFIG_SAMPLES_PER_FRAME=240 MIC_ARRAY_CONFIG_MIC_COUNT=2 diff --git a/examples/stlp/bsp_config/XK_VOICE_L71/platform/driver_instances.c b/examples/stlp/bsp_config/XK_VOICE_L71/platform/driver_instances.c index b849ebe6..6672a93b 100644 --- a/examples/stlp/bsp_config/XK_VOICE_L71/platform/driver_instances.c +++ b/examples/stlp/bsp_config/XK_VOICE_L71/platform/driver_instances.c @@ -29,3 +29,6 @@ rtos_i2c_slave_t *i2c_slave_ctx = &i2c_slave_ctx_s; static rtos_spi_slave_t spi_slave_ctx_s; rtos_spi_slave_t *spi_slave_ctx = &spi_slave_ctx_s; + +static rtos_dfu_image_t dfu_image_ctx_s; +rtos_dfu_image_t *dfu_image_ctx = &dfu_image_ctx_s; diff --git a/examples/stlp/bsp_config/XK_VOICE_L71/platform/driver_instances.h b/examples/stlp/bsp_config/XK_VOICE_L71/platform/driver_instances.h index 26100c4e..eb7c93b8 100644 --- a/examples/stlp/bsp_config/XK_VOICE_L71/platform/driver_instances.h +++ b/examples/stlp/bsp_config/XK_VOICE_L71/platform/driver_instances.h @@ -11,6 +11,7 @@ #include "rtos_i2s.h" #include "rtos_mic_array.h" #include "rtos_qspi_flash.h" +#include "rtos_dfu_image.h" #include "rtos_spi_slave.h" /* Tile specifiers */ @@ -56,5 +57,6 @@ extern rtos_i2c_master_t *i2c_master_ctx; extern rtos_i2c_slave_t *i2c_slave_ctx; extern rtos_spi_slave_t *spi_slave_ctx; extern rtos_i2s_t *i2s_ctx; +extern rtos_dfu_image_t *dfu_image_ctx; #endif /* DRIVER_INSTANCES_H_ */ diff --git a/examples/stlp/bsp_config/XK_VOICE_L71/platform/platform_conf.h b/examples/stlp/bsp_config/XK_VOICE_L71/platform/platform_conf.h index 53761ad3..86fb7e54 100644 --- a/examples/stlp/bsp_config/XK_VOICE_L71/platform/platform_conf.h +++ b/examples/stlp/bsp_config/XK_VOICE_L71/platform/platform_conf.h @@ -162,4 +162,56 @@ #define appconfSPI_TASK_PRIORITY (configMAX_PRIORITIES/2) #endif /* appconfSPI_TASK_PRIORITY */ +/*****************************************/ +/* DFU Settings */ +/*****************************************/ +#define FL_QUADDEVICE_W25Q64JW \ +{ \ + 0, /* Just specify 0 as flash_id */ \ + 256, /* page size */ \ + 32768, /* num pages */ \ + 3, /* address size */ \ + 4, /* log2 clock divider */ \ + 0x9F, /* QSPI_RDID */ \ + 0, /* id dummy bytes */ \ + 3, /* id size in bytes */ \ + 0xEF6017, /* device id (determined from xflash --spi-read-id 0x9F)*/ \ + 0x20, /* QSPI_SE */ \ + 4096, /* Sector erase is always 4KB */ \ + 0x06, /* QSPI_WREN */ \ + 0x04, /* QSPI_WRDI */ \ + PROT_TYPE_SR, /* Protection via SR */ \ + {{0x18,0x00},{0,0}}, /* QSPI_SP, QSPI_SU */ \ + 0x02, /* QSPI_PP */ \ + 0xEB, /* QSPI_READ_FAST */ \ + 1, /* 1 read dummy byte */ \ + SECTOR_LAYOUT_REGULAR, /* mad sectors */ \ + {4096,{0,{0}}}, /* regular sector sizes */ \ + 0x05, /* QSPI_RDSR */ \ + 0x01, /* QSPI_WRSR */ \ + 0x01, /* QSPI_WIP_BIT_MASK */ \ +} + +#ifndef BOARD_QSPI_SPEC +/* Set up a default SPI spec if the app has not provided + * one explicitly. + * Note: The version checks only work in XTC Tools >15.2.0 + * By default FL_QUADDEVICE_W25Q64JW is used + */ +#ifdef __XMOS_XTC_VERSION_MAJOR__ +#if (__XMOS_XTC_VERSION_MAJOR__ == 15) \ + && (__XMOS_XTC_VERSION_MINOR__ >= 2) \ + && (__XMOS_XTC_VERSION_PATCH__ >= 0) +/* In XTC >15.2.0 some SFDP support enables a generic + * default spec + */ +#define BOARD_QSPI_SPEC FL_QUADDEVICE_DEFAULT +#else +#define BOARD_QSPI_SPEC FL_QUADDEVICE_W25Q64JW +#endif +#else +#define BOARD_QSPI_SPEC FL_QUADDEVICE_W25Q64JW +#endif /* __XMOS_XTC_VERSION_MAJOR__ */ +#endif /* BOARD_QSPI_SPEC */ + #endif /* PLATFORM_CONF_H_ */ diff --git a/examples/stlp/bsp_config/XK_VOICE_L71/platform/platform_init.c b/examples/stlp/bsp_config/XK_VOICE_L71/platform/platform_init.c index 1ef5207e..6d0daf30 100644 --- a/examples/stlp/bsp_config/XK_VOICE_L71/platform/platform_init.c +++ b/examples/stlp/bsp_config/XK_VOICE_L71/platform/platform_init.c @@ -25,6 +25,20 @@ static void mclk_init(chanend_t other_tile_c) static void flash_init(void) { #if ON_TILE(FLASH_TILE_NO) + fl_QuadDeviceSpec qspi_spec = BOARD_QSPI_SPEC; + fl_QSPIPorts qspi_ports = { + .qspiCS = PORT_SQI_CS, + .qspiSCLK = PORT_SQI_SCLK, + .qspiSIO = PORT_SQI_SIO, + .qspiClkblk = FLASH_CLKBLK, + }; + + rtos_dfu_image_init( + dfu_image_ctx, + &qspi_ports, + &qspi_spec, + 1); + rtos_qspi_flash_init( qspi_flash_ctx, FLASH_CLKBLK, @@ -47,7 +61,7 @@ static void flash_init(void) qspi_io_sample_edge_falling, 0, - qspi_flash_page_program_1_4_4); + qspi_flash_page_program_1_1_4); #endif } diff --git a/examples/stlp/src/app_conf.h b/examples/stlp/src/app_conf.h index 4a6c3283..5d015cc6 100644 --- a/examples/stlp/src/app_conf.h +++ b/examples/stlp/src/app_conf.h @@ -28,6 +28,12 @@ /* If in channel sample format, appconfAUDIO_PIPELINE_FRAME_ADVANCE == MIC_ARRAY_CONFIG_SAMPLES_PER_FRAME*/ #define appconfAUDIO_PIPELINE_FRAME_ADVANCE MIC_ARRAY_CONFIG_SAMPLES_PER_FRAME +/** + * A positive delay will delay mics + * A negative delay will delay ref + */ +#define appconfINPUT_SAMPLES_MIC_DELAY_MS 0 + #ifdef appconfPIPELINE_BYPASS #define appconfAUDIO_PIPELINE_SKIP_STATIC_DELAY 1 #define appconfAUDIO_PIPELINE_SKIP_AEC 1 diff --git a/examples/stlp/src/main.c b/examples/stlp/src/main.c index 555759ef..54264a95 100644 --- a/examples/stlp/src/main.c +++ b/examples/stlp/src/main.c @@ -103,7 +103,7 @@ void audio_pipeline_input(void *input_app_data, } if (mic_from_usb) { - ch_count += 2; /* mic frames */ + ch_cnt += 2; /* mic frames */ } /* @@ -113,7 +113,7 @@ void audio_pipeline_input(void *input_app_data, usb_audio_recv(intertile_ctx, frame_count, usb_mic_audio_frame, - ch_count); + ch_cnt); #endif #if appconfI2S_ENABLED @@ -340,6 +340,7 @@ void startup_task(void *arg) #if ON_TILE(FS_TILE_NO) rtos_fatfs_init(qspi_flash_ctx); + rtos_dfu_image_print_debug(dfu_image_ctx); #endif #if appconfWW_ENABLED && ON_TILE(WW_TILE_NO) diff --git a/examples/stlp/src/usb/tusb_config.h b/examples/stlp/src/usb/tusb_config.h index 399c285b..c6b57604 100644 --- a/examples/stlp/src/usb/tusb_config.h +++ b/examples/stlp/src/usb/tusb_config.h @@ -44,7 +44,7 @@ #define CFG_TUSB_DEBUG 0 #endif -#define CFG_TUSB_MEM_ALIGN __attribute__ ((aligned(4))) +#define CFG_TUSB_MEM_ALIGN __attribute__ ((aligned(8))) #define CFG_TUSB_DEBUG_PRINTF rtos_printf @@ -67,6 +67,13 @@ #define CFG_TUD_MIDI 0 #define CFG_TUD_AUDIO 1 #define CFG_TUD_VENDOR 0 +#define CFG_TUD_DFU 1 + +//-------------------------------------------------------------------- +// DFU DRIVER CONFIGURATION +//-------------------------------------------------------------------- +// DFU buffer size, it has to be set to the buffer size used in TUD_DFU_DESCRIPTOR +#define CFG_TUD_DFU_XFER_BUFSIZE 4096 //-------------------------------------------------------------------- // AUDIO CLASS DRIVER CONFIGURATION diff --git a/examples/stlp/src/usb/usb_descriptors.c b/examples/stlp/src/usb/usb_descriptors.c index e2904d19..2df81fb7 100644 --- a/examples/stlp/src/usb/usb_descriptors.c +++ b/examples/stlp/src/usb/usb_descriptors.c @@ -115,11 +115,11 @@ const uint16_t tud_audio_desc_lengths[CFG_TUD_AUDIO] = { uac2_total_descriptors_length }; -#define CONFIG_TOTAL_LEN (TUD_CONFIG_DESC_LEN + CFG_TUD_AUDIO * uac2_total_descriptors_length) +#define CONFIG_TOTAL_LEN (TUD_CONFIG_DESC_LEN + (CFG_TUD_AUDIO * uac2_total_descriptors_length) + TUD_DFU_DESC_LEN(DFU_ALT_COUNT)) #define EPNUM_AUDIO 0x01 - #define AUDIO_INTERFACE_STRING_INDEX 4 +#define DFU_INTERFACE_STRING_INDEX 5 uint8_t const desc_configuration[] = { // Interface count, string index, total length, attribute, power in mA @@ -196,6 +196,9 @@ uint8_t const desc_configuration[] = { TUD_AUDIO_DESC_CS_AS_ISO_EP(/*_attr*/ AUDIO_CS_AS_ISO_DATA_EP_ATT_NON_MAX_PACKETS_OK, /*_ctrl*/ AUDIO_CTRL_NONE, /*_lockdelayunit*/ AUDIO_CS_AS_ISO_DATA_EP_LOCK_DELAY_UNIT_MILLISEC, /*_lockdelay*/ 0x0003), #endif + // Interface number, Alternate count, starting string index, attributes, detach timeout, transfer size + TUD_DFU_DESCRIPTOR(ITF_NUM_DFU_MODE, DFU_ALT_COUNT, DFU_INTERFACE_STRING_INDEX, DFU_FUNC_ATTRS, 1000, CFG_TUD_DFU_XFER_BUFSIZE), + }; // desc_configuration // Invoked when received GET CONFIGURATION DESCRIPTOR @@ -213,10 +216,13 @@ uint8_t const* tud_descriptor_configuration_cb(uint8_t index) // array of pointer to string descriptors char const *string_desc_arr[] = {(const char[]) {0x09, 0x04}, // 0: is supported language is English (0x0409) - "XMOS", // 1: Manufacturer - XCORE_VOICE_PRODUCT_STR, // 2: Product - "123456", // 3: Serials, should use chip ID - XCORE_VOICE_PRODUCT_STR // 4: Audio Interface + "XMOS", // 1: Manufacturer + XCORE_VOICE_PRODUCT_STR, // 2: Product + "123456", // 3: Serials, should use chip ID + XCORE_VOICE_PRODUCT_STR, // 4: Audio Interface + "DFU FACTORY", // 5: DFU device + "DFU UPGRADE", // 6: DFU device + "DFU DATAPARTITION", // 7: DFU device }; static uint16_t _desc_str[32]; diff --git a/examples/stlp/src/usb/usb_descriptors.h b/examples/stlp/src/usb/usb_descriptors.h index fdd9791f..72adefab 100644 --- a/examples/stlp/src/usb/usb_descriptors.h +++ b/examples/stlp/src/usb/usb_descriptors.h @@ -25,9 +25,15 @@ enum { #if AUDIO_INPUT_ENABLED ITF_NUM_AUDIO_STREAMING_MIC, #endif + ITF_NUM_DFU_MODE, ITF_NUM_TOTAL }; +// Number of DFU alt interfaces +#define DFU_ALT_COUNT 3 +// DFU functional attributes +#define DFU_FUNC_ATTRS (DFU_ATTR_CAN_UPLOAD | DFU_ATTR_CAN_DOWNLOAD | DFU_ATTR_WILL_DETACH | DFU_ATTR_MANIFESTATION_TOLERANT) + // Unit numbers are arbitrary selected #define UAC2_ENTITY_CLOCK 0x01 // Speaker path diff --git a/examples/stlp/src/usb/usb_dfu.c b/examples/stlp/src/usb/usb_dfu.c new file mode 100644 index 00000000..3dc5aceb --- /dev/null +++ b/examples/stlp/src/usb/usb_dfu.c @@ -0,0 +1,221 @@ +/* + * Copyright (c) 2022 XMOS LIMITED. This Software is subject to the terms of the + * XMOS Public License: Version 1 + * + * The MIT License (MIT) + * + * Copyright (c) 2019 Ha Thach (tinyusb.org) + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + * + */ + +#include +#include +#include +#include + +#include "FreeRTOS.h" +#include "timers.h" +#include "platform/driver_instances.h" +#include "tusb.h" + +//--------------------------------------------------------------------+ +// MACRO CONSTANT TYPEDEF PROTYPES +//--------------------------------------------------------------------+ +static void reboot(void); + +//--------------------------------------------------------------------+ +// DFU callbacks +// Note: alt is used as the partition number, in order to support multiple partitions like FLASH, EEPROM, etc. +//--------------------------------------------------------------------+ + +static size_t total_len = 0; +static size_t bytes_avail = 0; +static uint32_t dn_base_addr = 0; + +// Invoked right before tud_dfu_download_cb() (state=DFU_DNBUSY) or tud_dfu_manifest_cb() (state=DFU_MANIFEST) +// Application return timeout in milliseconds (bwPollTimeout) for the next download/manifest operation. +// During this period, USB host won't try to communicate with us. +uint32_t tud_dfu_get_timeout_cb(uint8_t alt, uint8_t state) +{ + if ( state == DFU_DNBUSY ) { + return 10; // 10 ms + } else if (state == DFU_MANIFEST) { + // since we don't buffer entire image and do any flashing in manifest stage + return 0; + } + + return 0; +} + +// Invoked when received DFU_DNLOAD (wLength>0) following by DFU_GETSTATUS (state=DFU_DNBUSY) requests +// This callback could be returned before flashing op is complete (async). +// Once finished flashing, application must call tud_dfu_finish_flashing() +void tud_dfu_download_cb(uint8_t alt, uint16_t block_num, uint8_t const* data, uint16_t length) +{ + rtos_printf("Received Alt %d BlockNum %d of length %d\n", alt, block_num, length); + + unsigned data_partition_base_addr = rtos_dfu_image_get_data_partition_addr(dfu_image_ctx); + switch(alt) { + default: + case 0: + tud_dfu_finish_flashing(DFU_STATUS_ERR_WRITE); + break; + case 1: + if (dn_base_addr == 0) { + total_len = 0; + dn_base_addr = rtos_dfu_image_get_upgrade_addr(dfu_image_ctx); + bytes_avail = data_partition_base_addr - dn_base_addr; + } + /* fallthrough */ + case 2: + if (dn_base_addr == 0) { + total_len = 0; + dn_base_addr = data_partition_base_addr; + bytes_avail = rtos_qspi_flash_size_get(qspi_flash_ctx) - dn_base_addr; + } + rtos_printf("Using addr 0x%x\nsize %u\n", dn_base_addr, bytes_avail); + if(length > 0) { + unsigned cur_addr = dn_base_addr + (block_num * CFG_TUD_DFU_XFER_BUFSIZE); + if((bytes_avail - total_len) >= length) { + rtos_printf("write %d at 0x%x\n", length, cur_addr); + + size_t sector_size = rtos_qspi_flash_sector_size_get(qspi_flash_ctx); + xassert(CFG_TUD_DFU_XFER_BUFSIZE == sector_size); + + uint8_t *tmp_buf = rtos_osal_malloc( sizeof(uint8_t) * sector_size); + rtos_qspi_flash_lock(qspi_flash_ctx); + { + rtos_qspi_flash_read( + qspi_flash_ctx, + tmp_buf, + cur_addr, + sector_size); + memcpy(tmp_buf, data, length); + rtos_qspi_flash_erase( + qspi_flash_ctx, + cur_addr, + sector_size); + rtos_qspi_flash_write( + qspi_flash_ctx, + (uint8_t *) tmp_buf, + cur_addr, + sector_size); + } + rtos_qspi_flash_unlock(qspi_flash_ctx); + rtos_osal_free(tmp_buf); + total_len += length; + } else { + rtos_printf("Insufficient space\n"); + tud_dfu_finish_flashing(DFU_STATUS_ERR_ADDRESS); + } + } + + tud_dfu_finish_flashing(DFU_STATUS_OK); + break; + } +} + +// Invoked when download process is complete, received DFU_DNLOAD (wLength=0) following by DFU_GETSTATUS (state=Manifest) +// Application can do checksum, or actual flashing if buffered entire image previously. +// Once finished flashing, application must call tud_dfu_finish_flashing() +void tud_dfu_manifest_cb(uint8_t alt) +{ + (void) alt; + rtos_printf("Download completed, enter manifestation\n"); + + /* Perform a read to ensure all writes have been flushed */ + uint32_t dummy = 0; + rtos_qspi_flash_read( + qspi_flash_ctx, + (uint8_t *)&dummy, + 0, + sizeof(dummy)); + + /* Reset download */ + dn_base_addr = 0; + + // flashing op for manifest is complete without error + // Application can perform checksum, should it fail, use appropriate status such as errVERIFY. + tud_dfu_finish_flashing(DFU_STATUS_OK); +} + +// Invoked when received DFU_UPLOAD request +// Application must populate data with up to length bytes and +// Return the number of written bytes +uint16_t tud_dfu_upload_cb(uint8_t alt, uint16_t block_num, uint8_t* data, uint16_t length) +{ + uint32_t endaddr = 0; + uint16_t retval = 0; + uint32_t addr = block_num * CFG_TUD_DFU_XFER_BUFSIZE; + + rtos_printf("Upload Alt %d BlockNum %d of length %d\n", alt, block_num, length); + + switch(alt) { + default: + break; + case 0: + if (rtos_dfu_image_get_factory_size(dfu_image_ctx) > 0) { + addr += rtos_dfu_image_get_factory_addr(dfu_image_ctx); + endaddr = rtos_dfu_image_get_factory_addr(dfu_image_ctx) + rtos_dfu_image_get_factory_size(dfu_image_ctx); + } + break; + case 1: + if (rtos_dfu_image_get_upgrade_size(dfu_image_ctx) > 0) { + addr += rtos_dfu_image_get_upgrade_addr(dfu_image_ctx); + endaddr = rtos_dfu_image_get_upgrade_addr(dfu_image_ctx) + rtos_dfu_image_get_upgrade_size(dfu_image_ctx); + } + break; + case 2: + if ((rtos_qspi_flash_size_get(qspi_flash_ctx) - rtos_dfu_image_get_data_partition_addr(dfu_image_ctx)) > 0) { + addr += rtos_dfu_image_get_data_partition_addr(dfu_image_ctx); + endaddr = rtos_qspi_flash_size_get(qspi_flash_ctx); /* End of flash */ + } + break; + } + + if (addr < endaddr) { + rtos_qspi_flash_read(qspi_flash_ctx, data, addr, length); + retval = length; + } + return retval; +} + +// Invoked when the Host has terminated a download or upload transfer +void tud_dfu_abort_cb(uint8_t alt) +{ + (void) alt; + rtos_printf("Host aborted transfer\n"); +} + +// Invoked when a DFU_DETACH request is received +void tud_dfu_detach_cb(void) +{ + rtos_printf("Host detach, we should probably reboot\n"); + reboot(); +} + +static void reboot(void) +{ + rtos_printf("Reboot initiated by tile:0x%x\n", get_local_tile_id()); + write_sswitch_reg_no_ack(get_local_tile_id(), XS1_SSWITCH_WATCHDOG_COUNT_NUM, 0x10000); + write_sswitch_reg_no_ack(get_local_tile_id(), XS1_SSWITCH_WATCHDOG_CFG_NUM, (1 << XS1_WATCHDOG_COUNT_ENABLE_SHIFT) | (1 << XS1_WATCHDOG_TRIGGER_ENABLE_SHIFT) ); + while(1) {;} +} diff --git a/examples/stlp/stlp.cmake b/examples/stlp/stlp.cmake index 52b80090..a51f87ca 100644 --- a/examples/stlp/stlp.cmake +++ b/examples/stlp/stlp.cmake @@ -46,18 +46,38 @@ set(APP_COMMON_LINK_LIBRARIES sdk::lib_src ) -set(STLP_PIPELINES - adec - adec_altarch -) +#********************** +# Pipeline Options +# By default only these targets are created: +# example_stlp_int_fixed_delay +# example_stlp_ua_adec +#********************** +option(ENABLE_ALL_STLP_PIPELINES "Create all STLP pipeline configurations" OFF) + +if(ENABLE_ALL_STLP_PIPELINES) + set(STLP_PIPELINES_INT + fixed_delay + adec + adec_altarch + ) + + set(STLP_PIPELINES_UA + fixed_delay + adec + adec_altarch + ) +else() + set(STLP_PIPELINES_INT + fixed_delay + ) + + set(STLP_PIPELINES_UA + adec + ) +endif() #********************** # XMOS Example Design Targets #********************** include(${CMAKE_CURRENT_LIST_DIR}/stlp_int.cmake) include(${CMAKE_CURRENT_LIST_DIR}/stlp_ua.cmake) - -#********************** -# XMOS Development Targets -#********************** -include(${CMAKE_CURRENT_LIST_DIR}/stlp_dev.cmake) diff --git a/examples/stlp/stlp_dev.cmake b/examples/stlp/stlp_dev.cmake deleted file mode 100644 index 5e8f4d33..00000000 --- a/examples/stlp/stlp_dev.cmake +++ /dev/null @@ -1,210 +0,0 @@ - -set(STLP_DEV_PIPELINES - adec - adec_altarch -) - -option(DEBUG_STLP_USB_MIC_INPUT "Enable stlp usb mic input" OFF) - -set(STLP_UA_COMPILE_DEFINITIONS - ${APP_COMPILE_DEFINITIONS} - appconfI2S_ENABLED=0 - appconfUSB_ENABLED=1 - appconfAEC_REF_DEFAULT=appconfAEC_REF_USB -) - -set(STLP_INT_COMPILE_DEFINITIONS - ${APP_COMPILE_DEFINITIONS} - appconfI2S_ENABLED=1 - appconfUSB_ENABLED=0 - appconfAEC_REF_DEFAULT=appconfAEC_REF_I2S -) - -if(DEBUG_STLP_USB_MIC_INPUT) - list(APPEND STLP_UA_COMPILE_DEFINITIONS appconfMIC_SRC_DEFAULT=appconfMIC_SRC_USB) - list(APPEND STLP_UA_COMPILE_DEFINITIONS appconfUSB_AUDIO_TESTING=appconfUSB_AUDIO_TESTING) -endif() - -foreach(STLP_AP ${STLP_PIPELINES}) - #********************** - # Tile Targets - #********************** - set(TARGET_NAME tile0_example_stlp_dev_ua_${STLP_AP}) - add_executable(${TARGET_NAME} EXCLUDE_FROM_ALL) - target_sources(${TARGET_NAME} PUBLIC ${APP_SOURCES}) - target_include_directories(${TARGET_NAME} PUBLIC ${APP_INCLUDES}) - target_compile_definitions(${TARGET_NAME} - PUBLIC - ${STLP_UA_COMPILE_DEFINITIONS} - THIS_XCORE_TILE=0 - ) - target_compile_options(${TARGET_NAME} PRIVATE ${APP_COMPILER_FLAGS}) - target_link_libraries(${TARGET_NAME} - PUBLIC - ${APP_COMMON_LINK_LIBRARIES} - sln_voice::app::stlp::xcore_ai_explorer - sln_voice::app::stlp::ap::${STLP_AP} - ) - target_link_options(${TARGET_NAME} PRIVATE ${APP_LINK_OPTIONS}) - unset(TARGET_NAME) - - set(TARGET_NAME tile1_example_stlp_dev_ua_${STLP_AP}) - add_executable(${TARGET_NAME} EXCLUDE_FROM_ALL) - target_sources(${TARGET_NAME} PUBLIC ${APP_SOURCES}) - target_include_directories(${TARGET_NAME} PUBLIC ${APP_INCLUDES}) - target_compile_definitions(${TARGET_NAME} - PUBLIC - ${STLP_UA_COMPILE_DEFINITIONS} - THIS_XCORE_TILE=1 - ) - target_compile_options(${TARGET_NAME} PRIVATE ${APP_COMPILER_FLAGS}) - target_link_libraries(${TARGET_NAME} - PUBLIC - ${APP_COMMON_LINK_LIBRARIES} - sln_voice::app::stlp::xcore_ai_explorer - sln_voice::app::stlp::ap::${STLP_AP} - ) - target_link_options(${TARGET_NAME} PRIVATE ${APP_LINK_OPTIONS}) - unset(TARGET_NAME) - - #********************** - # Merge binaries - #********************** - merge_binaries(example_stlp_dev_ua_${STLP_AP} tile0_example_stlp_dev_ua_${STLP_AP} tile1_example_stlp_dev_ua_${STLP_AP} 1) - - #********************** - # Create run and debug targets - #********************** - create_run_target(example_stlp_dev_ua_${STLP_AP}) - create_debug_target(example_stlp_dev_ua_${STLP_AP}) - create_flash_app_target(example_stlp_dev_ua_${STLP_AP}) - - #********************** - # Filesystem support targets - #********************** - if(${CMAKE_HOST_SYSTEM_NAME} STREQUAL Windows) - add_custom_command( - OUTPUT example_stlp_dev_fat_ua_${STLP_AP}.fs - COMMAND ${CMAKE_COMMAND} -E make_directory %temp%/fatmktmp/fs - COMMAND ${CMAKE_COMMAND} -E copy demo.txt %temp%/fatmktmp/fs/demo.txt - COMMAND fatfs_mkimage --input=%temp%/fatmktmp --output=example_stlp_dev_fat_ua_${STLP_AP}.fs - BYPRODUCTS %temp%/fatmktmp - DEPENDS example_stlp_dev_ua_${STLP_AP} - COMMENT - "Create filesystem" - WORKING_DIRECTORY - ${CMAKE_CURRENT_LIST_DIR}/filesystem_support - VERBATIM - ) - else() - add_custom_command( - OUTPUT example_stlp_dev_fat_ua_${STLP_AP}.fs - COMMAND bash -c "tmp_dir=$(mktemp -d) && fat_mnt_dir=$tmp_dir && mkdir -p $fat_mnt_dir && mkdir $fat_mnt_dir/fs && cp ./demo.txt $fat_mnt_dir/fs/demo.txt && fatfs_mkimage --input=$tmp_dir --output=example_stlp_dev_fat_ua_${STLP_AP}.fs" - DEPENDS example_stlp_dev_ua_${STLP_AP} - COMMENT - "Create filesystem" - WORKING_DIRECTORY - ${CMAKE_CURRENT_LIST_DIR}/filesystem_support - VERBATIM - ) - endif() - - add_custom_target(flash_fs_example_stlp_dev_ua_${STLP_AP} - COMMAND xflash --quad-spi-clock 50MHz --factory example_stlp_dev_ua_${STLP_AP}.xe --boot-partition-size 0x100000 --data ${CMAKE_CURRENT_LIST_DIR}/filesystem_support/example_stlp_dev_fat_ua_${STLP_AP}.fs - DEPENDS example_stlp_dev_fat_ua_${STLP_AP}.fs - COMMENT - "Flash filesystem" - VERBATIM - ) - - #********************** - # Tile Targets - #********************** - set(TARGET_NAME tile0_example_stlp_dev_int_${STLP_AP}) - add_executable(${TARGET_NAME} EXCLUDE_FROM_ALL) - target_sources(${TARGET_NAME} PUBLIC ${APP_SOURCES}) - target_include_directories(${TARGET_NAME} PUBLIC ${APP_INCLUDES}) - target_compile_definitions(${TARGET_NAME} - PUBLIC - ${STLP_INT_COMPILE_DEFINITIONS} - THIS_XCORE_TILE=0 - ) - target_compile_options(${TARGET_NAME} PRIVATE ${APP_COMPILER_FLAGS}) - target_link_libraries(${TARGET_NAME} - PUBLIC - ${APP_COMMON_LINK_LIBRARIES} - sln_voice::app::stlp::xcore_ai_explorer - sln_voice::app::stlp::ap::${STLP_AP} - ) - target_link_options(${TARGET_NAME} PRIVATE ${APP_LINK_OPTIONS}) - unset(TARGET_NAME) - - set(TARGET_NAME tile1_example_stlp_dev_int_${STLP_AP}) - add_executable(${TARGET_NAME} EXCLUDE_FROM_ALL) - target_sources(${TARGET_NAME} PUBLIC ${APP_SOURCES}) - target_include_directories(${TARGET_NAME} PUBLIC ${APP_INCLUDES}) - target_compile_definitions(${TARGET_NAME} - PUBLIC - ${STLP_INT_COMPILE_DEFINITIONS} - THIS_XCORE_TILE=1 - ) - target_compile_options(${TARGET_NAME} PRIVATE ${APP_COMPILER_FLAGS}) - target_link_libraries(${TARGET_NAME} - PUBLIC - ${APP_COMMON_LINK_LIBRARIES} - sln_voice::app::stlp::xcore_ai_explorer - sln_voice::app::stlp::ap::${STLP_AP} - ) - target_link_options(${TARGET_NAME} PRIVATE ${APP_LINK_OPTIONS}) - unset(TARGET_NAME) - - #********************** - # Merge binaries - #********************** - merge_binaries(example_stlp_dev_int_${STLP_AP} tile0_example_stlp_dev_int_${STLP_AP} tile1_example_stlp_dev_int_${STLP_AP} 1) - - #********************** - # Create run and debug targets - #********************** - create_run_target(example_stlp_dev_int_${STLP_AP}) - create_debug_target(example_stlp_dev_int_${STLP_AP}) - create_flash_app_target(example_stlp_dev_int_${STLP_AP}) - - #********************** - # Filesystem support targets - #********************** - if(${CMAKE_HOST_SYSTEM_NAME} STREQUAL Windows) - add_custom_command( - OUTPUT example_stlp_dev_fat_int_${STLP_AP}.fs - COMMAND ${CMAKE_COMMAND} -E make_directory %temp%/fatmktmp/fs - COMMAND ${CMAKE_COMMAND} -E copy demo.txt %temp%/fatmktmp/fs/demo.txt - COMMAND fatfs_mkimage --input=%temp%/fatmktmp --output=example_stlp_dev_fat_int_${STLP_AP}.fs - BYPRODUCTS %temp%/fatmktmp - DEPENDS example_stlp_dev_int_${STLP_AP} - COMMENT - "Create filesystem" - WORKING_DIRECTORY - ${CMAKE_CURRENT_LIST_DIR}/filesystem_support - VERBATIM - ) - else() - add_custom_command( - OUTPUT example_stlp_dev_fat_int_${STLP_AP}.fs - COMMAND bash -c "tmp_dir=$(mktemp -d) && fat_mnt_dir=$tmp_dir && mkdir -p $fat_mnt_dir && mkdir $fat_mnt_dir/fs && cp ./demo.txt $fat_mnt_dir/fs/demo.txt && fatfs_mkimage --input=$tmp_dir --output=example_stlp_dev_fat_int_${STLP_AP}.fs" - DEPENDS example_stlp_dev_int_${STLP_AP} - COMMENT - "Create filesystem" - WORKING_DIRECTORY - ${CMAKE_CURRENT_LIST_DIR}/filesystem_support - VERBATIM - ) - endif() - - add_custom_target(flash_fs_example_stlp_dev_int_${STLP_AP} - COMMAND xflash --quad-spi-clock 50MHz --factory example_stlp_dev_int_${STLP_AP}.xe --boot-partition-size 0x100000 --data ${CMAKE_CURRENT_LIST_DIR}/filesystem_support/example_stlp_dev_fat_int_${STLP_AP}.fs - DEPENDS example_stlp_dev_fat_int_${STLP_AP}.fs - COMMENT - "Flash filesystem" - VERBATIM - ) -endforeach() diff --git a/examples/stlp/stlp_int.cmake b/examples/stlp/stlp_int.cmake index 9dc2c548..94b91f37 100644 --- a/examples/stlp/stlp_int.cmake +++ b/examples/stlp/stlp_int.cmake @@ -11,7 +11,7 @@ set(STLP_INT_COMPILE_DEFINITIONS MIC_ARRAY_CONFIG_MCLK_FREQ=12288000 ) -foreach(STLP_AP ${STLP_PIPELINES}) +foreach(STLP_AP ${STLP_PIPELINES_INT}) #********************** # Tile Targets #********************** diff --git a/examples/stlp/stlp_ua.cmake b/examples/stlp/stlp_ua.cmake index e5e26a1d..ba4da195 100644 --- a/examples/stlp/stlp_ua.cmake +++ b/examples/stlp/stlp_ua.cmake @@ -25,7 +25,8 @@ if(DEBUG_STLP_USB_MIC_INPUT_PIPELINE_BYPASS) list(APPEND STLP_UA_COMPILE_DEFINITIONS appconfUSB_AUDIO_SAMPLE_RATE=48000) endif() -foreach(STLP_AP ${STLP_PIPELINES}) +query_tools_version() +foreach(STLP_AP ${STLP_PIPELINES_UA}) #********************** # Tile Targets #********************** @@ -79,7 +80,8 @@ foreach(STLP_AP ${STLP_PIPELINES}) create_debug_target(example_stlp_ua_${STLP_AP}) create_filesystem_target(example_stlp_ua_${STLP_AP}) create_flash_app_target(example_stlp_ua_${STLP_AP}) - + create_upgrade_img_target(example_stlp_ua_${STLP_AP} ${XTC_VERSION_MAJOR} ${XTC_VERSION_MINOR}) + #********************** # Filesystem support targets #********************** @@ -111,6 +113,7 @@ foreach(STLP_AP ${STLP_PIPELINES}) ) endif() + ## Note this takes place of create_flash_app_dfu_target() as there is a filesystem here add_custom_target(flash_fs_example_stlp_ua_${STLP_AP} COMMAND xflash --quad-spi-clock 50MHz --factory example_stlp_ua_${STLP_AP}.xe --boot-partition-size 0x100000 --data ${CMAKE_CURRENT_LIST_DIR}/filesystem_support/example_stlp_ua_${STLP_AP}_fat.fs DEPENDS example_stlp_ua_${STLP_AP}_fat.fs diff --git a/index.rst b/index.rst index 092f275f..86ad3144 100644 --- a/index.rst +++ b/index.rst @@ -14,8 +14,7 @@ XCORE |reg| -VOICE Solutions .. toctree:: :maxdepth: 1 - doc/product_description/index - doc/datasheet/index + doc/introduction/index doc/user_guide/index doc/copyright diff --git a/settings.json b/settings.json index 69ae50ef..233e5794 100644 --- a/settings.json +++ b/settings.json @@ -1,5 +1,5 @@ { "title": "XCORE-VOICE SOLUTION", "project": "sln_voice", - "version": "0.20.0" + "version": "0.21.0-beta.0" } diff --git a/test/README.rst b/test/README.rst index 7a779c76..e03f4363 100644 --- a/test/README.rst +++ b/test/README.rst @@ -37,7 +37,7 @@ Run the following command to deactivate the Conda environment: Install Python Packages ======================= -Install development packages: +Install development packages with the Python virtual environment activated: .. code-block:: console @@ -52,6 +52,5 @@ Functional tests exists for the following: - Audio processing pipelines - Speech recognition command dictionaries - Sample rate conversion -- Example applications To run tests, see the README files located in the directories containing the tests. diff --git a/test/sample_rate_conversion/README.rst b/test/sample_rate_conversion/README.rst index 473b0edf..28b65536 100644 --- a/test/sample_rate_conversion/README.rst +++ b/test/sample_rate_conversion/README.rst @@ -37,7 +37,7 @@ Run the test with the following command from the top of the repository: .. code-block:: console - bash test/commands/check_sample_rate_conversion.sh + bash test/sample_rate_conversion/check_sample_rate_conversion.sh All paths must be absolute. Relative paths may cause errors. diff --git a/test/sample_rate_conversion/test_sample_rate_conversion.py b/test/sample_rate_conversion/test_sample_rate_conversion.py index 3e0f1c0c..b09dc34a 100644 --- a/test/sample_rate_conversion/test_sample_rate_conversion.py +++ b/test/sample_rate_conversion/test_sample_rate_conversion.py @@ -7,7 +7,7 @@ from thdncalculator import THDN_and_freq TEST_CHAN = 0 # Note, the mic channels are swapped on the output -TEST_FREQ = [1000, 2000] # Note, the mic channels are swapped on the output +TEST_FREQ = [2000, 1000] # Note, the mic channels are swapped on the output TEST_SAMPLE_RATE = 48000 def test_48k_output(wav_file, wav_duration): @@ -18,6 +18,6 @@ def test_48k_output(wav_file, wav_duration): assert isclose(TEST_FREQ[ch], freq, rel_tol=(1 / (2 * wav_duration))) THDN_max = ( - -85.0 - ) # in decibels. Actual THD should be -160 or better but capturing artefacts sometimes lower it to -87 + -60.0 + ) # in decibels. Actual THD should be -160 or better but capturing artefacts sometimes lower it to -60 assert THDN_max > THDN diff --git a/tools/ci/build_examples.sh b/tools/ci/build_examples.sh index 41445af0..73686448 100644 --- a/tools/ci/build_examples.sh +++ b/tools/ci/build_examples.sh @@ -23,8 +23,10 @@ examples=( "audio_mux example_audio_mux No XCORE_AI_EXPLORER xmos_cmake_toolchain/xs3a.cmake" "stlp_int_adec example_stlp_int_adec Yes XK_VOICE_L71 xmos_cmake_toolchain/xs3a.cmake" "stlp_int_adec_altarch example_stlp_int_adec_altarch Yes XK_VOICE_L71 xmos_cmake_toolchain/xs3a.cmake" + "stlp_int_fixed_delay example_stlp_int_fixed_delay Yes XK_VOICE_L71 xmos_cmake_toolchain/xs3a.cmake" "stlp_ua_adec example_stlp_ua_adec Yes XK_VOICE_L71 xmos_cmake_toolchain/xs3a.cmake" "stlp_ua_adec_altarch example_stlp_ua_adec_altarch Yes XK_VOICE_L71 xmos_cmake_toolchain/xs3a.cmake" + "stlp_ua_fixed_delay example_stlp_ua_fixed_delay Yes XK_VOICE_L71 xmos_cmake_toolchain/xs3a.cmake" "ffd example_ffd Yes XK_VOICE_L71 xmos_cmake_toolchain/xs3a.cmake" ) @@ -43,7 +45,7 @@ for ((i = 0; i < ${#examples[@]}; i += 1)); do (cd ${path}; rm -rf build_${board}) (cd ${path}; mkdir -p build_${board}) - (cd ${path}/build_${board}; log_errors cmake ../ -DCMAKE_TOOLCHAIN_FILE=${toolchain_file} -DBOARD=${board}; log_errors make ${app_target} -j) + (cd ${path}/build_${board}; log_errors cmake ../ -DCMAKE_TOOLCHAIN_FILE=${toolchain_file} -DBOARD=${board} -DENABLE_ALL_STLP_PIPELINES=1; log_errors make ${app_target} -j) (cd ${path}/build_${board}; cp ${app_target}.xe ${DIST_DIR}) if [ "$run_fs_target" = "Yes" ]; then echo '======================================================' diff --git a/tools/ci/build_tests.sh b/tools/ci/build_tests.sh index a43a781b..a7a5696b 100644 --- a/tools/ci/build_tests.sh +++ b/tools/ci/build_tests.sh @@ -40,6 +40,6 @@ for ((i = 0; i < ${#examples[@]}; i += 1)); do (cd ${path}; rm -rf build_${board}) (cd ${path}; mkdir -p build_${board}) - (cd ${path}/build_${board}; log_errors cmake ../ -DCMAKE_TOOLCHAIN_FILE=${toolchain_file} -DBOARD=${board} ${optional_cache_entry}; log_errors make ${make_target} -j) + (cd ${path}/build_${board}; log_errors cmake ../ -DCMAKE_TOOLCHAIN_FILE=${toolchain_file} -DBOARD=${board} -DENABLE_ALL_STLP_PIPELINES=1 ${optional_cache_entry}; log_errors make ${make_target} -j) (cd ${path}/build_${board}; cp ${make_target}.xe ${DIST_DIR}/example_${name}_test.xe) done diff --git a/xcore_sdk b/xcore_sdk index 1dca91d4..65c544cb 160000 --- a/xcore_sdk +++ b/xcore_sdk @@ -1 +1 @@ -Subproject commit 1dca91d4ca537f448ba2ab779ac521e8748eaeac +Subproject commit 65c544cbdf9da32d7092e8c684b16c87d05960cb