UART Communication Delays and Data Loss

[mightChamp]

Board

ESP32 Wroom 32E 4MB Flash

Device Description

We are making an Uart Data logger, an external device send uart data, which stored in SD card.

Hardware Configuration

GPIO13- Uart TX, and GPIO14- Uart RX
Uart setup:
Baud Rate: 115200
Flow Control: Disabled
Buffer Size: 4096

Version

latest development Release Candidate (RC-X)

IDE Name

Arduino IDE

Operating System

Windows 11

Flash frequency

80Mhz

PSRAM enabled

no

Upload speed

921600

Description

We are encountering an issue with UART communication on ESP32 where data reception is delayed, and sometimes data is lost. This issue occurs intermittently and was not present in version v1.0.6.

Sketch

void setup() {
    Serial.begin(115200);  // UART initialized at 115200 baud rate
}

void loop() {
    if (Serial.available()) {
        String data = Serial.readString();
        Serial.println(data);
    }
}

Debug Message

No Debug Message

Other Steps to Reproduce

Set up UART communication between ESP32 and another device.
Send continuous data streams over UART.
Observe occasional delays and missed data.

I have checked existing issues, online documentation and the Troubleshooting Guide

• I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[lbernstone]

The HardwareSerial library has been modified to have more capabilities for handling intermittent and burst data flows. Take a look at the OnReceive and RxTimeout examples to see how to handle high activity and sporadic flows.
@SuGlider This could definitely use some doco

[SuGlider]

@mightChamp - The issue is related only to Core 3.1.0-RC1? Does it also happen with Core 3.0.5?

[SuGlider]

This sounds related to #10397
After testing the sketch from the issue, it seems clear that IDF 4.x and 5.x UART Driver used by ESP32 Arduino HardwareSerial is not efficient and causes a huge performance issue.

The evidence is that whenever Serial0.begin() is executed, there is a significative time slot reduction to the main Arduino Task, affecting the time that loop() has to process data.

In Arduino Core 1.0.6, the UART driver was local one, faster and way more efficient.
The solution for this issue is to refactor HardwareSerial and create its own Arduino UART Driver, the same way it is done for SPI.
@me-no-dev - FYI.

[SuGlider]

@mightChamp - Could you please check if PR #10428 solves this issue too?
This PR improves the loop() execution efficiency and may give it more "CPU cycles" in order to solve the data loss issue.

[mightChamp]

We have tested the fix from PR #10428, but unfortunately, we are still facing the same issue.

This issue is currently blocking our development progress, and we would greatly appreciate an estimate of how long it might take to investigate or resolve this. If there are any further tests or adjustments we could try, we are ready to assist.

[SuGlider]

@mightChamp - based on what @lbernstone wrote, there are a few new methods to deal with UART that were not available in 1.0.6.

• It is possible to create a callback function that is executed as soon as data is available in the UART FIFO using Serial.onReceive(calbackFunc) - please check the example and its commentaries in
  https://github.com/espressif/arduino-esp32/blob/master/libraries/ESP32/examples/Serial/OnReceive_Demo/OnReceive_Demo.ino
  https://github.com/espressif/arduino-esp32/blob/master/cores/esp32/HardwareSerial.h#L229-L239

• Change RX FIFO Timeout and RX FIFO Full events. Those control when the driver will call onReceive callback.
  Arduino Core 1.0.6 sets Timeout to 2 symbols (about 20 bits at the current baud rate ~ for 115200, its is 174 ms) and also sets FIFO Full to 1 character, which forces the backback function to be called as soon as the first byte arrives.
  Example and documentation:
  https://github.com/espressif/arduino-esp32/blob/master/libraries/ESP32/examples/Serial/RxFIFOFull_Demo/RxFIFOFull_Demo.ino
  https://github.com/espressif/arduino-esp32/blob/master/libraries/ESP32/examples/Serial/RxTimeout_Demo/RxTimeout_Demo.ino

• A example that reads UART data flow up to a certain timeout
  https://github.com/espressif/arduino-esp32/blob/master/libraries/ESP32/examples/Serial/onReceiveExample/onReceiveExample.ino

[SuGlider]

I'll build a Python script + the sketch example that is here listed to find out more about the issue.
If I find out something else, I'll write about it here.

[mightChamp]

@mightChamp - based on what @lbernstone wrote, there are a few new methods to deal with UART that were not available in 1.0.6.

• It is possible to create a callback function that is executed as soon as data is available in the UART FIFO using Serial.onReceive(calbackFunc) - please check the example and its commentaries in
  https://github.com/espressif/arduino-esp32/blob/master/libraries/ESP32/examples/Serial/OnReceive_Demo/OnReceive_Demo.ino
  https://github.com/espressif/arduino-esp32/blob/master/cores/esp32/HardwareSerial.h#L229-L239
• Change RX FIFO Timeout and RX FIFO Full events. Those control when the driver will call onReceive callback.
  Arduino Core 1.0.6 sets Timeout to 2 symbols (about 20 bits at the current baud rate ~ for 115200, its is 174 ms) and also sets FIFO Full to 1 character, which forces the backback function to be called as soon as the first byte arrives.
  Example and documentation:
  https://github.com/espressif/arduino-esp32/blob/master/libraries/ESP32/examples/Serial/RxFIFOFull_Demo/RxFIFOFull_Demo.ino
  https://github.com/espressif/arduino-esp32/blob/master/libraries/ESP32/examples/Serial/RxTimeout_Demo/RxTimeout_Demo.ino
• A example that reads UART data flow up to a certain timeout
  https://github.com/espressif/arduino-esp32/blob/master/libraries/ESP32/examples/Serial/onReceiveExample/onReceiveExample.ino

Despite testing all examples, we encountered significant performance degradation, and also faces missing data bytes and delay issue.

[SuGlider]

Despite testing all examples, we encountered significant performance degradation, and also faces missing data bytes and delay issue.

Please elaborate more about "significant performance degradation", "missing data bytes" and "delay issue".
If you could post an example (sketch + PC side script + details) for each issue, it would accelerate the resolution and also help us to validate a possible fix.

You can count on me for fixing it ASAP.

Thanks!

[mightChamp]

@SuGlider

We are experiencing a delay issue that creates the illusion of missing bytes due to timeout. Also PR #10429, reduces this delay, but still face some delay, as compared to V1.0.6.

Our project is quite large, using 1.6 MB of Flash and 35% of RAM, with two additional tasks running. The code works perfectly in v1.0.6, but we are looking to upgrade to the latest SDK. Due to the size and complexity of the code, providing a simplified version for testing is challenging.

[SuGlider]

@mightChamp - Could you please test it using this sketch:

void setup() {
    Serial.begin(115200);  // UART initialized at 115200 baud rate
    // settings used by Core 1.0.6:
    Serial.setRxFIFOFull(1); // it will push every single byte from FIFO directly to the Arduino RX Buffer
    Serial.setRxTimeout(2); // RX Timeout is set to 2 symbols (at 115200 bps, it is about 18us)
}

void loop() {
    if (Serial.available()) {
        String data = Serial.readString();
        Serial.println(data);
    }
}