diff --git a/README.md b/README.md index 21ec6ef397..50a7dc256d 100644 --- a/README.md +++ b/README.md @@ -77,6 +77,8 @@ Documentation for latest development version: https://arduino-esp8266.readthedoc If you find the forum useful, please consider supporting it with a donation.
[![Donate](https://img.shields.io/badge/paypal-donate-yellow.svg)](https://www.paypal.com/webscr?cmd=_s-xclick&hosted_button_id=4M56YCWV6PX66) +Feel free to ask questions at our [Gitter channel](https://app.gitter.im/#/room/#esp8266_Arduino:gitter.im). + If you encounter an issue which you think is a bug in the ESP8266 Arduino Core or the associated libraries, or if you want to propose an enhancement, you are welcome to submit it here on Github: https://github.com/esp8266/Arduino/issues. Please provide as much context as possible, as well as the information requested in the issue template: diff --git a/doc/Troubleshooting/debugging.rst b/doc/Troubleshooting/core_debugging.rst similarity index 98% rename from doc/Troubleshooting/debugging.rst rename to doc/Troubleshooting/core_debugging.rst index 4216a0c793..10bfcd8037 100644 --- a/doc/Troubleshooting/debugging.rst +++ b/doc/Troubleshooting/core_debugging.rst @@ -1,5 +1,6 @@ -Debugging -========= +Core Debugging +============== + Introduction ------------ diff --git a/doc/faq/pictures/a02-decode-stack-tace-1-2.png b/doc/Troubleshooting/decode-stack-trace-1-2.png similarity index 100% rename from doc/faq/pictures/a02-decode-stack-tace-1-2.png rename to doc/Troubleshooting/decode-stack-trace-1-2.png diff --git a/doc/faq/pictures/a02-decode-stack-tace-3-6.png b/doc/Troubleshooting/decode-stack-trace-3-6.png similarity index 100% rename from doc/faq/pictures/a02-decode-stack-tace-3-6.png rename to doc/Troubleshooting/decode-stack-trace-3-6.png diff --git a/doc/Troubleshooting/exception-cause-decoding.png b/doc/Troubleshooting/exception-cause-decoding.png new file mode 100644 index 0000000000..2ff357ebc6 Binary files /dev/null and b/doc/Troubleshooting/exception-cause-decoding.png differ diff --git a/doc/Troubleshooting/exception_causes.rst b/doc/Troubleshooting/exception_causes.rst new file mode 100644 index 0000000000..5767d283b0 --- /dev/null +++ b/doc/Troubleshooting/exception_causes.rst @@ -0,0 +1,96 @@ +Exception Causes (EXCCAUSE) +=========================== + + ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| EXCCAUSE | Cause Name | Cause Description | Required | EXCVADDR | +| Code | | | Option | Loaded | ++==========+================================+=========================================+=============+==========+ +| 0 | IllegalInstructionCause | Illegal instruction | Exception | No | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 1 | SyscallCause | SYSCALL instruction | Exception | No | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 2 | InstructionFetchErrorCause | Processor internal physical address or | Exception | Yes | +| | | data error during instruction fetch | | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 3 | LoadStoreErrorCause | Processor internal physical address or | Exception | Yes | +| | | data error during load or store | | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 4 | Level1InterruptCause | Level-1 interrupt as indicated by set | Interrupt | No | +| | | level-1 bits in the INTERRUPT register | | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 5 | AllocaCause | MOVSP instruction, if caller’s | Windowed | No | +| | | registers are not in the register file | Register | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 6 | IntegerDivideByZeroCause | QUOS, QUOU, REMS, or REMU divisor | 32-bit | No | +| | | operand is zero | Integer | | +| | | | Divide | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 7 | Reserved for Tensilica | | | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 8 | PrivilegedCause | Attempt to execute a privileged | MMU | No | +| | | operation when CRING != 0 | | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 9 | LoadStoreAlignmentCause | Load or store to an unaligned address | Unaligned | Yes | +| | | | Exception | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 10..11 | Reserved for Tensilica | | | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 12 | InstrPIFDateErrorCause | PIF data error during instruction fetch | Processor | Yes | +| | | | Interface | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 13 | LoadStorePIFDataErrorCause | Synchronous PIF data error during | Processor | Yes | +| | | LoadStore access | Interface | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 14 | InstrPIFAddrErrorCause | PIF address error during instruction | Processor | Yes | +| | | fetch | Interface | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 15 | LoadStorePIFAddrErrorCause | Synchronous PIF address error during | Processor | Yes | +| | | LoadStore access | Interface | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 16 | InstTLBMissCause | Error during Instruction TLB refill | MMU | Yes | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 17 | InstTLBMultiHitCause | Multiple instruction TLB entries | MMU | Yes | +| | | matched | | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 18 | InstFetchPrivilegeCause | An instruction fetch referenced a | MMU | Yes | +| | | virtual address at a ring level less | | | +| | | than CRING | | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 19 | Reserved for Tensilica | | | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 20 | InstFetchProhibitedCause | An instruction fetch referenced a page | Region | Yes | +| | | mapped with an attribute that does not | Protection | | +| | | permit instruction fetch | or MMU | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 21..23 | Reserved for Tensilica | | | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 24 | LoadStoreTLBMissCause | Error during TLB refill for a load or | MMU | Yes | +| | | store | | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 25 | LoadStoreTLBMultiHitCause | Multiple TLB entries matched for a load | MMU | Yes | +| | | or store | | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 26 | LoadStorePrivilegeCause | A load or store referenced a virtual | MMU | Yes | +| | | address at a ring level less than CRING | | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 27 | Reserved for Tensilica | | | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 28 | LoadProhibitedCause | A load referenced a page mapped with an | Region | Yes | +| | | attribute that does not permit loads | Protection | | +| | | | or MMU | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 29 | StoreProhibitedCause | A store referenced a page mapped with | Region | Yes | +| | | an attribute that does not permit | Protection | | +| | | | or MMU | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 30..31 | Reserved for Tensilica | | | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 32..39 | CoprocessornDisabled | Coprocessor n instruction when cpn | Coprocessor | No | +| | | disabled. n varies 0..7 as the cause | | | +| | | varies 32..39 | | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ +| 40..63 | Reserved | | | | ++----------+--------------------------------+-----------------------------------------+-------------+----------+ + +Infos from Xtensa Instruction Set Architecture (ISA) Reference Manual diff --git a/doc/Troubleshooting/hw-watchdog-example.png b/doc/Troubleshooting/hw-watchdog-example.png new file mode 100644 index 0000000000..bc81800d51 Binary files /dev/null and b/doc/Troubleshooting/hw-watchdog-example.png differ diff --git a/doc/Troubleshooting/improving_exception_decoder_results.rst b/doc/Troubleshooting/improving_exception_decoder_results.rst new file mode 100644 index 0000000000..e0bf3c304f --- /dev/null +++ b/doc/Troubleshooting/improving_exception_decoder_results.rst @@ -0,0 +1,60 @@ +Improving Exception Decoder Results +=================================== + + +Due to the limited resources on the device, our default compiler optimizations +focus on creating the smallest code size (``.bin`` file). The GCC compiler's +option ``-Os`` contains the base set of optimizations used. This set is fine for +release but not ideal for debugging. + +Our view of a crash is often the `Stack Dump `__ +which gets copy/pasted into an Exception Decoder. +For some situations, the optimizer doesn't write caller return addresses to the +stack. When we crash, the list of functions called is missing. And when the +crash occurs in a leaf function, there is seldom if ever any evidence of who +called. + +With the ``-Os`` option, functions called once are inlined into the calling +function. A chain of these functions can optimize down to the calling function. +When the crash occurs in one of these chain functions, the actual location in +the source code is no longer available. + +When you select ``Debug Optimization: Lite`` on the Arduino IDE Tools menu, it +turns off ``optimize-sibling-calls``. Turning off this optimization allows more +caller addresses to be written to the stack, improving the results from the +Exception Decoder. Without this option, the callers involved in the crash may be +missing from the Decoder results. Because of the limited stack space, there is +the remote possibility that removing this optimization could lead to more +frequent stack overflows. You only want to do this in a debug setting. This +option does not help the chained function issue. + +When you select ``Debug Optimization: Optimum``, you get an even more complete +stack trace. For example, chained function calls may show up. This selection +uses the compiler option ``-Og``. GCC considers this the ideal optimization for +the "edit-compile-debug cycle" ... "producing debuggable code." You can read the +specifics at `GCC's Optimize Options `__ + +When global optimization creates build size issues or stack overflow issues, +select ``Debug Optimization: None``, and use a targeted approach with +``#pragma GCC optimize("Og")`` at the module level. Or, if you want to use a +different set of optimizations, you can set optimizations through build options. +Read more at `Global Build Options <../faq/a06-global-build-options.rst>`__. + +For non-Arduino IDE build platforms, you may need to research how to add build +options. Some build platforms already use ``-Og`` for debug builds. + +A crash in a leaf function may not leave the caller's address on the stack. +The return address can stay in a register for the duration of the call. +Resulting in a crash report identifying the crashing function without a +trace of who called. You can encourage the compiler to save the caller's +return address by adding an inline assembly trick +``__asm__ __volatile__("" ::: "a0", "memory");`` at the beginning of the +function's body. Or instead, for a debug build conditional option, use the +macro ``DEBUG_LEAF_FUNCTION()`` from ``#include ``. For compiler +toolchain 3.2.0 and above, the ``-Og`` option is an alternative solution. + +In some cases, adding ``#pragma GCC optimize("Og,no-ipa-pure-const")`` to a +module as well as using ``DEBUG_LEAF_FUNCTION()`` in a leaf function were +needed to display a complete call chain. Or use +``#pragma GCC optimize("Os,no-inline,no-optimize-sibling-calls,no-ipa-pure-const")`` +if you require optimization ``-Os``. diff --git a/doc/faq/a02-my-esp-crashes.rst b/doc/Troubleshooting/index.rst similarity index 64% rename from doc/faq/a02-my-esp-crashes.rst rename to doc/Troubleshooting/index.rst index 0134dda666..6e1906363e 100644 --- a/doc/faq/a02-my-esp-crashes.rst +++ b/doc/Troubleshooting/index.rst @@ -1,4 +1,15 @@ -:orphan: +Troubleshooting +=============== + +.. toctree:: + :maxdepth: 2 + :caption: Contents: + + Core Debugging + Exception Causes + Stack Dumps + Improving Exception Decoder Results + My ESP crashes running some code. How to troubleshoot it? --------------------------------------------------------- @@ -10,11 +21,9 @@ My ESP crashes running some code. How to troubleshoot it? - `What is the Cause of Restart? <#what-is-the-cause-of-restart>`__ - `Exception <#exception>`__ - `Watchdog <#watchdog>`__ -- `Exception Decoder <#exception-decoder>`__ -- `Improving Exception Decoder Results <#improving-exception-decoder-results>`__ - `Other Common Causes for Crashes <#other-causes-for-crashes>`__ -- `If at the Wall, Enter an Issue - Report <#if-at-the-wall-enter-an-issue-report>`__ +- `If nothing else helps, file an issue report + <#if-nothing-else-helps-file-an-issue-repot>`__ - `Conclusion <#conclusion>`__ Introduction @@ -34,7 +43,7 @@ What ESP has to Say Start off by opening a Serial Monitor (Ctrl+Shift+M) to observe the output. Typical crash log looks as follows: -.. figure:: pictures/a02-typical-crash-log.png +.. figure:: typical-crash-log.png :alt: Typical crash log Typical crash log @@ -137,24 +146,36 @@ Exception Typical restart because of exception looks like follows: -.. figure:: pictures/a02-exception-cause-decoding.png +.. figure:: exception-cause-decoding.png :alt: Exception cause decoding Exception cause decoding Start with looking up exception code in the `Exception Causes -(EXCCAUSE) <../exception_causes.rst>`__ -table to understand what kind of issue it is. If you have no clues what -it's about and where it happens, then use `Arduino ESP8266/ESP32 -Exception Stack Trace -Decoder `__ to find -out in which line of application it is triggered. Please refer to `Check -Where the Code Crashes <#check-where-the-code-crashes>`__ point below -for a quick example how to do it. - -**NOTE:** When decoding exceptions be sure to include all lines between -the ``---- CUT HERE ----`` marks in the output to allow the decoder to also -provide the line of code that's actually causing the exception. +(EXCCAUSE) <../Troubleshooting/exception_causes.rst>`__ +table to understand what kind of issue it is. + +Stack dump +^^^^^^^^^^ + +If the ESP crashes the Exception Cause will be shown and the current stack will be dumped. + +:: + + Exception (0): epc1=0x402103f4 epc2=0x00000000 epc3=0x00000000 excvaddr=0x00000000 depc=0x00000000 + + ctx: sys + sp: 3ffffc10 end: 3fffffb0 offset: 01a0 + + + >>>stack>>> + + ........... + + <<`__ Watchdog ^^^^^^^^ @@ -176,7 +197,7 @@ serial monitor. An example of application crash triggered by software wdt is shown below. -.. figure:: pictures/a02-sw-watchdog-example.png +.. figure:: sw-watchdog-example.png :alt: Example of restart by s/w watchdog Example of restart by s/w watchdog @@ -187,7 +208,7 @@ particular line in code where wdt has been triggered. Reset by hardware watchdog timer is shown on picture below. -.. figure:: pictures/a02-hw-watchdog-example.png +.. figure:: hw-watchdog-example.png :alt: Example of restart by h/w watchdog Example of restart by h/w watchdog @@ -204,142 +225,6 @@ out before restart, you should be able to narrow down part of code firing the h/w wdt reset. If diagnosed application or library has debug option then switch it on to aid this troubleshooting. -Exception Decoder -~~~~~~~~~~~~~~~~~ - -Decoding of ESP stack trace is now easy and available to everybody -thanks to great `Arduino ESP8266/ESP32 Exception Stack Trace -Decoder `__ developed -by @me-no-dev. - -Installation for Arduino IDE is quick and easy following the -`installation `__ -instructions. - -If you don't have any code for troubleshooting, use the example below: - -:: - - void setup() - { - Serial.begin(115200); - Serial.println(); - Serial.println("Let's provoke the s/w wdt firing..."); - // - // provoke an OOM, will be recorded as the last occurred one - char* out_of_memory_failure = (char*)malloc(1000000); - // - // wait for s/w wdt in infinite loop below - while(true); - // - Serial.println("This line will not ever print out"); - } - - void loop(){} - - -Enable the Out-Of-Memory (*OOM*) debug option (in the *Tools > Debug Level* -menu), compile/flash/upload this code to your ESP (Ctrl+U) and start Serial -Monitor (Ctrl+Shift+M). You should shortly see ESP restarting every couple -of seconds and ``Soft WDT reset`` message together with stack trace showing -up on each restart. Click the Autoscroll check-box on Serial Monitor to -stop the messages scrolling up. Select and copy the stack trace, including -the ``last failed alloc call: ...`` line, go to the *Tools* and open the -*ESP Exception Decoder*. - -.. figure:: pictures/a02-decode-stack-tace-1-2.png - :alt: Decode the stack trace, steps 1 and 2 - - Decode the stack trace, steps 1 and 2 - -Now paste the stack trace to Exception Decoder's window. At the bottom -of this window you should see a list of decoded lines of sketch you have -just uploaded to your ESP. On the top of the list, like on the top of -the stack trace, there is a reference to the last line executed just -before the software watchdog timer fired causing the ESP's restart. -Check the number of this line and look it up on the sketch. It should be -the line ``Serial.println("Let's provoke the s/w wdt firing...")``, that -happens to be just before ``while(true)`` that made the watchdog fired -(ignore the lines with comments, that are discarded by compiler). - -.. figure:: pictures/a02-decode-stack-tace-3-6.png - :alt: Decode the stack trace, steps 3 through 6 - - Decode the stack trace, steps 3 through 6 - -Armed with `Arduino ESP8266/ESP32 Exception Stack Trace -Decoder `__ you can -track down where the module is crashing whenever you see the stack trace -dropped. The same procedure applies to crashes caused by exceptions. - - Note, to decode the exact line of code where the application - crashed, you need to use ESP Exception Decoder in context of sketch - you have just loaded to the module for diagnosis. Decoder is not - able to correctly decode the stack trace dropped by some other - application not compiled and loaded from your Arduino IDE. - - -Improving Exception Decoder Results -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Due to the limited resources on the device, our default compiler optimizations -focus on creating the smallest code size (``.bin`` file). The GCC compiler's -option ``-Os`` contains the base set of optimizations used. This set is fine for -release but not ideal for debugging. - -Our view of a crash is often the `Stack Dump <../Troubleshooting/stack_dump.rst>`__ -which gets copy/pasted into an Exception Decoder. -For some situations, the optimizer doesn't write caller return addresses to the -stack. When we crash, the list of functions called is missing. And when the -crash occurs in a leaf function, there is seldom if ever any evidence of who -called. - -With the ``-Os`` option, functions called once are inlined into the calling -function. A chain of these functions can optimize down to the calling function. -When the crash occurs in one of these chain functions, the actual location in -the source code is no longer available. - -When you select ``Debug Optimization: Lite`` on the Arduino IDE Tools menu, it -turns off ``optimize-sibling-calls``. Turning off this optimization allows more -caller addresses to be written to the stack, improving the results from the -Exception Decoder. Without this option, the callers involved in the crash may be -missing from the Decoder results. Because of the limited stack space, there is -the remote possibility that removing this optimization could lead to more -frequent stack overflows. You only want to do this in a debug setting. This -option does not help the chained function issue. - -When you select ``Debug Optimization: Optimum``, you get an even more complete -stack trace. For example, chained function calls may show up. This selection -uses the compiler option ``-Og``. GCC considers this the ideal optimization for -the "edit-compile-debug cycle" ... "producing debuggable code." You can read the -specifics at `GCC's Optimize Options `__ - -When global optimization creates build size issues or stack overflow issues, -select ``Debug Optimization: None``, and use a targeted approach with -``#pragma GCC optimize("Og")`` at the module level. Or, if you want to use a -different set of optimizations, you can set optimizations through build options. -Read more at `Global Build Options `__. - -For non-Arduino IDE build platforms, you may need to research how to add build -options. Some build platforms already use ``-Og`` for debug builds. - -A crash in a leaf function may not leave the caller's address on the stack. -The return address can stay in a register for the duration of the call. -Resulting in a crash report identifying the crashing function without a -trace of who called. You can encourage the compiler to save the caller's -return address by adding an inline assembly trick -``__asm__ __volatile__("" ::: "a0", "memory");`` at the beginning of the -function's body. Or instead, for a debug build conditional option, use the -macro ``DEBUG_LEAF_FUNCTION()`` from ``#include ``. For compiler -toolchain 3.2.0 and above, the ``-Og`` option is an alternative solution. - -In some cases, adding ``#pragma GCC optimize("Og,no-ipa-pure-const")`` to a -module as well as using ``DEBUG_LEAF_FUNCTION()`` in a leaf function were -needed to display a complete call chain. Or use -``#pragma GCC optimize("Os,no-inline,no-optimize-sibling-calls,no-ipa-pure-const")`` -if you require optimization ``-Os``. - - Other Causes for Crashes ~~~~~~~~~~~~~~~~~~~~~~~~ @@ -400,8 +285,8 @@ Memory, memory, memory ``ESP.getFreeHeap()`` / ``ESP.getHeapFragmentation()`` / ``ESP.getMaxFreeBlockSize()`` will help the process of finding memory issues. - Now is time to re-read about the `exception decoder - <#exception-decoder>`__. + Now is time to re-read about the `stack dump and exception decoder + <../Troubleshootng/stack_dump.rst>`__. *Some techniques for reducing memory usage* @@ -419,8 +304,8 @@ Stack * Objects that have large data members, such as large arrays, should also be avoided on the stack, and should be dynamically allocated (consider smart pointers). -If at the Wall, Enter an Issue Report -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +If nothing else helps, file an issue report +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Using the procedure above you should be able to troubleshoot all the code you write. It may happen that ESP is crashing inside some library @@ -472,12 +357,15 @@ Conclusion Do not be afraid to troubleshoot ESP exception and watchdog restarts. `Esp8266 / Arduino `__ core provides detailed diagnostics that will help you pin down the issue. Before -checking the s/w, get your h/w right. Use `ESP Exception -Decoder `__ to find -out where the code fails. If you do you homework and are still unable to -identify the root cause, submit an issue report. Provide enough details. -Be specific and isolate the issue. Then ask community for support. There -are plenty of people that like to work with ESP and willing to help with -your problem. +checking the s/w, get your h/w right. Use `our troubleshooting guide +<../Troubleshootng/stack_dump.rst>`__ to find out where the code fails. +If you do you homework and are still unable to identify the root cause, +submit an issue report. Provide enough details. Be specific and isolate the issue. +Then ask community for support. There are plenty of people that like to work with +ESP and willing to help with your problem. + +[ESP8266 Community Forum](https://www.esp8266.com/u/arduinoanswers) is a well-established community for questions and answers about Arduino for ESP8266. Stackoverflow is also an alternative. If you need help, have a "How do I..." type question, have a problem with a 3rd party library not hosted in this repo, or just want to discuss how to approach a problem, please ask there. + +Also check out our [ESP8266 Gitter channel](https://app.gitter.im/#/room/#esp8266_Arduino:gitter.im) `FAQ list :back: `__ diff --git a/doc/Troubleshooting/stack_dump.rst b/doc/Troubleshooting/stack_dump.rst index ef320392e7..f7d9bb628c 100644 --- a/doc/Troubleshooting/stack_dump.rst +++ b/doc/Troubleshooting/stack_dump.rst @@ -1,5 +1,5 @@ -Stack Dumps -=========== +Decoding Stack Dumps +==================== Introduction ------------ @@ -54,15 +54,180 @@ The first number after ``Exception`` gives the cause of the reset. a full list of all causes can be found `here <../exception_causes.rst>`__ the hex after are the stack dump. -Decode -~~~~~~ +Due to the limited resources on the device, our default compiler optimizations +can obfuscate the result. See `how to improve the decoder results `__. -It's possible to decode the Stack to readable information. -You can get a copy and read about the `Esp Exception Decoder `__ tool. -For a troubleshooting example using the Exception Decoder Tool, read `FAQ: My ESP Crashes <../faq/a02-my-esp-crashes.rst#exception-decoder>`__. +Decoding +-------- + +**NOTE:** When decoding exceptions be sure to include **all lines** between +the ``---- CUT HERE ----`` marks in the output to allow the decoder to also +provide the line of code that's actually causing the exception. + + +ESP Excepton Decoder +^^^^^^^^^^^^^^^^^^^^ + +Using https://github.com/me-no-dev/EspExceptionDecoder by @me-no-dev .. figure:: ESP_Exception_Decoderp.png :alt: ESP Exception Decoder ESP Exception Decoder + +`Installation instructions for Arduino IDE 1.x `__ + +If you don't have any code for troubleshooting, use the example below: + +.. code:: cpp + + void setup() + { + Serial.begin(115200); + Serial.println(); + Serial.println("Let's provoke the s/w wdt firing..."); + // + // provoke an OOM, will be recorded as the last occurred one + char* out_of_memory_failure = (char*)malloc(1000000); + // + // wait for s/w wdt in infinite loop below + while(true); + // + Serial.println("This line will not ever print out"); + } + + void loop(){} + + +Enable the Out-Of-Memory (*OOM*) debug option (in the *Tools > Debug Level* +menu), compile/flash/upload this code to your ESP (Ctrl+U) and start Serial +Monitor (Ctrl+Shift+M). You should shortly see ESP restarting every couple +of seconds and ``Soft WDT reset`` message together with stack trace showing +up on each restart. Click the Autoscroll check-box on Serial Monitor to +stop the messages scrolling up. Select and copy the stack trace, including +the ``last failed alloc call: ...`` line, go to the *Tools* and open the +*ESP Exception Decoder*. + +.. figure:: decode-stack-trace-1-2.png + :alt: Decode the stack trace, steps 1 and 2 + + Decode the stack trace, steps 1 and 2 + +Now paste the stack trace to Exception Decoder's window. At the bottom +of this window you should see a list of decoded lines of sketch you have +just uploaded to your ESP. On the top of the list, like on the top of +the stack trace, there is a reference to the last line executed just +before the software watchdog timer fired causing the ESP's restart. +Check the number of this line and look it up on the sketch. It should be +the line ``Serial.println("Let's provoke the s/w wdt firing...")``, that +happens to be just before ``while(true)`` that made the watchdog fired +(ignore the lines with comments, that are discarded by compiler). + +.. figure:: decode-stack-trace-3-6.png + :alt: Decode the stack trace, steps 3 through 6 + + Decode the stack trace, steps 3 through 6 + +Armed with `Arduino ESP8266/ESP32 Exception Stack Trace +Decoder `__ you can +track down where the module is crashing whenever you see the stack trace +dropped. The same procedure applies to crashes caused by exceptions. + +Note, to decode the exact line of code where the application +crashed, you need to use ESP Exception Decoder in context of sketch +you have just loaded to the module for diagnosis. Decoder is not +able to correctly decode the stack trace dropped by some other +application not compiled and loaded from your Arduino IDE. + +decoder.py script +^^^^^^^^^^^^^^^^^ + +Core also includes a standalone script that is able to decode + +.. code:: console + + $ python3 tools/decoder.py --help + usage: decoder.py [-h] [--tool {gdb,addr2line}] [--toolchain-path TOOLCHAIN_PATH] firmware_elf [postmortem] + + positional arguments: + firmware_elf + postmortem + + options: + -h, --help show this help message and exit + --tool {gdb,addr2line} + --toolchain-path TOOLCHAIN_PATH + Sets path to Xtensa tools, when they are not in PATH + +Where 'postmortem' is either path to the file or ``-`` to capture input of some other tool output. + +For example + +.. code:: cpp + + #include + + int* somewhere { nullptr }; + + void setup() { + delay(5000); + Serial.begin(115200); + Serial.printf("%d\n", *somewhere); + } + + void loop() { + } + +Right after booting, device would print the following to the default serial port + +.. code:: console + + --------------- CUT HERE FOR EXCEPTION DECODER --------------- + + Exception (28): + epc1=0x4020105f epc2=0x00000000 epc3=0x00000000 excvaddr=0x00000000 depc=0x00000000 + + >>>stack>>> + + ctx: cont + sp: 3ffffe00 end: 3fffffd0 offset: 0190 + 3fffff90: 0001c200 0000001c 00000000 402018d9 + 3fffffa0: 3fffdad0 00000000 3ffee4bc 40201055 + 3fffffb0: feefeffe feefeffe 3ffee510 40201954 + 3fffffc0: feefeffe feefeffe 3ffe85d8 40100c39 + << Show verbose output during [✓] compile* + +For example + +.. code:: + + C:\Users\USERNAME\AppData\Local\Temp\arduino\sketches\2D54B6F2B852F5DEF454A04EC8FA3CF5 + + +PlatformIO +^^^^^^^^^^ + +Use the built-in `'pio device monitor' exception filter ` or decoder.py script diff --git a/doc/Troubleshooting/sw-watchdog-example.png b/doc/Troubleshooting/sw-watchdog-example.png new file mode 100644 index 0000000000..002e6ade68 Binary files /dev/null and b/doc/Troubleshooting/sw-watchdog-example.png differ diff --git a/doc/Troubleshooting/typical-crash-log.png b/doc/Troubleshooting/typical-crash-log.png new file mode 100644 index 0000000000..1506464d76 Binary files /dev/null and b/doc/Troubleshooting/typical-crash-log.png differ diff --git a/doc/conf.py b/doc/conf.py index 3b05ae5617..254598531c 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -66,7 +66,7 @@ # # This is also used if you do content translation via gettext catalogs. # Usually you set "language" from the command line for these cases. -language = None +language = "en" # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. diff --git a/doc/exception_causes.rst b/doc/exception_causes.rst index 2951c62a4e..4752784e73 100644 --- a/doc/exception_causes.rst +++ b/doc/exception_causes.rst @@ -1,95 +1,3 @@ -Exception Causes (EXCCAUSE) -=========================== +:orphan: -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| EXCCAUSE | Cause Name | Cause Description | Required | EXCVADDR | -| Code | | | Option | Loaded | -+==========+================================+=========================================+=============+==========+ -| 0 | IllegalInstructionCause | Illegal instruction | Exception | No | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 1 | SyscallCause | SYSCALL instruction | Exception | No | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 2 | InstructionFetchErrorCause | Processor internal physical address or | Exception | Yes | -| | | data error during instruction fetch | | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 3 | LoadStoreErrorCause | Processor internal physical address or | Exception | Yes | -| | | data error during load or store | | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 4 | Level1InterruptCause | Level-1 interrupt as indicated by set | Interrupt | No | -| | | level-1 bits in the INTERRUPT register | | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 5 | AllocaCause | MOVSP instruction, if caller’s | Windowed | No | -| | | registers are not in the register file | Register | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 6 | IntegerDivideByZeroCause | QUOS, QUOU, REMS, or REMU divisor | 32-bit | No | -| | | operand is zero | Integer | | -| | | | Divide | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 7 | Reserved for Tensilica | | | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 8 | PrivilegedCause | Attempt to execute a privileged | MMU | No | -| | | operation when CRING != 0 | | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 9 | LoadStoreAlignmentCause | Load or store to an unaligned address | Unaligned | Yes | -| | | | Exception | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 10..11 | Reserved for Tensilica | | | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 12 | InstrPIFDateErrorCause | PIF data error during instruction fetch | Processor | Yes | -| | | | Interface | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 13 | LoadStorePIFDataErrorCause | Synchronous PIF data error during | Processor | Yes | -| | | LoadStore access | Interface | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 14 | InstrPIFAddrErrorCause | PIF address error during instruction | Processor | Yes | -| | | fetch | Interface | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 15 | LoadStorePIFAddrErrorCause | Synchronous PIF address error during | Processor | Yes | -| | | LoadStore access | Interface | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 16 | InstTLBMissCause | Error during Instruction TLB refill | MMU | Yes | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 17 | InstTLBMultiHitCause | Multiple instruction TLB entries | MMU | Yes | -| | | matched | | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 18 | InstFetchPrivilegeCause | An instruction fetch referenced a | MMU | Yes | -| | | virtual address at a ring level less | | | -| | | than CRING | | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 19 | Reserved for Tensilica | | | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 20 | InstFetchProhibitedCause | An instruction fetch referenced a page | Region | Yes | -| | | mapped with an attribute that does not | Protection | | -| | | permit instruction fetch | or MMU | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 21..23 | Reserved for Tensilica | | | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 24 | LoadStoreTLBMissCause | Error during TLB refill for a load or | MMU | Yes | -| | | store | | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 25 | LoadStoreTLBMultiHitCause | Multiple TLB entries matched for a load | MMU | Yes | -| | | or store | | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 26 | LoadStorePrivilegeCause | A load or store referenced a virtual | MMU | Yes | -| | | address at a ring level less than CRING | | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 27 | Reserved for Tensilica | | | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 28 | LoadProhibitedCause | A load referenced a page mapped with an | Region | Yes | -| | | attribute that does not permit loads | Protection | | -| | | | or MMU | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 29 | StoreProhibitedCause | A store referenced a page mapped with | Region | Yes | -| | | an attribute that does not permit | Protection | | -| | | | or MMU | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 30..31 | Reserved for Tensilica | | | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 32..39 | CoprocessornDisabled | Coprocessor n instruction when cpn | Coprocessor | No | -| | | disabled. n varies 0..7 as the cause | | | -| | | varies 32..39 | | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ -| 40..63 | Reserved | | | | -+----------+--------------------------------+-----------------------------------------+-------------+----------+ - -Infos from Xtensa Instruction Set Architecture (ISA) Reference Manual +Moved to `Troubleshooting `__ diff --git a/doc/faq/readme.rst b/doc/faq/readme.rst index cfd65eca90..97b358405d 100644 --- a/doc/faq/readme.rst +++ b/doc/faq/readme.rst @@ -38,7 +38,7 @@ My ESP crashes running some code. How to troubleshoot it? The code may crash because of s/w bug or issue with your h/w. Before entering an issue report, please perform initial troubleshooting. -`Read more `__. +`Read more <../Troubleshooting/index.rst>`__. How can I get some extra KBs in flash ? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/ideoptions.rst b/doc/ideoptions.rst index f591772548..7212235f01 100644 --- a/doc/ideoptions.rst +++ b/doc/ideoptions.rst @@ -150,7 +150,7 @@ Take note some sketches may start working after changing the optimization. Or fail less often. And it is also possible (not likely) that source code that was working with ``-Os`` may break with ``-Og``. -For more topic depth, read `Improving Exception Decoder Results `__ +For more topic depth, read `Improving Exception Decoder Results `__ lwIP variant diff --git a/doc/index.rst b/doc/index.rst index 5f3ec247c6..e276171583 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -19,7 +19,5 @@ Welcome to ESP8266 Arduino Core's documentation! Boards FAQ - Exception causes - Debugging - Stack Dump + Troubleshooting Using with Eclipse