LMIC and sleeping MCUs

[Mark-Wills]

Question: what is the suggested method/technique to use this library with an MCU that sleeps? I'm using it with AVR, and I sleep the MCU for blocks of 8 seconds. I've used this technique successfully before, however, while attempting to use this technique with the LMIC library the system crashes and resets.
Are there any special considerations / rules that I need to follow to allow the MCU to sleep? Do I need to adjust the LMIC clock in some way?
Thanks
Mark

[terrillmoore]

Hi @Mark-Wills, when I use this library while sleeping, there are two things I worry about.

I am careful not to sleep with the LMIC is active. This can be done in a number of ways, because on a Class A device, after joining, the LMIC only becomes active when the application transmits a message, and it will become inactive after a fairly predictable delay (worst case). The delay can vary based on network behavior, etc.

However, as we add the rest of the obscure features of the spec, this becomes less predictable, because the network can do things like sending a chain of confirmed downlinks. This will keep the LMIC active indefinitely. Because of this concern, there's a newish API, os_queryTimeCriticalJobs(ostime_t delay). This lets the application ask "is it safe not to service the LMIC for delay ticks. If there are time critical jobs sheduled before delay from now, it's not safe to stop polling the LMIC. This can be used to control decisions whether to sleep the CPU.

The second critical issue with the current LMIC HAL is the value of millis() -- this must advance in sync with real time. You must arrange to update this after you wake up, or you must modify the code that implements os_getTime() to allow it to return a value that is otherwise locked to real time.

On the Adafruit / M0 BSP, I had to modify the BSP to add an API, adjust_millis_forward(); I tried pushing this upstream, but got no interest, so MCCI did our own BSP (which is how we got a BSP that adds the ability to adjust all the LMIC config things from the Tools menu).

[Mark-Wills]

@terrillmoore

Thanks very much for the heads up regarding os_queryTimeCriticalJobs(ostime_t delay).
I've added some code to my application to check it and it seems to work beautifully. The only issue is that it doesn't work just prior to a network join. It says it's okay to sleep the MCU but milliseconds later it starts the join process. However, I can get around this by setting a flag in the EV_JOINED state, such that the sleep code will only allow the MCU to sleep when LMIC says it's okay, AND we have successfully joined the network.

Your comments regarding changing millis() are interesting. I looked through the LMIC code to see how easy it would be to modify the library. However, after giving it some thought I figured it might be better to acheive this at the application level - afterall, it's the application that is putting the MCU to sleep, so it could be argued that it's the application's responsibility to ensure that the clock/millis is updated correctly.

I came up with the following code (which is specific to the AVR, I'm afraid):

extern volatile unsigned long timer0_millis;
...
...
void advanceMillis(unsigned long offset) {
  uint8_t oldSREG = SREG;
  cli();
  timer0_millis += offset;
  SREG = oldSREG;
}

I put the ATMEGA to sleep in blocks of 8 seconds, which the largest amount of time it can sleep for. So it could be used like this:

setLowPowerMode(SLEEP_8_SECONDS);
goToSleep();
advanceMillis(8000);

setLowPowerMode() and goToSleep() are my own functions, but you get the idea.

I have tested the above code in isolation and it appears to work perfectly:

extern volatile unsigned long timer0_millis;

void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);
}

void loop() {
  // put your main code here, to run repeatedly:
  Serial.println(millis());
  delay(2000);
  advanceMillis(10000);
}

void advanceMillis(unsigned long offset) {
  uint8_t oldSREG = SREG;
  cli();
  timer0_millis += offset;
  SREG = oldSREG;
}

Next step will be to integrate this into my application and see if I can pull it all together. Adding this functionality to the library might be tricky as there is no gurantee that millis() is implemented in the same way across different MCU families. Therefore it is probably best left to the application programmer to solve this. If you're interested, I could write some AVR example code that uses LMIC and sleeps the MCU and send you a pull request for inclusion in the example applications, assuming my GitHub foo is up to the job :-) .

Regards

Mark

[terrillmoore]

@Mark-Wills, would be happy to include such an example.

[cstratton]

What I did when I was still using LMiC was to sleep in the OS scheduler itself, until just before the next scheduled event, in effect by replacing the delay function that was being used with a state-preserving sleep (my sleep code busy waits instead if given a short interval). For a while I had odd code to patchup the running ticks counter from the RTC on wake, eventually I just used a fresh RTC reading anywhere the time was needed (and because the RTC on my chip wants to do leap years and other nastiness that would make converting back to monotonic time painful, I only let it run from midnight 1/1/1970 for about ten minutes, then accumulate that to an offset maintained by the reading code and reset the RTC)

I'd avoid not sleeping when unjoined, as that means that a node that powers up out of range of a gateway with working backhaul will set about quite enthusiastically draining its battery.

[terrillmoore]

@cstratton That will work. There is a known bug (#2) with join implementation that causes battery drainage as well as network issues if you have a dense deployment of thousands of devices; I’ve just identified the root cause so perhaps I’ll have a fix for that someday soon. This bug is (like many LMIC things) much worse for US-like bandplans. For EU, it honors the duty cycle.

[OstertagM]

Oh was just simple.
Do I something wrong? I use ABP.
I have added only some code in the EV_TXCOMPLETE case:

   wake_count = LMIC.seqnoUp;
            ets_printf(fmt_str, wake_count);

            //Sleep for some seconds 
            //Work with a fixed sleep cycle
            #ifdef FORCE_DEEPSLEEP
            Serial.println("Going to sleep now");
            delay(1000);
            Serial.flush(); 
            esp_deep_sleep_start();
            Serial.println("This will never be printed");         
            #endif

the wake_count I'll reload in setup.
It seems to work... I don't know if I can get an downlink when... Perhaps not. But this is not needed in this case...

[terrillmoore]

@OstertagM yes, as long as you save wake_count across boots, you should be fine. EV_TXCOMPLETE is a good place to do this. However, it's best to let the LMIC get to an idle state by using os_queryTimeCriticalJobs(), passing sec2osticks(sleepIntervalInSeconds) as the paramter, and waiting until it says there are no jobs scheduled. Also a good idea to save the channel configuration, as even for ABP devices, the network is allowed to add channels and otherwise re-configure you. Unfortunately there's not yet a simple API for doing this. I'm hoping to do that once I'm convinced 3.0.99 is stable.

[geppoleppo]

Hi Guys, I ask you for help on a firmware problem.
I have created a code for the opening / closing of solenoid valves.
In the loop I check if there are TX / RX activities that are running.
If nothing is found, the device goes to sleep until the next job (which corresponds to sending an Uplink.
The code worked with version 2.3. of Lmic while with the new code it doesn't work anymore.
Do you have any ideas?
Below is the routine code.
Thanks

void loop(void) {
os_runloop_once(); // Execute the LMIC loop
const int32_t millis_to_next_task = run_one_task();
// If there is a TX/RX job running, don't sleep
if (LMIC.opmode != OP_NEXTCHNL) {
return;
}
//Serial.println(F("NEXT TASK"));
//Serial.println(millis_to_next_task);
if (millis_to_next_task == -1) {
// No task is scheduled!
Serial.println(F("ERROR: no task scheduled"));
sleep_millis(8192);
} else if (millis_to_next_task > 0) {
sleep_millis(millis_to_next_task);
}
}

[OstertagM]

What does not work? The sleep period the downlink / uplink?

[geppoleppo]

LMIC.opmode
is always ! = by OP_NEXTCHNL
and therefore the node never goes to sleep.

[OstertagM]

switch on the debug of lmic and post it here. What activation method do you use? Are you joined?

[Mark-Wills]

You can use os_queryTimeCriticalJobs to determine if the unit can sleep or not.

if(!os_queryTimeCriticalJobs(ms2osticksRound(9000)))
  Serial.println("Can sleep");
  else Serial.println("Cannot sleep");

The code above determines if there are any scheduled jobs within the next 9 seconds.

Regards

Mark

[OstertagM]

@Mark-Wills OK thanks also I will try. Could be better for me.

[OstertagM]

One thing is setting to deep sleep to very long time you if you Arduino youd should do:

#define uS_TO_S_FACTOR 1000000ULL  /* Conversion factor for micro seconds to seconds */
#define TIME_TO_SLEEP 3600 /* Time ESP32 will go to sleep (in seconds) */

Because all examples shows it without ULL and I wondered why it didn't worked even it should.
Now it works.
Both sleep modes. With external Button wake up and Timer at same time.
I tested it from Mark-Wills and found out that this is not so good. as I thought.
I schedule it it main loop this but that was not so good. Now I do this check in the

        case EV_TXCOMPLETE:
            Serial.println(F("EV_TXCOMPLETE (includes waiting for RX windows)"));
             if (LMIC.dataLen) {
        // data received in rx slot after tx
        Serial.print(F("Received "));
        Serial.print(LMIC.dataLen);
        Serial.print(F(" bytes of payload: 0x"));
        for (int i = 0; i < LMIC.dataLen; i++) {
          if (LMIC.frame[LMIC.dataBeg + i] < 0x10) {
            Serial.print(F("0"));
        }
        Serial.print(LMIC.frame[LMIC.dataBeg + i], HEX);
    }
    Serial.println();
             }       
            
            if (LMIC.txrxFlags & TXRX_ACK)
              Serial.println(F("Received ack"));
            if (LMIC.dataLen) {
              Serial.println(F("Received "));
              Serial.println(LMIC.dataLen);
              Serial.println(F(" bytes of payload"));
            }
            
            //Session:
            Serial.println(F("Value of seqnoUp: "));
            //Zustand speichern für deep Sleep
            wake_count = LMIC.seqnoUp;
            ets_printf(fmt_str, wake_count);

            //Sleep for some seconds 
            //Work with a fixed sleep cycle
            #ifdef FORCE_DEEPSLEEP
            Serial.println("Going to sleep now");
            delay(1000);
            Serial.flush(); 
            esp_deep_sleep_start();
            //Serial.println("This will never be printed");         
            #endif
                        
            // Schedule next transmission
            os_setTimedCallback(&sendjob, os_getTime()+sec2osticks(TX_INTERVAL), do_send);
            break;

But without testing this query. I saw that I am here always in the case to can a sleep.

[cyberman54]

@terrillmoore

Also a good idea to save the channel configuration

What would be a best practice to do this? Saving and reloading (which?) parts of LMIC struct?

[OstertagM]

As you can see I just save
LMIC.seqnoUp
And reload it

[Beliwars]

@terrillmoore

Thanks very much for the heads up regarding os_queryTimeCriticalJobs(ostime_t delay).
I've added some code to my application to check it and it seems to work beautifully. The only issue is that it doesn't work just prior to a network join. It says it's okay to sleep the MCU but milliseconds later it starts the join process. However, I can get around this by setting a flag in the EV_JOINED state, such that the sleep code will only allow the MCU to sleep when LMIC says it's okay, AND we have successfully joined the network.

Your comments regarding changing millis() are interesting. I looked through the LMIC code to see how easy it would be to modify the library. However, after giving it some thought I figured it might be better to acheive this at the application level - afterall, it's the application that is putting the MCU to sleep, so it could be argued that it's the application's responsibility to ensure that the clock/millis is updated correctly.

I came up with the following code (which is specific to the AVR, I'm afraid):

extern volatile unsigned long timer0_millis;
...
...
void advanceMillis(unsigned long offset) {
  uint8_t oldSREG = SREG;
  cli();
  timer0_millis += offset;
  SREG = oldSREG;
}

I put the ATMEGA to sleep in blocks of 8 seconds, which the largest amount of time it can sleep for. So it could be used like this:

setLowPowerMode(SLEEP_8_SECONDS);
goToSleep();
advanceMillis(8000);

setLowPowerMode() and goToSleep() are my own functions, but you get the idea.

I have tested the above code in isolation and it appears to work perfectly:

extern volatile unsigned long timer0_millis;

void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);
}

void loop() {
  // put your main code here, to run repeatedly:
  Serial.println(millis());
  delay(2000);
  advanceMillis(10000);
}

void advanceMillis(unsigned long offset) {
  uint8_t oldSREG = SREG;
  cli();
  timer0_millis += offset;
  SREG = oldSREG;
}

Next step will be to integrate this into my application and see if I can pull it all together. Adding this functionality to the library might be tricky as there is no gurantee that millis() is implemented in the same way across different MCU families. Therefore it is probably best left to the application programmer to solve this. If you're interested, I could write some AVR example code that uses LMIC and sleeps the MCU and send you a pull request for inclusion in the example applications, assuming my GitHub foo is up to the job :-) .

Regards

Mark

Hello Mark,

Do you know how could this be done on ESP32?

[OstertagM]

I think there are many different ways to do it with an Esp32. Essentially is to store the state know how many time is gone and restore the state. Perhaps somebody can tell here which variables or functions returns the state in joined case.

[Beliwars]

Regarding that, #460 (comment)
I describe there what I've saved, but only thinking about LMIC session so I don't need to Join again; maybe should I save the value of osticks before sleep and then write a function which will add the defined offset? how long before it rolls-over? There are some bitwise operations to handle the overflow of the micros() on the hal_ticks() function which is the return value of os_getTime()

[terrillmoore]

As mentioned in #460, see https://github.com/mcci-catena/arduino-lorawan for a (c++) example of saving and restoring session information, and https://githhub.com/mcci-catena/Catena-Arduino-Platform for a (very elaborate) library that actually uses the apis to save and restore info as needed.

[Beliwars]

thanks for your reply @terrillmoore, I've already looked at https://github.com/mcci-catena/arduino-lorawan but only at the examples level and well let's say my code was already at an advanced state when I found this wrapper library so I didn't want to redo all my work and go to a C++ styled coding; but know looking at the structure inside the classes I can see this:

struct SessionInfoV2
{
// to ensure packing, we just repeat the header.
uint8_t Tag; // kSessionInfoTag_V1
uint8_t Size; // sizeof(SessionInfoV1)
uint8_t Region; // selected region.
uint8_t LinkADR; // Current link ADR (per [1.0.2] 5.2)
uint32_t NetID; // the network ID
uint32_t DevAddr; // device address
uint8_t NwkSKey[16]; // network session key
uint8_t AppSKey[16]; // app session key
uint32_t FCntUp; // uplink frame count
uint32_t FCntDown; // downlink frame count
SessionChannelMask Channels; // info about the enabled
// channels.
uint16_t Country; // Country code
int16_t LinkIntegrity; // the link-integrity counter.
uint8_t Redundancy; // NbTrans (in bits 3:0)
uint8_t DutyCycle; // Duty cycle (per [1.0.2] 5.3)
// TODO([email protected]) complete
};

I guess I have to save all this variables from the LMIC, on bold is what I'm at the moment saving before going into deepsleep, I guess I can add the other variables as well and save them on RTC memory space of ESP32.

Regarding https://githhub.com/mcci-catena/Catena-Arduino-Platform, could you be a little more specific, it's a huge library and I'm a little bit lost on what should I be looking.

Thanks again and regards from Paris