Scan process hangs in Docker container after downtime

[marcus67]

There seems to be a problem when a Docker container is put into hibernation. After wake up LittleBrother resumes its work but gets stuck in the next call contacting the master process. See the final lines of the log output below:

2021-01-29 09:27:56,483 - App - DEBUG - Executing task app_control.check 0.228 [s] behind schedule... *** START ***
2021-01-29 09:27:56,484 - MasterConnector - DEBUG - Executing POST API call 'http://casimir.chaos:5555/api/events'
2021-01-29 09:27:56,745 - App - DEBUG - Executing task app_control.check 0.228 [s] behind schedule... *** END ***
2021-01-29 09:27:56,746 - App - DEBUG - Sleeping for 4.493694 seconds (or until next signal)
2021-01-29 09:28:01,244 - App - DEBUG - Woken by signal
2021-01-29 09:28:01,244 - App - DEBUG - Executing task app_control.scan_processes(ProcessHandler) 0.005 [s] behind schedule... *** START ***
2021-01-29 09:28:01,244 - ClientProcessHandler - DEBUG - Scanning processes...
2021-01-29 09:28:01,259 - App - DEBUG - Executing task app_control.scan_processes(ProcessHandler) 0.005 [s] behind schedule... *** END ***
2021-01-29 09:28:01,259 - App - DEBUG - Sleeping for 0.223581 seconds (or until next signal)
2021-01-29 09:28:01,483 - App - DEBUG - Woken by signal
2021-01-29 09:28:01,484 - App - DEBUG - Executing task app_control.check 0.001 [s] behind schedule... *** START ***
2021-01-29 09:28:01,484 - MasterConnector - DEBUG - Executing POST API call 'http://casimir.chaos:5555/api/events'
2021-01-29 09:28:21,500 - MasterConnector - ERROR - Unknown error code 'None' with message 'None' while accessing artifact 'http://casimir.chaos:5555/api/events'
2021-01-29 09:28:21,500 - AppControl - ERROR - Exception 'HTTPConnectionPool(host='casimir.chaos', port=5555): Max retries exceeded with url: /api/events (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6c87099f60>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))' while sending events to master
2021-01-29 09:28:21,500 - App - DEBUG - Executing task app_control.check 0.001 [s] behind schedule... *** END ***
2021-01-29 09:28:21,501 - App - DEBUG - Executing task app_control.scan_processes(DeviceHandler) 15.024 [s] behind schedule... *** START ***
2021-01-29 09:28:21,509 - App - DEBUG - Executing task app_control.scan_processes(DeviceHandler) 15.024 [s] behind schedule... *** END ***
2021-01-29 09:28:21,510 - App - DEBUG - Executing task app_control.check 15.026 [s] behind schedule... *** START ***
2021-01-29 09:28:21,510 - MasterConnector - DEBUG - Executing POST API call 'http://casimir.chaos:5555/api/events'
2021-01-29 09:46:54,484 - MasterConnector - ERROR - Unknown error code 'None' with message 'None' while accessing artifact 'http://casimir.chaos:5555/api/events'
2021-01-29 09:46:54,484 - AppControl - ERROR - Exception '('Connection aborted.', OSError(113, 'No route to host'))' while sending events to master
2021-01-29 09:46:54,484 - App - DEBUG - Executing task app_control.check 15.026 [s] behind schedule... *** END ***
2021-01-29 09:46:54,484 - App - DEBUG - Executing task app_control.scan_processes(ProcessHandler) 1123.240 [s] behind schedule... *** START ***
2021-01-29 09:46:54,484 - ClientProcessHandler - DEBUG - Scanning processes...
2021-01-29 09:46:54,497 - App - DEBUG - Executing task app_control.scan_processes(ProcessHandler) 1123.240 [s] behind schedule... *** END ***
2021-01-29 09:46:54,497 - App - INFO - Detected delay of 1123.240473 seconds -> adding to downtime timer
2021-01-29 09:46:54,498 - App - DEBUG - Sleeping for 4.999957 seconds (or until next signal)
2021-01-29 09:46:59,501 - App - DEBUG - Woken by signal
2021-01-29 09:46:59,501 - App - DEBUG - Executing task app_control.check 0.003 [s] behind schedule... *** START ***
2021-01-29 09:46:59,501 - MasterConnector - DEBUG - Executing POST API call 'http://casimir.chaos:5555/api/events'

[marcus67]

Remark: after a restart of the container the process is fine again.