Docker: Replace sleep with s6-notifyoncheck (#6475)

For the `aiida-core-with-services` image where the services are part of
the image, we cannot rely on the health check for the services provided
by docker-build as is used for the `aiida-core-base` case. Instead, a
simple sleep was added to the `aiida-prepare.sh` script that sets up 
the profile, to make sure the services are up before the profile is
created.

This solution is neither elegant nor robust. Here the sleep approach is
replaced by `s6-notifyoncheck`. This hook allows blocking the startup
from continuing until a script returns a 0 exit code. The script in
question first calls `rabbitmq-diagnostics ping` to make sure the
RabbitMQ server is even up, followed by a call to
`rabbitmq-diagnostics check_running`. If the latter returns 0, it means
RabbitMQ is up and running and the script returns 0 as well, which will
trigger `s6-notifyoncheck` to continue with the rest of the startup.

Note that `rabbitmq-diagnostics is_running` could not be used as that
command sometimes returns 0 even if the service is not ready at all.

Co-authored-by: Sebastiaan Huber <[email protected]>

.docker/aiida-core-base/s6-assets/init/aiida-prepare.sh

@@ -18,11 +18,6 @@ verdi config set warnings.development_version False
 # If the environment variable `SETUP_DEFAULT_AIIDA_PROFILE` is not set, set it to `true`.
 if [[ ${SETUP_DEFAULT_AIIDA_PROFILE:-true} == true ]] && ! verdi profile show ${AIIDA_PROFILE_NAME} &> /dev/null; then
 
-    # For the container that includes the services, this script is called as soon as the RabbitMQ startup script has
-    # been launched, but it can take a while for the service to come up. If ``verdi presto`` is called straight away
-    # it is possible it tries to connect to the service before that and it will configure the profile without a broker.
-    sleep 5
-
     # Create AiiDA profile.
     verdi presto \
         --verbosity info \

.docker/aiida-core-with-services/s6-assets/s6-rc.d/rabbitmq/data/check

@@ -0,0 +1,15 @@
+#!/bin/bash
+
+rabbitmq-diagnostics ping
+
+if [ $? -ne 0 ]; then
+    exit 1
+fi
+
+rabbitmq-diagnostics check_running
+
+if [ $? -ne 0 ]; then
+    exit 1
+fi
+
+exit 0

.docker/aiida-core-with-services/s6-assets/s6-rc.d/rabbitmq/notification-fd

@@ -0,0 +1 @@
+3

.docker/aiida-core-with-services/s6-assets/s6-rc.d/rabbitmq/run

@@ -2,5 +2,14 @@
 
 with-contenv
 
-foreground { s6-echo "Calling /etc/init/rabbitmq.sh" }
-rabbitmq-server
+foreground { s6-echo "Starting RMQ server and notifying back when the service is ready" }
+
+
+# For the container that includes the services, aiida-prepare.sh script is called as soon as the RabbitMQ startup script has
+# been launched, but it can take a while for the RMQ service to come up. If ``verdi presto`` is called straight away
+# it is possible it tries to connect to the service before that and it will configure the profile without a broker.
+# Here we use s6-notifyoncheck to do the polling healthy check of the readyness of RMQ service.
+#
+# -w 500: 500 ms between two invocations of ./data/check
+
+s6-notifyoncheck -w 500 rabbitmq-server