add high-level architecture diagram

neutrons · Sep 5, 2024 · eae3bd2 · eae3bd2
1 parent c41defb
commit eae3bd2
Show file tree

Hide file tree

Showing 2 changed files with 149 additions and 9 deletions.
diff --git a/docs/developer/design/communication_flows.rst b/docs/developer/design/communication_flows.rst
@@ -1,6 +1,10 @@
+.. _communication_flows:
+
 ActiveMQ Communication Flows
 ============================
 
+.. contents:: :local:
+
 Autoreducers
 ------------
 
@@ -95,7 +99,7 @@ parameters for instruments that have implemented
 
 DASMON
 ------
-DASMON, from Data Acquisition System (DAS) Monitor, provides process variable (PV) updates from the
+DASMON, from Data Acquisition (DAQ) System Monitor, provides process variable (PV) updates from the
 beamlines to WebMon. Due to the high volume of PV updates, DASMON writes directly to the database.
 
 .. mermaid::
@@ -105,13 +109,12 @@ beamlines to WebMon. Due to the high volume of PV updates, DASMON writes directl
         participant Workflow DB
         participant Dasmon listener
         DASMON->>Workflow DB: PV update
-        Workflow DB->>Dasmon listener: PV update
         DASMON->>Dasmon listener: Heartbeats
 
 Dasmon listener
 ---------------
 
-Streaming Message Service (SMS)
+Stream Management Service (SMS)
 ...............................
 
 .. mermaid::
@@ -138,12 +141,35 @@ admins by email when a service has missed heartbeats (needs to be verified that
         participant Dasmon listener
         participant Workflow DB
         actor Subscribed users
-        Other services->>Dasmon listener: Heartbeat
-        Dasmon listener->>Workflow DB: Status update
-        opt There are missed heartbeats
+        loop Every N s
+            Other services->>Dasmon listener: Heartbeat
+            Dasmon listener->>Workflow DB: Status update
+        end
+        opt Service has 3 missed heartbeats
             Dasmon listener->>Subscribed users: Email
         end
 
+
+.. mermaid::
+
+    flowchart LR
+        SMS["SMS (per beamline)"]
+        PVSD["PVSD (per beamline)"]
+        DASMON["DASMON (per beamline)"]
+        STC
+        Autoreducers
+        DasmonListener
+        WorkflowDB[(DB)]
+        SMS-->|heartbeat|DasmonListener
+        PVSD-->|heartbeat|DasmonListener
+        DASMON-->|heartbeat|DasmonListener
+        STC-->|heartbeat|DasmonListener
+        Autoreducers-->|heartbeat|DasmonListener
+        WorkflowManager-->|heartbeat|DasmonListener
+        DasmonListener-->|heartbeat|DasmonListener
+        DasmonListener-->WorkflowDB
+        DasmonListener-.->|if missed 3 heartbeats|InstrumentScientist
+
 WebMon interaction with Live Data Server
 ----------------------------------------
 
@@ -164,13 +190,15 @@ or HTML div) to Live Data Server.
         participant Live Data Server
 
         WebMon->>Autoreducer: REDUCTION.DATA_READY
-        Autoreducer->>Live Data Server: publish_plot
-        Note over Live Data Server: Store plot in DB
+        opt Publish plot
+            Autoreducer->>Live Data Server: publish_plot
+            Note over Live Data Server: Store plot in DB
+        end
 
 Display plot from Live Data Server
 ................................
 
-Run overview pages (monitor.sns.gov/report/<instrument>/<run number>/) will query the Live
+Run overview pages (``monitor.sns.gov/report/<instrument>/<run number>/``) will query the Live
 Data Server for a plot for that instrument and run number and display it if available.
 
 .. mermaid::

diff --git a/docs/developer/design/overview.rst b/docs/developer/design/overview.rst
@@ -5,3 +5,115 @@ Design Overview
    :maxdepth: 1
 
    communication_flows
+
+
+High-level architecture
+-----------------------
+
+The diagram below describes the high-level architecture of WebMon. The gray box labeled "DAS" are
+services owned by the Data Acquisition System team that feed information to WebMon but are not part
+of WebMon. The services communicate mainly through an ActiveMQ message broker. The communication
+flow is described in :ref:`communication_flows`.
+
+.. mermaid::
+
+    flowchart LR
+        FileArchive[("`SNS/HFIR
+        File archive`")]
+        subgraph DAS
+            DASMON
+            TranslationService["`Streaming
+            Translation
+            Client
+            (STC)`"]
+            SMS["`Stream
+            Management
+            Service
+            (SMS)`"]
+            PVSD
+        end
+        subgraph "Workflow Manager"
+            WorkflowManager[Workflow Manager]
+            DasmonListener[Dasmon listener]
+            Database[(DB)]
+        end
+        subgraph Autoreduction
+            Autoreducers-->|cataloging|ONCat
+            LiveDataServer<-->LiveDataDB[(DB)]
+            LiveReduce
+        end
+        subgraph Frontend
+            WebMon["`WebMon
+            monitor.sns.gov`"]
+        end
+        SMS-->|live data stream|LiveReduce
+        TranslationService-->|trigger reduction|WorkflowManager
+        DASMON-->|status|DasmonListener
+        DASMON-->|PV updates|Database
+        WorkflowManager<-->|instructions|Autoreducers
+        Autoreducers-->|publish plots|LiveDataServer
+        Database-->WebMon
+        ONCat-->|run metadata|WebMon
+        LiveDataServer-->|plots|WebMon
+        LiveReduce-->|publish plots|LiveDataServer
+        DasmonListener-->Database
+        WorkflowManager-->Database
+        TranslationService-->|NeXus file|FileArchive
+        FileArchive<-->Autoreducers
+        style DAS fill:#D3D3D3, stroke-dasharray: 5 5
+
+
+High-level architecture
+-----------------------
+
+The diagram below describes the high-level architecture of WebMon. The gray box labeled "DAS" are
+services owned by the Data Acquisition System team that feed information to WebMon, e.g. to trigger
+autoreduction when a run is finished. The services mainly communicate through an ActiveMQ message
+broker. The communication flow is described in :ref:`communication_flows`.
+
+.. mermaid::
+
+    flowchart LR
+        FileArchive[("`SNS/HFIR
+        File archive`")]
+        subgraph DAS
+            DASMON
+            TranslationService["`Streaming
+            Translation
+            Client
+            (STC)`"]
+            SMS["`Stream
+            Management
+            Service
+            (SMS)`"]
+            PVSD
+        end
+        subgraph "Workflow Manager"
+            WorkflowManager[Workflow Manager]
+            DasmonListener[Dasmon listener]
+            Database[(DB)]
+        end
+        subgraph Autoreduction
+            Autoreducers-->|REST API|ONCat
+            LiveDataServer<-->LiveDataDB[(DB)]
+            LiveReduce
+        end
+        subgraph Frontend
+            WebMon["`WebMon
+            monitor.sns.gov`"]
+        end
+        SMS-->LiveReduce
+        TranslationService-.->|ActiveMQ|WorkflowManager
+        DASMON-.->|ActiveMQ|DasmonListener
+        DASMON-->Database
+        WorkflowManager<-.->|ActiveMQ|Autoreducers
+        Autoreducers-->|REST API|LiveDataServer
+        Database-->WebMon
+        ONCat-->|REST API|WebMon
+        LiveDataServer-->|REST API|WebMon
+        LiveReduce-->|REST API|LiveDataServer
+        DasmonListener-->Database
+        WorkflowManager-->Database
+        TranslationService-->FileArchive
+        FileArchive<-->Autoreducers
+        style DAS fill:#D3D3D3, stroke-dasharray: 5 5