Merge pull request #62 from mach3-software/feature_DockerFile

Docker File and CI

.github/workflows/CDImage.yml

@@ -0,0 +1,52 @@
+name: Image CD
+
+# The events that trigger the workflow
+on:
+  push:
+    branches: [ develop ]
+  release:
+    types: [ published ]
+
+permissions:
+  contents: read
+  packages: write
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        include:
+          - os: Ubuntu22.04
+            file: Doc/MaCh3DockerFiles/Ubuntu22.04/Dockerfile
+            tag_latest: ubuntu22.04latest
+            tag_release: ubuntu22.04v1.1.0
+          - os: Alma9
+            file: Doc/MaCh3DockerFiles/Alma9/Dockerfile
+            tag_latest: alma9latest
+            tag_release: alma9v1.1.0
+
+    name: Image CD ${{ matrix.os }}
+
+    steps:
+    - name: Checkout repository
+      uses: actions/checkout@v3
+
+    - name: Log in to GitHub Container Registry
+      run: echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u ${{ github.actor }} --password-stdin
+
+    - name: Build Docker image
+      run: |
+        if [ "${{ github.event_name }}" == 'release' ]; then
+          docker build . --file ${{ matrix.file }} --tag ghcr.io/${{ github.repository_owner }}/mach3:${{ matrix.tag_release }} --build-arg MACH3_VERSION=develop
+        else
+          docker build . --file ${{ matrix.file }} --tag ghcr.io/${{ github.repository_owner }}/mach3:${{ matrix.tag_latest }} --build-arg MACH3_VERSION=develop
+        fi
+
+    - name: Push Docker image
+      run: |
+        if [ "${{ github.event_name }}" == 'release' ]; then
+          docker push ghcr.io/${{ github.repository_owner }}/mach3:${{ matrix.tag_release }}
+        else
+          docker push ghcr.io/${{ github.repository_owner }}/mach3:${{ matrix.tag_latest }}
+        fi

.github/workflows/CIBuild.yml

@@ -0,0 +1,34 @@
+name: Build CI
+
+# The events that trigger the workflow
+on:
+  pull_request:
+    branches: [ develop ]
+
+permissions:
+  contents: read
+  packages: write
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        include:
+          - os: Ubuntu22.04
+            file: Doc/MaCh3DockerFiles/Ubuntu22.04/Dockerfile
+            tag: ubuntu22.04latest
+          - os: Alma9
+            file: Doc/MaCh3DockerFiles/Alma9/Dockerfile
+            tag: alma9latest
+
+    name: Build CI ${{ matrix.os }}
+
+    steps:
+    - uses: actions/checkout@v3
+
+    - name: Log in to GitHub Container Registry
+      run: echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u ${{ github.actor }} --password-stdin
+
+    - name: Build the Docker image
+      run: docker build . --file ${{ matrix.file }} --tag ghcr.io/${{ github.repository_owner }}/mach3:${{ matrix.tag }} --build-arg MACH3_VERSION=${{ github.head_ref }}

.gitignore

@@ -1,5 +1,6 @@
 #ignore any object files
 *.o
+*.sif
 
 #ignore some stuff in libconfig
 *.lo

Diagnostics/README.md

@@ -0,0 +1,51 @@
+### Plotting and Diagnostic
+**ProcessMCMC** - The main app you want to use for analysing the ND280 chain. It prints posterior distribution after burn-in the cut. Moreover, you can compare two/three different chains. There are a few options you can modify easily inside the app like selection, burn-in cut, and whether to plot xse+flux or only flux. Other functionality
+<ol>
+<li> Produce a covariance matrix with multithreading (which requires lots of RAM due to caching) </li>
+<li> Violin plots </li>
+<li> Credible intervals and regions </li>
+<li> Calculate Bayes factor and give significance based on Jeffreys scale </li>
+<li> Produce triangle plots </li>
+<li> Study covariance matrix stability </li>
+</ol>
+
+**GetPostfitParamPlots** - This will plot output from ProcessMCMC for nice plots which can go to TN. Bits are hardcoded to make plots nicer users should be careful when using the non-conventional xsec model. If you used `ProcessMCMC` with `PlotDet` you will also get an overlay of detector parameters (ND or ND+FD depending on chain type). If Violin plot was produced in `ProcessMCMC` you will get fancy plots here as well.
+
+**GetPenaltyTerm** - Since xsec and flux and ND spline systematic are treated as the same systematic object we cannot just take log_xsec, hence we need this script, use `GetFluxPenaltyTerm MCMCChain.root config`. Parameters of relevance are loaded via config, thus you can study any combination you want. Time needed increases with number of sets :(
+
+**DiagMCMC** - Perform MCMC diagnostic like autocorrelation or trace plots.
+
+**RHat** - Performs RHat diagnostic to study if all used chains converged to the same stationary distribution.
+```
+./RHat Ntoys MCMCchain_1.root MCMCchain_2.root MCMCchain_3.root ... [how many you like]
+```
+
+**PlotLLH** - Plot LLH scans, flexible and configurable in command line. can take any number of LLH scans as input, will use the first one as a baseline when making e.g. ratio plots. The first file must be a MaCh3 scan.
+options:
+
+    -r overlay ratio plots
+
+    -s also make plots split by sample contribution, to use this option, the LLH scan must have been run with the option `LLH_SCAN_BY_SAMPLE = true` in the config file
+
+    -g draw a grid on the plots
+
+    -l a string specifying the labels for each scan to plot in the legent. this should be a string of labels separated by semi colons, e.g.: -`l "label1;label2;label3"`
+
+    -o the name of the output pdf
+
+    -d a string specifying additional drawing options to pass to the histogram draw calls, e.g. `-d "C"` will plot smooth curves through the histogram bins. See https://root.cern/doc/master/classTHistPainter.html#HP01a for possible options.
+
+
+**CombineMaCh3Chains** - will combine chains files produced by **MCMC**, enforcing the condition that all the files to combine were made using the exact same software versions and config files
+```
+CombineMaCh3Chains [-h] [-c [0-9]] [-f] [-o <output file>] file1.root [file2.root, file3.root ...]
+```
+*fileX.root* are the individual spline files to combine, can specify any number, need at least one
+
+-c target compression level for the combined file, default is 1, in line with hadd
+
+-f force overwrite of the combined file if it exists already
+
+-h print usage message and exit
+
+*Output file* (optional) name of the combined file. If not specified, will just use *file1.root*, the first in the list of files, same as *hadd*.

Doc/MaCh3DockerFiles/Alma9/Dockerfile

@@ -0,0 +1,30 @@
+#To run use:  docker build -t mach3 .
+# KS: Get glorious container from Luke which will work as a base
+FROM picker24/root_v6_26_10:alma9 AS mach3_build
+
+# Declare the build argument
+ARG MACH3_VERSION
+ENV MACH3_VERSION=${MACH3_VERSION:-develop}
+ENV MACH3_WORK_DIR=/opt/MaCh3/
+ENV MACH3_INSTALL_DIR=/opt/MaCh3/build/
+
+RUN mkdir -p ${MACH3_WORK_DIR}
+
+WORKDIR /opt/
+# KS: Let's clone MaCh3
+RUN --mount=type=ssh git clone https://github.com/mach3-software/MaCh3 ${MACH3_WORK_DIR}
+WORKDIR ${MACH3_WORK_DIR}
+RUN git checkout ${MACH3_VERSION}
+
+RUN mkdir -p ${MACH3_INSTALL_DIR}
+WORKDIR ${MACH3_INSTALL_DIR}
+RUN cmake ${MACH3_WORK_DIR}
+RUN make VERBOSE=1 && make install
+
+# KS: Need to set them here, otherwise container using this container will not be able to find MaCh3
+ENV MaCh3_ROOT=${MACH3_INSTALL_DIR}
+ENV PATH=${MaCh3_ROOT}/bin:${PATH} \
+    LD_LIBRARY_PATH=${MaCh3_ROOT}/lib:${LD_LIBRARY_PATH}
+
+# Start from MaCh3 install dir
+WORKDIR ${MACH3_INSTALL_DIR}

Doc/MaCh3DockerFiles/Alma9/MaCh3.def

@@ -0,0 +1,34 @@
+#Run like singularity build mach3.sif MaCh3.def
+
+Bootstrap: docker
+From: picker24/root_v6_26_10:alma9
+
+%environment
+    export MaCh3_ROOT=${MACH3_INSTALL_DIR}
+    export PATH=${MaCh3_ROOT}/bin:${PATH}
+    export LD_LIBRARY_PATH=${MaCh3_ROOT}/lib:${LD_LIBRARY_PATH}
+
+%post
+    export MACH3_VERSION="develop"
+    export MACH3_WORK_DIR=/opt/MaCh3/
+    export MACH3_INSTALL_DIR=${MACH3_WORK_DIR}/build/
+
+    # Create directories
+    mkdir -p ${MACH3_WORK_DIR}
+    mkdir -p ${MACH3_INSTALL_DIR}
+
+    # Clone MaCh3 repository
+    cd /opt/
+    git clone https://github.com/mach3-software/MaCh3 ${MACH3_WORK_DIR}
+    cd ${MACH3_WORK_DIR}
+    git checkout ${MACH3_VERSION}
+
+    # Build and install MaCh3
+    cd ${MACH3_INSTALL_DIR}
+    cmake ${MACH3_WORK_DIR}
+    make VERBOSE=1 && make install
+
+%runscript
+    # This section defines the default behavior when the container is run
+    cd ${MACH3_INSTALL_DIR}
+

Doc/MaCh3DockerFiles/Ubuntu22.04/Dockerfile

@@ -0,0 +1,38 @@
+#To run use:  docker build -t mach3 .
+# KS: Get glorious container from ROOT which will work as a base
+FROM rootproject/root:6.32.02-ubuntu22.04 AS mach3_build
+
+
+RUN apt update && apt upgrade -y
+
+RUN apt-get install -y nlohmann-json3-dev
+#MISC_SW
+RUN apt install -y --no-install-recommends vim less nano gdb csh tcsh ed quota python3 python3-dev cvs procmail ca-certificates
+
+# Declare the build argument
+ARG MACH3_VERSION
+ENV MACH3_VERSION=${MACH3_VERSION:-develop}
+ENV MACH3_WORK_DIR=/opt/MaCh3/
+ENV MACH3_INSTALL_DIR=/opt/MaCh3/build/
+
+RUN mkdir -p ${MACH3_WORK_DIR}
+
+WORKDIR /opt/
+# KS: Let's clone MaCh3
+RUN --mount=type=ssh git clone https://github.com/mach3-software/MaCh3 ${MACH3_WORK_DIR}
+WORKDIR ${MACH3_WORK_DIR}
+RUN git checkout ${MACH3_VERSION}
+
+RUN mkdir -p ${MACH3_INSTALL_DIR}
+WORKDIR ${MACH3_INSTALL_DIR}
+RUN cmake ${MACH3_WORK_DIR}
+RUN make VERBOSE=1
+RUN make install
+
+# KS: Need to set them here, otherwise container using this container will not be able to find MaCh3
+ENV MaCh3_ROOT=${MACH3_INSTALL_DIR}
+ENV PATH=${MaCh3_ROOT}/bin:${PATH} \
+    LD_LIBRARY_PATH=${MaCh3_ROOT}/lib:${LD_LIBRARY_PATH}
+
+# Start from MaCh3 install dir
+WORKDIR ${MACH3_INSTALL_DIR}

README.md

@@ -3,15 +3,15 @@ The Markov Chain 3 flavour is a framework born in 2013 as a Bayesian MCMC fitter
 
 The framework has also evolved to allow non MCMC modules to interrogate the likelihoods implemented.
 
-
 [![License](https://img.shields.io/badge/License-BSD_3--Clause-blue.svg)](https://github.com/mach3-software/MaCh3/blob/develop/LICENSE.txt)
 [![Release - v1.0.0](https://img.shields.io/badge/Release-v1.0.0-2ea44f)](https://github.com/mach3-software/MaCh3/releases)
+[![Container Image](https://img.shields.io/badge/Container-Image-brightgreen)](https://github.com/mach3-software/MaCh3/pkgs/container/mach3)
 [![Code - Documented](https://img.shields.io/badge/Code-Documented-2ea44f)](https://github.com/mach3-software/MaCh3/wiki)
 [![Code - Doxygen](https://img.shields.io/badge/Code-Doxygen-2ea44f)](https://mach3-software.github.io/MaCh3/index.html)
+[![Build Status](https://github.com/mach3-software/MaCh3/workflows/Docker%20CI%20Alma9/badge.svg)](https://github.com/mach3-software/MaCh3/actions?query=workflow%3A%22Docker+CI+Alma9%22)
 
 ## Famous Plots
-TODO this should be expanded
-
+Example of plots made using MaCh3 apparent in scientific publications, for more see [here](https://github.com/mach3-software/MaCh3/wiki/14.-MaCh3-in-the-Field)
 <img src="Doc/Plots/delta.png" alt="MaCh3" align="left" width="200"/>
 <img src="Doc/Plots/Jarlskog.png" alt="MaCh3" align="center" width="200"/>
 
@@ -110,10 +110,21 @@ Most of external libraries are being handled through CPM. The only external libr
 
 Based on several test here are recommended version:
 ```
-  GCC: ...
+  GCC: >= 8.5 [lower versions may work]
   CMake: >= 3.14
   ROOT: >= 6.18
 ```
+### Supported operational systems
+| Name        | Status |
+|-------------|--------|
+| Alma9       | ✅     |
+| Ubuntu22.04 | ✅     |
+| CentOS7     | ❔     |
+| Windows     | ❌     |
+
+✅ - Part of CI/CD <br>
+❔ - Not part of CI/CD but used by some users/developers so it might work <br>
+❌ - Not supported and no plans right now <br>
 
 # How To Use
 This is an example how your executable can look like using MaCh3:
@@ -143,55 +154,4 @@ This is an example how your executable can look like using MaCh3:
 
 
 ### Plotting and Diagnostic
-Example of chain diagnostic utils can be found [here](https://github.com/mach3-software/MaCh3/tree/develop/Diagnostics) with example of config. Currently available utils include:
-
-**ProcessMCMC** - The main app you want to use for analysing the ND280 chain. It prints posterior distribution after burn-in the cut. Moreover, you can compare two/three different chains. There are a few options you can modify easily inside the app like selection, burn-in cut, and whether to plot xse+flux or only flux. Other functionality
-<ol>
-<li> Produce a covariance matrix with multithreading (which requires lots of RAM due to caching) </li>
-<li> Violin plots </li>
-<li> Credible intervals and regions </li>
-<li> Calculate Bayes factor and give significance based on Jeffreys scale </li>
-<li> Produce triangle plots </li>
-<li> Study covariance matrix stability </li>
-</ol>
-
-**GetPostfitParamPlots** - This will plot output from ProcessMCMC for nice plots which can go to TN. Bits are hardcoded to make plots nicer users should be careful when using the non-conventional xsec model. If you used `ProcessMCMC` with `PlotDet` you will also get an overlay of detector parameters (ND or ND+FD depending on chain type). If Violin plot was produced in `ProcessMCMC` you will get fancy plots here as well.
-
-**GetPenaltyTerm** - Since xsec and flux and ND spline systematic are treated as the same systematic object we cannot just take log_xsec, hence we need this script, use `GetFluxPenaltyTerm MCMCChain.root config`. Parameters of relevance are loaded via config, thus you can study any combination you want. Time needed increases with number of sets :(
-
-**DiagMCMC** - Perform MCMC diagnostic like autocorrelation or trace plots.
-
-**RHat** - Performs RHat diagnostic to study if all used chains converged to the same stationary distribution.
-```
-./RHat Ntoys MCMCchain_1.root MCMCchain_2.root MCMCchain_3.root ... [how many you like]
-```
-
-**PlotLLH** - Plot LLH scans, flexible and configurable in command line. can take any number of LLH scans as input, will use the first one as a baseline when making e.g. ratio plots. The first file must be a MaCh3 scan.
-options:
-
-    -r overlay ratio plots
-
-    -s also make plots split by sample contribution, to use this option, the LLH scan must have been run with the option `LLH_SCAN_BY_SAMPLE = true` in the config file
-
-    -g draw a grid on the plots
-
-    -l a string specifying the labels for each scan to plot in the legent. this should be a string of labels separated by semi colons, e.g.: -`l "label1;label2;label3"`
-
-    -o the name of the output pdf
-
-    -d a string specifying additional drawing options to pass to the histogram draw calls, e.g. `-d "C"` will plot smooth curves through the histogram bins. See https://root.cern/doc/master/classTHistPainter.html#HP01a for possible options.
-
-
-**CombineMaCh3Chains** - will combine chains files produced by **MCMC**, enforcing the condition that all the files to combine were made using the exact same software versions and config files
-```
-CombineMaCh3Chains [-h] [-c [0-9]] [-f] [-o <output file>] file1.root [file2.root, file3.root ...]
-```
-*fileX.root* are the individual spline files to combine, can specify any number, need at least one
-
--c target compression level for the combined file, default is 1, in line with hadd
-
--f force overwrite of the combined file if it exists already
-
--h print usage message and exit
-
-*Output file* (optional) name of the combined file. If not specified, will just use *file1.root*, the first in the list of files, same as *hadd*.
+Example of chain diagnostic utils can be found [here](https://github.com/mach3-software/MaCh3/tree/develop/Diagnostics) with example of config.

mcmc/CMakeLists.txt

@@ -2,7 +2,7 @@ set(HEADERS
     FitterBase.h
     mcmc.h
     LikelihoodFit.h
-    $<$<NOT:$<BOOL:${MaCh3_MINUIT2_ENABLED}>>:MinuitFit.h>
+    MinuitFit.h
     PSO.h
     MCMCProcessor.h
     SampleSummary.h
@@ -13,7 +13,7 @@ add_library(MCMC SHARED
     FitterBase.cpp
     mcmc.cpp
     LikelihoodFit.cpp
-    $<$<NOT:$<BOOL:${MaCh3_MINUIT2_ENABLED}>>:MinuitFit.cpp>
+    $<$<BOOL:${MaCh3_MINUIT2_ENABLED}>:MinuitFit.cpp>
     PSO.cpp
     MCMCProcessor.cpp
     SampleSummary.cpp