diff --git a/.gitignore b/.gitignore index ec65bcb5..7ad252f8 100644 --- a/.gitignore +++ b/.gitignore @@ -19,11 +19,11 @@ test/out/ test/.nfs* # TeX -/docs/*.aux -/docs/*.log -/docs/*.out -/docs/*.synctex.gz -/docs/*.toc +/docs/**/*.aux +/docs/**/*.log +/docs/**/*.out +/docs/**/*.synctex.gz +/docs/**/*.toc # Misc tmp/ \ No newline at end of file diff --git a/Changelog.md b/Changelog.md index c3be4c4b..70c4c4b7 100644 --- a/Changelog.md +++ b/Changelog.md @@ -1,3 +1,55 @@ +# Version 2.6.0 + +- New **INCOMPATIBLE** format for sdp.zip. + Instead of a single `block_XXX.json` file now we use `block_info_XXX.json` and `block_data_XXX.json` ( + or `block_data_XXX.bin`), see [SDPB_input_format.md](docs/SDPB_input_format.md). + See [#114](https://github.com/davidsd/sdpb/pull/114). + +- Compact binary format instead of JSON for `block_data_XXX` in sdp.zip. + When generating sdp.zip by `sdp2input` or `pvm2sdp`, you can use optional command-line argument to choose between + binary and JSON formats: + +``` +sdp2input --outputFormat FORMAT +pvm2sdp FORMAT PRECISION INPUT... OUTPUT +``` + +where `FORMAT` is either `bin` (used by default) or `json`. +We recommend using the new binary format since it is more compact and efficient. +See [#114](https://github.com/davidsd/sdpb/pull/114), [#128](https://github.com/davidsd/sdpb/pull/128), [#119](https://github.com/davidsd/sdpb/pull/119), [#149](https://github.com/davidsd/sdpb/pull/149). + +- New lightweight Docker image does not contain `scalar_blocks` and `blocks_3d` anymore. + Separate images for + [sdpb](https://hub.docker.com/r/bootstrapcollaboration/sdpb/tags), + [scalar_blocks](https://hub.docker.com/r/bootstrapcollaboration/scalar_blocks/tags) + and [blocks_3d](https://hub.docker.com/r/bootstrapcollaboration/blocks_3d/tags) + are now uploaded to https://hub.docker.com/u/bootstrapcollaboration. + See [#130](https://github.com/davidsd/sdpb/pull/130). + +- CI/CD pipelines on [CircleCI](https://app.circleci.com/pipelines/github/davidsd/sdpb). + Tests are run for each `git push` to the repo and for each pull request. + Docker images for the `master` branch and for each release starting from 2.6.0 + are uploaded to [Docker Hub](https://hub.docker.com/r/bootstrapcollaboration/sdpb/tags) automatically. + See [#133](https://github.com/davidsd/sdpb/pull/133), + [#136](https://github.com/davidsd/sdpb/pull/136). + +- Tests are reorganized and rewritten from shell scripts to C++ [Catch2](https://github.com/catchorg/Catch2) framework. + Added unit tests and realistic end-to-end tests. + See [#91](https://github.com/davidsd/sdpb/pull/91), + [#102](https://github.com/davidsd/sdpb/pull/102), + [#109](https://github.com/davidsd/sdpb/pull/109), + [#119](https://github.com/davidsd/sdpb/pull/119). + +- Switched from C++14 to C++17. + See [#118](https://github.com/davidsd/sdpb/pull/118), + [#121](https://github.com/davidsd/sdpb/pull/121). + +- Updated installations + for [BU](docs/site_installs/Boston.md), [Caltech](docs/site_installs/Caltech.md), [Expanse](docs/site_installs/Expanse.md), [Harvard](docs/site_installs/Harvard.md), + and [Imperial College](docs/site_installs/Imperial.md) clusters. + +See https://github.com/davidsd/sdpb/releases/tag/2.6.0 for the full changelog. + # Version 2.5.1 ## outer_limits diff --git a/Install.md b/Install.md index 3c602c78..55989225 100644 --- a/Install.md +++ b/Install.md @@ -1,7 +1,8 @@ This guide is for building SDPB. To just run SDPB, it may be easier to use [Docker](docs/Docker.md) or [Singularity](docs/Singularity.md). In addition, there are [site specific -notes](docs/site_installs/Readme.md) for different HPC machines. +notes](docs/site_installs/Readme.md) for different HPC machines and also +for [Apple MacBooks](docs/site_installs/Apple_MacBook.md). * [Requirements](#requirements) * [Installation](#installation) @@ -13,7 +14,7 @@ notes](docs/site_installs/Readme.md) for different HPC machines. SDPB requires - A modern C++ compiler with C++ 17 support. SDPB has been tested with - GCC versions 7 through 10. + GCC versions 8 through 12. - An MPI implementation such as [OpenMPI](https://www.open-mpi.org/) or [MPICH](https://www.mpich.org/) @@ -39,7 +40,7 @@ SDPB requires SDPB has only been tested on Linux (Debian buster and Centos 7). On Centos 7, the system compiler (gcc 4.8.5) is too old to support -C++ 14. So you will have to install a newer compiler and Boost. The +C++ 17. So you will have to install a newer compiler and Boost. The system versions of GMP, MPFR, and libxml2 have been tested to work. In principle, SDPB should be installable on Mac OS X using a package @@ -47,7 +48,7 @@ manager such as [Homebrew](https://brew.sh). # Installation -1. Download the the fork of [Elemental](https://gitlab.com/bootstrapcollaboration/elemental) +1. Download the fork of [Elemental](https://gitlab.com/bootstrapcollaboration/elemental) git clone https://gitlab.com/bootstrapcollaboration/elemental.git diff --git a/Readme.md b/Readme.md index 3d1b53e8..eea3fbb7 100644 --- a/Readme.md +++ b/Readme.md @@ -28,7 +28,7 @@ Let $S^{m\times m}[x]$ be the space of symmetric $m\times m$ matrices whose entr Here, $M\succeq 0$ means "M is positive semidefinite." For more information, see [A Semidefinite Program Solver for the Conformal Bootstrap](http://arxiv.org/abs/1502.02033) -and [the manual](/docs/SDPB-Manual.pdf). +and [the manual](/docs/SDPB_Manual/SDPB-Manual.pdf). Authors: David Simmons-Duffin (dsd@caltech.edu), Walter Landry (wlandry@caltech.edu). diff --git a/docs/Docker.md b/docs/Docker.md index 105842fd..d2fdd506 100644 --- a/docs/Docker.md +++ b/docs/Docker.md @@ -1,22 +1,55 @@ # Docker -Docker is a relatively easy way to run binaries without having to -build SDPB. This can be a convenient way to run SDPB on your laptop -or desktop. For HPC centers, most of them support Singularity rather +[Docker](https://www.docker.com/) (or its alternative, [Podman](https://podman.io)) +is a relatively easy way to run binaries without having to build SDPB. +This can be a convenient way to run SDPB on your laptop or desktop. + +For HPC centers, most of them support Singularity rather than Docker, so you should read the guide for -[Singularity](Singularity.md). There are instructions at the -[Docker website](https://www.docker.com/get-started) on how to install -it on Windows, Mac, or Linux. +[Singularity](Singularity.md). + +There are instructions at the [Docker website](https://www.docker.com/get-started) on how to install it on Windows, Mac, +or Linux. + +## Get Docker image + +### Download from Docker Hub + +Docker image built from the `master` branch (updated regularly) +can be downloaded from [Docker Hub](https://hub.docker.com/r/bootstrapcollaboration/sdpb/tags) as + + docker pull bootstrapcollaboration/sdpb:master + +You can also download image for a specific release, e.g., 2.6.0: + + docker pull bootstrapcollaboration/sdpb:2.6.0 + +The list of all available tags can be found in +[bootstrapcollaboration/sdpb](https://hub.docker.com/r/bootstrapcollaboration/sdpb/tags) repo on Docker Hub. +Images for SDPB 2.5.1 and earlier can be downloaded from [wlandry/sdpb](https://hub.docker.com/r/wlandry/sdpb/tags) +repo. + +### Build from sources -There is a docker image on [Docker Hub](https://hub.docker.com/) named -`wlandry/sdpb:2.5.1` containing binaries for `scalar_blocks`, -`pvm2sdp`, `sdp2input`, and `sdpb`. +You can also [build](https://docs.docker.com/engine/reference/commandline/build/) Docker image from sources by yourself +using [Dockerfile](../Dockerfile): + + docker build . --tag sdpb:master + docker run sdpb:master sdpb --help + +Our [Dockerfile](../Dockerfile) also contains separate `test` target which allows +to [run built-in tests](../test/run_all_tests.sh) inside Docker containter: + + docker build . --tag sdpb-test --target test + docker run sdpb-test ./test/run_all_tests.sh + +## Run Docker image Suppose you have an input file `/my/project/input.xml`. To use this file, and be able to write output, we will make the `/my/project` directory visible to the docker image in the location `/usr/local/share/sdpb`. -The structure of the command to run docker is +The structure of the [docker run](https://docs.docker.com/engine/reference/commandline/run/) command is docker run @@ -24,10 +57,8 @@ In this case, `` will be used to mount your directory in a place that docker will see it. -v /my/project/:/usr/local/share/sdpb/ - -`` is the image name. - wlandry/sdpb:2.5.1 +`` is the image name, e.g. `bootstrapcollaboration/sdpb:master` `` is the command that you would normally use to run the SDPB commands (see [Usage.md](Usage.md)). The directory containing the @@ -38,17 +69,29 @@ convert from xml `mpirun` runs as root inside the docker container. Running `mpirun` as root is normally dangerous, but it is safe to do so inside the -container. To allow mpirun to run as root, we add the option +container. To allow `mpirun` to run as root, we add the option `--allow-run-as-root`. This uses 4 cores when running pvm2sdp. You can change that number to match your own machine. Putting it all together on a single line - docker run -v /my/project/:/usr/local/share/sdpb wlandry/sdpb:2.5.1 mpirun --allow-run-as-root -n 4 pvm2sdp 1024 /usr/local/share/sdpb/input.xml /usr/local/share/sdpb/input + docker run -v /my/project/:/usr/local/share/sdpb bootstrapcollaboration/sdpb:master mpirun --allow-run-as-root -n 4 pvm2sdp 1024 /usr/local/share/sdpb/input.xml /usr/local/share/sdpb/sdp.zip -Running this command will populate the directory `/my/project/input`. +Running this command will create `/my/project/sdp.zip`. To search for primal-dual solutions - docker run -v /my/project/:/usr/local/share/sdpb wlandry/sdpb:2.5.1 mpirun --allow-run-as-root -n 4 sdpb --precision=1024 --procsPerNode=4 -s /usr/local/share/sdpb/input + docker run -v /my/project/:/usr/local/share/sdpb bootstrapcollaboration/sdpb:master mpirun --allow-run-as-root -n 4 sdpb --precision=1024 --procsPerNode=4 -s /usr/local/share/sdpb/sdp.zip + +The results will be in `/my/project/`. + +Note that the newly created files may be owned by root. +If you cannot remove them outside the container, run `rm` from the container, e.g.: + + docker run -v /my/project/:/usr/local/share/sdpb bootstrapcollaboration/sdpb:master rm /usr/local/share/sdpb/sdp.zip + +# Podman + +Instead of Docker, one can also use [Podman](https://podman.io), which is compatible with Docker images and uses the +same syntax (the only difference is the additional `docker.io/` prefix): -The results will be in `/my/project/`. Note that the files may be -owned by root. + podman pull docker.io/bootstrapcollaboration/sdpb:master + podman run -v /my/project/:/usr/local/share/sdpb bootstrapcollaboration/sdpb:master mpirun --allow-run-as-root -n 4 pvm2sdp 1024 /usr/local/share/sdpb/input.xml /usr/local/share/sdpb/input \ No newline at end of file diff --git a/docs/Outer_Limits.pdf b/docs/Outer_Limits/Outer_Limits.pdf similarity index 80% rename from docs/Outer_Limits.pdf rename to docs/Outer_Limits/Outer_Limits.pdf index ed9935fa..54db38d2 100644 Binary files a/docs/Outer_Limits.pdf and b/docs/Outer_Limits/Outer_Limits.pdf differ diff --git a/docs/Outer_Limits.tex b/docs/Outer_Limits/Outer_Limits.tex similarity index 99% rename from docs/Outer_Limits.tex rename to docs/Outer_Limits/Outer_Limits.tex index 5f615222..cd7af242 100644 --- a/docs/Outer_Limits.tex +++ b/docs/Outer_Limits/Outer_Limits.tex @@ -232,7 +232,7 @@ \section{Input} \end{lstlisting} The exact format is specified in the JSON schema -\texttt{docs/functions\_schema.json}. At its heart, there is a set of +\texttt{docs/json\_schema/functions\_schema.json}. At its heart, there is a set of nested arrays. The outermost array corresponds to the $j$ index of $m_{j,rs}^{n}(\Delta)$, followed by $r$, $s$, and $n$. The toy example has $N=2$, $J=1$, and $R_{1}=1$, so $j=r=s=1$ and $n=1$ or @@ -270,7 +270,7 @@ \section{Input} \end{lstlisting} The format for this file is specified in the JSON schema -\texttt{docs/points\_schema.json}. The outer array corresponds to the +\texttt{docs/json\_schema/points\_schema.json}. The outer array corresponds to the $j$ index of $m_{j,rs}^{n}(\Delta)$, and the inner array is a list of points for that index. diff --git a/docs/dsdshorthand.sty b/docs/Outer_Limits/dsdshorthand.sty similarity index 100% rename from docs/dsdshorthand.sty rename to docs/Outer_Limits/dsdshorthand.sty diff --git a/docs/outer_plots.py b/docs/Outer_Limits/outer_plots.py similarity index 100% rename from docs/outer_plots.py rename to docs/Outer_Limits/outer_plots.py diff --git a/docs/outer_plots_1.png b/docs/Outer_Limits/outer_plots_1.png similarity index 100% rename from docs/outer_plots_1.png rename to docs/Outer_Limits/outer_plots_1.png diff --git a/docs/outer_plots_2.png b/docs/Outer_Limits/outer_plots_2.png similarity index 100% rename from docs/outer_plots_2.png rename to docs/Outer_Limits/outer_plots_2.png diff --git a/docs/outer_plots_3.png b/docs/Outer_Limits/outer_plots_3.png similarity index 100% rename from docs/outer_plots_3.png rename to docs/Outer_Limits/outer_plots_3.png diff --git a/docs/Outer_Limits_Usage.md b/docs/Outer_Limits_Usage.md index 25505e55..23c10bcf 100644 --- a/docs/Outer_Limits_Usage.md +++ b/docs/Outer_Limits_Usage.md @@ -1,7 +1,7 @@ # Outer_Limits Usage The exact details of how `outer_limits` works is described in the -`outer_limits` [manual](Outer_Limits.pdf). To briefly summarize, +`outer_limits` [manual](Outer_Limits/Outer_Limits.pdf). To briefly summarize, `outer_limits` reads in function evaluated at Chebyshev zeros and initial points. It applies constant constraints at those initial points and computes a solution. It then iteratively adds constraints @@ -29,9 +29,9 @@ of options. There are four main differences: format. Defaults to `functions` with the ending `_out.json`. The format for `functions` is detailed in the -[schema](functions_schema.json), and there is an +[schema](json_schema/functions_schema.json), and there is an [example](../test/data/outer_limits/toy_functions.json) included with the source code. -Similarly, `points` has a [schema](points_schema.json) and an +Similarly, `points` has a [schema](json_schema/points_schema.json) and an [example](../test/data/outer_limits/toy_functions_points.json). If you have an existing input file for `sdpb`, you can use the bundled diff --git a/docs/SDPB-PMP-Description.png b/docs/SDPB-PMP-Description.png deleted file mode 100644 index 2ff11442..00000000 Binary files a/docs/SDPB-PMP-Description.png and /dev/null differ diff --git a/docs/SDPB-Manual.pdf b/docs/SDPB_Manual/SDPB-Manual.pdf similarity index 94% rename from docs/SDPB-Manual.pdf rename to docs/SDPB_Manual/SDPB-Manual.pdf index f59c2713..b03d1b10 100644 Binary files a/docs/SDPB-Manual.pdf and b/docs/SDPB_Manual/SDPB-Manual.pdf differ diff --git a/docs/SDPB-Manual.tex b/docs/SDPB_Manual/SDPB-Manual.tex similarity index 99% rename from docs/SDPB-Manual.tex rename to docs/SDPB_Manual/SDPB-Manual.tex index ba628bfb..4b055a94 100644 --- a/docs/SDPB-Manual.tex +++ b/docs/SDPB_Manual/SDPB-Manual.tex @@ -38,7 +38,7 @@ {\Large \begin{center} -{\bf SDPB 2.5.1 \\\vspace{.1in}} +{\bf SDPB 2.6.0 \\\vspace{.1in}} \end{center} } \begin{center} @@ -253,7 +253,7 @@ \subsection{\texttt{JSON} and \texttt{Mathematica} Interface} As an example bootstrap application, the included notebook \texttt{Bootstrap2dExample.m} computes a single-correlator dimension bound for 2d CFTs with a $\Z_2$ symmetry, as in \cite{Rychkov:2009ij}. -The \texttt{JSON} format is described by the schema in \texttt{docs/sdp2input\_schema.json}. +The \texttt{JSON} format is described by the schema in \texttt{docs/json\_schema/sdp2input\_schema.json}. \subsection{An Example} \label{sec:example} diff --git a/docs/SDPB_Manual/dsdshorthand.sty b/docs/SDPB_Manual/dsdshorthand.sty new file mode 100644 index 00000000..e59efe94 --- /dev/null +++ b/docs/SDPB_Manual/dsdshorthand.sty @@ -0,0 +1,250 @@ +%% +%% `dsdshorthand.sty' -- Macros for everyday use, generally to improve +%% code readability with a consistent shorthand. Macro names are +%% usually based more on the appearance of the symbol rather than the +%% meaning (e.g. \Z for \mathbb{Z}, instead of \Integers, or \cA for +%% \mathcal{A}). Choices are geared towards mathematical physics, +%% sort of. +%% +%% David Simmons-Duffin +%% Last modified May 17, 2006 +%% +\NeedsTeXFormat{LaTeX2e} +\ProvidesPackage{dsdshorthand}[2006/05/17 David Simmons-Duffin's macros] +\RequirePackage{amsmath, amssymb, amsthm} +\RequirePackage{ifthen} % LaTeX logic + +%%%%%%% Not Required, but Oft-Used Packages %%%%%%% +\usepackage{amsmath} +\usepackage{amssymb} +\usepackage{amsthm} +\usepackage{ifthen} +\usepackage{graphicx} % \includegraphics +\usepackage{epstopdf} % allow inclusion of eps files +\usepackage{bm} % access to bold math fonts, via \bm +\usepackage{xspace} % \xspace puts space where needed +\usepackage{mathrsfs} % fancy script: \mathscr + +\DeclareGraphicsRule{*}{mps}{*}{} % not sure what this does, but it's +% necessary to make feynmp work correctly + +%%%%%%% Argument-taking Macros, Size Control %%%%%%% +\newcommand \pdr [2] {\frac{\partial #1}{\partial #2}} +\newcommand \pddr [3] { + \ifthenelse{\equal{#3}{}} + { \frac{\partial^2 #1}{\partial #2^2} } + { \frac{\partial^2 #1}{\partial #2 \partial #3} } +} +\newcommand \rdr [2] {\frac{d #1}{d #2}} +\newcommand \rddr [3] { + \ifthenelse{\equal{#3}{}} + { \frac{d^2 #1}{d #2^2} } + { \frac{d^2 #1}{d #2 d #3} } +} + +\newcommand \lr [3] {\left #1 #2 \right #3} +\newcommand \p [1] {\lr({#1})} +\newcommand \tl [1] {\widetilde{#1}} +\renewcommand \bar [1] {\overline{#1}} +%\let \smallbar = \bar +%\let \bar = \overline +%\let \smallhat = \hat +%\let \hat = \widehat + +%%%%%%% Notation Shortcuts %%%%%%% +\newcommand \< {\langle} +\renewcommand \> {\rangle} +\newcommand \bra [1] {\< #1 |} +\newcommand \ket [1] {| #1 \>} +\renewcommand \. {\cdot} +\newcommand \inv {^{-1}} +\newcommand \x {\times} +\newcommand \we {\wedge} +\newcommand \oo {\infty} +\newcommand \aeq {\thickapprox} +\newcommand \ptl {\partial} +\newcommand \grad {\nabla} +\newcommand \mto {\mapsto} +\newcommand \hc {\mathrm{h.c.}} +\newcommand \half{\frac 1 2} +\newcommand \Id {\mathbf{1}} +\newcommand \Dslash {\!\not\!\! D} +\newcommand \dslash {\!\not\! \ptl} +\newcommand \pslash {\!\not\! p} +\newcommand \qslash {\!\not\! q} +\newcommand \kslash {\!\not\! k} + +%%%%%%% Greek %%%%%%% +\renewcommand \a {\alpha} +\renewcommand \b {\beta} +\newcommand \g {\gamma} +\newcommand \G {\Gamma} +\newcommand \de {\delta} +\newcommand \De {\Delta} +\newcommand \e {\epsilon} +\renewcommand \l {\lambda} +\renewcommand \L {\Lambda} +\renewcommand \r {\rho} +\newcommand \f {\phi} +\newcommand \vf {\varphi} +\newcommand \s {\sigma} +\newcommand \w {\omega} +\renewcommand \O {\Omega} +\renewcommand \th {\theta} +\newcommand \ka {\kappa} +\newcommand \z {\zeta} + +%%%%%%% Spaces and Fields %%%%%%% +\newcommand \R {\mathbb{R}} +\newcommand \Z {\mathbb{Z}} +\newcommand \N {\mathbb{N}} +\newcommand \C {\mathbb{C}} +\newcommand \Q {\mathbb{Q}} +\newcommand \Qp {\mathbb{Q}_p} +\renewcommand \P {\mathbb{P}} +\newcommand \A {\mathbb{A}} + +%%%%%%% Multi-letter Functors/Functions %%%%%%% +% how many of these should be mathops? +\newcommand \SU {\mathrm{SU}} +\newcommand \SO {\mathrm{SO}} +\newcommand \GL {\mathrm{GL}} +\newcommand \SL {\mathrm{SL}} +\newcommand \PSL {\mathrm{PSL}} +\newcommand \PGL {\mathrm{PGL}} +\newcommand \Sp {\mathrm{Sp}} +\newcommand \Span {\mathrm{Span}} +\newcommand \Aut {\mathrm{Aut}} +\newcommand \Res {\mathop{\mathrm{Res}}} +\renewcommand \Re {\mathop{\mathrm{Re}}} +\renewcommand \Im {\mathop{\mathrm{Im}}} +\newcommand \Tr {\mathrm{Tr}} +\newcommand \Pf {\mathrm{Pf}} +\newcommand \Ker {\mathop{\mathrm{Ker}}} +\newcommand \Coker {\mathop{\mathrm{Coker}}} +\newcommand \Hom {\mathrm{Hom}} +\newcommand \Mor {\mathrm{Mor}} +\newcommand \Gal {\mathrm{Gal}} +\newcommand \Sym {\mathrm{Sym}} +\newcommand \codim {\mathop{\mathrm{codim}}} +\newcommand \ind {\mathop{\mathrm{ind}}} + +%%%%%%% Single-letter Shortcuts %%%%%%% +%% note: I've defined bold and caligraphed shortcuts only for the +%% letters I used most often. This is not the whole alphabet (and nor +%% can it be -- consider, e.g., {\bf...}) + +%%%%%%% Bold Letters %%%%%%% +%% lowercase +\newcommand \ba {\mathbf{a}} +\newcommand \bi {\mathbf{i}} +\newcommand \bj {\mathbf{j}} +\newcommand \bk {\mathbf{k}} +\newcommand \bl {\mathbf{l}} +\newcommand \bn {\mathbf{n}} +\newcommand \bp {\mathbf{p}} +\newcommand \bq {\mathbf{q}} +\newcommand \br {\mathbf{r}} +\newcommand \bs {\mathbf{s}} +\newcommand \bt {\mathbf{t}} +\newcommand \bu {\mathbf{u}} +\newcommand \bv {\mathbf{v}} +\newcommand \bw {\mathbf{w}} +\newcommand \bx {\mathbf{x}} +\newcommand \by {\mathbf{y}} +\newcommand \bz {\mathbf{z}} +%% Uppercase +\newcommand \bA {\mathbf{A}} +\newcommand \bB {\mathbf{B}} +\newcommand \bE {\mathbf{E}} +\newcommand \bF {\mathbf{F}} +\newcommand \bG {\mathbf{G}} +\newcommand \bH {\mathbf{H}} +\newcommand \bI {\mathbf{I}} +\newcommand \bJ {\mathbf{J}} +\newcommand \bK {\mathbf{K}} +\newcommand \bL {\mathbf{L}} +\newcommand \bP {\mathbf{P}} +\newcommand \bQ {\mathbf{Q}} +\newcommand \bR {\mathbf{R}} +\newcommand \bS {\mathbf{S}} + +%%%%%%% Caligraphic Letters %%%%%%% +\newcommand \cA {\mathcal{A}} +\newcommand \cB {\mathcal{B}} +\newcommand \cC {\mathcal{C}} +\newcommand \cD {\mathcal{D}} +\newcommand \cE {\mathcal{E}} +\newcommand \cF {\mathcal{F}} +\newcommand \cG {\mathcal{G}} +\newcommand \cH {\mathcal{H}} +\newcommand \cL {\mathcal{L}} +\newcommand \cM {\mathcal{M}} +\newcommand \cO {\mathcal{O}} +\newcommand \cP {\mathcal{P}} +\newcommand \cT {\mathcal{T}} +\newcommand \cZ {\mathcal{Z}} + +%%%%%%% Unit Vectors %%%%%%% +\newcommand \xhat {\hat{\bx}} +\newcommand \yhat {\hat{\by}} +\newcommand \zhat {\hat{\bz}} +\newcommand \rhat {\hat{\br}} +\newcommand \nhat {\hat{\bn}} + +%%%%%%% General Environment Shortcuts %%%%%%% +%\newcommand \be {\begin{align*}} % These are broken -- macros +%\newcommand \ee {\end {align*}} % for \end{..} don't work in align +%\newcommand \ben {\begin{align}} +%\newcommand \een {\end {align}} +\newcommand \be {\begin{eqnarray*}} % these should be replaced +\newcommand \ee {\end {eqnarray*}} % by align, but can't be... +\newcommand \ben {\begin{eqnarray}} +\newcommand \een {\end {eqnarray}} +\newcommand \bitem {\begin{itemize}} +\newcommand \eitem {\end {itemize}} +\newcommand \benum {\begin{enumerate}} +\newcommand \eenum {\end {enumerate}} + + +%%%%%%%% Theorem Types %%%%%%%% +% last arg means number by sections (e.g. Theorem 2.3) +\newtheorem {theorem} {Theorem} [section] +% the second arg means share the same counter with 'theorem' +\newtheorem {lemma} [theorem] {Lemma} +\newtheorem {corollary} [theorem] {Corollary} +\newtheorem {proposition} [theorem] {Proposition} +% individually counted, for use in problem sets +\newtheorem {lemma*} {Lemma} + +% not numbered +\newtheorem* {definition} {Definition} +\newtheorem* {claim} {Claim} +\newtheorem* {remark} {Remark} + +%%%%%%% Theorem Environment Shortcuts %%%%%%% +\newcommand \blem {\begin{lemma}} +\newcommand \elem {\end {lemma}} +\newcommand \bpf {\begin{proof}} +\newcommand \epf {\end {proof}} +\newcommand \bprop {\begin{proposition}} +\newcommand \eprop {\end {proposition}} +\newcommand \bthm {\begin{theorem}} +\newcommand \ethm {\end {theorem}} +\newcommand \bedef {\begin{definition}} +\newcommand \stopdef {\end {definition}} +\newcommand \bclaim {\begin{claim}} +\newcommand \eclaim {\end {claim}} +\newcommand \bcor {\begin{corollary}} +\newcommand \ecor {\end {corollary}} +\newcommand \brmk {\begin{remark}} +\newcommand \ermk {\end {remark}} + +%%%%%%%%% new maketitle for problem sets %%%%%%%%%%%% +\newcommand \makepsettitle { +\begin{center} +{\sc \@title } \\ +\@date +\end{center} +\markright{ {\rm \@author} } % should probably be shipped out of this macro +} diff --git a/docs/optimizationplot.pdf b/docs/SDPB_Manual/optimizationplot.pdf similarity index 100% rename from docs/optimizationplot.pdf rename to docs/SDPB_Manual/optimizationplot.pdf diff --git a/docs/SDPB_input_format.md b/docs/SDPB_input_format.md index 6f62e3ae..5c8df1d2 100644 --- a/docs/SDPB_input_format.md +++ b/docs/SDPB_input_format.md @@ -36,10 +36,10 @@ but it includes zip, tar, tar.gz, 7z. If you like, you can also leave everything in a plain old directory. The JSON schema for these input files are in -[sdpb_input_control_schema.json](sdpb_input_control_schema.json), -[sdpb_input_objectives_schema.json](sdpb_input_objectives_schema.json), -[sdpb_input_block_info_schema.json](sdpb_input_block_info_schema.json), -[sdpb_input_block_data_schema.json](sdpb_input_block_data_schema.json). Running `SDPB` on the +[sdpb_input_control_schema.json](json_schema/sdpb_input_control_schema.json), +[sdpb_input_objectives_schema.json](json_schema/sdpb_input_objectives_schema.json), +[sdpb_input_block_info_schema.json](json_schema/sdpb_input_block_info_schema.json), +[sdpb_input_block_data_schema.json](json_schema/sdpb_input_block_data_schema.json). Running `SDPB` on the test example pvm2sdp 1024 test/data/pvm2sdp/pvm.xml test/out/pvm2sdp/sdp.zip diff --git a/docs/Singularity.md b/docs/Singularity.md index 232c51db..68336a0f 100644 --- a/docs/Singularity.md +++ b/docs/Singularity.md @@ -1,17 +1,21 @@ # Singularity -[Singularity](https://www.sylabs.io/docs/) is the preferred method for -running containers on HPC systems. For laptop and desktop Linux -machines, it is often a more user friendly way of running containers -than Docker. Singularity can use existing Docker images. +Singularity +(since 2021 split into two forks, [SingularityCE](https://www.sylabs.io/docs/) and [Apptainer](https://apptainer.org/)) +is the preferred method for running containers on HPC systems. +For laptop and desktop Linux machines, it is often a more user-friendly way of running containers than Docker. +Singularity can use existing Docker images. +## Install Singularity On Debian and Ubuntu, Singularity is available in the default repository in the `singularity-container` package. So installing Singularity should be as simple as running sudo apt-get install singularity-container - -On Redhat and Centos, Singularity is available through EPEL. The Singularity web site has detailed [installation instructions](https://www.sylabs.io/guides/3.2/user-guide/installation.html#install-the-centos-rhel-package-using-yum). + +On Redhat and Centos, Singularity is available through EPEL. +The Singularity website has +detailed [installation instructions](https://www.sylabs.io/guides/3.2/user-guide/installation.html#install-the-centos-rhel-package-using-yum). On HPC systems, if Singularity is available, it is probably installed as a module. Run @@ -20,30 +24,51 @@ as a module. Run and look for `singularity`. +## Convert from Docker image Once Singularity is installed, the next step is to download and convert the image - singularity pull docker://wlandry/sdpb:2.5.1 + singularity pull docker://bootstrapcollaboration/sdpb:master Depending on your version of Singularity, this will create a file -named `sdpb-2.5.1.simg` or `sdpb_2.5.1.sif`. +named `sdpb-master.simg` or `sdpb_master.sif`. + +If you want to build Singularity image from current sources, you should build Docker image first +and push it to [local registry](https://docs.docker.com/registry/): + + # Build Docker image + docker build . --tag sdpb + + # Run local registry + docker run -d -p 5000:5000 --name registry registry:2.8 + + # Push to local regisrty + docker image tag sdpb localhost:5000/sdpb:master + docker push localhost:5000/sdpb:master + + # Pull from local registry and convert to Singularity image + singularity pull --no-https docker://localhost:5000/sdpb:master + + # Stop local registry and remove all data + docker container stop registry && docker container rm -v registry +## Run Singularity image Singularity should automatically mount your home directory. If your -Singularity image is named `sdpb-2.5.1.simg`, you can invoke the SDPB +Singularity image is named `sdpb-2.6.0.sif`, you can invoke the SDPB programs by prepending the command with - singularity exec sdpb-2.5.1.simg + singularity exec sdpb-2.6.0.sif So to convert the XML input file at `/home/user/input.xml`, run the command - singularity exec sdpb-2.5.1.simg mpirun -n 4 pvm2sdp 1024 /home/user/input.xml /home/user/input + singularity exec sdpb-2.6.0.sif mpirun -n 4 pvm2sdp 1024 /home/user/input.xml /home/user/input This uses 4 cores when running pvm2sdp. You can change that number to match your own machine. To find a primal-dual solution, - singularity exec sdpb-2.5.1.simg mpirun -n 4 sdpb --precision=1024 --procsPerNode=4 -s /home/user/input + singularity exec sdpb-2.6.0.sif mpirun -n 4 sdpb --precision=1024 --procsPerNode=4 -s /home/user/input In theory, Singularity can be used to run jobs across multiple nodes. We have not been able to make that work yet. So for large, multi-node diff --git a/docs/Usage.md b/docs/Usage.md index 59f79ceb..2ef03f96 100644 --- a/docs/Usage.md +++ b/docs/Usage.md @@ -1,9 +1,13 @@ # Usage Details of how SDPB works are described in the -[manual](SDPB-Manual.pdf). An example input file +[manual](SDPB_Manual/SDPB-Manual.pdf). An example input file [pvm.xml](../test/data/pvm2sdp/pvm.xml) is included with the source code. +Some known issues and workaround are described [below](#common-issues-and-workarounds). +You may also [find](https://github.com/davidsd/sdpb/issues) unresolved issues +or [report](https://github.com/davidsd/sdpb/issues/new) a new one in the GitHub repository. + The build system creates the executables `pvm2sdp`, `sdp2input`, and `sdpb` in the `build` directory. There are two steps when running SDPB. @@ -31,8 +35,8 @@ conversion. `[INPUT]` is a single Mathematica, JSON, or NSV (Null Separated Value) file. `[OUTPUT]` is an output directory. The single file Mathematica and JSON formats are described in Section -3.2 of the [manual](SDPB-Manual.pdf). In addition, for JSON there -is a [schema](sdp2input_schema.json). +3.2 of the [manual](SDPB_Manual/SDPB-Manual.pdf). In addition, for JSON there +is a [schema](json_schema/sdp2input_schema.json). The NSV format allows you to load an SDP from multiple JSON and/or Mathematica files. NSV files contain a list of files, separated by @@ -64,7 +68,7 @@ SDP, with the NSV example loading the SDP from two Mathematica files: Use `pvm2sdp` to create input files from Polynomial Vector Matrix files in XML. The format for these XML files is described in Section 3.1 of -[the manual](SDPB-Manual.pdf). The usage is +[the manual](SDPB_Manual/SDPB-Manual.pdf). The usage is pvm2sdp [PRECISION] [INPUT] ... [OUTPUT] @@ -130,6 +134,15 @@ uses the srun command The documentation for your HPC system will tell you how to write a batch script and invoke MPI programs. +Note also that usually you have to **load modules** on your HPC before running SDPB. You can find the corresponding +command in installations instructions for your HPC (see [docs/site_installs](site_installs) folder). For example, +on [Expanse](site_installs/Expanse.md) this command reads + + module load cpu/0.15.4 gcc/10.2.0 openmpi/4.0.4 gmp/6.1.2 mpfr/4.0.2 cmake/3.18.2 openblas/dynamic/0.3.7 + + +Note that most computation for different blocks can be done in parallel, and optimal performance is generally achieved +when the number of MPI jobs is comparable to the number of blocks. To efficiently run large MPI jobs, SDPB needs an accurate measurement of the time to evaluate each block. If `block_timings` does not @@ -156,20 +169,6 @@ In addition to having the same block structure, the runs must also use the same `precision`, `procsPerNode`, and number and distribution of cores. -## Optimizing Memory Use - -SDPB's defaults are set for optimal performance. This may result in -using more memory than is available. Running SDPB on more nodes will -reduce the amount of memory required on each node. If this is not -sufficient, you can also use the option `--procGranularity`. -This option sets minimum number of processes that a block group can -have, so it must evenly divide the `--procsPerNode` option. Using a -larger granularity will result in less memory use (up to a point) -because SDPB will make fewer local copies of the matrix Q. However, -larger granularity is also slower because even small blocks will be -distributed among multiple cores. So you should use -`--procGranularity` only when absolutely needed. - ## Running approx_objective If you have a family of SDP's and a solution to one of these SDP's, @@ -204,7 +203,7 @@ A full example of the whole sequence is The output is a JSON list with each element including the location of the new SDP, the approximate objective, and the first and second order -terms. There is a [JSON schema](approx_objective_schema.json) +terms. There is a [JSON schema](json_schema/approx_objective_schema.json) describing the format. The objective can be surprisingly sensitive to small changes, @@ -254,5 +253,60 @@ This will output the spectra into `test/out/spectrum/spectrum.json` and should l } ] -It is a json file with arrays of zeros. There is a [JSON schema](spectrum_schema.json) +It is a json file with arrays of zeros. There is a [JSON schema](json_schema/spectrum_schema.json) describing the format. + +## Common issues and workarounds + +### SDPB is slow, how many cores should I use for optimal performance? + +Most computation for different blocks can be done in parallel, and optimal performance is generally achieved when the +number of MPI jobs approaches the number of blocks. + +Note, however, that increasing number of MPI processes increases also communication overhead, especially between +different machines. Thus, sometimes single-node computation can outperform multi-node ones. + +You may use these considerations as a starting point, and run benchmarks in your environment to find the best +configuration for your problem. + +### SDPB fails with out-of-memory, std::bad_alloc etc. + +SDPB's defaults are set for optimal performance. This may result in using more memory than is available. + +Two ways to reduce memory usage: + +1. Running SDPB on more nodes will reduce the amount of memory required on each node. +2. You can also use the option `--procGranularity`. + This option sets minimum number of processes that a block group can have, so it must evenly divide + the `--procsPerNode` option. Using a larger granularity will result in less memory use (up to a point) because SDPB + will make fewer local copies of the matrix Q. However, larger granularity is also slower because even small blocks + will be distributed among multiple cores. So you should use `--procGranularity` only when absolutely needed. + +### SDPB crashes when using all available cores on the node + +We observed unexpected crashes for large SDPB runs even with enough memory, e.g. `--procsPerNode=128` on Expanse HPC ( +having 128 cores per node). +In such cases, reducing this option e.g. to `--procsPerNode=64` may help. + +### SDPB fails to read large sdp.zip + +Sometimes this happens if sdp.zip size exceeds 4GB. You may try to unzip it to some folder and pass the folder instead +of zip archive to sdpb: + +``` +unzip -o path/to/sdp.zip -d path/to/sdp_dir +sdpb -s path/to/sdp_dir <...> +``` + +### Spectrum does not work in parallel + +See https://github.com/davidsd/sdpb/issues/152. + +If this happens, replace, e.g. `mpirun -n 6 build/spectrum <...>` with `mpirun -n 1 build/spectrum <...>` or +simply `build/spectrum <...>`. + +### Spectrum does not find zeros + +Try to set `--threshold` option for `spectrum` larger than `--dualityGapThreshold` for `sdpb`. + +Note that currently spectrum [cannot find isolated zeros](https://github.com/davidsd/sdpb/issues/153). \ No newline at end of file diff --git a/docs/approx_objective_schema.json b/docs/json_schema/approx_objective_schema.json similarity index 100% rename from docs/approx_objective_schema.json rename to docs/json_schema/approx_objective_schema.json diff --git a/docs/functions_schema.json b/docs/json_schema/functions_schema.json similarity index 100% rename from docs/functions_schema.json rename to docs/json_schema/functions_schema.json diff --git a/docs/points_schema.json b/docs/json_schema/points_schema.json similarity index 100% rename from docs/points_schema.json rename to docs/json_schema/points_schema.json diff --git a/docs/sdp2input_schema.json b/docs/json_schema/sdp2input_schema.json similarity index 100% rename from docs/sdp2input_schema.json rename to docs/json_schema/sdp2input_schema.json diff --git a/docs/sdpb_input_block_data_schema.json b/docs/json_schema/sdpb_input_block_data_schema.json similarity index 100% rename from docs/sdpb_input_block_data_schema.json rename to docs/json_schema/sdpb_input_block_data_schema.json diff --git a/docs/sdpb_input_block_info_schema.json b/docs/json_schema/sdpb_input_block_info_schema.json similarity index 100% rename from docs/sdpb_input_block_info_schema.json rename to docs/json_schema/sdpb_input_block_info_schema.json diff --git a/docs/sdpb_input_control_schema.json b/docs/json_schema/sdpb_input_control_schema.json similarity index 100% rename from docs/sdpb_input_control_schema.json rename to docs/json_schema/sdpb_input_control_schema.json diff --git a/docs/sdpb_input_objectives_schema.json b/docs/json_schema/sdpb_input_objectives_schema.json similarity index 100% rename from docs/sdpb_input_objectives_schema.json rename to docs/json_schema/sdpb_input_objectives_schema.json diff --git a/docs/spectrum_schema.json b/docs/json_schema/spectrum_schema.json similarity index 100% rename from docs/spectrum_schema.json rename to docs/json_schema/spectrum_schema.json diff --git a/docs/site_installs/Apple_MacBook.md b/docs/site_installs/Apple_MacBook.md new file mode 100644 index 00000000..73db5add --- /dev/null +++ b/docs/site_installs/Apple_MacBook.md @@ -0,0 +1,48 @@ +# Apple MacBook Air M2 + + Chip: Apple M2 + macOS: Ventura 13.6 + +# Build SDPB from sources + +## Homebrew packages + +If you don't have Homebrew package manager, install it following official instructions from https://brew.sh/. Add brew +to path following instructions shown after installing. + +Then you can install packages required for SDPB: + + brew install gmp mpfr boost rapidjson libarchive openblas cmake open-mpi + +You can see installation directory and another information for a package (e.g. `boost`) +by calling `brew info ` (e.g. `brew info boost`). + +## Python 2 + +If Python 2 is missing, install it using macOS installer from https://www.python.org/downloads/release/python-2718/ + +You may try to skip this step, but note that waf fails with `bunzip2: Data integrity error when decompressing` when +using default Python 3, at least on some laptops. + +## Elemental + + git clone https://gitlab.com/bootstrapcollaboration/elemental.git + cd elemental + mkdir build + cd build + cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/install -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc + make && make install + cd ../.. + +## sdpb + + git clone https://github.com/davidsd/sdpb.git + cd sdpb + CXXFLAGS="${CXXFLAGS} -arch arm64" LDFLAGS="${LDFLAGS} -arch arm64" ./waf configure --elemental-dir=$HOME/install --boost-dir=/opt/homebrew/opt/boost/ --gmpxx-dir=/opt/homebrew/Cellar/gmp/6.3.0/ --mpfr-dir=/opt/homebrew/Cellar/mpfr/4.2.1/ --rapidjson-dir=/opt/homebrew/Cellar/rapidjson/1.1.0 --libarchive-dir=/opt/homebrew/Cellar/libarchive/3.7.2/ --prefix=$HOME/install/sdpb-master + ./waf # build binaries + ./build/sdpb --help # run SDPB binary + ./test/run_all_tests.sh # run tests to check correctness + ./waf install # install sdpb to --prefix=$HOME/install/sdpb-master + +If waf fails to find some package, e.g. `boost`, check the installation directory by calling, e.g. `brew info boost` and +update `--boost-dir` argument above. \ No newline at end of file diff --git a/docs/site_installs/Boston.md b/docs/site_installs/Boston.md new file mode 100644 index 00000000..393fdd2c --- /dev/null +++ b/docs/site_installs/Boston.md @@ -0,0 +1,94 @@ +# System info + + AlmaLinux release 8.8 (Sapphire Caracal) + Linux scc2 4.18.0-477.15.1.el8_8.x86_64 #1 SMP Mon Jul 24 04:58:00 EDT 2023 x86_64 x86_64 x86_64 GNU/Linux + +### Useful links + +- [BU Shared Computing Cluster Documentation](https://www.bu.edu/tech/support/research/system-usage/) +- [Running MPI jobs](https://www.bu.edu/tech/support/research/system-usage/running-jobs/) + +# Load modules + +For compiling and/or running SDPB, you have to load modules first: + + module load python3 gcc/12.2.0 openmpi/4.1.5 cmake/3.22.2 gmplib/6.2.1 mpfr/4.0.1 boost/1.79.0_gcc5+ openblas/0.3.23 + +You may run `module -t list` to view loaded modules, +and `module purge` to unload all modules. + +You may also add this command to your `~/.bashrc` file, so that modules will load automatically. + +# Use existing SDPB installation + +## Choose SDPB version + +SDPB is installed in `/usr2/collab/vdommes/install/sdpb-` folder, +where `` denotes specific version. + +You may list all available versions via + + ls /usr2/collab/vdommes/install | grep sdpb + +Fo example, `sdpb-master` is built from the latest [master](https://github.com/davidsd/sdpb/tree/master) branch ( +run `sdpb --version` to see commit hash, e.g. `SDPB 2.5.1-130-g88b1c9ae`), +and `sdpb-2.6.0` is a stable [2.6.0](https://github.com/davidsd/sdpb/releases/tag/2.6.0) release. + +Examples below are for `sdpb-master`. +You may replace it with another version, e.g. `sdpb-2.6.0`. +In that case, please refer +to [2.6.0 documentation](https://github.com/davidsd/sdpb/blob/2.6.0/docs/site_installs/Boston.md). + +## Run SDPB + + /usr2/collab/vdommes/install/sdpb-master/bin/sdpb --help + +### Batch script example + + qsub /usr2/collab/vdommes/install/sdpb-master/share/sdpb_example.sh + +This command submits `sdpb_example.sh` to +the [queueing system](https://www.bu.edu/tech/support/research/system-usage/running-jobs/). + +`sdpb_example.sh` loads modules and runs `sdp2input`+`sdpb` for a simple problem. +See script code and comments for more details. + +Script output is written to the log file in the current directory, e.g.: +`./run_sdpb_example.sh.o8388881`. +SDPB output files are written to the `./out/` folder in the current directory. + +# Build SDPB from sources + +Use `RPATH` instead of `RUNPATH` in `mpicxx` linker, to fix shared library loading in SDPB: + + export OMPI_LDFLAGS="$(mpicxx --showme:link) -Wl,--disable-new-dtags" + +## Elemental + git clone https://gitlab.com/bootstrapcollaboration/elemental.git + cd elemental + mkdir build + cd build + cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/install -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc + make && make install + cd ../.. + +## RapidJSON + git clone https://github.com/Tencent/rapidjson.git + cp -r rapidjson/include $HOME/install + +## libarchive + wget http://www.libarchive.org/downloads/libarchive-3.7.1.tar.xz + tar -xf libarchive-3.7.1.tar.xz + cd libarchive-3.7.1 + ./configure --prefix=$HOME/install + make && make install + cd .. + +## sdpb + git clone https://github.com/davidsd/sdpb.git + cd sdpb + ./waf configure --elemental-dir=$HOME/install --rapidjson-dir=$HOME/install --libarchive-dir=$HOME/install --prefix=$HOME/install/sdpb-master + ./waf # -j 1 + ./test/run_all_tests.sh + ./waf install + cd .. diff --git a/docs/site_installs/Caltech.md b/docs/site_installs/Caltech.md index f13abdbf..09ef67ad 100644 --- a/docs/site_installs/Caltech.md +++ b/docs/site_installs/Caltech.md @@ -1,95 +1,95 @@ -System info ------------ - CentOS Linux release 7.6.1810 (Core) - Linux login1 3.10.0-957.1.3.el7.x86_64 #1 SMP Thu Nov 29 14:49:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux - glibc-devel-2.17-260.el7_6.3.x86_64 - -sdpb -==== - module load cmake/3.10.2 gcc/7.3.0 openmpi/3.0.0 boost/1_68_0-gcc730 eigen/eigen - # Can not use openblas module - # It is compiled with the old gcc, and that would require loading - # two different libgfortran.so - git clone https://gitlab.com/bootstrapcollaboration/elemental.git libelemental - cd libelemental - mkdir build - cd build - cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/install -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc - make - make install +# System info -RapidJSON ---------- - git clone https://github.com/Tencent/rapidjson.git - cd rapidjson - mkdir build - cd build - cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/install - make - make install + CentOS Linux release 7.9.2009 (Core) + Linux login2.cm.cluster 3.10.0-1160.53.1.el7.x86_64 #1 SMP Fri Jan 14 13:59:45 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux -libarchive ----------- - wget http://www.libarchive.org/downloads/libarchive-3.5.1.tar.xz - ./configure --prefix=$HOME/install - make -j 16 - make install +### Useful links: -sdpb ----- - ./waf configure --elemental-dir=$HOME/install --rapidjson-dir=$HOME/install --libarchive-dir=$HOME/install --prefix=$HOME/install - ./waf # I needed to do './waf -j 1' (single threaded) to get it to compile without crashing - ./waf install +- [Caltech HPC documentation](https://hpc.sites.caltech.edu/documentation) +- [How to run MPI jobs](https://hpc.sites.caltech.edu/documentation/slurm-commands) +- [Software and Modules](https://hpc.sites.caltech.edu/documentation/software-and-modules) -scalar_blocks -============= +# Load modules -Trilinos --------- - git clone --branch trilinos-release-12-12-branch https://github.com/trilinos/Trilinos.git - mkdir build - cd build - cmake -DCMAKE_CXX_COMPILER=g++ -DCMAKE_C_COMPILER=gcc -DTrilinos_ENABLE_Sacado=ON -DTrilinos_ENABLE_Kokkos=OFF -DTrilinos_ENABLE_Teuchos=OFF -DCMAKE_INSTALL_PREFIX=$HOME/install .. - make - make install - -scalar_blocks -------------- - git clone https://gitlab.com/bootstrapcollaboration/scalar_blocks.git - cd scalar_blocks - ./waf configure --prefix=$HOME/install --trilinos-dir=$HOME/install --eigen-incdir=/software/eigen-b3f3d4950030/ - ./waf # maybe -j 1 - ./waf install +For compiling and/or running SDPB, you have to load modules first: + + module load cmake/3.25.1 gcc/9.2.0 openmpi/4.1.5 boost/1_81_0_openmpi-4.1.1_gcc-9.2.0 eigen/eigen mpfr/4.0.2 + +You may run `module -t list` to view loaded modules, +and `module purge` to unload all modules. + +You may also add this command to your `~/.bashrc` file, so that modules will load automatically. + +# Use existing SDPB installation + +## Choose SDPB version + +SDPB is installed in `/central/home/vdommes/install/sdpb-` folder, +where `` denotes specific version. + +You may list all available versions via + + ls /central/home/vdommes/install | grep sdpb -blocks_3d -========= +Fo example, `sdpb-master` is built from the latest [master](https://github.com/davidsd/sdpb/tree/master) branch ( +run `sdpb --version` to see commit hash, e.g. `SDPB 2.5.1-130-g88b1c9ae`), +and `sdpb-2.6.0` is a stable [2.6.0](https://github.com/davidsd/sdpb/releases/tag/2.6.0) release. -fmt ---- - wget https://github.com/fmtlib/fmt/releases/download/6.2.1/fmt-6.2.1.zip +Examples below are for `sdpb-master`. +You may replace it with another version, e.g. `sdpb-2.6.0`. +In that case, please refer +to [2.6.0 documentation](https://github.com/davidsd/sdpb/blob/2.6.0/docs/site_installs/Caltech.md). + +## Run SDPB + + /central/home/vdommes/install/sdpb-master/bin/sdpb --help + +### Batch script example + + sbatch /central/home/vdommes/install/sdpb-master/share/sdpb_example.sh + +This command submits `sdpb_example.sh` to +the [queueing system](https://hpc.sites.caltech.edu/documentation/slurm-commands). + +`sdpb_example.sh` loads modules and runs `sdp2input`+`sdpb` for a simple problem. +See script code and comments for more details. + +Script output is written to the log file in the current directory, e.g.: +`./sdpb_example.sh.26306151.out`. +SDPB output files are written to the `./out/` folder in the current directory. + +# Build SDPB from sources + +TODO: Note that `/usr/include/gmp.h` is present on login node, but may be absent on compute nodes. In that case one +should compile on a login node. + +## Elemental + + git clone https://gitlab.com/bootstrapcollaboration/elemental.git + cd elemental mkdir build cd build - cmake -DCMAKE_CXX_COMPILER=g++ -DCMAKE_INSTALL_PREFIX=$HOME/install .. - make - make install - -blocks_3d ---------- - git clone git@gitlab.com:bootstrapcollaboration/blocks_3d.git - cd blocks_3d - ./waf configure --prefix=$HOME/install --eigen-incdir=/software/eigen-b3f3d4950030/ --fmt-dir=$HOME/install --fmt-libdir=$HOME/install/lib64 - ./waf # maybe -j - ./waf install + cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/install -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc + make && make install + cd ../.. -Note: If you are trying to build the "profiling" branch of `blocks_3d`, you will need to load the following modules before attempting the above: +## RapidJSON + git clone https://github.com/Tencent/rapidjson.git + cp -r rapidjson/include $HOME/install - module purge - module load gcc/9.2.0 - module load boost/1_76_0_gcc-9.2.0 - module load python3/3.7.0 - module load eigen/eigen +## libarchive -Batch scripts -------------- - /home/wlandry/sdpb/runs/TTTT_small.sh - /home/wlandry/scalar_blocks/runs/scalar_blocks.sh + wget http://www.libarchive.org/downloads/libarchive-3.7.1.tar.xz + tar -xf libarchive-3.7.1.tar.xz + cd libarchive-3.7.1 + ./configure --prefix=$HOME/install + make && make install + cd ../.. + +## sdpb + + ./waf configure --elemental-dir=$HOME/install --rapidjson-dir=$HOME/install --libarchive-dir=$HOME/install --mpfr-dir=/central/software/mpfr/4.0.2 --prefix=$HOME/install/sdpb-master + ./waf # -j 1 + ./test/run_all_tests.sh + ./waf install + cd .. diff --git a/docs/site_installs/Docker.org b/docs/site_installs/Docker.org deleted file mode 100644 index 5d73d157..00000000 --- a/docs/site_installs/Docker.org +++ /dev/null @@ -1,63 +0,0 @@ -There are two sets of instructions in parallel, one for Podman, one -for Docker. Podman has been tested more recently and is recommended. - -These instructions assume that you have a rootfs.tar.xz image of -Debian's 10.10 release (buster). It should work with minimal changes -for the most recent 11 release (bullseye). You should be able to get -the image from https://docker.debian.net/ - -It also assumes that you have a copy of the MPSolve 2.2 tar ball. You -can get that from https://numpi.dm.unipi.it/mpsolve-2.2/ - -The instructions have you build SDPB, scalar_blocks, blocks_3d, and -spectrum_extraction in one container in the directory -$HOME/docker/sdpb_dev/. Then you will copy the necessary bits to a -second container built in $HOME/docker/sdpb/. Doing it this way -allows the second container to be significantly smaller. - -The instructions assume that $HOME/docker/sdpb has rootfs.tar.xz and -sdpb_dev_Dockerfile, and $HOME/docker/sdpb_dev has rootfs.tar.xz, -mpsolve.tgz, and sdpb_Dockerfile. You will need to rename -sdpb_dev_Dockerfile and sdpb_Dockerfile to Dockerfile in their -respective directories. - -You will probably need to change the mount directories (-v option) for -your local system. - -There are also instructions for pushing the container to -hub.docker.com. - -* Build -** In $HOME/docker/sdpb_dev/ -*** Update date for base OS -*** Update 'git log' hash -*** Podman -**** podman build --tag=sdpb_dev . -**** podman run -v $HOME/docker/sdpb:/usr/local/src/tgz sdpb_dev tar -zcvf tgz/sdpb.tgz ../bin/ ../lib/ -*** Docker -**** docker build --tag=sdpb_dev . -**** docker run -v $HOME/docker/sdpb:/usr/local/src/tgz sdpb_dev tar -zcvf tgz/sdpb.tgz ../bin/ ../lib/ -** In $HOME/docker/sdpb/ -*** Update date for base OS -*** Podman -**** podman build --tag=sdpb:2.5.1 . -**** podman login -u wlandry docker.io -**** podman tag sdpb:2.5.1 docker.io/wlandry/sdpb:2.5.1 -**** podman push docker.io/wlandry/sdpb:2.5.1 -*** Docker -**** docker build --tag=sdpb:2.5.1 . -**** docker login -u wlandry -**** docker tag sdpb:2.5.1 wlandry/sdpb:2.5.1 -**** docker push wlandry/sdpb:2.5.1 -* Run -** Podman -*** podman run -v $HOME/scalar_blocks:/usr/local/share/scalar_blocks sdpb:2.5.1 scalar_blocks --dim 3 --order 5 --max-derivs 6 --spin-ranges 0-21 --poles 6 --delta-12 0.8791 --delta-34 0.8791 --num-threads=4 -o /usr/local/share/scalar_blocks/tests/benchmark --precision=665 -*** podman run -v $HOME/blocks_3d:/usr/local/share/blocks_3d sdpb:2.5.1 blocks_3d --j-external "0, 0, 0, 0" --j-internal "0-4" --j-12 0 --j-43 0 --delta-12 0.12 --delta-43 0.543 --four-pt-struct "0, 0, 0, 0" --four-pt-sign=1 --order 20 --lambda 5 --kept-pole-order 10 --num-threads 4 --precision 665 -o /usr/local/share/blocks_3d/test/benchmark/derivs_{}.json --coordinates="zzb,yyb,xt,ws,xt_radial,ws_radial" --delta-1-plus-2="2.3" -*** podman run -v $HOME/sdpb/:/usr/local/share/sdpb sdpb:2.5.1 mpirun --allow-run-as-root pvm2sdp 1024 /usr/local/share/sdpb/test/test.xml /usr/local/share/sdpb/test/test -*** podman run -v $HOME/sdpb/:/usr/local/share/sdpb sdpb:2.5.1 mpirun --allow-run-as-root sdpb --precision=1024 --noFinalCheckpoint --procsPerNode=4 -s /usr/local/share/sdpb/test/test -*** podman run -v $HOME/sdpb/:/usr/local/share/spectrum sdpb:2.5.1 spectrum.py -s /usr/local/share/spectrum/test/spectrum_test.xml -o /usr/local/share/spectrum/test/spectrum_test_out -m /usr/local/share/spectrum/test/spectrum_test.spectrum.m -p 1024 -** Docker -*** docker run -v $HOME/scalar_blocks:/usr/local/share/scalar_blocks sdpb:2.5.1 scalar_blocks --dim 3 --order 5 --max-derivs 6 --spin-ranges 0-21 --poles 6 --delta-12 0.8791 --delta-34 0.8791 --num-threads=4 -o /usr/local/share/scalar_blocks/tests/benchmark --precision=665 -*** docker run -v $HOME/sdpb/:/usr/local/share/sdpb sdpb:2.5.1 mpirun --allow-run-as-root pvm2sdp 1024 /usr/local/share/sdpb/test/test.xml /usr/local/share/sdpb/test/test -*** docker run -v $HOME/sdpb/:/usr/local/share/sdpb sdpb:2.5.1 mpirun --allow-run-as-root sdpb --precision=1024 --noFinalCheckpoint --procsPerNode=4 -s /usr/local/share/sdpb/test/test -*** docker run -v $HOME/sdpb/:/usr/local/share/spectrum sdpb:2.5.1 spectrum.py -s /usr/local/share/spectrum/test/spectrum_test.xml -o /usr/local/share/spectrum/test/spectrum_test_out -m /usr/local/share/spectrum/test/spectrum_test.spectrum.m -p 1024 diff --git a/docs/site_installs/Expanse.md b/docs/site_installs/Expanse.md new file mode 100644 index 00000000..69a4cb19 --- /dev/null +++ b/docs/site_installs/Expanse.md @@ -0,0 +1,96 @@ +# System info + + Rocky Linux release 8.8 (Green Obsidian) + Linux exp-1-33 4.18.0-477.15.1.el8_8.x86_64 #1 SMP Wed Jun 28 15:04:18 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux + +### Useful links + +- [Expanse HPC User Guide](https://www.sdsc.edu/support/user_guides/expanse.html) +- [Running MPI jobs](https://www.sdsc.edu/support/user_guides/expanse.html#running) + +# Load modules + +For compiling and/or running SDPB, you have to load modules first: + + module load cpu/0.15.4 gcc/10.2.0 openmpi/4.0.4 gmp/6.1.2 mpfr/4.0.2 cmake/3.18.2 openblas/dynamic/0.3.7 boost/1.74.0 slurm + +You may run `module -t list` to view loaded modules, +and `module purge` to unload all modules. + +You may also add this command to your `~/.bashrc` file, so that modules will load automatically. + +# Use existing SDPB installation + +## Choose SDPB version + +SDPB is installed in `/home/vdommes/install/sdpb-` folder, +where `` denotes specific version. + +You may list all available versions via + + ls /home/vdommes/install | grep sdpb + +Fo example, `sdpb-master` is built from the latest [master](https://github.com/davidsd/sdpb/tree/master) branch ( +run `sdpb --version` to see commit hash, e.g. `SDPB 2.5.1-130-g88b1c9ae`), +and `sdpb-2.6.0` is a stable [2.6.0](https://github.com/davidsd/sdpb/releases/tag/2.6.0) release. + +Examples below are for `sdpb-master`. +You may replace it with another version, e.g. `sdpb-2.6.0`. +In that case, please refer +to [2.6.0 documentation](https://github.com/davidsd/sdpb/blob/2.6.0/docs/site_installs/Expanse.md). + +## Run SDPB + + /home/vdommes/install/sdpb-master/bin/sdpb --help + +### Batch script example + + sbatch /home/vdommes/install/sdpb-master/share/sdpb_example.sh + +This command submits `sdpb_example.sh` to +the [queueing system](https://www.sdsc.edu/support/user_guides/expanse.html#running). + +`sdpb_example.sh` loads modules and runs `sdp2input`+`sdpb` for a simple problem. +See script code and comments for more details. + +Script output is written to the log file in the current directory, e.g.: +`./sdpb_example.sh.26306151.out`. +SDPB output files are written to the `./out/` folder in the current directory. + +# Build SDPB from sources + +TODO: on compute nodes compiler fails to find some libs + +## Elemental + + git clone https://gitlab.com/bootstrapcollaboration/elemental.git + cd elemental + mkdir build + cd build + cmake .. -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc -DCMAKE_Fortran_COMPILER=mpif90 -DCMAKE_INSTALL_PREFIX=$HOME/install -DMATH_LIBS="-L$BLASDIR -lopenblas" + make -j 64 && make install + cd ../.. + +## libarchive + + wget http://www.libarchive.org/downloads/libarchive-3.7.1.tar.xz + tar -xf libarchive-3.7.1.tar.xz + cd libarchive-3.7.1 + ./configure --prefix=$HOME/install + make -j 64 && make install + cd .. + +## RapidJSON + + git clone https://github.com/Tencent/rapidjson.git + cp -r rapidjson/include $HOME/install + +## sdpb + + git clone https://github.com/davidsd/sdpb.git + cd sdpb + CXX=mpicxx ./waf configure --prefix=$HOME/install/sdpb-master --elemental-dir=$HOME/install --rapidjson-dir=$HOME/install --boost-dir=$HOME/install --libarchive-dir=$HOME/install + ./waf # -j 1 + ./test/run_all_tests.sh + ./waf install + cd .. diff --git a/docs/site_installs/Expanse.org b/docs/site_installs/Expanse.org deleted file mode 100644 index 4b13ba5c..00000000 --- a/docs/site_installs/Expanse.org +++ /dev/null @@ -1,113 +0,0 @@ -* CentOS Linux release 8.1.1911 (Core) - I made two installations: OpenMPI and MVAPICH. I would recommend - the OpenMPI version, since that is what is used at almost all other - installations. There was some concern that the OpenMPI version was - buggy, but that seems not to be the case. I am keeping the notes - here in case it is useful to others. - -* gcc openmpi -** module - module load cpu gcc/10.2.0 openmpi/4.0.4 gmp/6.1.2 mpfr/4.0.2 cmake/3.18.2 openblas/dynamic/0.3.7 - Already loaded cpu, gcc/10.2.0 - -** boost - # UPDATE: Maybe we can just do this: - module load boost/1.74.0 - # OLD instructions: - wget https://boostorg.jfrog.io/artifactory/main/release/1.74.0/source/boost_1_74_0.tar.bz2 - tar -jxf boost_1_74_0.tar.bz2 - cd boost_1_74_0 - ./bootstrap.sh --prefix=$HOME/gnu_openmpi/install --without-libraries=python - ./b2 --prefix=$HOME/gnu_openmpi/install -j 64 - ./b2 --prefix=$HOME/gnu_openmpi/install install - -** Elemental - git clone https://gitlab.com/bootstrapcollaboration/elemental.git - cmake .. -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc -DCMAKE_Fortran_COMPILER=mpif90 -DCMAKE_INSTALL_PREFIX=$HOME/gnu_openmpi/install -DMATH_LIBS="-L$BLASDIR -lopenblas" - -** libarchive - wget http://www.libarchive.org/downloads/libarchive-3.5.1.tar.xz - ./configure --prefix=$HOME/gnu_openmpi/install - -** RapidJSON - git clone https://github.com/Tencent/rapidjson.git - cmake .. -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_INSTALL_PREFIX=$HOME/gnu_openmpi/install - -** sdpb - git clone https://github.com/davidsd/sdpb.git - CXX=mpicxx ./waf configure --prefix=$HOME/gnu_openmpi/install --elemental-dir=$HOME/gnu_openmpi/install --rapidjson-dir=$HOME/gnu_openmpi/install --boost-dir=$HOME/gnu_openmpi/install --libarchive-dir=$HOME/gnu_openmpi/install - -** Trilinos - git clone --branch trilinos-release-12-12-branch https://github.com/trilinos/Trilinos.git - cmake -DCMAKE_CXX_COMPILER=g++ -DCMAKE_C_COMPILER=gcc -DCMAKE_Fortran_COMPILER=gfortran -DTrilinos_ENABLE_Sacado=ON -DTrilinos_ENABLE_Kokkos=OFF -DTrilinos_ENABLE_Teuchos=OFF -DCMAKE_INSTALL_PREFIX=$HOME/gnu_openmpi/install .. - -** scalar_blocks - git clone https://gitlab.com/bootstrapcollaboration/scalar_blocks - ./waf configure --boost-dir=$HOME/gnu_openmpi/install --gmp-dir=$HOME/gnu_openmpi/install --trilinos-dir=$HOME/gnu_openmpi/install --eigen-incdir=/cm/shared/apps/spack/cpu/opt/spack/linux-centos8-zen2/intel-19.1.1.217/eigen-3.3.7-plaog3szjnn3gh6wq5co55xxjuswwo7f/include/eigen3 --prefix=$HOME/gnu_openmpi/install - -** fmt - wget https://github.com/fmtlib/fmt/releases/download/6.2.1/fmt-6.2.1.zip - unzip fmt-6.2.1.zip - cd fmt-6.2.1 - mkdir build - cd build - cmake -DCMAKE_CXX_COMPILER=g++ -DCMAKE_INSTALL_PREFIX=$HOME/gnu_openmpi/install .. - make - make install -** blocks_3d - module load boost/1.74.0 - git clone https://gitlab.com/bootstrapcollaboration/blocks_3d - ./waf configure --prefix=$HOME/gnu_openmpi/install --fmt-dir=$HOME/gnu_openmpi/install --fmt-libdir=$HOME/gnu_openmpi/install/lib64 --boost-dir=/cm/shared/apps/spack/cpu/opt/spack/linux-centos8-zen2/gcc-10.2.0/boost-1.74.0-ubdskthwa2edpqy4bx6deq6c3qy2xubs/include --eigen-incdir=/cm/shared/apps/spack/cpu/opt/spack/linux-centos8-zen2/intel-19.1.1.217/eigen-3.3.7-plaog3szjnn3gh6wq5co55xxjuswwo7f/include/eigen3 - ./waf - ./waf install - -* Batch Scripts - /home/wlandry/gnu_openmpi/runs/TTTT_small.sh - /home/wlandry/gnu_openmpi/runs/blocks.sh - -* gcc mvapich -** module - module load cpu gcc/10.2.0 mvapich2/2.3.4 gmp/6.1.2 mpfr/4.0.2 cmake/3.18.2 openblas/dynamic/0.3.7 - -** boost - wget https://dl.bintray.com/boostorg/release/1.74.0/source/boost_1_74_0.tar.bz2 - tar -jxf boost_1_74_0.tar.bz2 - cd boost_1_74_0 - ./bootstrap.sh --prefix=$HOME/gnu_mvapich/install --without-libraries=python - ./b2 --prefix=$HOME/gnu_mvapich/install -j 64 - ./b2 --prefix=$HOME/gnu_mvapich/install install - -** Elemental - git clone https://gitlab.com/bootstrapcollaboration/elemental.git - cmake .. -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc -DCMAKE_Fortran_COMPILER=mpif90 -DCMAKE_INSTALL_PREFIX=$HOME/gnu_mvapich/install -DMATH_LIBS="-L$BLASDIR -lopenblas" - -** libarchive - wget http://www.libarchive.org/downloads/libarchive-3.5.1.tar.xz - ./configure --prefix=$HOME/gnu_mvapich/install - -** RapidJSON - git clone https://github.com/Tencent/rapidjson.git - cmake .. -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_INSTALL_PREFIX=$HOME/gnu_mvapich/install - -** sdpb - git clone https://github.com/davidsd/sdpb.git - CXX=mpicxx ./waf configure --prefix=$HOME/gnu_mvapich/install --elemental-dir=$HOME/gnu_mvapich/install --rapidjson-dir=$HOME/gnu_mvapich/install --boost-dir=$HOME/gnu_mvapich/install --libarchive-dir=$HOME/gnu_mvapich/install - -** Trilinos - git clone --branch trilinos-release-12-12-branch https://github.com/trilinos/Trilinos.git - cmake .. -DCMAKE_CXX_COMPILER=g++ -DCMAKE_C_COMPILER=gcc -DCMAKE_Fortran_COMPILER=gfortran -DTrilinos_ENABLE_Sacado=ON -DTrilinos_ENABLE_Kokkos=OFF -DTrilinos_ENABLE_Teuchos=OFF -DCMAKE_INSTALL_PREFIX=$HOME/gnu_mvapich/install - -** scalar_blocks - git clone https://gitlab.com/bootstrapcollaboration/scalar_blocks - ./waf configure --boost-dir=$HOME/gnu_mvapich/install --gmp-dir=$HOME/gnu_mvapich/install --trilinos-dir=$HOME/gnu_mvapich/install --eigen-incdir=/cm/shared/apps/spack/cpu/opt/spack/linux-centos8-zen2/intel-19.1.1.217/eigen-3.3.7-plaog3szjnn3gh6wq5co55xxjuswwo7f/include/eigen3 --prefix=$HOME/gnu_mvapich/install - -** fmt - wget https://github.com/fmtlib/fmt/releases/download/6.2.1/fmt-6.2.1.zip - cmake .. -DCMAKE_CXX_COMPILER=g++ -DCMAKE_INSTALL_PREFIX=$HOME/gnu_mvapich/install -** blocks_3d - git clone https://gitlab.com/bootstrapcollaboration/blocks_3d - ./waf configure --prefix=$HOME/gnu_mvapich/install --fmt-dir=$HOME/gnu_mvapich/install --fmt-libdir=$HOME/gnu_mvapich/install/lib64 --boost-dir=$HOME/gnu_mvapich/install --eigen-incdir=/cm/shared/apps/spack/cpu/opt/spack/linux-centos8-zen2/intel-19.1.1.217/eigen-3.3.7-plaog3szjnn3gh6wq5co55xxjuswwo7f/include/eigen3 - -** Batch scripts - /home/wlandry/gnu_mvapich/runs/TTTT_small.sh - diff --git a/docs/site_installs/Harvard.md b/docs/site_installs/Harvard.md new file mode 100644 index 00000000..a1e4649c --- /dev/null +++ b/docs/site_installs/Harvard.md @@ -0,0 +1,105 @@ +# System info + + Rocky Linux release 8.7 (Green Obsidian) + Linux holylogin02.rc.fas.harvard.edu 4.18.0-425.10.1.el8_7.x86_64 #1 SMP Thu Jan 12 16:32:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux + +### Useful links + +- [Harvard FAS RC User Guide](https://docs.rc.fas.harvard.edu/) +- [Running MPI jobs](https://docs.rc.fas.harvard.edu/kb/running-jobs/) + +# Load modules + +For compiling and/or running SDPB, you have to load modules first: + + module load python gcc/12.2.0-fasrc01 cmake/3.25.2-fasrc01 openmpi/4.1.5-fasrc01 gmp/6.2.1-fasrc01 mpfr/4.2.0-fasrc01 + +You may run `module -t list` to view loaded modules, +and `module purge` to unload all modules. + +You may also add this command to your `~/.bashrc` file, so that modules will load automatically. + +# Use existing SDPB installation + +## Choose SDPB version + +SDPB is installed in `/n/home02/vdommes/install/sdpb-` folder, +where `` denotes specific version. + +You may list all available versions via + + ls /n/home02/vdommes/install | grep sdpb + +Fo example, `sdpb-master` is built from the latest [master](https://github.com/davidsd/sdpb/tree/master) branch ( +run `sdpb --version` to see commit hash, e.g. `SDPB 2.5.1-130-g88b1c9ae`), +and `sdpb-2.6.0` is a stable [2.6.0](https://github.com/davidsd/sdpb/releases/tag/2.6.0) release. + +Examples below are for `sdpb-master`. +You may replace it with another version, e.g. `sdpb-2.6.0`. +In that case, please refer +to [2.6.0 documentation](https://github.com/davidsd/sdpb/blob/2.6.0/docs/site_installs/Harvard.md). + +## Run SDPB + + /n/home02/vdommes/install/sdpb-master/bin/sdpb --help + +### Batch script example + + sbatch /n/home02/vdommes/install/sdpb-master/share/sdpb_example.sh + +This command submits `sdpb_example.sh` to +the [queueing system](https://docs.rc.fas.harvard.edu/kb/running-jobs/). + +`sdpb_example.sh` loads modules and runs `sdp2input`+`sdpb` for a simple problem. +See script code and comments for more details. + +Script output is written to the log file in the current directory, e.g.: +`./sdpb_example.sh.26306151.out`. +SDPB output files are written to the `./out/` folder in the current directory. + +# Build SDPB from sources + +Use `RPATH` instead of `RUNPATH` in `mpicxx` linker, to fix shared library loading in SDPB: + + export OMPI_LDFLAGS="$(mpicxx --showme:link) -Wl,--disable-new-dtags" + +## Boost + + wget https://boostorg.jfrog.io/artifactory/main/release/1.83.0/source/boost_1_83_0.tar.bz2 + tar -jxf boost_1_83_0.tar.bz2 + cd boost_1_83_0 + ./bootstrap.sh --prefix=$HOME/install --without-libraries=python + ./b2 -j 16 --prefix=$HOME/install + ./b2 --prefix=$HOME/install install + cd .. + +## Elemental + + git clone https://gitlab.com/bootstrapcollaboration/elemental.git + mkdir build + cd build + cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/install -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc + make && make install + +## RapidJSON + + git clone https://github.com/Tencent/rapidjson.git + cp -r rapidjson/include $HOME/install + +## libarchive + + wget http://www.libarchive.org/downloads/libarchive-3.7.1.tar.xz + tar -xf libarchive-3.7.1.tar.xz + cd libarchive-3.7.1 + ./configure --prefix=$HOME/install + make -j 16 && make install + cd .. + +## sdpb + + git clone https://github.com/davidsd/sdpb.git + cd sdpb + ./waf configure --boost-dir=$HOME/install --elemental-dir=$HOME/install --rapidjson-dir=$HOME/install --libarchive-dir=$HOME/install --prefix=$HOME/install/sdpb-master + ./waf # -j 1 + ./test/run_all_tests.sh + ./waf install diff --git a/docs/site_installs/Harvard.org b/docs/site_installs/Harvard.org deleted file mode 100644 index 1f834cad..00000000 --- a/docs/site_installs/Harvard.org +++ /dev/null @@ -1,44 +0,0 @@ -* Cannon -* CentOS Linux release 7.6.1810 (Core) - Linux holylogin01.rc.fas.harvard.edu 3.10.0-957.12.1.el7.x86_64 #1 SMP Mon Apr 29 14:59:59 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux - glibc-devel-2.17-260.el7_6.4.x86_64 -** modules -module load gcc/9.2.0-fasrc01 cmake/3.16.1-fasrc01 OpenBLAS/0.3.7-fasrc02 openmpi/4.0.2-fasrc01 eigen/3.3.7-fasrc02 libxml2/2.7.8-fasrc02 metis/5.1.0-fasrc01 -module unload boost - - Eigen automatically loads boost, but do not use boost module. It - is built with gcc 4.8.5, and 1.63 does not work with scalar blocks - for weird reasons anyway. - -** Boost - wget https://sourceforge.net/projects/boost/files/boost/1.68.0/boost_1_68_0.tar.bz2/download - ./bootstrap.sh --prefix=$HOME/install --without-libraries=python - ./b2 --prefix=$HOME/install -j 16 - ./b2 --prefix=$HOME/install install - -** Elemental - git clone https://gitlab.com/bootstrapcollaboration/elemental.git libelemental - cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/install -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc -DMETIS_INCLUDE_DIR=/n/helmod/apps/centos7/Core/metis/5.1.0-fasrc01/include -DMETIS_LIBRARY=/n/helmod/apps/centos7/Core/metis/5.1.0-fasrc01/lib/libmetis.a - -** RapidJSON - git clone https://github.com/Tencent/rapidjson.git - cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/install - -** libarchive - wget http://www.libarchive.org/downloads/libarchive-3.5.1.tar.xz - ./configure --prefix=$HOME/install - make -j 16 - make install - -** sdpb - ./waf configure --boost-dir=$HOME/install --elemental-dir=$HOME/install --rapidjson-dir=$HOME/install --libarchive-dir=$HOME/install --prefix=$HOME/install - Lots and lots of deprecated warnings. - -** Trilinos - cmake -DCMAKE_CXX_COMPILER=g++ -DCMAKE_C_COMPILER=gcc -DTrilinos_ENABLE_Sacado=ON -DTrilinos_ENABLE_Kokkos=OFF -DTrilinos_ENABLE_Teuchos=OFF -DCMAKE_INSTALL_PREFIX=$HOME/install .. -** scalar_blocks - ./waf configure --prefix=/n/home09/wlandry/install --trilinos-dir=$HOME/install --boost-dir=$HOME/install - -* Batch Scripts - /n/home09/wlandry/sdpb/runs/TTTT_small.sh - /n/home09/wlandry/scalar_blocks/runs/scalar_blocks.sh diff --git a/docs/site_installs/Imperial.md b/docs/site_installs/Imperial.md new file mode 100644 index 00000000..06888fbe --- /dev/null +++ b/docs/site_installs/Imperial.md @@ -0,0 +1,95 @@ +# System info + + Red Hat Enterprise Linux release 8.5 (Ootpa) + Linux login-c 4.18.0-348.20.1.el8_5.x86_64 #1 SMP Tue Mar 8 12:56:54 EST 2022 x86_64 x86_64 x86_64 GNU/Linux + +### Useful links for Imperial College London HPC: + +- [HPC documentation](https://wiki.imperial.ac.uk/display/HPC) +- [MPI jobs](https://wiki.imperial.ac.uk/display/HPC/MPI+Jobs) +- [Applications](https://wiki.imperial.ac.uk/display/HPC/Applications) + +# Load modules + +For compiling and/or running SDPB, you have to load modules first: + + module load cmake/3.18.2 oneapi/mpi/2021.4.0 gcc/11.2.0 boost/1.78.0 gmp/5.0.1 mpfr/3.1.1 metis/5.1.0 + +You may run `module -t list` to view loaded modules, +and `module purge` to unload all modules. + +You may also add this command to your `~/.bashrc` file, so that modules will load automatically. + +# Use existing SDPB installation + +## Choose SDPB version + +SDPB is installed in `/rds/general/user/vdommes/home/install/sdpb-` folder, +where `` denotes specific version. + +You may list all available versions via + + ls /rds/general/user/vdommes/home/install | grep sdpb + +Fo example, `sdpb-master` is built from the latest [master](https://github.com/davidsd/sdpb/tree/master) branch ( +run `sdpb --version` to see commit hash, e.g. `SDPB 2.5.1-130-g88b1c9ae`), +and `sdpb-2.6.0` is a stable [2.6.0](https://github.com/davidsd/sdpb/releases/tag/2.6.0) release. + +Examples below are for `sdpb-master`. +You may replace it with another version, e.g. `sdpb-2.6.0`. +In that case, please refer +to [2.6.0 documentation](https://github.com/davidsd/sdpb/blob/2.6.0/docs/site_installs/Imperial.md). + +## Run SDPB + + /rds/general/user/vdommes/home/install/sdpb-master/bin/sdpb --help + +### Batch script example + + qsub /rds/general/user/vdommes/home/install/sdpb-master/share/sdpb_example.sh + +This command submits `sdpb_example.sh` to +the [queueing system](https://wiki.imperial.ac.uk/display/HPC/Queueing+System). + +`sdpb_example.sh` loads modules and runs `sdp2input`+`sdpb` for a simple problem. +See script code and comments for more details. + +Script output is written to the log file in the current directory, e.g.: +`./run_sdpb_example.sh.o8388881`. +SDPB output files are written to the `./out/` folder in the current directory. + +# Build SDPB from sources + +## Elemental + + git clone https://gitlab.com/bootstrapcollaboration/elemental.git + cd elemental + mkdir build + cd build + cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/install -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc + make && make install + cd ../.. + +## RapidJSON + + git clone https://github.com/Tencent/rapidjson.git + cp -r rapidjson/include $HOME/install + +## libarchive + + git clone -b v3.6.2 https://github.com/libarchive/libarchive.git + cd libarchive + ./build/autogen.sh + ./configure --prefix=$HOME/install + make && make install + cd ../.. + +## sdpb + + git clone https://github.com/davidsd/sdpb.git + cd sdpb + python3 ./waf configure --mpfr-dir=$MPFR_HOME --elemental-dir=$HOME/install --rapidjson-dir=$HOME/install --libarchive-dir=$HOME/install --prefix=$HOME/install/sdpb-master + python3 ./waf # -j 1 + ./test/run_all_tests.sh mpirun + python3 ./waf install + cd .. diff --git a/docs/site_installs/Readme.md b/docs/site_installs/Readme.md index 39a01092..57125b30 100644 --- a/docs/site_installs/Readme.md +++ b/docs/site_installs/Readme.md @@ -1,5 +1,7 @@ -The *.org files in this directory contain notes on building and -running SDPB on various HPC machines. Most of the systems use +The *.org and *.md files in this directory contain notes on building and +running SDPB on various HPC machines, as well as [Apple MacBooks](Apple_MacBook). + +Most of the HPC systems use modules, so there is typically a module invocation like module load gcc/8.3.0 openmpi/3.1.4 openblas/0.3.6 @@ -8,7 +10,7 @@ to load all necessary modules. Generally, I tried to use as many of the builtin modules and system libraries as possible. This was not always possible. For example, `GMP` might be built without C++ support. More commonly, `Boost` was built with the system compiler, -so it is incompatible with the C++ 14 compiler required for the rest +so it is incompatible with the C++ 17 compiler required for the rest of the code. Some of the systems run into problems if you try to build too much in @@ -17,6 +19,14 @@ For example, 'make -j 4' or './waf -j 4' will only use 4 cores. On some systems I found it expedient to submit a compile job to the queue. +You can check SDPB installation by running + + ./test/run_all_tests.sh + +More details about testing can be found in [Install.md](../../Install.md) + +(**TODO**: old note from Walter below, scripts can be outdated or missing:) + At the end of the file, there is an example of running TTTT_small. It should take about 510 iterations and several hours, ending with primal and dual optimal solution. The values for the primal and dual objectives should be diff --git a/docs/site_installs/sdpb_Dockerfile b/docs/site_installs/sdpb_Dockerfile deleted file mode 100644 index c721bc96..00000000 --- a/docs/site_installs/sdpb_Dockerfile +++ /dev/null @@ -1,7 +0,0 @@ -FROM scratch -ADD rootfs.tar.xz / - -RUN apt-get update && apt-get -y dist-upgrade && echo "Wed Sep 22 10:23:41 2021" -RUN apt-get -y install openmpi-bin libgmp10 libgmpxx4ldbl libmpfr6 libboost-system1.74.0 libboost-filesystem1.74.0 libboost-program-options1.74.0 libboost-date-time1.74.0 libboost-iostreams1.74.0 libopenblas-base libmetis5 libtrilinos-teuchos12 libtrilinos-kokkos12 python3-lxml python3-mpmath libarchive13 - -ADD sdpb.tgz /usr/local diff --git a/docs/site_installs/sdpb_dev_Dockerfile b/docs/site_installs/sdpb_dev_Dockerfile deleted file mode 100644 index 4cbd3f8d..00000000 --- a/docs/site_installs/sdpb_dev_Dockerfile +++ /dev/null @@ -1,25 +0,0 @@ -FROM scratch -ADD rootfs.tar.xz / - -RUN apt-get update && apt-get -y dist-upgrade && echo "Wed Sep 22 10:23:41 2021" -RUN apt-get -y install openmpi-bin libopenmpi-dev libgmp-dev libmpfr-dev libmpfrc++-dev libboost-all-dev g++ cmake libopenblas-dev libxml2-dev git libmetis-dev pkg-config libeigen3-dev libtrilinos-sacado-dev libtrilinos-kokkos-dev libtrilinos-teuchos-dev rapidjson-dev libarchive-dev libfmt-dev -WORKDIR /usr/local/src -RUN git clone https://gitlab.com/bootstrapcollaboration/elemental.git && cd elemental && git log -1 cc51ddb0f33a467fbf580989d9951620ff2a7877 -RUN mkdir -p elemental/build -RUN cd elemental/build && cmake .. -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc -DCMAKE_INSTALL_PREFIX=/usr/local/ -RUN cd elemental/build && make -j `nproc` -RUN cd elemental/build && make install -RUN git clone https://github.com/davidsd/sdpb && cd sdpb && git log -1 69ea607373557f45bebc324f329b277f28394bd8 -RUN cd sdpb && python3 ./waf configure --elemental-dir=/usr/local --prefix=/usr/local -RUN cd sdpb && python3 ./waf && python3 ./waf install -RUN git clone https://gitlab.com/bootstrapcollaboration/scalar_blocks.git && cd scalar_blocks && git log -1 6698eeb209726f086096900b34866319bc029964 -RUN cd scalar_blocks && CXX=mpicxx python3 ./waf configure --prefix=/usr/local -RUN cd scalar_blocks && python3 ./waf && python3 ./waf install -RUN git clone https://gitlab.com/bootstrapcollaboration/blocks_3d.git && cd blocks_3d && git log -1 4b18e2420249b443b1560f4c0db09ec52b9508e6 -RUN cd blocks_3d && python3 ./waf configure --prefix=/usr/local -RUN cd blocks_3d && python3 ./waf && python3 ./waf install -RUN git clone https://gitlab.com/bootstrapcollaboration/spectrum-extraction.git && cd spectrum-extraction -RUN install spectrum-extraction/spectrum.py /usr/local/bin -RUN ln -s /usr/bin/python3 /usr/local/bin/python -ADD mpsolve.tgz . -RUN cd MPSolve-2.2 && make -j && install unisolve /usr/local/bin