From e54a0ffc6e2645c4617145f38a45e6ac9e51d353 Mon Sep 17 00:00:00 2001 From: Caspar van Leeuwen Date: Fri, 20 Oct 2023 17:46:12 +0200 Subject: [PATCH 1/7] Added documentation on how to build on top of EESSI --- docs/using_eessi/building_on_eessi.md | 69 +++++++++++++++++++++++++++ mkdocs.yml | 1 + 2 files changed, 70 insertions(+) create mode 100644 docs/using_eessi/building_on_eessi.md diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md new file mode 100644 index 000000000..1a2bc0c18 --- /dev/null +++ b/docs/using_eessi/building_on_eessi.md @@ -0,0 +1,69 @@ +# Building software on top of EESSI + +## Building software on top of EESSI with EasyBuild +Building on top of EESSI with EasyBuild is relatively straightforward. One crucial feature is that EasyBuild supports building against operating system libraries that are _not_ in a standard prefix (such as `/usr/lib`). This is required when building against EESSI, since all of the software in EESSI is build against the [compatibility layer](../compatibility_layer.md). + +### Starting the EESSI software environment +Start your environment as described [here](../using_eessi/setting_up_environment.md) + +### Configure EasyBuild +To configure EasyBuild, first, check out the Github repository. We advice you check out the branch corresponding to the version of EESSI you would like to use. + +If you are unsure which version you are using, you can run +``` +echo ${EESSI_PILOT_VERSION} +``` +to check it. + +To build on top of e.g. version `2023.06` of the EESSI software stack, we check it out, and go into that directory: + +``` +git clone https://github.com/EESSI/software-layer/ --branch 2023.06 +cd software-layer +``` +Then, you have to pick a working directory (that you have write access to) where EasyBuild can do the build, and an install directory, where EasyBuild can install it. In this example, we use `/tmp/easybuild` as our working directory, and `$HOME/.local/easybuild` as our installpath: +``` +export WORKDIR=/tmp/easybuild +export EASYBUILD_INSTALLPATH="${HOME}/.local/easybuild" +source configure_easybuild +``` +Next, you load the EasyBuild module that you want to use, e.g. +``` +module load EasyBuild/4.8.1 +``` +Finally, you can check the current configuration for EasyBuild using +``` +eb --show-config +``` + +!!! Note + We use EasyBuild's default behaviour in optimizing for the host architecture. Since the EESSI initalization script also loads the EESSI stack that is optimized for your host architecture, this matches nicely. However, if you work on a cluster with heterogeneous node types, you have to realize you can only use these builds on the same architecture as where you build them. You can use different `EASYBUILD_INSTALLPATH`s if you want to build for different host architectures. + +### Building +Now, you are ready to build. For example, at the time of writing, `netCDF-4.9.0-gompi-2022a.eb` was not in the EESSI environment yet, so you can build it yourself: +``` +eb netCDF-4.9.0-gompi-2022a.eb +``` + +!!! Note + If this netCDF module is available by the time you are trying, you can force a local rebuild by adding the `--force` argument in order to experiment with building locally, or pick a different EasyConfig to build. + +### Adding your module directory to the `MODULEPATH` +Finally, we need to make sure this module is available on our `MODULEPATH`. The path where EasyBuild installs modules by default is `${EASYBUILD_INSTALLPATH}/modules/all`. You can add it to your modulepath by running: +``` +module use ${EASYBUILD_INSTALLPATH}/modules/all +``` +You may want to do this as part of your `.bashrc`. + +Now, we should be able to load our newly build module: +``` +module load netCDF/4.9.0-gompi-2022a +``` + +!!! Note + Be careful adding to the `MODULEPATH` in your `.bashrc` if you are on a cluster with heterogenous architectures. You don't want to pick up on a module that was not compiled for the correct architectures accidentally. + +## Manually building software op top of EESSI +Building software on top of EESSI would require your linker to use the same system-dependencies as the software in EESSI does. In other words: it requires you to link against libraries from the compatibility layer, instead of from your host OS. + +While we plan to support this in the future, manually building on top of EESSI is not supported yet. diff --git a/mkdocs.yml b/mkdocs.yml index 4ed1996f9..140e09280 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -40,6 +40,7 @@ nav: - Using EESSI: - using_eessi/setting_up_environment.md - using_eessi/basic_commands.md + - using_eessi/building_on_eessi.md - using_eessi/eessi_demos.md - Getting support: support.md - Meetings: From d518b915ab7c702cdef416690a5cbc87c6105544 Mon Sep 17 00:00:00 2001 From: Caspar van Leeuwen Date: Fri, 20 Oct 2023 17:53:51 +0200 Subject: [PATCH 2/7] Fixed typo --- docs/using_eessi/building_on_eessi.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 1a2bc0c18..8534f05dd 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -37,7 +37,7 @@ eb --show-config ``` !!! Note - We use EasyBuild's default behaviour in optimizing for the host architecture. Since the EESSI initalization script also loads the EESSI stack that is optimized for your host architecture, this matches nicely. However, if you work on a cluster with heterogeneous node types, you have to realize you can only use these builds on the same architecture as where you build them. You can use different `EASYBUILD_INSTALLPATH`s if you want to build for different host architectures. + We use EasyBuild's default behaviour in optimizing for the host architecture. Since the EESSI initialization script also loads the EESSI stack that is optimized for your host architecture, this matches nicely. However, if you work on a cluster with heterogeneous node types, you have to realize you can only use these builds on the same architecture as where you build them. You can use different `EASYBUILD_INSTALLPATH`s if you want to build for different host architectures. ### Building Now, you are ready to build. For example, at the time of writing, `netCDF-4.9.0-gompi-2022a.eb` was not in the EESSI environment yet, so you can build it yourself: @@ -61,7 +61,7 @@ module load netCDF/4.9.0-gompi-2022a ``` !!! Note - Be careful adding to the `MODULEPATH` in your `.bashrc` if you are on a cluster with heterogenous architectures. You don't want to pick up on a module that was not compiled for the correct architectures accidentally. + Be careful adding to the `MODULEPATH` in your `.bashrc` if you are on a cluster with heterogeneous architectures. You don't want to pick up on a module that was not compiled for the correct architectures accidentally. ## Manually building software op top of EESSI Building software on top of EESSI would require your linker to use the same system-dependencies as the software in EESSI does. In other words: it requires you to link against libraries from the compatibility layer, instead of from your host OS. From c5b59a304d80f1709ac42759b9b645f5afb28b01 Mon Sep 17 00:00:00 2001 From: Caspar van Leeuwen Date: Mon, 30 Oct 2023 15:14:01 +0100 Subject: [PATCH 3/7] More detail on how to handle heterogenous nodes. Took other comments from Thomas into account --- docs/using_eessi/building_on_eessi.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 8534f05dd..29d7e0818 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -7,7 +7,7 @@ Building on top of EESSI with EasyBuild is relatively straightforward. One cruci Start your environment as described [here](../using_eessi/setting_up_environment.md) ### Configure EasyBuild -To configure EasyBuild, first, check out the Github repository. We advice you check out the branch corresponding to the version of EESSI you would like to use. +To configure EasyBuild, first, check out the [EESSI software-layer repository](https://github.com/EESSI/software-layer.git). We advise you to check out the branch corresponding to the version of EESSI you would like to use. If you are unsure which version you are using, you can run ``` @@ -21,9 +21,9 @@ To build on top of e.g. version `2023.06` of the EESSI software stack, we check git clone https://github.com/EESSI/software-layer/ --branch 2023.06 cd software-layer ``` -Then, you have to pick a working directory (that you have write access to) where EasyBuild can do the build, and an install directory, where EasyBuild can install it. In this example, we use `/tmp/easybuild` as our working directory, and `$HOME/.local/easybuild` as our installpath: +Then, you have to pick a working directory (that you have write access to) where EasyBuild can do the build, and an install directory (with sufficient storage space), where EasyBuild can install it. In this example, we create a temporary directory in `/tmp/` as our working directory, and use `$HOME/.local/easybuild` as our installpath: ``` -export WORKDIR=/tmp/easybuild +export WORKDIR=$(mktemp --directory --tmpdir=/tmp -t eessi-build.XXXXXXXXXX) export EASYBUILD_INSTALLPATH="${HOME}/.local/easybuild" source configure_easybuild ``` @@ -37,7 +37,7 @@ eb --show-config ``` !!! Note - We use EasyBuild's default behaviour in optimizing for the host architecture. Since the EESSI initialization script also loads the EESSI stack that is optimized for your host architecture, this matches nicely. However, if you work on a cluster with heterogeneous node types, you have to realize you can only use these builds on the same architecture as where you build them. You can use different `EASYBUILD_INSTALLPATH`s if you want to build for different host architectures. + We use EasyBuild's default behaviour in optimizing for the host architecture. Since the EESSI initialization script also loads the EESSI stack that is optimized for your host architecture, this matches nicely. However, if you work on a cluster with heterogeneous node types, you have to realize you can only use these builds on the same architecture as where you build them. You can use different `EASYBUILD_INSTALLPATH`s if you want to build for different host architectures. For example, when you are on a system that has a mix of `AMD zen3` and `AMD zen4` nodes, you might want to use `EASYBUILD_INSTALLPATH=$HOME/.local/easybuild/zen3` when building on a `zen3` node, `EASYBUILD_INSTALLPATH=$HOME/.local/easybuild/zen4` when building on a `zen4` node. Then, in the step beloww, instead of the `module use` command listed there, you can use `module use $HOME/.local/easybuild/zen3/modules/all` when you want to run on a `zen3` node and `module use $HOME/.local/easybuild/zen4/modules/all` when you want to run on a `zen4` node. ### Building Now, you are ready to build. For example, at the time of writing, `netCDF-4.9.0-gompi-2022a.eb` was not in the EESSI environment yet, so you can build it yourself: @@ -53,7 +53,8 @@ Finally, we need to make sure this module is available on our `MODULEPATH`. The ``` module use ${EASYBUILD_INSTALLPATH}/modules/all ``` -You may want to do this as part of your `.bashrc`. + +You may want to do this as part of your `.bashrc` (unless your system is heterogenous, since then the `module use` path may then be different per run - see the note above). Now, we should be able to load our newly build module: ``` From 863fa026ca6c1fdf63d4654c939226a7f0e0e9fd Mon Sep 17 00:00:00 2001 From: Caspar van Leeuwen Date: Mon, 30 Oct 2023 15:21:05 +0100 Subject: [PATCH 4/7] Removed some duplicate comments, as that was already in a node. Make sure it's clear the EESSI environment needs to be loaded before loading the module that was build on top --- docs/using_eessi/building_on_eessi.md | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 29d7e0818..225a621df 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -48,22 +48,25 @@ eb netCDF-4.9.0-gompi-2022a.eb !!! Note If this netCDF module is available by the time you are trying, you can force a local rebuild by adding the `--force` argument in order to experiment with building locally, or pick a different EasyConfig to build. -### Adding your module directory to the `MODULEPATH` -Finally, we need to make sure this module is available on our `MODULEPATH`. The path where EasyBuild installs modules by default is `${EASYBUILD_INSTALLPATH}/modules/all`. You can add it to your modulepath by running: +### Using the newly build module +First, you'll need to add the subdirectory of the `EASYBUILD_INSTALLPATH` that contains the modules to the `MODULEPATH`. You can do that using: + ``` module use ${EASYBUILD_INSTALLPATH}/modules/all ``` -You may want to do this as part of your `.bashrc` (unless your system is heterogenous, since then the `module use` path may then be different per run - see the note above). +you may want to do this as part of your `.bashrc`. + +!!! Note + Be careful adding to the `MODULEPATH` in your `.bashrc` if you are on a cluster with heterogeneous architectures. You don't want to pick up on a module that was not compiled for the correct architectures accidentally. + +Since your module is build on top of the EESSI environment, that needs to be loaded first (as described [here](../using_eessi/setting_up_environment.md)), if you haven't already done so. -Now, we should be able to load our newly build module: +Finally, you should be able to load our newly build module: ``` module load netCDF/4.9.0-gompi-2022a ``` -!!! Note - Be careful adding to the `MODULEPATH` in your `.bashrc` if you are on a cluster with heterogeneous architectures. You don't want to pick up on a module that was not compiled for the correct architectures accidentally. - ## Manually building software op top of EESSI Building software on top of EESSI would require your linker to use the same system-dependencies as the software in EESSI does. In other words: it requires you to link against libraries from the compatibility layer, instead of from your host OS. From c509868734ae3626f7903b737ce12140430df349 Mon Sep 17 00:00:00 2001 From: ocaisa Date: Tue, 7 Nov 2023 09:32:45 +0100 Subject: [PATCH 5/7] Update docs/using_eessi/building_on_eessi.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Thomas Röblitz --- docs/using_eessi/building_on_eessi.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 225a621df..d0960a4fd 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -24,8 +24,8 @@ cd software-layer Then, you have to pick a working directory (that you have write access to) where EasyBuild can do the build, and an install directory (with sufficient storage space), where EasyBuild can install it. In this example, we create a temporary directory in `/tmp/` as our working directory, and use `$HOME/.local/easybuild` as our installpath: ``` export WORKDIR=$(mktemp --directory --tmpdir=/tmp -t eessi-build.XXXXXXXXXX) -export EASYBUILD_INSTALLPATH="${HOME}/.local/easybuild" source configure_easybuild +export EASYBUILD_INSTALLPATH="${HOME}/.local/easybuild" ``` Next, you load the EasyBuild module that you want to use, e.g. ``` From ef238de539a76860bd6401dbe0d9398eb4086b57 Mon Sep 17 00:00:00 2001 From: ocaisa Date: Tue, 7 Nov 2023 09:34:09 +0100 Subject: [PATCH 6/7] Address review comments --- docs/using_eessi/building_on_eessi.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index d0960a4fd..530311c06 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -29,7 +29,7 @@ export EASYBUILD_INSTALLPATH="${HOME}/.local/easybuild" ``` Next, you load the EasyBuild module that you want to use, e.g. ``` -module load EasyBuild/4.8.1 +module load EasyBuild/4.8.2 ``` Finally, you can check the current configuration for EasyBuild using ``` @@ -48,7 +48,7 @@ eb netCDF-4.9.0-gompi-2022a.eb !!! Note If this netCDF module is available by the time you are trying, you can force a local rebuild by adding the `--force` argument in order to experiment with building locally, or pick a different EasyConfig to build. -### Using the newly build module +### Using the newly built module First, you'll need to add the subdirectory of the `EASYBUILD_INSTALLPATH` that contains the modules to the `MODULEPATH`. You can do that using: ``` From 485eab57697f2e42900f09e7aee31b5dc7071fdb Mon Sep 17 00:00:00 2001 From: ocaisa Date: Tue, 7 Nov 2023 09:38:07 +0100 Subject: [PATCH 7/7] Minor tweaks and corrections --- docs/using_eessi/building_on_eessi.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 530311c06..4d1c67e8d 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -1,7 +1,7 @@ # Building software on top of EESSI ## Building software on top of EESSI with EasyBuild -Building on top of EESSI with EasyBuild is relatively straightforward. One crucial feature is that EasyBuild supports building against operating system libraries that are _not_ in a standard prefix (such as `/usr/lib`). This is required when building against EESSI, since all of the software in EESSI is build against the [compatibility layer](../compatibility_layer.md). +Building on top of EESSI with EasyBuild is relatively straightforward. One crucial feature is that EasyBuild supports building against operating system libraries that are _not_ in a standard prefix (such as `/usr/lib`). This is required when building against EESSI, since all of the software in EESSI is built against the [compatibility layer](../compatibility_layer.md). ### Starting the EESSI software environment Start your environment as described [here](../using_eessi/setting_up_environment.md) @@ -46,7 +46,7 @@ eb netCDF-4.9.0-gompi-2022a.eb ``` !!! Note - If this netCDF module is available by the time you are trying, you can force a local rebuild by adding the `--force` argument in order to experiment with building locally, or pick a different EasyConfig to build. + If this netCDF module is available by the time you are trying, you can force a local rebuild by adding the `--rebuild` argument in order to experiment with building locally, or pick a different EasyConfig to build. ### Using the newly built module First, you'll need to add the subdirectory of the `EASYBUILD_INSTALLPATH` that contains the modules to the `MODULEPATH`. You can do that using: @@ -60,7 +60,7 @@ you may want to do this as part of your `.bashrc`. !!! Note Be careful adding to the `MODULEPATH` in your `.bashrc` if you are on a cluster with heterogeneous architectures. You don't want to pick up on a module that was not compiled for the correct architectures accidentally. -Since your module is build on top of the EESSI environment, that needs to be loaded first (as described [here](../using_eessi/setting_up_environment.md)), if you haven't already done so. +Since your module is built on top of the EESSI environment, that needs to be loaded first (as described [here](../using_eessi/setting_up_environment.md)), if you haven't already done so. Finally, you should be able to load our newly build module: ``` @@ -70,4 +70,4 @@ module load netCDF/4.9.0-gompi-2022a ## Manually building software op top of EESSI Building software on top of EESSI would require your linker to use the same system-dependencies as the software in EESSI does. In other words: it requires you to link against libraries from the compatibility layer, instead of from your host OS. -While we plan to support this in the future, manually building on top of EESSI is not supported yet. +While we plan to support this in the future, manually building on top of EESSI is currently not supported yet in a trivial way.