Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new init scripts for new initialization module #668

Merged
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
d512675
Add init scripts
MaKaNu Aug 12, 2024
7f71c28
Fix fish init script
MaKaNu Aug 12, 2024
1e2f154
Add init script
MaKaNu Aug 12, 2024
a0c4377
convert bash init script
MaKaNu Aug 12, 2024
68aea9f
Add init script
MaKaNu Aug 12, 2024
788375d
Export ${debian_chroot:+($debian_chroot)}\u@\h:\w\$ for all init scr…
MaKaNu Aug 12, 2024
b14cef8
Implement module load following doc
MaKaNu Aug 13, 2024
1aa8cb8
Add init scripts test
MaKaNu Aug 16, 2024
0b6bc54
Cleanup script and FIX init module path
MaKaNu Aug 16, 2024
4aa7d00
Use haswell for testing
MaKaNu Aug 16, 2024
71543e3
Fix Pipeline
MaKaNu Aug 16, 2024
ea32ffe
Add StdEnv.lua module
MaKaNu Aug 16, 2024
bb3c8bd
Install Shells on github actions
MaKaNu Aug 16, 2024
385aee0
Fix requirements install on github action
MaKaNu Aug 16, 2024
461c941
Fix Fish command call issue
MaKaNu Aug 19, 2024
aee306c
Merge pull request #1 from EESSI/2023.06-software.eessi.io
MaKaNu Aug 21, 2024
6d7fde5
Merge branch 'EESSI:2023.06-software.eessi.io' into 2023.06-software.…
MaKaNu Sep 5, 2024
fcf4937
move init scripts into init/lmod
MaKaNu Sep 6, 2024
032cdae
Revert a0c4377e
MaKaNu Sep 6, 2024
33b6b92
Use EESSI as LMOD SYSTEM DEFAULT
MaKaNu Sep 6, 2024
d4bc3bc
Revert "Add StdEnv.lua module"
MaKaNu Sep 6, 2024
63482f5
Remove PS1 export
MaKaNu Sep 6, 2024
7f95d5e
Update init scripts path in test script
MaKaNu Sep 6, 2024
a2ce628
Remove Prompt test from unittest
MaKaNu Sep 6, 2024
2ab75c3
Rename Test Numbers
MaKaNu Sep 6, 2024
7c798d4
Remove PROMPT variable from action
MaKaNu Sep 6, 2024
8d74aea
Set correct arch override
MaKaNu Sep 6, 2024
596f605
Fix arch differences
MaKaNu Sep 11, 2024
462f180
Add Exception for CSH
MaKaNu Sep 12, 2024
68ad3af
Add Check for not testable shells
MaKaNu Sep 12, 2024
bccd55a
Add csh to test
MaKaNu Sep 12, 2024
7ad306a
Update init/lmod/csh
MaKaNu Sep 12, 2024
1b38cec
Update install_scripts.sh
MaKaNu Sep 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions .github/workflows/scripts/test_init_scripts.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
#!/bin/bash
EESSI_VERSION="2023.06"
export LMOD_PAGER=cat

# initialize assert framework
if [ ! -d assert.sh ]; then
echo "assert.sh not cloned."
echo ""
echo "run \`git clone https://github.com/lehmannro/assert.sh.git\`"
exit 1
fi
. assert.sh/assert.sh

SHELLS=$@

for shell in ${SHELLS[@]}; do
echo = | awk 'NF += (OFS = $_) + 100'
echo RUNNING TESTS FOR SHELL: $shell
echo = | awk 'NF += (OFS = $_) + 100'

# TEST 1: Source Script and check Module Output
assert "$shell -c 'source init/$shell' 2>&1 " "EESSI/$EESSI_VERSION loaded successfully"
# TEST 2: Check PS1 Prompt (Previous exported)
PROMPT="${shell^^}_PROMPT"
assert "echo ${!PROMPT}" "{EESSI $EESSI_VERSION}"
# Test 3: Check module overview
MODULE_SECTIONS=($($shell -c "source init/$shell 2>/dev/null; module ov 2>&1 | grep -e '---'"))
assert "echo ${MODULE_SECTIONS[1]}" "/cvmfs/software.eessi.io/versions/$EESSI_VERSION/software/linux/x86_64/intel/haswell/modules/all"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You've got no guarantees what architecture you will end up with so better to override the architecture to give you something predictable.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

export EESSI_ARCHDETECT_OPTIONS="x86_64/generic"

assert "echo ${MODULE_SECTIONS[4]}" "/cvmfs/software.eessi.io/versions/$EESSI_VERSION/init/modules"
# Test 4: Load Python module and check version and path
command="$shell -c 'source init/$shell 2>/dev/null; module load Python/3.10.8-GCCcore-12.2.0; python --version'"
expected="Python 3.10.8"
assert "$command" "$expected"
command="$shell -c 'source init/$shell 2>/dev/null; module load Python/3.10.8-GCCcore-12.2.0; which python'"
expected="/cvmfs/software.eessi.io/versions/$EESSI_VERSION/software/linux/x86_64/intel/haswell/software/Python/3.10.8-GCCcore-12.2.0/bin/python"
assert "$command" "$expected"
done

assert_end source_eessi

# RESET PAGER
export LMOD_PAGER=
50 changes: 50 additions & 0 deletions .github/workflows/tests_init_module.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# documentation: https://help.github.com/en/articles/workflow-syntax-for-github-actions
name: Check for EESSI init shell scripts to load eessi software module in software.eessi.io
on:
push:
branches: [ "*-software.eessi.io" ]
pull_request:
workflow_dispatch:
permissions:
contents: read # to fetch code (actions/checkout)
jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
EESSI_VERSION:
- 2023.06
EESSI_SOFTWARE_SUBDIR_OVERRIDE:
- x86_64/haswell
MaKaNu marked this conversation as resolved.
Show resolved Hide resolved
steps:
- name: Check out software-layer repository
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1

- name: Mount EESSI CernVM-FS pilot repository
uses: cvmfs-contrib/github-action-cvmfs@55899ca74cf78ab874bdf47f5a804e47c198743c # v4.0
with:
cvmfs_config_package: https://github.com/EESSI/filesystem-layer/releases/download/latest/cvmfs-config-eessi_latest_all.deb
cvmfs_http_proxy: DIRECT
cvmfs_repositories: software.eessi.io

- name: Clone assert.sh script
run: git clone https://github.com/lehmannro/assert.sh.git

- name: Install missing shells
run: |
sudo apt update
sudo apt install zsh ksh fish
echo "# INIT ZSH" > ~/.zshrc

- name: Prepare PROMPT variables
run: |
export BASH_PROMPT=$(bash -c 'source init/bash; echo "$PS1"')
export ZSH_PROMPT=$(zsh -c 'source init/zsh; echo "$PS1"')
export KSH_PROMPT=$(ksh -c 'source init/ksh; echo "$PS1"')
export FISH_PROMPT=$(fish -c 'source init/fish; echo "$PS1"')

- name: Run tests for available shells
run: |
.github/workflows/scripts/test_init_scripts.sh "bash" "zsh" "ksh" "fish"

56 changes: 16 additions & 40 deletions init/bash
Original file line number Diff line number Diff line change
@@ -1,41 +1,17 @@
function show_msg {
# only echo msg if EESSI_SILENT is unset
msg=$1
if [[ -z ${EESSI_SILENT+x} ]]; then
echo "$msg"
fi
}

# The following method should be safe, but might break if file is a symlink
# (could switch to $(dirname "$(readlink -f "$BASH_SOURCE")") in that case)
source $(dirname "$BASH_SOURCE")/eessi_environment_variables

# only continue if setting EESSI environment variables worked fine
if [ $? -eq 0 ]; then

export PS1="{EESSI $EESSI_VERSION} $PS1"

# add location of commands provided by compat layer to $PATH;
# see https://github.com/EESSI/software-layer/issues/52
export PATH=$EPREFIX/usr/bin:$EPREFIX/bin:$PATH

# init Lmod
show_msg "Initializing Lmod..."
source $EESSI_EPREFIX/usr/share/Lmod/init/bash

# prepend location of modules for EESSI software stack to $MODULEPATH
show_msg "Prepending $EESSI_MODULEPATH to \$MODULEPATH..."
module use $EESSI_MODULEPATH
show_msg "Prepending site path $EESSI_SITE_MODULEPATH to \$MODULEPATH..."
module use $EESSI_SITE_MODULEPATH

#show_msg ""
#show_msg "*** Known problems in the ${EESSI_VERSION} software stack ***"
#show_msg ""
#show_msg "1) ..."
#show_msg ""
#show_msg ""

echo "Environment set up to use EESSI (${EESSI_VERSION}), have fun!"

# Choose an EESSI version
EESSI_VERSION="${EESSI_VERSION:-2023.06}"
ocaisa marked this conversation as resolved.
Show resolved Hide resolved
# Path to top-level module tree
export MODULEPATH=/cvmfs/software.eessi.io/versions/"$EESSI_VERSION"/init/modules
. /cvmfs/software.eessi.io/versions/"$EESSI_VERSION"/compat/linux/$(uname -m)/usr/share/Lmod/init/bash

export PS1
if [ -z "$__Init_Default_Modules" ]; then
export __Init_Default_Modules=1;

## ability to predefine elsewhere the default list
LMOD_SYSTEM_DEFAULT_MODULES=${LMOD_SYSTEM_DEFAULT_MODULES:-"StdEnv"}
export LMOD_SYSTEM_DEFAULT_MODULES
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
LMOD_SYSTEM_DEFAULT_MODULES=${LMOD_SYSTEM_DEFAULT_MODULES:-"StdEnv"}
LMOD_SYSTEM_DEFAULT_MODULES=${LMOD_SYSTEM_DEFAULT_MODULES:-"EESSI"}

You'll need this everywhere

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some sites also force full name matching so you probably can't leave it versionless

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is now implemented with Version number. It doesn't matter to me but like to understand the difference why StdEnv.lua is not the correct solution.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no existing StdEnv.lua when Lmod is configured (we don't ship one)

module --initial_load --no_redirect restore
else
module refresh
fi
18 changes: 18 additions & 0 deletions init/csh
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Choose an EESSI version
if (! $?EESSI_VERSION) then; set EESSI_VERSION = "2023.06"; endif
# Path to top-level module tree
setenv MODULEPATH /cvmfs/software.eessi.io/versions/"$EESSI_VERSION"/init/modules
source /cvmfs/software.eessi.io/versions/"$EESSI_VERSION"/compat/linux/`uname -m`/usr/share/Lmod/init/csh

setenv PS1 $prompt
if (! $?__Init_Default_Modules ) then
setenv __Init_Default_Modules 1;

## ability to predefine elsewhere the default list
if (! $?LMOD_SYSTEM_DEFAULT_MODULES) then; setenv LMOD_SYSTEM_DEFAULT_MODULES "StdEnv"; endif
module --initial_load --no_redirect restore
else
module refresh
endif
#module load "$LMOD_SYSTEM_DEFAULT_MODULES"

16 changes: 16 additions & 0 deletions init/fish
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Choose an EESSI version
set EESSI_VERSION (set -q EESSI_VERSION; and echo "$EESSI_VERSION"; or echo "2023.06")
# Path to top-level module tree
set -x MODULEPATH /cvmfs/software.eessi.io/versions/"$EESSI_VERSION"/init/modules
. /cvmfs/software.eessi.io/versions/"$EESSI_VERSION"/compat/linux/(uname -m)/usr/share/Lmod/init/fish

export PS1
if test -z "$__Init_Default_Modules"
export __Init_Default_Modules=1;

## ability to predefine elsewhere the default list
set -x LMOD_SYSTEM_DEFAULT_MODULES (set -q LMOD_SYSTEM_DEFAULT_MODULE; and echo "$LMOD_SYSTEM_DEFAULT_MODULE"; or echo "StdEnv")
module --initial_load --no_redirect restore
else
module refresh
end
17 changes: 17 additions & 0 deletions init/ksh
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Choose an EESSI version
EESSI_VERSION="${EESSI_VERSION:-2023.06}"
# Path to top-level module tree
export MODULEPATH=/cvmfs/software.eessi.io/versions/"$EESSI_VERSION"/init/modules
. /cvmfs/software.eessi.io/versions/"$EESSI_VERSION"/compat/linux/$(uname -m)/usr/share/Lmod/init/ksh

export PS1
if [ -z "$__Init_Default_Modules" ]; then
export __Init_Default_Modules=1;

## ability to predefine elsewhere the default list
LMOD_SYSTEM_DEFAULT_MODULES=${LMOD_SYSTEM_DEFAULT_MODULES:-"StdEnv"}
export LMOD_SYSTEM_DEFAULT_MODULES
module --initial_load --no_redirect restore
else
module refresh
fi
1 change: 1 addition & 0 deletions init/modules/StdEnv.lua
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
load("EESSI/2023.06")
17 changes: 17 additions & 0 deletions init/zsh
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Choose an EESSI version
EESSI_VERSION="${EESSI_VERSION:-2023.06}"
# Path to top-level module tree
export MODULEPATH=/cvmfs/software.eessi.io/versions/"$EESSI_VERSION"/init/modules
. /cvmfs/software.eessi.io/versions/"$EESSI_VERSION"/compat/linux/$(uname -m)/usr/share/Lmod/init/zsh

export PS1
if [ -z "$__Init_Default_Modules" ]; then
export __Init_Default_Modules=1;

## ability to predefine elsewhere the default list
LMOD_SYSTEM_DEFAULT_MODULES=${LMOD_SYSTEM_DEFAULT_MODULES:-"StdEnv"}
export LMOD_SYSTEM_DEFAULT_MODULES
module --initial_load --no_redirect restore
else
module refresh
fi
Loading