Conference call notes 20221207

(back to Conference calls)

Notes on the 211th EasyBuild conference call, Wednesday 7 December 2022 (09:00 UTC)

Attendees

List of attendees (11):

• Simon Branford (Univ. of Birmingham, UK)
• Alex Domingo (Vrije Universiteit Brussel, Belgium)
• Jasper Grimm (University of York, UK)
• Kenneth Hoste (HPC-UGent)
• Alain van Hoof (TU Eindhoven, Netherlands)
• Lev Lafayette (Univ. of Melbourne, Australia)
• Sebastien Moretti (SIB, Switzerland)
• Jurij Pečar (EMBL, Germany)
• Åke Sandgren (Umeå University, Sweden)
• Jörg Saßmannshausen (Imperial College London, UK)
• Abdoul Wahid Mainassara (LuxProvide, Luxembourg)

Agenda

• overview of recent developments + outlook to next EasyBuild release
• update on porting of EasyBuild docs to MarkDown
• 2022b update for common toolchains
• Q&A

Recent developments

• release timeline
  • latest release: EasyBuild v4.6.2 (21 Oct 2022)
  • ETA next release (EasyBuild v4.7.0): by mid Dec'22
• recent changes
  • framework
    • bug fixes
      • ...
    • enhancements
      • check whether nvidia-smi/rocm-smi command is available before trying to run it in get_gpu_info (PR #4131)
      • automatically cancel Github Action workflow runs for outdated commits (PR #4132)
    • changes
      • ...
    • docs
      • correct uses of :param name: and :return: in docstrings (PR #4129)
      • docstrings - fixes to formatting or variables names (PR #4130)
  • easyblocks
    • bug fixes
      • create $XDG_CACHE_HOME for PyTorch tests (PR #2806)
      • fix counting of failures in PyTorch tests (PR #2834)
      • fix Python 2.7 compatibility of PyTorch easyblock (PR #2840)
    • enhancements
      • automatically cancel Github Action workflow runs for outdated commits (PR #2835)
      • use fixed names for bazel/wrapper subdirectories used when building TensorFlow, to make debugging easier (PR #2841)
    • updates
      • update Xmipp easyblock to handle effects of CUDA at SYSTEM level and newer CUDA version requirements for stdc++ (PR #2831)
      • update LLVM easyblock to put 'cmake' symlink in place so separate CMake modules requires for LLVM 15+ can be found (PR #2832)
    • changes
      • ...
    • new software
      • add generic JuliaBundle easyblock (PR #2830)
  • easyconfigs
    • ~75 easyconfig PRs were merged since last conf call
    • bug fixes
      • add patches to fix PyTorch 1.10.0 build on POWER (PR #15904)
      • make sure that Python dependency is actually used for VTK 9.0.1 (PR #16741)
      • fix source_urls for colossalai 0.1.8 (no longer available via PyPI, only via GitHub repo) (PR #16693)
    • enhancements
      • install subtree support for recent git versions (PR #16784 + PR #16785)
      • add libmad dependency to SoX v14.4.2 (PR #16758)
      • automatically cancel Github Action workflow runs for outdated commits (PR #16754)
    • (noteworthy) new software
      • see also overview at https://github.com/easybuilders/easybuild-easyconfigs/issues?q=is%3Amerged+label%3Anew
      • Alfred (PR #16780)
      • NVSHMEM (PR #14832)
      • scvi-tools (PR #16457 + PR #16458)
    • noteworthy software updates
      • TensorFlow v2.8.4 (PR #16779)
      • PyTorch v1.12.1 (PR #16453 + PR #16484)
    • changes
      • switch IJulia and MultilevelEstimators to JuliaBundle easyblock (PR #16665)
      • only give read permissions in GitHub Actions workflows (PR #16263)
• work in progress
  • framework
    • reported bugs / bug fixes
      • ...
    • enhancements
      • add easybuild.tools.LooseVersion (since distutils is deprecated in Python 3.10) (PR #3794)
      • add support for using --output-format=md (MarkDown) (PR #4117)
      • add support for --filter-rpath-sanity-libs to skip RPATH sanity check for designated libraries (PR #4119)
      • add sanity_check_load_module method to EasyBlock to provide control over when module is loaded during sanity check step (PR #4125)
      • add checksums.json support for Github integration (PR #4133)
    • changes
      • also run unit tests with Python 3.11 (WIP) (PR #4092)
      • auto-enable use of oneAPI C/C++ compilers for intel-compilers >= 2022.2.0 (PR #4135)
      • deprecate support for running EasyBuild with Python 2 (PR #4136)
  • easyblocks
    • bug reports/fixes
      • load module early during PythonPackage's sanity check step via sanity_check_load_module (PR #2828)
        • to fix --sanity-check-only not working for software installed with PythonPackage (see issue #2745)
    • enhancements
      • set $TEMPDIRPATH for testsuite in the BerkeleyGW easyblock, to avoid polluting /tmp (PR #2836)
      • allow use of test_cmd without runtest for ConfigureMake (PR #2837)
      • enhance CMakeMake easyblock to run ctest command if runtest is True (PR #2838)
    • updates
      • ...
    • new software
      • ...
    • changes
      • ...
  • easyconfigs
    • still over 700 open easyconfig PRs...
    • bug fixes/reports
      • skip remaining PyTorch 1.12.1 test timeout (PR #16793)
    • new software
      • see https://github.com/easybuilders/easybuild-easyconfigs/labels/new
      • ...
    • noteworthy software updates
      • CP2K v9.1 & v2022.1 for foss 2022a (PR #16007)
      • TensorFlow 2.9.1 (PR #16008)
      • Qt5 v5.15.7 with GCCcore/12.2.0 (PR #16810)
    • changes
      • rename hyperopt to Hyperopt (PR #16697)

Update on porting of EasyBuild docs to MarkDown

• See https://github.com/easybuilders/easybuild-docs (preview @ https://easybuilders.github.io/easybuild-docs)
• status
  • ...
• WIP framework PR #4117 to add support for --output-format=md (used for auto-generated documentation)
• API docs are WIP @ https://github.com/easybuilders/easybuild-docs/pull/79

2022b common toolchains

• proposal for foss/2022b
  • see foss/2022.10
  • GCC 12.2.0 + binutils 2.39 (current latest)
  • UCX 1.13.1 + libfabric 1.16.1 + OpenMPI v4.1.4 (current latest) - same as in foss/2022a
  • OpenBLAS 0.3.21 + FlexiBLAS 3.2.1 (current latest)
  • FFTW 3.3.10 (current latest) - same as in foss/2022a
  • ScaLAPACK 2.2.0 (current latest) - same as in foss/2022a
• proposal for intel/2022b:
  • see intel/2022.11
  • upstream docs: https://software.intel.com/content/www/us/en/develop/articles/oneapi-standalone-components.html
  • base: GCC 12.2.0 + binutils 2.39 (current latest)
  • Intel C/C++/Fortran compilers 2022.2.1 (classic + oneAPI) (current latest)
  • Intel MPI 2021.7.1 (current latest)
  • Intel MKL 2022.2.1 (current latest)
  • we should enable the use of oneAPI C/C++ compilers by default for this Intel compilers versions
    • this will require separate toolchain options to enable C/C++ oneAPI compilers and Fortran oneAPI compilers
• key software
  • Perl 5.36.0 (done, see PR #16204)
  • Python 3.10.8 (done, see PR #16632)
  • X11 v20221110 (done, see PR #16664)
  • Qt5 v5.15.7 with GCCcore/12.2.0 (PR #16810)
    • PRs for required dependencies are still missing
    • What do we do with JasPer? Use v4.0.0? Use latest v3.x?
      • need to keep an eye on grib (for WRF), ...
• mpi4py should be split off from SciPy-bundle

Q&A

• Jörg: support for IPUs in EasyBuild
  • we can set up a meeting to discuss how support for IPUs can be implemented in EasyBuild
• software pulling in lots of dependencies automatically
  • like PSI4, FreeFem++
  • how to deal with this...
  • getting control over dependencies is better for multiple reasons (testing, offline installation, ...)
  • it's often also worthwhile to check what other packages are doing: https://salsa.debian.org/science-team/freefempp/-/tree/master/debian