Conference call notes 20170831

(back to Conference calls)

Notes on the 84th EasyBuild conference call, Wednesday August 31st 2017 (5pm - 6pm CEST)

Attendees

Alphabetical list of attendees (5):

• Pablo Escobar (UniBas, Switzerland)
• Kenneth Hoste (HPC-UGent, Belgium)
• Adam Huffman (Francis Crick Institute, UK)
• Alan O'Cais (JSC, Germany)
• Bart Oldeman (McGill University, Canada)

Agenda

• update on EasyBuild v3.4.0
• 2017b toolchains
• Python at GCCcore
• support for eb --trace
• Q&A

Notes

EasyBuild v3.4.0

• will hopefully be released sometime next week
• support for --inject-checksums (merged into develop)
  • http://easybuild.readthedocs.io/en/develop/Writing_easyconfig_files.html#adding-or-replacing-checksums-using-inject-checksums
  • Pablo: enable automatically with --new-pr, but only if sources are there?
• support for --trace --experimental (WIP)
  • https://github.com/easybuilders/easybuild-framework/pull/2285
• clean error on easyconfig parse failure (merged)
• easyblocks: minor updates & bug fixes
• easyconfigs: easyconfigs for 2017b toolchains

2017b toolchains

• foss/2017b: https://github.com/easybuilders/easybuild-easyconfigs/pull/4768
• intel/2017b https://github.com/easybuilders/easybuild-easyconfigs/pull/4618
• PRs for Python (2.7.13, 3.6.2) & Perl (5.26.0) with 2017b coming up, R (3.4.1) in the works (many deps...)

Python at GCCcore

• see discussions at:
  • Python @ GCCcore by Jack: https://github.com/easybuilders/easybuild-easyconfigs/pull/4962
    • a bit controversial, because of hackish forced RPATH linking
    • Python packages installed on top of it had Python as a build dep only...
      • cfr. https://github.com/easybuilders/easybuild-easyconfigs/pull/4917
      • so resulting module doesn't work until the right (full) Python is loaded along with it
      • Alan: Python packages on top of a Python GCCcore could specify prereq Python?
    • reverted in https://github.com/easybuilders/easybuild-easyconfigs/pull/5075, to avoid including this in EasyBuild v3.4.0
  • possible alternative proposed by KH in https://github.com/easybuilders/easybuild-easyconfigs/pull/5072
    • Python@GCCcore renamed to Python-core to allow a full Python shadowing it
      • much like Python module currently shadows system python
      • Alan: what about mixing Python-core 2 & Python 3 (or vice-versa)?
        • not that different from loading Python 2/3 and shadowing system Python...
    • this approach also has some concerns
      • Davide: confusion between Python-core & Python: which should be used when
      • Jack: bare vs extensions included...
• would be nice to get feedback from Damian on a good setup with a Python at GCCcore level
• main intention is to take control over which Python is used, avoid relying on system Python

Support for eb --trace

• https://github.com/easybuilders/easybuild-framework/pull/2285
• plan to merge this in EB v3.4.0 under --experimental to make it more easy to get suggestions/feedback

Other

• Pablo: declare RPATH stable?

  • working fine for 815 installations
    • that's with $LD_LIBRARY_PATH still in place
    • RPATH was disabled for about 5 of these, mostly bioinfo stuff that includes lots of tools
  • https://github.com/easybuilders/easybuild-framework/issues/1992
    • K: feedback from John @ SARA?
  • Bart: OpenFOAM basically always requires $LD_LIBRARY_PATH because of esoteric linking style (unless a lot of effort is put into it)

• Bart: depends_on vs load supported by Lmod

  • cfr. http://lmod.readthedocs.io/en/latest/098_dependent_modules.html#depends-on-a
  • depends_on kind of fixes behaviour of load
  • could be done optionally via configuration setting

• Bart: sys.path magic done in easy_install for Python 3.5 causes problems with virtualenv

  • no longer there in 3.6.1, not there in 2.7.13?
  • K: more related to setuptools version?
  • Bart: enhance Python(Package?) easyblock to avoid use of sys.path magic in easy_install.pth, to avoid that the Python packages installed there get preference over packages installed in virtualenv?
  • related recent change to PythonPackage easyblock: https://github.com/easybuilders/easybuild-easyblocks/pull/711

• Alan: libtinfo split from ncurses @ dummy

  • Ubuntu 16.10 has libtinfo in OS, which should be shadowed in ncurses to avoid problems
  • but, building ncurses using --with-termlib was created problems too
    • see discussion at https://github.com/easybuilders/easybuild-easyconfigs/pull/3545#r132546663 + https://github.com/easybuilders/easybuild-easyblocks/pull/1088
  • fixed by symlinking libtinfo to libncurses https://github.com/easybuilders/easybuild-easyconfigs/pull/5067
  • should also be done for non-dummy ncurses?
    • Alan will open issue to check with Davide/Åke
  • also fixed linkage to ncurses for libreadline
  • via SHLIB_LIBS in buildopts rather than LDFLAGS in preconfigopts
  • cfr. https://github.com/easybuilders/easybuild-easyconfigs/pull/5074