Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

{2023.06,zen4}[foss/2023a] R-bundle-Bioconductor 3.18 #613

Conversation

boegel
Copy link
Contributor

@boegel boegel commented Jun 15, 2024

No description provided.

@boegel boegel added 2023.06-software.eessi.io 2023.06 version of software.eessi.io zen4 labels Jun 15, 2024
Copy link

eessi-bot bot commented Jun 15, 2024

Instance eessi-bot-mc-aws is configured to build for:

  • architectures: x86_64/generic, x86_64/intel/haswell, x86_64/intel/skylake_avx512, x86_64/amd/zen2, x86_64/amd/zen3, aarch64/generic, aarch64/neoverse_n1, aarch64/neoverse_v1
  • repositories: eessi.io-2023.06-compat, eessi-hpc.org-2023.06-software, eessi.io-2023.06-software, eessi-hpc.org-2023.06-compat

Copy link

eessi-bot bot commented Jun 15, 2024

Instance eessi-bot-mc-azure is configured to build for:

  • architectures: x86_64/amd/zen4
  • repositories: eessi.io-2023.06-software, eessi-hpc.org-2023.06-compat, eessi-hpc.org-2023.06-software, eessi.io-2023.06-compat

@boegel
Copy link
Contributor Author

boegel commented Jun 15, 2024

bot: build repo:eessi.io-2023.06-software arch:x86_64/amd/zen4

Copy link

eessi-bot bot commented Jun 15, 2024

Updates by the bot instance eessi-bot-mc-aws (click for details)
  • received bot command build repo:eessi.io-2023.06-software arch:x86_64/amd/zen4 from boegel

    • expanded format: build repository:eessi.io-2023.06-software architecture:x86_64/amd/zen4
  • handling command build repository:eessi.io-2023.06-software architecture:x86_64/amd/zen4 resulted in:

    • no jobs were submitted

Copy link

eessi-bot bot commented Jun 15, 2024

Updates by the bot instance eessi-bot-mc-azure (click for details)
  • received bot command build repo:eessi.io-2023.06-software arch:x86_64/amd/zen4 from boegel

    • expanded format: build repository:eessi.io-2023.06-software architecture:x86_64/amd/zen4
  • handling command build repository:eessi.io-2023.06-software architecture:x86_64/amd/zen4 resulted in:

Copy link

eessi-bot bot commented Jun 15, 2024

New job on instance eessi-bot-mc-azure for architecture x86_64-amd-zen4 for repository eessi.io-2023.06-software in job dir /project/def-users/SHARED/jobs/2024.06/pr_613/114

date job status comment
Jun 15 07:26:30 UTC 2024 submitted job id 114 awaits release by job manager
Jun 15 07:27:10 UTC 2024 released job awaits launch by Slurm scheduler
Jun 15 07:31:13 UTC 2024 running job 114 is running
Jun 15 08:24:24 UTC 2024 finished
😢 FAILURE (click triangle for details)
Details
✅ job output file slurm-114.out
❌ found message matching ERROR:
❌ found message matching FAILED:
❌ found message matching required modules missing:
❌ no message matching No missing installations
✅ found message matching .tar.gz created!
Artefacts
eessi-2023.06-software-linux-x86_64-amd-zen4-1718439743.tar.gzsize: 151 MiB (159198101 bytes)
entries: 27850
modules under 2023.06/software/linux/x86_64/amd/zen4/modules/all
ATK/2.38.0-GCCcore-12.3.0.lua
Arrow/14.0.1-gfbf-2023a.lua
Boost/1.82.0-GCC-12.3.0.lua
FLAC/1.4.2-GCCcore-12.3.0.lua
GEOS/3.12.0-GCC-12.3.0.lua
GLPK/5.0-GCCcore-12.3.0.lua
GMP/6.2.1-GCCcore-12.3.0.lua
Gdk-Pixbuf/2.42.10-GCCcore-12.3.0.lua
JasPer/4.0.0-GCCcore-12.3.0.lua
LAME/3.100-GCCcore-12.3.0.lua
LittleCMS/2.15-GCCcore-12.3.0.lua
MPFR/4.2.0-GCCcore-12.3.0.lua
NLopt/2.7.1-GCCcore-12.3.0.lua
Pango/1.50.14-GCCcore-12.3.0.lua
PostgreSQL/16.1-GCCcore-12.3.0.lua
RapidJSON/1.1.0-20230928-GCCcore-12.3.0.lua
UDUNITS/2.2.28-GCCcore-12.3.0.lua
Xvfb/21.1.8-GCCcore-12.3.0.lua
libogg/1.3.5-GCCcore-12.3.0.lua
libopus/1.4-GCCcore-12.3.0.lua
libsndfile/1.2.2-GCCcore-12.3.0.lua
libvorbis/1.3.7-GCCcore-12.3.0.lua
netCDF/4.9.2-gompi-2023a.lua
nettle/3.9.1-GCCcore-12.3.0.lua
nodejs/18.17.1-GCCcore-12.3.0.lua
utf8proc/2.8.0-GCCcore-12.3.0.lua
software under 2023.06/software/linux/x86_64/amd/zen4/software
ATK/2.38.0-GCCcore-12.3.0
Arrow/14.0.1-gfbf-2023a
Boost/1.82.0-GCC-12.3.0
FLAC/1.4.2-GCCcore-12.3.0
GEOS/3.12.0-GCC-12.3.0
GLPK/5.0-GCCcore-12.3.0
GMP/6.2.1-GCCcore-12.3.0
Gdk-Pixbuf/2.42.10-GCCcore-12.3.0
JasPer/4.0.0-GCCcore-12.3.0
LAME/3.100-GCCcore-12.3.0
LittleCMS/2.15-GCCcore-12.3.0
MPFR/4.2.0-GCCcore-12.3.0
NLopt/2.7.1-GCCcore-12.3.0
Pango/1.50.14-GCCcore-12.3.0
PostgreSQL/16.1-GCCcore-12.3.0
RapidJSON/1.1.0-20230928-GCCcore-12.3.0
UDUNITS/2.2.28-GCCcore-12.3.0
Xvfb/21.1.8-GCCcore-12.3.0
libogg/1.3.5-GCCcore-12.3.0
libopus/1.4-GCCcore-12.3.0
libsndfile/1.2.2-GCCcore-12.3.0
libvorbis/1.3.7-GCCcore-12.3.0
netCDF/4.9.2-gompi-2023a
nettle/3.9.1-GCCcore-12.3.0
nodejs/18.17.1-GCCcore-12.3.0
utf8proc/2.8.0-GCCcore-12.3.0
other under 2023.06/software/linux/x86_64/amd/zen4
no other files in tarball
Jun 15 08:24:24 UTC 2024 test result
😢 FAILURE (click triangle for details)
Reason
EESSI test suite was not run, test step itself failed to execute.
Details
✅ job output file slurm-114.out
❌ found message matching ERROR:
✅ no message matching [\s*FAILED\s*].*Ran .* test case

@bedroge bedroge changed the title {2023.06,zen4}[foss/2023b] R-bundle-Bioconductor 3.18 {2023.06,zen4}[foss/2023a] R-bundle-Bioconductor 3.18 Jun 15, 2024
@boegel
Copy link
Contributor Author

boegel commented Jun 15, 2024

failed due to corrupt download, probably another job (like #614) was downloading that exact same file at the same time...

Checksum verification for /project/def-users/bot/shared/easybuild/sources/a/at-spi2-core/at-spi2-core-2.49.91.tar.xz using aa72bbb12188ee3d0152cc6ea935415e6dc623ffa751b6a7cc23e9025f0410fd failed

Tarball was indeed corrupt, trying to manually unpack it resulted in

xz: (stdin): Unexpected end of input

@boegel
Copy link
Contributor Author

boegel commented Jun 15, 2024

bot: build repo:eessi.io-2023.06-software arch:x86_64/amd/zen4

Copy link

eessi-bot bot commented Jun 15, 2024

Updates by the bot instance eessi-bot-mc-aws (click for details)
  • received bot command build repo:eessi.io-2023.06-software arch:x86_64/amd/zen4 from boegel

    • expanded format: build repository:eessi.io-2023.06-software architecture:x86_64/amd/zen4
  • handling command build repository:eessi.io-2023.06-software architecture:x86_64/amd/zen4 resulted in:

    • no jobs were submitted

Copy link

eessi-bot bot commented Jun 15, 2024

Updates by the bot instance eessi-bot-mc-azure (click for details)
  • received bot command build repo:eessi.io-2023.06-software arch:x86_64/amd/zen4 from boegel

    • expanded format: build repository:eessi.io-2023.06-software architecture:x86_64/amd/zen4
  • handling command build repository:eessi.io-2023.06-software architecture:x86_64/amd/zen4 resulted in:

Copy link

eessi-bot bot commented Jun 15, 2024

New job on instance eessi-bot-mc-azure for architecture x86_64-amd-zen4 for repository eessi.io-2023.06-software in job dir /project/def-users/SHARED/jobs/2024.06/pr_613/116

date job status comment
Jun 15 08:29:59 UTC 2024 submitted job id 116 awaits release by job manager
Jun 15 08:30:34 UTC 2024 released job awaits launch by Slurm scheduler
Jun 15 09:29:49 UTC 2024 running job 116 is running
Jun 15 10:23:00 UTC 2024 finished
😢 FAILURE (click triangle for details)
Details
✅ job output file slurm-116.out
❌ found message matching ERROR:
❌ found message matching FAILED:
❌ found message matching required modules missing:
❌ no message matching No missing installations
✅ found message matching .tar.gz created!
Artefacts
eessi-2023.06-software-linux-x86_64-amd-zen4-1718446887.tar.gzsize: 156 MiB (164221579 bytes)
entries: 28580
modules under 2023.06/software/linux/x86_64/amd/zen4/modules/all
ATK/2.38.0-GCCcore-12.3.0.lua
Arrow/14.0.1-gfbf-2023a.lua
Boost/1.82.0-GCC-12.3.0.lua
FLAC/1.4.2-GCCcore-12.3.0.lua
GEOS/3.12.0-GCC-12.3.0.lua
GLPK/5.0-GCCcore-12.3.0.lua
GMP/6.2.1-GCCcore-12.3.0.lua
Gdk-Pixbuf/2.42.10-GCCcore-12.3.0.lua
JasPer/4.0.0-GCCcore-12.3.0.lua
LAME/3.100-GCCcore-12.3.0.lua
LittleCMS/2.15-GCCcore-12.3.0.lua
MPFR/4.2.0-GCCcore-12.3.0.lua
NLopt/2.7.1-GCCcore-12.3.0.lua
Pango/1.50.14-GCCcore-12.3.0.lua
PostgreSQL/16.1-GCCcore-12.3.0.lua
RapidJSON/1.1.0-20230928-GCCcore-12.3.0.lua
UDUNITS/2.2.28-GCCcore-12.3.0.lua
Wayland/1.22.0-GCCcore-12.3.0.lua
Xvfb/21.1.8-GCCcore-12.3.0.lua
at-spi2-atk/2.38.0-GCCcore-12.3.0.lua
at-spi2-core/2.49.91-GCCcore-12.3.0.lua
googletest/1.13.0-GCCcore-12.3.0.lua
libepoxy/1.5.10-GCCcore-12.3.0.lua
libogg/1.3.5-GCCcore-12.3.0.lua
libopus/1.4-GCCcore-12.3.0.lua
libsndfile/1.2.2-GCCcore-12.3.0.lua
libvorbis/1.3.7-GCCcore-12.3.0.lua
netCDF/4.9.2-gompi-2023a.lua
nettle/3.9.1-GCCcore-12.3.0.lua
nodejs/18.17.1-GCCcore-12.3.0.lua
utf8proc/2.8.0-GCCcore-12.3.0.lua
software under 2023.06/software/linux/x86_64/amd/zen4/software
ATK/2.38.0-GCCcore-12.3.0
Arrow/14.0.1-gfbf-2023a
Boost/1.82.0-GCC-12.3.0
FLAC/1.4.2-GCCcore-12.3.0
GEOS/3.12.0-GCC-12.3.0
GLPK/5.0-GCCcore-12.3.0
GMP/6.2.1-GCCcore-12.3.0
Gdk-Pixbuf/2.42.10-GCCcore-12.3.0
JasPer/4.0.0-GCCcore-12.3.0
LAME/3.100-GCCcore-12.3.0
LittleCMS/2.15-GCCcore-12.3.0
MPFR/4.2.0-GCCcore-12.3.0
NLopt/2.7.1-GCCcore-12.3.0
Pango/1.50.14-GCCcore-12.3.0
PostgreSQL/16.1-GCCcore-12.3.0
RapidJSON/1.1.0-20230928-GCCcore-12.3.0
UDUNITS/2.2.28-GCCcore-12.3.0
Wayland/1.22.0-GCCcore-12.3.0
Xvfb/21.1.8-GCCcore-12.3.0
at-spi2-atk/2.38.0-GCCcore-12.3.0
at-spi2-core/2.49.91-GCCcore-12.3.0
googletest/1.13.0-GCCcore-12.3.0
libepoxy/1.5.10-GCCcore-12.3.0
libogg/1.3.5-GCCcore-12.3.0
libopus/1.4-GCCcore-12.3.0
libsndfile/1.2.2-GCCcore-12.3.0
libvorbis/1.3.7-GCCcore-12.3.0
netCDF/4.9.2-gompi-2023a
nettle/3.9.1-GCCcore-12.3.0
nodejs/18.17.1-GCCcore-12.3.0
utf8proc/2.8.0-GCCcore-12.3.0
other under 2023.06/software/linux/x86_64/amd/zen4
no other files in tarball
Jun 15 10:23:00 UTC 2024 test result
😢 FAILURE (click triangle for details)
Reason
EESSI test suite was not run, test step itself failed to execute.
Details
✅ job output file slurm-116.out
❌ found message matching ERROR:
✅ no message matching [\s*FAILED\s*].*Ran .* test case

@boegel
Copy link
Contributor Author

boegel commented Jun 15, 2024

Failed again due to another checksum failure:

Checksum verification for /project/def-users/bot/shared/easybuild/sources/g/GTK3/adwaita-icon-theme-44.0.tar.xz using 4889c5601bbfecd25d80ba342209d0a936dcf691ee56bd6eca4cde361f1a664c failed

I've removed the corrupt adwaita-icon-theme-44.0.tar.xz, but we should try and figure out why this happens.

I also don't understand how job 116 could start while 115 (for #614) was still running, since there's only 1 zen4 node available currently...

Any ideas here @trz42 ?

@boegel
Copy link
Contributor Author

boegel commented Jun 15, 2024

bot: build repo:eessi.io-2023.06-software arch:x86_64/amd/zen4

Copy link

eessi-bot bot commented Jun 15, 2024

Updates by the bot instance eessi-bot-mc-azure (click for details)
  • received bot command build repo:eessi.io-2023.06-software arch:x86_64/amd/zen4 from boegel

    • expanded format: build repository:eessi.io-2023.06-software architecture:x86_64/amd/zen4
  • handling command build repository:eessi.io-2023.06-software architecture:x86_64/amd/zen4 resulted in:

Copy link

eessi-bot bot commented Jun 15, 2024

New job on instance eessi-bot-mc-azure for architecture x86_64-amd-zen4 for repository eessi.io-2023.06-software in job dir /project/def-users/SHARED/jobs/2024.06/pr_613/117

date job status comment
Jun 15 12:01:04 UTC 2024 submitted job id 117 awaits release by job manager
Jun 15 12:01:12 UTC 2024 released job awaits launch by Slurm scheduler
Jun 15 12:05:15 UTC 2024 running job 117 is running
Jun 15 13:27:01 UTC 2024 finished
😢 FAILURE (click triangle for details)
Details
✅ job output file slurm-117.out
❌ found message matching ERROR:
❌ found message matching FAILED:
❌ found message matching required modules missing:
❌ no message matching No missing installations
✅ found message matching .tar.gz created!
Artefacts
eessi-2023.06-software-linux-x86_64-amd-zen4-1718457852.tar.gzsize: 272 MiB (285878394 bytes)
entries: 35638
modules under 2023.06/software/linux/x86_64/amd/zen4/modules/all
ATK/2.38.0-GCCcore-12.3.0.lua
Armadillo/12.6.2-foss-2023a.lua
Arrow/14.0.1-gfbf-2023a.lua
Boost/1.82.0-GCC-12.3.0.lua
CFITSIO/4.3.0-GCCcore-12.3.0.lua
FLAC/1.4.2-GCCcore-12.3.0.lua
GEOS/3.12.0-GCC-12.3.0.lua
GLPK/5.0-GCCcore-12.3.0.lua
GMP/6.2.1-GCCcore-12.3.0.lua
GTK3/3.24.37-GCCcore-12.3.0.lua
Gdk-Pixbuf/2.42.10-GCCcore-12.3.0.lua
Ghostscript/10.01.2-GCCcore-12.3.0.lua
HDF/4.2.16-2-GCCcore-12.3.0.lua
ImageMagick/7.1.1-15-GCCcore-12.3.0.lua
Imath/3.1.7-GCCcore-12.3.0.lua
JasPer/4.0.0-GCCcore-12.3.0.lua
LAME/3.100-GCCcore-12.3.0.lua
LittleCMS/2.15-GCCcore-12.3.0.lua
MPFR/4.2.0-GCCcore-12.3.0.lua
NLopt/2.7.1-GCCcore-12.3.0.lua
OpenEXR/3.1.7-GCCcore-12.3.0.lua
PROJ/9.2.0-GCCcore-12.3.0.lua
Pango/1.50.14-GCCcore-12.3.0.lua
PostgreSQL/16.1-GCCcore-12.3.0.lua
Qhull/2020.2-GCCcore-12.3.0.lua
RapidJSON/1.1.0-20230928-GCCcore-12.3.0.lua
UDUNITS/2.2.28-GCCcore-12.3.0.lua
Wayland/1.22.0-GCCcore-12.3.0.lua
Xerces-C++/3.2.4-GCCcore-12.3.0.lua
Xvfb/21.1.8-GCCcore-12.3.0.lua
arpack-ng/3.9.0-foss-2023a.lua
at-spi2-atk/2.38.0-GCCcore-12.3.0.lua
at-spi2-core/2.49.91-GCCcore-12.3.0.lua
googletest/1.13.0-GCCcore-12.3.0.lua
json-c/0.16-GCCcore-12.3.0.lua
libepoxy/1.5.10-GCCcore-12.3.0.lua
libgeotiff/1.7.1-GCCcore-12.3.0.lua
libogg/1.3.5-GCCcore-12.3.0.lua
libopus/1.4-GCCcore-12.3.0.lua
libsndfile/1.2.2-GCCcore-12.3.0.lua
libtirpc/1.3.3-GCCcore-12.3.0.lua
libvorbis/1.3.7-GCCcore-12.3.0.lua
netCDF/4.9.2-gompi-2023a.lua
nettle/3.9.1-GCCcore-12.3.0.lua
nlohmann_json/3.11.2-GCCcore-12.3.0.lua
nodejs/18.17.1-GCCcore-12.3.0.lua
utf8proc/2.8.0-GCCcore-12.3.0.lua
software under 2023.06/software/linux/x86_64/amd/zen4/software
ATK/2.38.0-GCCcore-12.3.0
Armadillo/12.6.2-foss-2023a
Arrow/14.0.1-gfbf-2023a
Boost/1.82.0-GCC-12.3.0
CFITSIO/4.3.0-GCCcore-12.3.0
FLAC/1.4.2-GCCcore-12.3.0
GEOS/3.12.0-GCC-12.3.0
GLPK/5.0-GCCcore-12.3.0
GMP/6.2.1-GCCcore-12.3.0
GTK3/3.24.37-GCCcore-12.3.0
Gdk-Pixbuf/2.42.10-GCCcore-12.3.0
Ghostscript/10.01.2-GCCcore-12.3.0
HDF/4.2.16-2-GCCcore-12.3.0
ImageMagick/7.1.1-15-GCCcore-12.3.0
Imath/3.1.7-GCCcore-12.3.0
JasPer/4.0.0-GCCcore-12.3.0
LAME/3.100-GCCcore-12.3.0
LittleCMS/2.15-GCCcore-12.3.0
MPFR/4.2.0-GCCcore-12.3.0
NLopt/2.7.1-GCCcore-12.3.0
OpenEXR/3.1.7-GCCcore-12.3.0
PROJ/9.2.0-GCCcore-12.3.0
Pango/1.50.14-GCCcore-12.3.0
PostgreSQL/16.1-GCCcore-12.3.0
Qhull/2020.2-GCCcore-12.3.0
RapidJSON/1.1.0-20230928-GCCcore-12.3.0
UDUNITS/2.2.28-GCCcore-12.3.0
Wayland/1.22.0-GCCcore-12.3.0
Xerces-C++/3.2.4-GCCcore-12.3.0
Xvfb/21.1.8-GCCcore-12.3.0
arpack-ng/3.9.0-foss-2023a
at-spi2-atk/2.38.0-GCCcore-12.3.0
at-spi2-core/2.49.91-GCCcore-12.3.0
googletest/1.13.0-GCCcore-12.3.0
json-c/0.16-GCCcore-12.3.0
libepoxy/1.5.10-GCCcore-12.3.0
libgeotiff/1.7.1-GCCcore-12.3.0
libogg/1.3.5-GCCcore-12.3.0
libopus/1.4-GCCcore-12.3.0
libsndfile/1.2.2-GCCcore-12.3.0
libtirpc/1.3.3-GCCcore-12.3.0
libvorbis/1.3.7-GCCcore-12.3.0
netCDF/4.9.2-gompi-2023a
nettle/3.9.1-GCCcore-12.3.0
nlohmann_json/3.11.2-GCCcore-12.3.0
nodejs/18.17.1-GCCcore-12.3.0
utf8proc/2.8.0-GCCcore-12.3.0
other under 2023.06/software/linux/x86_64/amd/zen4
no other files in tarball
Jun 15 13:27:01 UTC 2024 test result
😢 FAILURE (click triangle for details)
Reason
EESSI test suite was not run, test step itself failed to execute.
Details
✅ job output file slurm-117.out
❌ found message matching ERROR:
✅ no message matching [\s*FAILED\s*].*Ran .* test case

@boegel
Copy link
Contributor Author

boegel commented Jun 15, 2024

make check in test step fails for Highway-1.0.4-GCCcore-12.3.0.eb:

The following tests FAILED:
        537 - HwyConvertTestGroup/HwyConvertTest.TestAllTruncate/AVX3_ZEN4  # GetParam() = 64 (Subprocess aborted)
 537/1187 Test  #537: HwyConvertTestGroup/HwyConvertTest.TestAllTruncate/AVX3_ZEN4  # GetParam() = 64 ........................................Subprocess aborted***Exception:   0.35 sec
Running main() from /tmp/bot/easybuild/build/googletest/1.13.0/GCCcore-12.3.0/googletest-1.13.0/googletest/src/gtest_main.cc
Note: Google Test filter = HwyConvertTestGroup/HwyConvertTest.TestAllTruncate/AVX3_ZEN4
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from HwyConvertTestGroup/HwyConvertTest
[ RUN      ] HwyConvertTestGroup/HwyConvertTest.TestAllTruncate/AVX3_ZEN4


u8x16 expect [0+ ->]:
  0x00,0x01,0x02,0x03,0x04,0x05,0x06,
u8x16 actual [0+ ->]:
  0x00,0x1A,0x02,0x1A,0x04,0x1A,0x06,
Abort at /tmp/bot/easybuild/build/Highway/1.0.4/GCCcore-12.3.0/highway-1.0.4/hwy/tests/convert_test.cc:423: AVX3_ZEN4, u8x16 lane 1 mismatch: expected '0x01', got '0x1A'.

@bedroge You have some experience here?

@bedroge
Copy link
Collaborator

bedroge commented Jun 15, 2024

make check in test step fails for Highway-1.0.4-GCCcore-12.3.0.eb:

The following tests FAILED:
        537 - HwyConvertTestGroup/HwyConvertTest.TestAllTruncate/AVX3_ZEN4  # GetParam() = 64 (Subprocess aborted)
 537/1187 Test  #537: HwyConvertTestGroup/HwyConvertTest.TestAllTruncate/AVX3_ZEN4  # GetParam() = 64 ........................................Subprocess aborted***Exception:   0.35 sec
Running main() from /tmp/bot/easybuild/build/googletest/1.13.0/GCCcore-12.3.0/googletest-1.13.0/googletest/src/gtest_main.cc
Note: Google Test filter = HwyConvertTestGroup/HwyConvertTest.TestAllTruncate/AVX3_ZEN4
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from HwyConvertTestGroup/HwyConvertTest
[ RUN      ] HwyConvertTestGroup/HwyConvertTest.TestAllTruncate/AVX3_ZEN4


u8x16 expect [0+ ->]:
  0x00,0x01,0x02,0x03,0x04,0x05,0x06,
u8x16 actual [0+ ->]:
  0x00,0x1A,0x02,0x1A,0x04,0x1A,0x06,
Abort at /tmp/bot/easybuild/build/Highway/1.0.4/GCCcore-12.3.0/highway-1.0.4/hwy/tests/convert_test.cc:423: AVX3_ZEN4, u8x16 lane 1 mismatch: expected '0x01', got '0x1A'.

@bedroge You have some experience here?

Someone else reported the same issue, but it looks like there's no solution (other than trying a newer version):
google/highway#1913

@boegel
Copy link
Contributor Author

boegel commented Jun 18, 2024

@bedroge I guess we could try and pinpoint where the fix was made in between Highway 1.0.4 and 1.0.7, so we can come up with a patch, but I took a quick stab at that, and doesn't seem easy...

@bedroge
Copy link
Collaborator

bedroge commented Jun 18, 2024

@bedroge I guess we could try and pinpoint where the fix was made in between Highway 1.0.4 and 1.0.7, so we can come up with a patch, but I took a quick stab at that, and doesn't seem easy...

I tried something similar for the issue I ran into, but also gave up at some point. In some versions quite a lot of code was changed, and backporting stuff was really not feasible...

@boegel
Copy link
Contributor Author

boegel commented Jun 18, 2024

@bedroge So how do you think we should proceed? Ignore the test failure? Disable that particular test?
Do we have any idea whether it's a bug in the test or in the Highway code itself?

@bedroge
Copy link
Collaborator

bedroge commented Jun 18, 2024

@boegel Perhaps we can try changing the optimization flags?

@trz42
Copy link
Collaborator

trz42 commented Jun 25, 2024

Analysed this a bit...

The failing test TestAllTruncate in hwy/tests/convert_test.cc seems to have been changed as follows from version 1.0.4 to 1.0.7:

@@ -432,15 +435,61 @@ struct TestTruncateTo {
 };

 HWY_NOINLINE void TestAllTruncate() {
-  ForUnsignedTypes(ForPartialVectors<TestTruncateTo>());
+  ForU163264(ForDemoteVectors<TestTruncateTo>());
 }

The change from ForUnsignedTypes to ForU163264 seems minimal (both defined in tests/test_util-inl.h)

ForUnsignedTypes is defined as follows (in 1.0.4, no change in 1.0.7):

template <class Func>
void ForUnsignedTypes(const Func& func) {
  func(uint8_t());
  func(uint16_t());
  func(uint32_t());
#if HWY_HAVE_INTEGER64
  func(uint64_t());
#endif
}

ForU153264 is defined as follows (not defined in 1.0.4, first introduced in 1.0.5):

template <class Func>
void ForU163264(const Func& func) {
  func(uint16_t());
  func(uint32_t());
#if HWY_HAVE_INTEGER64
  func(uint64_t());
#endif
}

ForPartialVectors hasn't changed from version 1.0.4 to 1.0.7. However, ForPartialVectors and ForDemoteVectors are quite a bit different, particularly, between version 1.0.4 and 1.0.7.

  • ForPartialVectors in 1.0.4:

    // Calls Test for all power of two N in [1, Lanes(d)]. This is the default
    // for ops that do not narrow nor widen their input, nor require 128 bits.
    template <class Test>
    class ForPartialVectors {
      mutable bool called_ = false;
    
     public:
      ~ForPartialVectors() {
        if (!called_) {
          HWY_ABORT("Test is incorrect, ensure operator() is called");
        }
      }
    
      template <typename T>
      void operator()(T t) const {
        called_ = true;
    #if HWY_TARGET == HWY_SCALAR
        (void)t;
        detail::ForeachCappedR<T, 1, 1, Test>::Do(1, 1);
    #else
        ForExtendableVectors<Test, 0>()(t);
    #endif
      }
    };
  • ForDemoteVectors in version 1.0.4:

    // Calls Test for all N than can be demoted (not the same as Shrinkable because
    // HWY_SCALAR has one lane).
    template <class Test, int kPow2 = 1>
    class ForDemoteVectors {
      mutable bool called_ = false;
    
     public:
      ~ForDemoteVectors() {
        if (!called_) {
          HWY_ABORT("Test is incorrect, ensure operator() is called");
        }
      }
    
      template <typename T>
      void operator()(T /*unused*/) const {
        called_ = true;
        constexpr size_t kMinLanes = size_t{1} << kPow2;
        constexpr size_t kMaxCapped = HWY_LANES(T);
        // For shrinking, an upper limit is unnecessary.
        constexpr size_t max_lanes = kMaxCapped;
    
        (void)kMinLanes;
        (void)max_lanes;
        (void)max_lanes;
    #if HWY_TARGET == HWY_SCALAR
        detail::ForeachCappedR<T, 1, 1, Test>::Do(1, 1);
    #else
        detail::ForeachCappedR<T, (kMaxCapped >> kPow2), kMinLanes, Test>::Do(
            kMinLanes, max_lanes);
    
    // TODO(janwas): call Extendable if kMinLanes check not required?
    #if HWY_TARGET == HWY_RVV
        // For each [MinPow2 + kPow2, 3]; counter is [MinPow2 + kPow2, 3].
        detail::ForeachShiftR<T, detail::MinPow2<T>() + kPow2, 0, Test>::Do(
            kMinLanes);
    #elif HWY_HAVE_SCALABLE
        // For each [MinPow2 + kPow2, 0]; counter is [MinPow2 + kPow2 + 3, 3].
        detail::ForeachShiftR<T, detail::MinPow2<T>() + kPow2 + 3, -3, Test>::Do(
            kMinLanes);
    #endif
    #endif  // HWY_TARGET == HWY_SCALAR
      }
    };
  • ForDemoteVectors in version 1.0.7:

    // Calls Test for all N than can be demoted (not the same as Shrinkable because
    // HWY_SCALAR has one lane and as a one-lane vector with a lane size of at least
    // 2 bytes can always be demoted to a vector with a smaller lane type).
    template <class Test, int kPow2 = 1>
    class ForDemoteVectors {
      mutable bool called_ = false;
    
     public:
      ~ForDemoteVectors() {
        if (!called_) {
          HWY_ABORT("Test is incorrect, ensure operator() is called");
        }
      }
    
      template <typename T>
      void operator()(T /*unused*/) const {
        called_ = true;
    
    #if HWY_HAVE_SCALABLE
        // kMinTVecPow2 is the smallest Pow2 for a vector with lane type T that is
        // supported by detail::ForeachPow2Trim
        constexpr int kMinTVecPow2 = detail::MinPow2<T>();
    
        // detail::MinPow2<T>() + kMinPow2Adj is the smallest Pow2 for a vector with
        // lane type T that can be demoted to a vector with a lane size of
        // (sizeof(T) >> kPow2)
        constexpr int kMinPow2Adj = HWY_MAX(-3 - kMinTVecPow2 + kPow2, 0);
    
        detail::ForeachPow2Trim<T, kMinPow2Adj, 0, Test>::Do(1);
    
        // On targets with scalable vectors, detail::ForeachCappedR below only
        // needs to be executed for vectors that have less than
        // Lanes(ScalableTag<T>()) as full vectors were already checked by the
        // detail::ForeachPow2Trim above.
        constexpr size_t kMaxCapped = HWY_LANES(T) >> 1;
        const size_t max_lanes = Lanes(ScalableTag<T>()) >> 1;
    #else
        // On targets where HWY_HAVE_SCALABLE is 0, any vector with HWY_LANES(T)
        // or fewer lanes can always be demoted to a vector with a smaller lane
        // type.
        constexpr size_t kMaxCapped = HWY_LANES(T);
        const size_t max_lanes = kMaxCapped;
    #endif
    
        detail::ForeachCappedR<T, kMaxCapped, 1, Test>::Do(1, max_lanes);
      }
    };
  • diff for ForDemoteVectors

     // Calls Test for all N than can be demoted (not the same as Shrinkable because
    -// HWY_SCALAR has one lane).
    +// HWY_SCALAR has one lane and as a one-lane vector with a lane size of at least
    +// 2 bytes can always be demoted to a vector with a smaller lane type).
     template <class Test, int kPow2 = 1>
     class ForDemoteVectors {
       mutable bool called_ = false;
    @@ -467,31 +479,34 @@ class ForDemoteVectors {
       template <typename T>
       void operator()(T /*unused*/) const {
         called_ = true;
    -    constexpr size_t kMinLanes = size_t{1} << kPow2;
    -    constexpr size_t kMaxCapped = HWY_LANES(T);
    -    // For shrinking, an upper limit is unnecessary.
    -    constexpr size_t max_lanes = kMaxCapped;
    
    -    (void)kMinLanes;
    -    (void)max_lanes;
    -    (void)max_lanes;
    -#if HWY_TARGET == HWY_SCALAR
    -    detail::ForeachCappedR<T, 1, 1, Test>::Do(1, 1);
    +#if HWY_HAVE_SCALABLE
    +    // kMinTVecPow2 is the smallest Pow2 for a vector with lane type T that is
    +    // supported by detail::ForeachPow2Trim
    +    constexpr int kMinTVecPow2 = detail::MinPow2<T>();
    +
    +    // detail::MinPow2<T>() + kMinPow2Adj is the smallest Pow2 for a vector with
    +    // lane type T that can be demoted to a vector with a lane size of
    +    // (sizeof(T) >> kPow2)
    +    constexpr int kMinPow2Adj = HWY_MAX(-3 - kMinTVecPow2 + kPow2, 0);
    +
    +    detail::ForeachPow2Trim<T, kMinPow2Adj, 0, Test>::Do(1);
    +
    +    // On targets with scalable vectors, detail::ForeachCappedR below only
    +    // needs to be executed for vectors that have less than
    +    // Lanes(ScalableTag<T>()) as full vectors were already checked by the
    +    // detail::ForeachPow2Trim above.
    +    constexpr size_t kMaxCapped = HWY_LANES(T) >> 1;
    +    const size_t max_lanes = Lanes(ScalableTag<T>()) >> 1;
     #else
    -    detail::ForeachCappedR<T, (kMaxCapped >> kPow2), kMinLanes, Test>::Do(
    -        kMinLanes, max_lanes);
    -
    -// TODO(janwas): call Extendable if kMinLanes check not required?
    -#if HWY_TARGET == HWY_RVV
    -    // For each [MinPow2 + kPow2, 3]; counter is [MinPow2 + kPow2, 3].
    -    detail::ForeachShiftR<T, detail::MinPow2<T>() + kPow2, 0, Test>::Do(
    -        kMinLanes);
    -#elif HWY_HAVE_SCALABLE
    -    // For each [MinPow2 + kPow2, 0]; counter is [MinPow2 + kPow2 + 3, 3].
    -    detail::ForeachShiftR<T, detail::MinPow2<T>() + kPow2 + 3, -3, Test>::Do(
    -        kMinLanes);
    +    // On targets where HWY_HAVE_SCALABLE is 0, any vector with HWY_LANES(T)
    +    // or fewer lanes can always be demoted to a vector with a smaller lane
    +    // type.
    +    constexpr size_t kMaxCapped = HWY_LANES(T);
    +    const size_t max_lanes = kMaxCapped;
     #endif
    -#endif  // HWY_TARGET == HWY_SCALAR
    +
    +    detail::ForeachCappedR<T, kMaxCapped, 1, Test>::Do(1, max_lanes);
       }
     };
  • The changes to ForDemoteVectors may be ok or not ... in the sense that they just fix a non-working test.

    • A first attempt to add ForU163264 and used that instead of ForUnsignedTypes as well as replacing ForPartialVectors with the original ForDemoteVectors (version 1.0.4) hit a out of memory error when recompiling Highway.
    • We could repeat that...
  • Generally, possible ways forward:

    • Disabling the failing test "hoping" that only the test was wrong (but then shouldn't it have failed for other architectures too?)
    • Building version 1.0.7 to verify if this builds and tests.
    • If version 1.0.7 builds, have a second look at the modifications (what was changed in the actual code that is being tested).
    • Alternatively (if version 1.0.7 builds and tests), one could build 1.0.7 but use the original test from 1.0.4.
    • Check if versions 1.0.5 and 1.0.6 fail with the same issue. For example, if 1.0.6 fails with the same issue, but 1.0.7 succeeds the changes to consider between 1.0.6 and 1.0.7 may be smaller.

@boegel
Copy link
Contributor Author

boegel commented Jun 26, 2024

@trz42 Isn't the change from ForUnsignedTypes to ForU163264 for TestAllTruncate a sign that they simply stopped running that test for uint8_t?
If so, that seems like an acceptable patch to apply for Highway 1.0.4 (maybe even without changing ForPartialVectors to ForDemoteVectors)?

@trz42
Copy link
Collaborator

trz42 commented Jun 29, 2024

easybuilders/easybuild-easyconfigs#20942 should fix the issue, however, the patch has been only tested for zen4 (it may not apply to other architectures we have been building for so far).

… 2023.06-software.eessi.io_zen4_R-bundle-Bioconductor-3.18-foss-2023a-R-4.3.2
@trz42
Copy link
Collaborator

trz42 commented Aug 5, 2024

Rebuilding after Highway got fixed (and ingested)...

bot: build repo:eessi.io-2023.06-software arch:x86_64/amd/zen4

Copy link

eessi-bot bot commented Aug 5, 2024

Updates by the bot instance eessi-bot-mc-aws (click for details)
  • received bot command build repo:eessi.io-2023.06-software arch:x86_64/amd/zen4 from trz42

    • expanded format: build repository:eessi.io-2023.06-software architecture:x86_64/amd/zen4
  • handling command build repository:eessi.io-2023.06-software architecture:x86_64/amd/zen4 resulted in:

    • no jobs were submitted

Copy link

eessi-bot bot commented Aug 5, 2024

Updates by the bot instance eessi-bot-mc-azure (click for details)
  • received bot command build repo:eessi.io-2023.06-software arch:x86_64/amd/zen4 from trz42

    • expanded format: build repository:eessi.io-2023.06-software architecture:x86_64/amd/zen4
  • handling command build repository:eessi.io-2023.06-software architecture:x86_64/amd/zen4 resulted in:

Copy link

eessi-bot bot commented Aug 5, 2024

New job on instance eessi-bot-mc-azure for architecture x86_64-amd-zen4 for repository eessi.io-2023.06-software in job dir /project/def-users/SHARED/jobs/2024.08/pr_613/177

date job status comment
Aug 05 08:03:00 UTC 2024 submitted job id 177 awaits release by job manager
Aug 05 08:03:41 UTC 2024 released job awaits launch by Slurm scheduler
Aug 05 08:09:44 UTC 2024 running job 177 is running
Aug 05 18:04:46 UTC 2024 finished
😁 SUCCESS (click triangle for details)
Details
✅ job output file slurm-177.out
✅ no message matching ERROR:
✅ no message matching FAILED:
✅ no message matching required modules missing:
✅ found message(s) matching No missing installations
✅ found message matching .tar.gz created!
Artefacts
eessi-2023.06-software-linux-x86_64-amd-zen4-1722880243.tar.gzsize: 8951 MiB (9386568651 bytes)
entries: 116427
modules under 2023.06/software/linux/x86_64/amd/zen4/modules/all
ATK/2.38.0-GCCcore-12.3.0.lua
Arrow/14.0.1-gfbf-2023a.lua
GLPK/5.0-GCCcore-12.3.0.lua
GTK3/3.24.37-GCCcore-12.3.0.lua
Gdk-Pixbuf/2.42.10-GCCcore-12.3.0.lua
Ghostscript/10.01.2-GCCcore-12.3.0.lua
ImageMagick/7.1.1-15-GCCcore-12.3.0.lua
NLopt/2.7.1-GCCcore-12.3.0.lua
PostgreSQL/16.1-GCCcore-12.3.0.lua
R-bundle-Bioconductor/3.18-foss-2023a-R-4.3.2.lua
R-bundle-CRAN/2023.12-foss-2023a.lua
RapidJSON/1.1.0-20230928-GCCcore-12.3.0.lua
Wayland/1.22.0-GCCcore-12.3.0.lua
Xvfb/21.1.8-GCCcore-12.3.0.lua
arrow-R/14.0.1-foss-2023a-R-4.3.2.lua
at-spi2-atk/2.38.0-GCCcore-12.3.0.lua
at-spi2-core/2.49.91-GCCcore-12.3.0.lua
libepoxy/1.5.10-GCCcore-12.3.0.lua
nettle/3.9.1-GCCcore-12.3.0.lua
utf8proc/2.8.0-GCCcore-12.3.0.lua
software under 2023.06/software/linux/x86_64/amd/zen4/software
ATK/2.38.0-GCCcore-12.3.0
Arrow/14.0.1-gfbf-2023a
GLPK/5.0-GCCcore-12.3.0
GTK3/3.24.37-GCCcore-12.3.0
Gdk-Pixbuf/2.42.10-GCCcore-12.3.0
Ghostscript/10.01.2-GCCcore-12.3.0
ImageMagick/7.1.1-15-GCCcore-12.3.0
NLopt/2.7.1-GCCcore-12.3.0
PostgreSQL/16.1-GCCcore-12.3.0
R-bundle-Bioconductor/3.18-foss-2023a-R-4.3.2
R-bundle-CRAN/2023.12-foss-2023a
RapidJSON/1.1.0-20230928-GCCcore-12.3.0
Wayland/1.22.0-GCCcore-12.3.0
Xvfb/21.1.8-GCCcore-12.3.0
arrow-R/14.0.1-foss-2023a-R-4.3.2
at-spi2-atk/2.38.0-GCCcore-12.3.0
at-spi2-core/2.49.91-GCCcore-12.3.0
libepoxy/1.5.10-GCCcore-12.3.0
nettle/3.9.1-GCCcore-12.3.0
utf8proc/2.8.0-GCCcore-12.3.0
other under 2023.06/software/linux/x86_64/amd/zen4
no other files in tarball
Aug 05 18:04:46 UTC 2024 test result
😢 FAILURE (click triangle for details)
Reason
EESSI test suite was not run, test step itself failed to execute.
Details
✅ job output file slurm-177.out
❌ found message matching ERROR:
✅ no message matching [\s*FAILED\s*].*Ran .* test case
Aug 05 19:13:53 UTC 2024 uploaded transfer of eessi-2023.06-software-linux-x86_64-amd-zen4-1722880243.tar.gz to S3 bucket succeeded

@ocaisa ocaisa added the bot:deploy Ask bot to deploy missing software installations to EESSI label Aug 5, 2024
@ocaisa
Copy link
Member

ocaisa commented Aug 5, 2024

Staging PR merged and ingested

@ocaisa ocaisa merged commit 782eb66 into EESSI:2023.06-software.eessi.io Aug 5, 2024
33 checks passed
@boegel boegel deleted the 2023.06-software.eessi.io_zen4_R-bundle-Bioconductor-3.18-foss-2023a-R-4.3.2 branch August 6, 2024 08:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2023.06-software.eessi.io 2023.06 version of software.eessi.io bot:deploy Ask bot to deploy missing software installations to EESSI zen4
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants