Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test bot PR155 with software-layer PR237 #60

Open
wants to merge 52 commits into
base: main
Choose a base branch
from
Open

Test bot PR155 with software-layer PR237 #60

wants to merge 52 commits into from

Conversation

poksumdo
Copy link
Collaborator

@poksumdo poksumdo commented Mar 14, 2023 •
edited
Loading

Tested bot code PR#155
Tested software-layer PR#237
Testing if it works at all and particularly with EESSI-pilot as repository target.

truib and others added 30 commits January 24, 2023 21:59
@truib
added a function to obtain full path to a tool
447bb2d
@truib
partial bot/build.sh for eessi_container.sh
e2c24cf
@truib
added function to determine IPv4 address
a71f8e4
@truib
various updates for bot/build.sh and eessi_container.sh
b5bf008
@truib
provide CVMFS_HTTP_PROXY if necessary
b5c07ee
@truib
add functions to derive hostname and port number from proxy URL
4ff3721
@truib
add function to determine config sections
7a58fbf
@truib
add parameter to list repositories, fix bugs
a5ea7d8 
fixed bugs

- determining IP address for proxy host was wrong
- handling of positional parameters for executing commands with arguments was wrong
@truib
fix bash pattern matching operator
6ed3d85
@truib
various updates to bot/build.sh
fd5cc3d 
- added handling of HTTPS_PROXY
- moved CONTAINER setting from site_config to repository section (different
  repositories could require different containers)
- added handling of CPU_TARGET and EESSI_SOFTWARE_SUBDIR_OVERRIDE
- updated TODOs
- added setting of EESSI_REPOS_CFG_FILE_OVERRIDE
- bug fixes
@truib
Merge branch 'enhancement/bot-build-with-swl-216' of github-trz:trz42…
234bab3 
…/software-layer into enhancement/bot-build-with-swl-216
@truib
replace FILE with DIR in EESSI_REPOS_CFG_FILE_OVERRIDE
b2eab08
@truib
Merge branch 'main' of github-trz:eessi/software-layer into enhanceme…
7663fbc 
…nt/bot-build-with-swl-216
@truib
fix whitespace issues
7e200a1
@truib
fix handling of HTTP(S)_PROXY settings
57b6f27
@truib
include changes made to test PR for building with the bot
fce504f
@trz42
Merge branch 'nessi.no-2022.11-dev' into enhancement/bot-build-with-s…
910d079 
…wl-216
@trz42
Merge branch 'update_for_eessi_container_sh' of github-trz:trz42/soft…
7c418cd 
…ware-layer into enhancement/bot-build-with-eessi-container
@trz42
improvements to resuming job environment
712c40e 
cherry-picked via 23e773c
Note only applied part for bot/build.sh
@trz42
improved handling of container cache + explicit pull of image
16d9857 
cherry-picked via bfb1b29
Note, only applied part for bot/build.sh
@trz42
changed --info -> --verbose
9e8ca62
@trz42
improved messages for resume info and adjusted parsing in bot/build.sh
c0350a9
@trz42
delete one of two tmp storage tarballs created by bot/build.sh
ef9a552
@trz42
Merge branch 'main' of github-trz:EESSI/software-layer into enhanceme…
1c0081d 
…nt/bot-build-with-eessi-container
@trz42
support cmd line arg flag terminator
7cc6575
@trz42
just print contents of cfg/job.cfg
0def4e9
@trz42
switching from json (YQ) to ini (cfg_files.sh)
ef608d1
@trz42
add --generic arg when running install script
429d5a0
@trz42
fix error in regex
bd31faa
@trz42
use command separator --
1bafb78
@trz42
Copy link
Owner

trz42 commented Apr 1, 2023

bot: bar

@trz42
Copy link
Owner

trz42 commented Apr 1, 2023

bot: bot

@trz42
Copy link
Owner

trz42 commented Apr 1, 2023 •
edited by eessi-bot-devel-trz42 bot
Loading

bot: next

  • received bot command next from trz42 (expanded command format: next )
  • handling bot_command='next' failed with ("unknown bot_command.command='next'; use bot: help for usage information",)

@trz42
Copy link
Owner

trz42 commented Apr 1, 2023 •
edited by eessi-bot-devel-trz42 bot
Loading

bot: more

  • received bot command more from trz42 (expanded command format: more )
  • handling bot_command='more' failed with ('unknown command; use bot: help for usage information',)

@trz42
Copy link
Owner

trz42 commented Apr 1, 2023 •
edited by eessi-bot-devel-trz42 bot
Loading

bot: help

  • how to send commands to bot instances:
    • commands must be send with a new comment
    • every command has the syntax bot: COMMAND [ARGUMENTS]*
    • currently supported COMMANDs: help

@trz42
Copy link
Owner

trz42 commented Apr 1, 2023 •
edited by eessi-bot-devel-trz42 bot
Loading

bot: foo
bot: help

  • received bot command foo from trz42 (expanded command format: foo )
  • received bot command help from trz42 (expanded command format: help )
  • handling bot_command='help' failed with ('unknown command; use bot: help for usage information',)
  • how to send commands to bot instances:
    • commands must be sent with a new comment
    • a comment may contain multiple commands, one per line
    • every command begins at the start of a line and has the syntax bot: COMMAND [ARGUMENTS]*
    • currently supported COMMANDs: help

@trz42
Copy link
Owner

trz42 commented Apr 1, 2023 •
edited by eessi-bot-devel-trz42 bot
Loading

bot: bar
bot: help

  • received bot command bar from trz42 (expanded command format: bar )
  • received bot command help from trz42 (expanded command format: help )
  • handling bot_command='help' failed with ('unknown command; use bot: help for usage information',)
  • how to send commands to bot instances:
    • commands must be sent with a new comment
    • a comment may contain multiple commands, one per line
    • every command begins at the start of a line and has the syntax bot: COMMAND [ARGUMENTS]*
    • currently supported COMMANDs: help

@trz42
Copy link
Owner

trz42 commented Apr 1, 2023 •
edited by eessi-bot-devel-trz42 bot
Loading

bot: fff
bot: xxx
bot: help

  • received bot command fff from trz42 (expanded command format: fff )
  • received bot command xxx from trz42 (expanded command format: xxx )
  • received bot command help from trz42 (expanded command format: help )
  • handling bot_command='help' failed with ("unknown command 'fff'; use bot: help for usage information",)
  • handling bot_command='help' failed with ("unknown command 'xxx'; use bot: help for usage information",)
  • how to send commands to bot instances:
    • commands must be sent with a new comment
    • a comment may contain multiple commands, one per line
    • every command begins at the start of a line and has the syntax bot: COMMAND [ARGUMENTS]*
    • currently supported COMMANDs: help

@trz42
Copy link
Owner

trz42 commented Apr 1, 2023 •
edited by eessi-bot-devel-trz42 bot
Loading

bot: fff
bot: xxx
bot: help

  • received bot command fff from trz42 (expanded command format: fff )
  • received bot command xxx from trz42 (expanded command format: xxx )
  • received bot command help from trz42 (expanded command format: help )
  • handling fff failed with ("unknown command 'fff'; use bot: help for usage information",)
  • handling xxx failed with ("unknown command 'xxx'; use bot: help for usage information",)
  • how to send commands to bot instances:
    • commands must be sent with a new comment
    • a comment may contain multiple commands, one per line
    • every command begins at the start of a line and has the syntax bot: COMMAND [ARGUMENTS]*
    • currently supported COMMANDs: help

@trz42
Copy link
Owner

trz42 commented Apr 1, 2023 •
edited by eessi-bot-devel-trz42 bot
Loading

bot: fff
bot: xxx
bot: help

  • received bot command fff from trz42 (expanded command format: fff )
  • received bot command xxx from trz42 (expanded command format: xxx )
  • received bot command help from trz42 (expanded command format: help )
  • line is not considered to contain a bot command
  • bot commands begin with bot: , make sure
    there is no whitespace at the beginning of a line

@trz42
Copy link
Owner

trz42 commented Apr 1, 2023 •
edited by eessi-bot-devel-trz42 bot
Loading

bot: fff
bot: xxx
bot: help

  • received bot command fff from trz42 (expanded command format: fff )
  • received bot command xxx from trz42 (expanded command format: xxx )
  • received bot command help from trz42 (expanded command format: help )
  • line is not considered to contain a bot command
  • bot commands begin with bot: , make sure
    there is no whitespace at the beginning of a line
  • handling fff failed with ("unknown command 'fff'; use bot: help for usage information",)
  • handling xxx failed with ("unknown command 'xxx'; use bot: help for usage information",)
  • handling cmd.command resulted in:
    How to send commands to bot instances
    • Commands must be sent with a new comment (edits of existing comments are ignored).
    • A comment may contain multiple commands, one per line.
    • Every command begins at the start of a line and has the syntax bot: COMMAND [ARGUMENTS]*
    • Currently supported COMMANDs are: help

@trz42
Copy link
Owner

trz42 commented Apr 1, 2023 •
edited by eessi-bot-devel-trz42 bot
Loading

bot: fff

bot: xxx

bot: help

  • received bot command fff from trz42 (expanded command format: fff )
  • received bot command xxx from trz42 (expanded command format: xxx )
  • received bot command help from trz42 (expanded command format: help )
  • line is not considered to contain a bot command
  • bot commands begin with bot: , make sure
    there is no whitespace at the beginning of a line
  • handling fff failed with ("unknown command 'fff'; use bot: help for usage information",)
  • handling xxx failed with ("unknown command 'xxx'; use bot: help for usage information",)
  • handling help resulted in:
    How to send commands to bot instances
    • Commands must be sent with a new comment (edits of existing comments are ignored).
    • A comment may contain multiple commands, one per line.
    • Every command begins at the start of a line and has the syntax bot: COMMAND [ARGUMENTS]*
    • Currently supported COMMANDs are: help

@trz42
Copy link
Owner

trz42 commented Apr 1, 2023 •
edited by eessi-bot-devel-trz42 bot
Loading

bot: fff

bot: xxx

bot: help

  • received bot command fff from trz42 (expanded command format: fff )
  • received bot command xxx from trz42 (expanded command format: xxx )
  • received bot command help from trz42 (expanded command format: help )
  • handling fff failed with ("unknown command 'fff'; use bot: help for usage information",)
  • handling xxx failed with ("unknown command 'xxx'; use bot: help for usage information",)
  • handling help resulted in:
    How to send commands to bot instances
    • Commands must be sent with a new comment (edits of existing comments are ignored).
    • A comment may contain multiple commands, one per line.
    • Every command begins at the start of a line and has the syntax bot: COMMAND [ARGUMENTS]*
    • Currently supported COMMANDs are: help

@trz42
Copy link
Owner

trz42 commented Apr 4, 2023 •
edited by eessi-bot-devel-trz42 bot
Loading

bot: help

  • received bot command help from trz42 (expanded command format: help )
  • handling help resulted in:
    How to send commands to bot instances
    • Commands must be sent with a new comment (edits of existing comments are ignored).
    • A comment may contain multiple commands, one per line.
    • Every command begins at the start of a line and has the syntax bot: COMMAND [ARGUMENTS]*
    • Currently supported COMMANDs are: help

@trz42
Copy link
Owner

trz42 commented Apr 4, 2023 •
edited by eessi-bot-devel-trz42 bot
Loading

bot: help

  • received bot command help from trz42 (expanded command format: help )
  • handling help resulted in:
    How to send commands to bot instances
    • Commands must be sent with a new comment (edits of existing comments are ignored).
    • A comment may contain multiple commands, one per line.
    • Every command begins at the start of a line and has the syntax bot: COMMAND [ARGUMENTS]*
    • Currently supported COMMANDs are: help, build, showconfig

@trz42
Copy link
Owner

trz42 commented Apr 4, 2023 •
edited by eessi-bot-devel-trz42 bot
Loading

bot: showconfig

  • received bot command showconfig from trz42 (expanded command format: showconfig )

@trz42
Copy link
Owner

trz42 commented Apr 4, 2023 •
edited by eessi-bot-devel-trz42 bot
Loading

bot: showconfig

  • received bot command showconfig from trz42 (expanded command format: showconfig )
  • handling showconfig resulted in:added comment #1496635143 to show configuration

@eessi-bot-devel-trz42
Copy link

Instance CitC-commander is configured to build:

  • arch x86_64/generic for repo nessi.no-2022.11
  • arch x86_64/generic for repo nessi.no-2023.04
  • arch x86_64/generic for repo eessi-2023.04
  • arch x86_64/intel/skylake_avx512 for repo nessi.no-2022.11
  • arch x86_64/intel/cascadelake for repo nessi.no-2022.11
  • arch x86_64/amd/zen2 for repo nessi.no-2022.11
  • arch aarch64/generic for repo nessi.no-2022.11
  • arch aarch64/generic for repo nessi.no-2023.04
  • arch aarch64/generic for repo eessi-2023.04

@trz42
Copy link
Owner

trz42 commented Apr 4, 2023 •
edited by eessi-bot-devel-trz42 bot
Loading

bot: build arch:cascade
bot: build arch:aarch64 repo:eessi-2023.04

  • received bot command build arch:cascade from trz42 (expanded command format: build architecture:cascade)
  • received bot command build arch:aarch64 repo:eessi-2023.04 from trz42 (expanded command format: build architecture:aarch64 repository:eessi-2023.04)

@trz42
Copy link
Owner

trz42 commented Apr 4, 2023 •
edited by eessi-bot-devel-trz42 bot
Loading

bot: build arch:cascade
bot: build arch:aarch64 repo:eessi-2023.04

  • received bot command build arch:cascade from trz42 (expanded command format: build architecture:cascade)
  • received bot command build arch:aarch64 repo:eessi-2023.04 from trz42 (expanded command format: build architecture:aarch64 repository:eessi-2023.04)
  • handling build resulted in:no jobs were prepared
  • handling build resulted in:no jobs were prepared

@trz42
Copy link
Owner

trz42 commented Apr 4, 2023 •
edited by eessi-bot-devel-trz42 bot
Loading

bot: showconfig

@eessi-bot-devel-trz42
Copy link

Instance CitC-commander is configured to build:

  • arch x86_64/generic for repo nessi.no-2022.11
  • arch x86_64/generic for repo nessi.no-2023.04
  • arch x86_64/generic for repo eessi-2023.04
  • arch x86_64/amd/zen2 for repo nessi.no-2022.11
  • arch x86_64/intel/skylake_avx512 for repo nessi.no-2022.11
  • arch x86_64/intel/cascadelake for repo nessi.no-2022.11
  • arch aarch64/generic for repo nessi.no-2022.11
  • arch aarch64/generic for repo nessi.no-2023.04
  • arch aarch64/generic for repo eessi-2023.04

@trz42
Copy link
Owner

trz42 commented Apr 4, 2023 •
edited by eessi-bot-devel-trz42 bot
Loading

bot: build arch:cascade
bot: build arch:aarch64 repo:eessi-2023.04

  • received bot command build arch:cascade from trz42 (expanded command format: build architecture:cascade)
  • received bot command build arch:aarch64 repo:eessi-2023.04 from trz42 (expanded command format: build architecture:aarch64 repository:eessi-2023.04)
  • handling build resulted in:created jobs: 4510
  • handling build resulted in:created jobs: 4511

@eessi-bot-devel-trz42
Copy link

eessi-bot-devel-trz42 bot commented Apr 4, 2023 •
edited
Loading

New job on instance CitC-commander for architecture x86_64-intel-cascadelake for repository nessi.no-2022.11 in job dir /mnt/shared/home/trz42/pilot.nessi/bot-rebuild/jobs/2023.04/pr_60/4510

  • cancelled
date job status comment
Apr 04 10:02:06 PM UTC 2023 submitted job id 4510 awaits release by job manager
Apr 04 10:03:10 PM UTC 2023 released job awaits launch by Slurm scheduler
Apr 04 10:08:14 PM UTC 2023 running job 4510 is running
Apr 04 10:13:20 PM UTC 2023 finished 😢 FAILURE
  • Found slurm output slurm-4510.out in job dir
  • Slurm output lacks message "No missing modules!".
  • Slurm output lacks message about created tarball.
  • No tarball matching eessi-*software-*.tar.gz found in job dir.

@eessi-bot-devel-trz42
Copy link

eessi-bot-devel-trz42 bot commented Apr 4, 2023 •
edited by trz42
Loading

New job on instance CitC-commander for architecture aarch64-generic for repository eessi-2023.04 in job dir /mnt/shared/home/trz42/pilot.nessi/bot-rebuild/jobs/2023.04/pr_60/4511

  • cancelled
date job status comment
Apr 04 10:02:16 PM UTC 2023 submitted job id 4511 awaits release by job manager
Apr 04 10:03:07 PM UTC 2023 released job awaits launch by Slurm scheduler
Apr 04 10:08:12 PM UTC 2023 running job 4511 is running
Apr 04 10:13:17 PM UTC 2023 finished 😢 FAILURE
  • Found slurm output slurm-4511.out in job dir
  • Slurm output lacks message "No missing modules!".
  • Slurm output lacks message about created tarball.
  • No tarball matching eessi-*software-*.tar.gz found in job dir.

@trz42
Copy link
Owner

trz42 commented Apr 4, 2023 •
edited by eessi-bot-devel-trz42 bot
Loading

bot: build r:2023

  • received bot command build r:2023 from trz42 (expanded command format: build repository:2023)
  • handling build resulted in:created jobs: 4512, 4513, 4514, 4515

@eessi-bot-devel-trz42
Copy link

New job on instance CitC-commander for architecture x86_64-generic for repository nessi.no-2023.04 in job dir /mnt/shared/home/trz42/pilot.nessi/bot-rebuild/jobs/2023.04/pr_60/4512

date job status comment
Apr 04 10:14:51 PM UTC 2023 submitted job id 4512 awaits release by job manager

@eessi-bot-devel-trz42
Copy link

New job on instance CitC-commander for architecture x86_64-generic for repository eessi-2023.04 in job dir /mnt/shared/home/trz42/pilot.nessi/bot-rebuild/jobs/2023.04/pr_60/4513

date job status comment
Apr 04 10:14:53 PM UTC 2023 submitted job id 4513 awaits release by job manager

@eessi-bot-devel-trz42
Copy link

New job on instance CitC-commander for architecture aarch64-generic for repository nessi.no-2023.04 in job dir /mnt/shared/home/trz42/pilot.nessi/bot-rebuild/jobs/2023.04/pr_60/4514

date job status comment
Apr 04 10:14:55 PM UTC 2023 submitted job id 4514 awaits release by job manager

@eessi-bot-devel-trz42
Copy link

New job on instance CitC-commander for architecture aarch64-generic for repository eessi-2023.04 in job dir /mnt/shared/home/trz42/pilot.nessi/bot-rebuild/jobs/2023.04/pr_60/4515

date job status comment
Apr 04 10:14:56 PM UTC 2023 submitted job id 4515 awaits release by job manager

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bot:build Instruct bot to build software stack
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@poksumdo @trz42 @truib