Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added 24h default time limit #156

Merged
merged 6 commits into from
Mar 10, 2023
Merged

Added 24h default time limit #156

Changes from 4 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
c2cc287
Added 24h default time limit
jonas-lq Feb 28, 2023
e9169f4
Fixed whitespace to be pep8 compliant
jonas-lq Feb 28, 2023
78ecdb8
Added checks for short and long form of time flag
jonas-lq Mar 6, 2023
2e2abc9
Added whitespace around operator
jonas-lq Mar 6, 2023
3fbc97a
Catching more cases in the time limit check
jonas-lq Mar 9, 2023
8bf9b1a
Code polishing
jonas-lq Mar 10, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion tasks/build.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@

BUILDENV = "buildenv"
BUILD_JOB_SCRIPT = "build_job_script"
DEFAULT_JOB_TIME_LIMIT = "24:00:00"
CVMFS_CUSTOMIZATIONS = "cvmfs_customizations"
HTTP_PROXY = "http_proxy"
HTTPS_PROXY = "https_proxy"
Expand Down Expand Up @@ -264,13 +265,20 @@ def submit_job(job, submitted_jobs, build_env_cfg, ym, pr_id):
- job_id(string): job_id of submitted job
- symlink(string): symlink from main pr_<ID> dir to job dir (job[0])
"""

# determine CPU target to use: job.arch_target, but without the linux/ part
cpu_target = '/'.join(job.arch_target.split('/')[1:])

# Add a default time limit of 24h to the command if nothing else is specified by the user
all_opts = " ".join([build_env_cfg[SLURM_PARAMS], job.slurm_opts, '--export=ALL,CPU_TARGET=%s' % cpu_target])
if ("--time" in all_opts) or ("-t" in all_opts):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a big improvement. However, there are a few cases this way of checking is not catching. See test cases 8 & 9 in trz42/software-layer#58 (comment)

So, we may need to iterate over all options and check if they begin with --time or -t, e.g.,

opt.startswith("--time")

time_limit = ""
else:
time_limit = f"--time={DEFAULT_JOB_TIME_LIMIT}"

command_line = ' '.join([
build_env_cfg[SUBMIT_COMMAND],
build_env_cfg[SLURM_PARAMS],
time_limit,
job.slurm_opts,
# set $CPU_TARGET in job environment, which can be picked up by build job script;
'--export=ALL,CPU_TARGET=%s' % cpu_target,
Expand Down