Skip to content

Commit

Permalink
pathogen-repo-build: Move build.log outside of working directory
Browse files Browse the repository at this point in the history
The entire working directory gets zipped and uploaded to AWS Batch when
using the `aws-batch` runtime, which includes an early `build.log` file.
When the build completes and downloads the build outputs, the early
`build.log` file overwrites the local `build.log` of the GH Action.
This results in a truncated file that does not include the
AWS_BATCH_JOB_ID and prevents the generation of the AWS Batch summary.

This commit moves the hard-coded `build.log` file to outside of the
current working directory to prevent it from being uploaded with the
pathogen repo build.
  • Loading branch information
joverlee521 committed Mar 15, 2024
1 parent 5cd0ba1 commit bd0a88b
Show file tree
Hide file tree
Showing 2 changed files with 16 additions and 16 deletions.
18 changes: 9 additions & 9 deletions .github/workflows/pathogen-repo-build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -98,14 +98,14 @@ on:
The default paths included in the artifact are:
build.log
../build.log
auspice/
results/
benchmarks/
logs/
.snakemake/log/
The "build.log" contains log messages from the `nextstrain build` command. The other paths are common output paths for Nextstrain builds. If a path does not exist in your build, then the action will still succeed and will print out a warning for the non-existent file(s). Use an exclude pattern for any of the default paths that you would like to exclude from the artifact (e.g. !build.log).
The "../build.log" contains log messages from the `nextstrain build` command. The other paths are common output paths for Nextstrain builds. If a path does not exist in your build, then the action will still succeed and will print out a warning for the non-existent file(s). Use an exclude pattern for any of the default paths that you would like to exclude from the artifact (e.g. !build.log).
This is not supported for builds on AWS Batch because the workflow detaches from the build. Please use the `nextstrain build` command locally to reattach to AWS Batch builds to download outputs.
type: string
Expand Down Expand Up @@ -189,14 +189,14 @@ on:
The default paths included in the artifact are:
build.log
../build.log
auspice/
results/
benchmarks/
logs/
.snakemake/log/
The "build.log" contains log messages from the `nextstrain build` command. The other paths are common output paths for Nextstrain builds. If a path does not exist in your build, then the action will still succeed and will print out a warning for the non-existent file(s). Use an exclude pattern for any of the default paths that you would like to exclude from the artifact (e.g. !build.log).
The "../build.log" contains log messages from the `nextstrain build` command. The other paths are common output paths for Nextstrain builds. If a path does not exist in your build, then the action will still succeed and will print out a warning for the non-existent file(s). Use an exclude pattern for any of the default paths that you would like to exclude from the artifact (e.g. !build.log).
This is not supported for builds on AWS Batch because the workflow detaches from the build. Please use the `nextstrain build` command locally to reattach to AWS Batch builds to download outputs.
type: string
Expand Down Expand Up @@ -274,14 +274,14 @@ jobs:
# shellcheck disable=SC2154
set -x
eval "$NEXTSTRAIN_BUILD_COMMAND" |& tee build.log
eval "$NEXTSTRAIN_BUILD_COMMAND" |& tee ../build.log
# Attempt to get the AWS Batch ID even if the run build command failed
# as long as the runtime is `aws-batch` and the `build.log` file exists
- if: ${{ always() && inputs.runtime == 'aws-batch' && hashFiles('build.log') != '' }}
# as long as the runtime is `aws-batch` and the `../build.log` file exists
- if: ${{ always() && inputs.runtime == 'aws-batch' && hashFiles('../build.log') != '' }}
name: Get AWS Batch job id
id: aws-batch
run: |
echo "AWS_BATCH_JOB_ID=$(sed -nE 's/.+AWS Batch Job ID\:.+ ([-a-f0-9]+)$/\1/p' < build.log)" | tee -a "$GITHUB_ENV"
echo "AWS_BATCH_JOB_ID=$(sed -nE 's/.+AWS Batch Job ID\:.+ ([-a-f0-9]+)$/\1/p' < ../build.log)" | tee -a "$GITHUB_ENV"
- if: ${{ always() && env.AWS_BATCH_JOB_ID }}
name: Generate AWS Batch summary
run: |
Expand All @@ -293,7 +293,7 @@ jobs:
if-no-files-found: warn
name: ${{ inputs.artifact-name }}
path: |
build.log
../build.log
auspice/
results/
benchmarks/
Expand Down
14 changes: 7 additions & 7 deletions .github/workflows/pathogen-repo-build.yaml.in
Original file line number Diff line number Diff line change
Expand Up @@ -128,14 +128,14 @@ on:

The default paths included in the artifact are:

build.log
../build.log
auspice/
results/
benchmarks/
logs/
.snakemake/log/

The "build.log" contains log messages from the `nextstrain build` command.
The "../build.log" contains log messages from the `nextstrain build` command.
The other paths are common output paths for Nextstrain builds.
If a path does not exist in your build, then the action will still
succeed and will print out a warning for the non-existent file(s).
Expand Down Expand Up @@ -240,15 +240,15 @@ jobs:
# shellcheck disable=SC2154
set -x

eval "$NEXTSTRAIN_BUILD_COMMAND" |& tee build.log
eval "$NEXTSTRAIN_BUILD_COMMAND" |& tee ../build.log

# Attempt to get the AWS Batch ID even if the run build command failed
# as long as the runtime is `aws-batch` and the `build.log` file exists
- if: ${{ always() && inputs.runtime == 'aws-batch' && hashFiles('build.log') != '' }}
# as long as the runtime is `aws-batch` and the `../build.log` file exists
- if: ${{ always() && inputs.runtime == 'aws-batch' && hashFiles('../build.log') != '' }}
name: Get AWS Batch job id
id: aws-batch
run: |
echo "AWS_BATCH_JOB_ID=$(sed -nE 's/.+AWS Batch Job ID\:.+ ([-a-f0-9]+)$/\1/p' < build.log)" | tee -a "$GITHUB_ENV"
echo "AWS_BATCH_JOB_ID=$(sed -nE 's/.+AWS Batch Job ID\:.+ ([-a-f0-9]+)$/\1/p' < ../build.log)" | tee -a "$GITHUB_ENV"

- if: ${{ always() && env.AWS_BATCH_JOB_ID }}
name: Generate AWS Batch summary
Expand All @@ -262,7 +262,7 @@ jobs:
if-no-files-found: warn
name: ${{ inputs.artifact-name }}
path: |
build.log
../build.log
auspice/
results/
benchmarks/
Expand Down

0 comments on commit bd0a88b

Please sign in to comment.