Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

release dist-manifest.json describes artifacts that do not exist #4195

Closed
turbocrime opened this issue Apr 11, 2024 · 1 comment · Fixed by #4282
Closed

release dist-manifest.json describes artifacts that do not exist #4195

turbocrime opened this issue Apr 11, 2024 · 1 comment · Fixed by #4282
Assignees
Labels
A-CI/CD Relates to continuous integration & deployment of Penumbra

Comments

@turbocrime
Copy link
Contributor

turbocrime commented Apr 11, 2024

source.tar.gz and source.tar.gz.sha256 are not uploaded as release artifacts, though they are generated by cargo dist and described in dist-manifest.json

this seems to be the relevant section of the workflow

- id: cargo-dist
name: Post-build
# We force bash here just because github makes it really hard to get values up
# to "real" actions without writing to env-vars, and writing to env-vars has
# inconsistent syntax between shell and powershell.
shell: bash
run: |
# Parse out what we just built and upload it to the Github Release™
echo "paths<<EOF" >> "$GITHUB_OUTPUT"
jq --raw-output ".artifacts[]?.path | select( . != null )" dist-manifest.json >> "$GITHUB_OUTPUT"
echo "EOF" >> "$GITHUB_OUTPUT"
cp dist-manifest.json "$BUILD_MANIFEST_NAME"
- name: "Upload artifacts"
uses: actions/upload-artifact@v3
with:
name: artifacts
path: |
${{ steps.cargo-dist.outputs.paths }}
${{ env.BUILD_MANIFEST_NAME }}

it seems the tar's artifact object in dist-manifest.json doesn't contain an 'assets' field so the tar and its checksum are ignored

@conorsch
Copy link
Contributor

In #4130 (comment), I enabled attaching full source tarballs to the github release objects. We haven't released since that change was made, so I don't have an artifact to point to, but we should be releasing again this week, after which I'll confirm we have info there.

Relatedly (for #4255), I'm testing newer versions of cargo-dist in a private fork, and it looks like the newer versions do properly include the source tarball created by cargo-dist—which is separate from the automatically created github one—and corresponding checksum:

source-tar-checksums

It'll probably be release n+2 that these changes make it to the monorepo, but they're coming. Thanks for the clear report, @turbocrime.

@conorsch conorsch self-assigned this Apr 24, 2024
@conorsch conorsch added the A-CI/CD Relates to continuous integration & deployment of Penumbra label Apr 24, 2024
conorsch added a commit that referenced this issue Apr 29, 2024
Bumps the cargo-dist dependency, and regenerates the relevant CI files.
Tested these changes in a private fork, so we can be confident they
won't break builds.

Motivated by deprecation warnings in cargo-dist declaring that pinning
rust versions should use `rust-toolchain.toml`, updated ours to pin
1.75. Standardizing on the toolchain file lets us remove repeated
versions from a few different places, such as the containerbuild.
Removed custom action helpers from Rust CI workflows, because GHA
runners already come with rustup, and will honor the toolchain file.
This change ensures that our dev, build, and test environments all
use the same version of rust, which should minimize surprises like we
saw in #4231.

Included in the release artifacts now is a complete source tarball
with associated checksum, as requested in #4195. We're also distributing
binaries compressed as .tar.gz, rather than .tar.xz, to satisfy #4256,
and opting into a custom install script to satisfy #4255. Therefore:

* closes #4231
* closes #4255
* closes #4256
conorsch added a commit that referenced this issue Apr 29, 2024
Bumps the cargo-dist dependency, and regenerates the relevant CI files.
Tested these changes in a private fork, so we can be confident they
won't break builds.

Motivated by deprecation warnings in cargo-dist declaring that pinning
rust versions should use `rust-toolchain.toml`, updated ours to pin
1.75. Standardizing on the toolchain file lets us remove repeated
versions from a few different places, such as the containerbuild.
Removed custom action helpers from Rust CI workflows, because GHA
runners already come with rustup, and will honor the toolchain file.
This change ensures that our dev, build, and test environments all
use the same version of rust, which should minimize surprises like we
saw in #4231.

Included in the release artifacts now is a complete source tarball
with associated checksum, as requested in #4195. We're also distributing
binaries compressed as .tar.gz, rather than .tar.xz, to satisfy #4256,
and opting into a custom install script to satisfy #4255. Therefore:

* closes #4231
* closes #4255
* closes #4256
conorsch added a commit that referenced this issue Apr 29, 2024
Bumps the cargo-dist dependency, and regenerates the relevant CI files.
Tested these changes in a private fork, so we can be confident they
won't break builds.

Motivated by deprecation warnings in cargo-dist declaring that pinning
rust versions should use `rust-toolchain.toml`, updated ours to pin
1.75. Standardizing on the toolchain file lets us remove repeated
versions from a few different places, such as the containerbuild.
Removed custom action helpers from Rust CI workflows, because GHA
runners already come with rustup, and will honor the toolchain file.
This change ensures that our dev, build, and test environments all
use the same version of rust, which should minimize surprises like we
saw in #4231.

Included in the release artifacts now is a complete source tarball
with associated checksum, as requested in #4195. We're also distributing
binaries compressed as .tar.gz, rather than .tar.xz, to satisfy #4256,
and opting into a custom install script to satisfy #4255. Therefore:

* closes #4195
* closes #4255
* closes #4256
conorsch added a commit that referenced this issue Apr 29, 2024
Bumps the cargo-dist dependency, and regenerates the relevant CI files.
Tested these changes in a private fork, so we can be confident they
won't break builds.

Motivated by deprecation warnings in cargo-dist declaring that pinning
rust versions should use `rust-toolchain.toml`, updated ours to pin
1.75. Standardizing on the toolchain file lets us remove repeated
versions from a few different places, such as the containerbuild.
Removed custom action helpers from Rust CI workflows, because GHA
runners already come with rustup, and will honor the toolchain file.
This change ensures that our dev, build, and test environments all
use the same version of rust, which should minimize surprises like we
saw in #4231.

Included in the release artifacts now is a complete source tarball
with associated checksum, as requested in #4195. We're also distributing
binaries compressed as .tar.gz, rather than .tar.xz, to satisfy #4256,
and opting into a custom install script to satisfy #4255. Therefore:

* closes #4195
* closes #4255
* closes #4256
conorsch added a commit that referenced this issue Apr 29, 2024
Bumps the cargo-dist dependency, and regenerates the relevant CI files.
Tested these changes in a private fork, so we can be confident they
won't break builds.

Motivated by deprecation warnings in cargo-dist declaring that pinning
rust versions should use `rust-toolchain.toml`, updated ours to pin
1.75. Standardizing on the toolchain file lets us remove repeated
versions from a few different places, such as the containerbuild.
Removed custom action helpers from Rust CI workflows, because GHA
runners already come with rustup, and will honor the toolchain file.
This change ensures that our dev, build, and test environments all
use the same version of rust, which should minimize surprises like we
saw in #4231.

Included in the release artifacts now is a complete source tarball
with associated checksum, as requested in #4195. We're also distributing
binaries compressed as .tar.gz, rather than .tar.xz, to satisfy #4256,
and opting into a custom install script to satisfy #4255. Therefore:

* closes #4195
* closes #4255
* closes #4256
@github-project-automation github-project-automation bot moved this from Backlog to Done in Penumbra Apr 29, 2024
conorsch added a commit that referenced this issue Apr 29, 2024
Bumps the cargo-dist dependency, and regenerates the relevant CI files.
Tested these changes in a private fork, so we can be confident they
won't break builds.

Motivated by deprecation warnings in cargo-dist declaring that pinning
rust versions should use `rust-toolchain.toml`, updated ours to pin
1.75. Standardizing on the toolchain file lets us remove repeated
versions from a few different places, such as the containerbuild.
Removed custom action helpers from Rust CI workflows, because GHA
runners already come with rustup, and will honor the toolchain file.
This change ensures that our dev, build, and test environments all
use the same version of rust, which should minimize surprises like we
saw in #4231.

Included in the release artifacts now is a complete source tarball
with associated checksum, as requested in #4195. We're also distributing
binaries compressed as .tar.gz, rather than .tar.xz, to satisfy #4256,
and opting into a custom install script to satisfy #4255. Therefore:

* closes #4195
* closes #4255
* closes #4256

(cherry picked from commit 36280c4)

Involved some manual conflict-resolution, particularly around RUSTFLAGS,
preserving the overrides since we haven't dropped the requirement for
the tokio unstable flag in the release branches yet.

ci: fix rust-toolchain.toml in container

Follow-up to #4282, ensuring that the rust-toolchain.toml makes it into
the container image, otherwise it'll throw an error on image build.
Observed this on the post-merge preview deploy.

(cherry picked from commit 8d3f4f9)
conorsch added a commit that referenced this issue Apr 29, 2024
Bumps the cargo-dist dependency, and regenerates the relevant CI files.
Tested these changes in a private fork, so we can be confident they
won't break builds.

Motivated by deprecation warnings in cargo-dist declaring that pinning
rust versions should use `rust-toolchain.toml`, updated ours to pin
1.75. Standardizing on the toolchain file lets us remove repeated
versions from a few different places, such as the containerbuild.
Removed custom action helpers from Rust CI workflows, because GHA
runners already come with rustup, and will honor the toolchain file.
This change ensures that our dev, build, and test environments all
use the same version of rust, which should minimize surprises like we
saw in #4231.

Included in the release artifacts now is a complete source tarball
with associated checksum, as requested in #4195. We're also distributing
binaries compressed as .tar.gz, rather than .tar.xz, to satisfy #4256,
and opting into a custom install script to satisfy #4255. Therefore:

* closes #4195
* closes #4255
* closes #4256

(cherry picked from commit 36280c4)

Involved some manual conflict-resolution, particularly around RUSTFLAGS,
preserving the overrides since we haven't dropped the requirement for
the tokio unstable flag in the release branches yet.

ci: fix rust-toolchain.toml in container

Follow-up to #4282, ensuring that the rust-toolchain.toml makes it into
the container image, otherwise it'll throw an error on image build.
Observed this on the post-merge preview deploy.

(cherry picked from commit 8d3f4f9)
conorsch added a commit that referenced this issue Apr 29, 2024
Bumps the cargo-dist dependency, and regenerates the relevant CI files.
Tested these changes in a private fork, so we can be confident they
won't break builds.

Motivated by deprecation warnings in cargo-dist declaring that pinning
rust versions should use `rust-toolchain.toml`, updated ours to pin
1.75. Standardizing on the toolchain file lets us remove repeated
versions from a few different places, such as the containerbuild.
Removed custom action helpers from Rust CI workflows, because GHA
runners already come with rustup, and will honor the toolchain file.
This change ensures that our dev, build, and test environments all
use the same version of rust, which should minimize surprises like we
saw in #4231.

Included in the release artifacts now is a complete source tarball
with associated checksum, as requested in #4195. We're also distributing
binaries compressed as .tar.gz, rather than .tar.xz, to satisfy #4256,
and opting into a custom install script to satisfy #4255. Therefore:

* closes #4195
* closes #4255
* closes #4256

(cherry picked from commit 36280c4)

Involved some manual conflict-resolution, particularly around RUSTFLAGS,
preserving the overrides since we haven't dropped the requirement for
the tokio unstable flag in the release branches yet.

ci: fix rust-toolchain.toml in container

Follow-up to #4282, ensuring that the rust-toolchain.toml makes it into
the container image, otherwise it'll throw an error on image build.
Observed this on the post-merge preview deploy.

(cherry picked from commit 8d3f4f9)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-CI/CD Relates to continuous integration & deployment of Penumbra
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants