Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workbench session init go cp #871

Merged
merged 15 commits into from
Nov 20, 2024
Merged

Workbench session init go cp #871

merged 15 commits into from
Nov 20, 2024

Conversation

skyeturriff
Copy link
Contributor

@skyeturriff skyeturriff commented Nov 15, 2024

Addresses https://github.com/rstudio/rstudio-pro/issues/7006

This pull request introduces changes to the workbench-session-init image, including the addition of a Go-based entrypoint, updates to the Dockerfile, and new tests for the Go code. The main changes are grouped into Dockerfile updates, Go code additions, and test additions.

Dockerfile Updates:

  • Updated the Dockerfile.ubuntu2204 to include a multi-stage build process, adding Go installation and setting up the Go workspace. The final image now includes the compiled Go binary and session components. [1] [2]
  • The first stage, labeled as builder, sets up the environment to compile the Go application and download necessary components.
  • The second stage, labeled as build, starts with another Ubuntu 22.04 base image. I was originally using the base scratch image here, but this wouldn't work with the goss tests and there is not much difference in final image size. This stage just copies the compiled Go binary and session components from the builder stage.

Go Code Additions:

  • Added a new main.go file that contains the logic for copying session components based on the session type, validating the target directory, and ensuring the copy operation preserves permissions, times, and owner.
  • Introduced a go.mod file to manage Go module dependencies for the project.
  • It uses the otiai10/copy package to handle the copying process, preserving important file attributes and optimizing performance with parallel execution. (I believe hosted used this at one point, or still does).

Test Additions:

  • Added main_test.go with tests for the getFilesToCopy function and the copyFiles function, ensuring correct behaviour for various session types and verifying file copying operations.

Removal of Old Shell Script:

  • Removed the old run.sh shell script that previously handled the copying of session components.

Test Configuration Update:

  • Updated the goss.yaml test configuration to reflect the new entrypoint binary instead of the old shell script.

Testing

  • Verified that the Docker image builds successfully and includes the compiled Go binary and session components.
  • Tested the Go program to ensure it correctly copies files and directories while preserving permissions and ownership.
  • Confirmed that the entrypoint script executes the Go program and logs relevant information for debugging.
just preview-bake workbench-session-init-daily
mkdir init
docker run -v $(pwd)/init:/mnt/init rstudio/workbench-session-init-preview:workbench-session-init-go-cp-jammy-daily

@skyeturriff
Copy link
Contributor Author

The Release - Build, Test, and Push / Workbench Session Init (pull_request) check is failing as the Get Version step is returning the version as 2024.12.0+439.pro1, which doesn't exist:

Fetch Workbench session init container version

Run WORKBENCH_SESSION_INIT_VERSION=$(just -f ci.Justfile get-version workbench --type=daily --local)
  WORKBENCH_SESSION_INIT_VERSION=$(just -f ci.Justfile get-version workbench --type=daily --local)
  echo "WORKBENCH_SESSION_INIT_VERSION=$WORKBENCH_SESSION_INIT_VERSION" >> $GITHUB_OUTPUT
  shell: /usr/bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.12.7/x64
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.12.7/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.7/x64
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.7/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.7/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.12.7/x64/lib
./tools/get-version.py workbench --type=daily --local
Swapping product 'workbench' for 'workbench'
Providing version for product: 'workbench' and version type: 'daily'
Got daily version: '2024.12.0+439.pro1'

It should be 2024.12.0+439.pro2. And so the constructed URL is invalid:

Workbench Session Init

#9 [builder 3/8] RUN mkdir -p /pwb-staging &&     RSW_VERSION_URL=$(echo -n "2024.12.0+439.pro1" | sed 's/+/-/g') &&     curl -fsSL -o /pwb-staging/rsp-session-multi-linux.tar.gz "https://s3.amazonaws.com/rstudio-ide-build/session/multi/x86_64/rsp-session-multi-linux-${RSW_VERSION_URL}-x86_64.tar.gz" &&     mkdir -p /opt/session-components &&     tar -C /opt/session-components -xpf /pwb-staging/rsp-session-multi-linux.tar.gz &&     chmod 755 /opt/session-components &&     curl -fsSL -o /pwb-staging/go.tar.gz "https://go.dev/dl/go1.22.2.linux-amd64.tar.gz" &&     tar -C /usr/local -xf /pwb-staging/go.tar.gz &&     rm -rf /pwb-staging
#9 0.150 curl: (22) The requested URL returned error: 404

When I run Get Version step locally, I get the correct version:

❯ just -f ci.Justfile get-version workbench --type=daily --local
./tools/get-version.py workbench --type=daily --local
Swapping product 'workbench' for 'workbench'
Providing version for product: 'workbench' and version type: 'daily'
Got daily version: '2024.12.0+439.pro2'
2024.12.0+439.pro2

@ianpittwood
Copy link
Collaborator

The Release - Build, Test, and Push / Workbench Session Init (pull_request) check is failing as the Get Version step is returning the version as 2024.12.0+439.pro1, which doesn't exist:

Fetch Workbench session init container version

Run WORKBENCH_SESSION_INIT_VERSION=$(just -f ci.Justfile get-version workbench --type=daily --local)
  WORKBENCH_SESSION_INIT_VERSION=$(just -f ci.Justfile get-version workbench --type=daily --local)
  echo "WORKBENCH_SESSION_INIT_VERSION=$WORKBENCH_SESSION_INIT_VERSION" >> $GITHUB_OUTPUT
  shell: /usr/bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.12.7/x64
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.12.7/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.7/x64
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.7/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.7/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.12.7/x64/lib
./tools/get-version.py workbench --type=daily --local
Swapping product 'workbench' for 'workbench'
Providing version for product: 'workbench' and version type: 'daily'
Got daily version: '2024.12.0+439.pro1'

It should be 2024.12.0+439.pro2. And so the constructed URL is invalid:

Workbench Session Init

#9 [builder 3/8] RUN mkdir -p /pwb-staging &&     RSW_VERSION_URL=$(echo -n "2024.12.0+439.pro1" | sed 's/+/-/g') &&     curl -fsSL -o /pwb-staging/rsp-session-multi-linux.tar.gz "https://s3.amazonaws.com/rstudio-ide-build/session/multi/x86_64/rsp-session-multi-linux-${RSW_VERSION_URL}-x86_64.tar.gz" &&     mkdir -p /opt/session-components &&     tar -C /opt/session-components -xpf /pwb-staging/rsp-session-multi-linux.tar.gz &&     chmod 755 /opt/session-components &&     curl -fsSL -o /pwb-staging/go.tar.gz "https://go.dev/dl/go1.22.2.linux-amd64.tar.gz" &&     tar -C /usr/local -xf /pwb-staging/go.tar.gz &&     rm -rf /pwb-staging
#9 0.150 curl: (22) The requested URL returned error: 404

When I run Get Version step locally, I get the correct version:

❯ just -f ci.Justfile get-version workbench --type=daily --local
./tools/get-version.py workbench --type=daily --local
Swapping product 'workbench' for 'workbench'
Providing version for product: 'workbench' and version type: 'daily'
Got daily version: '2024.12.0+439.pro2'
2024.12.0+439.pro2

@skyeturriff I suggest we merge versions into workbench-session-init to help make reruns easier in the future: #875

@skyeturriff skyeturriff merged commit 6363aef into dev Nov 20, 2024
39 of 41 checks passed
@skyeturriff skyeturriff deleted the workbench-session-init-go-cp branch November 20, 2024 20:12
zachhannum pushed a commit that referenced this pull request Nov 26, 2024
* add go program

* add multi-stage build dockerfile

* add positron

* fix path

* update err msgs and reuse file info

* merge comments

* enhance file copying with otiai10/copy package and update Dockerfile

* remove uneeded components

* add shared_run to common deps

* fix component filename

* update Dockerfile to use ubuntu:22.04 base image, as the scratch image doesn't work nicely with GOSS

* remove test var

---------

Co-authored-by: Ian Pittwood <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants