Skip to content

Commit

Permalink
feat: Add free-disk-space action (#13)
Browse files Browse the repository at this point in the history
* feat: add free-disk-space action
* chore: use free-disk-space action
  • Loading branch information
NickLarsenNZ authored Oct 23, 2024
1 parent 0c5dbc4 commit 9a7313d
Show file tree
Hide file tree
Showing 6 changed files with 153 additions and 30 deletions.
1 change: 1 addition & 0 deletions .github/workflows/pr_actions-smoke-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ on:
- .github/workflows/pr_actions-smoke-test.yml
- build-container-image/action.yml
- build-product-image/action.yml
- free-disk-space/action.yml
- publish-image/action.yml
- publish-index-manifest/action.yml
- shard/action.yml
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ particular step in a workflow.

- [build-container-image](./build-container-image/README.md)
- [build-product-image](./build-product-image/README.md)
- [free-disk-space](./free-disk-space/README.md)
- [publish-image](./publish-image/README.md)
- [publish-index-manifest](./publish-index-manifest/README.md)
- [run-pre-commit](./run-pre-commit/README.md)
Expand Down
17 changes: 2 additions & 15 deletions build-container-image/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,21 +42,8 @@ outputs:
runs:
using: composite
steps:
- name: Free Disk Space (Ubuntu)
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1
with:
# This might remove tools that are actually needed, if set to "true" but
# frees about 6 GB.
tool-cache: false

# All of these default to true, but feel free to set to "false" if
# necessary for your workflow.
android: true
dotnet: true
haskell: true
large-packages: true
docker-images: true
swap-storage: true
- name: Free Disk Space
uses: ./free-disk-space

- name: Setup Docker Buildx
uses: docker/setup-buildx-action@988b5a0280414f521da01fcc63a27aeeb4b104db # v3.6.1
Expand Down
17 changes: 2 additions & 15 deletions build-product-image/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,21 +29,8 @@ outputs:
runs:
using: composite
steps:
- name: Free Disk Space (Ubuntu)
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1
with:
# This might remove tools that are actually needed, if set to "true" but
# frees about 6 GB.
tool-cache: false

# All of these default to true, but feel free to set to "false" if
# necessary for your workflow.
android: true
dotnet: true
haskell: true
large-packages: true
docker-images: true
swap-storage: true
- name: Free Disk Space
uses: ./free-disk-space

- name: Setup Docker Buildx
uses: docker/setup-buildx-action@988b5a0280414f521da01fcc63a27aeeb4b104db # v3.6.1
Expand Down
27 changes: 27 additions & 0 deletions free-disk-space/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# `free-disk-space`

> Manifest: [free-disk-space/action.yml][free-disk-space]
This action This action frees up disk space on a runner.

It is based on `jlumbroso/free-disk-space`and runs cleanup tasks in parallel where possible, and hides the STDOUT output of each task to reduce noise in workflow runs.

> [!NOTE]
> This action is used by the [build-container-image] and [build-product-image] actions.
## Inputs and Outputs

> [!TIP]
> For descriptions of the inputs and outputs, see the complete [free-disk-space] action.
### Inputs

None

### Outputs

None

[free-disk-space]: ./action.yml
[build-container-image]: ../build-container-image/action.yml
[build-product-image]: ../build-product-image/action.yml
120 changes: 120 additions & 0 deletions free-disk-space/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
---
name: Free Disk Space
description: This action frees up disk space on a runner.
inputs:
product-name:
description: The name of the product to build via bake (directory name)
required: true
config-file:
description: Path the the config file used to generate the shard indices
default: ./conf.py
outputs:
versions:
description: A list of product versions
value: ${{ steps.generate_shards.outputs.VERSIONS }}
runs:
using: composite
steps:
- name: Free Disk Space (Parallel)
shell: bash
run: |
# WARNING: `set -e` is not set as we purposefully want this to run optimistically.
function cleanup_android() {
(
sudo rm -rf /usr/local/lib/android || true
)>/dev/null
}
function cleanup_apt() {
(
sudo apt-get remove --fix-missing -y '^aspnetcore-.*' \
'^dotnet-.*' \
'^llvm-.*' \
'php.*' \
'^mongodb-.*' \
'^mysql-.*' \
azure-cli \
google-chrome-stable \
firefox \
powershell \
mono-devel \
libgl1-mesa-dri \
google-cloud-sdk \
google-cloud-cli || true
sudo apt-get autoremove -y
sudo apt-get clean
)>/dev/null
}
function cleanup_dotnet() {
(
sudo rm -rf /usr/share/dotnet || true
)>/dev/null
}
function cleanup_haskell() {
(
sudo rm -rf /opt/ghc || true
sudo rm -rf /usr/local/.ghcup || true
)>/dev/null
}
function cleanup_docker() {
(
sudo docker image prune --all --force || true
)>/dev/null
}
function cleanup_agenttools() {
(
sudo rm -rf "$AGENT_TOOLSDIRECTORY" || true
)>/dev/null
}
function disable_swap() {
(
sudo swapoff -a || true
sudo rm -f /mnt/swapfile || true
free -h
)>/dev/null
}
# The bash `time` built-in can be given subshells, and the output format
# is configured via the TIMEFORMAT environment variable.
# The binary `time` output format can be configured via a `--format`
# flag.
# Since we are calling the cleanup functions in parallel, we need to
# wrap the built-in `time` so that each concurrent invokation doesn't
# affect the output format of another.
function time_it() {
local MESSAGE="$1"
local FUNCTION="$2"
local TIMEFORMAT="$MESSAGE (%Rs)"
time "$FUNCTION"
}
echo "::group::Disk usage before"
df -h /
echo "::endgroup::"
echo "::group::Parallel cleanup"
time_it 'Removed Android libraries' cleanup_android &
time_it 'Removed apt packages' cleanup_apt &
time_it 'Removed dotnet packages' cleanup_dotnet &
time_it 'Removed haskell' cleanup_haskell &
time_it 'Pruned docker images' cleanup_docker &
time_it 'Disabled swap' disable_swap &
# This might remove tools that are actually needed, if set to "true" but
# frees about 6 GB.
# time_it 'Removed agent tools' cleanup_agenttools &
echo "Waiting for cleanup tasks"
wait
echo "::endgroup::"
echo "::group::Disk usage after"
df -h /
echo "::endgroup::"

0 comments on commit 9a7313d

Please sign in to comment.