Skip to content

Commit

Permalink
Issue/1756 mithril script updates (#27)
Browse files Browse the repository at this point in the history
* syntactic bug in entrypoint.sh cardano-community#1757

* Remove uneeded UPDATE_CHECK=N exports now that it is a default of the image ENV variables

* export G_ACCOUNT so guild-deploy.sh inherits it during forked builds

* Set defaults for G_ACCOUNT and GUILD_DEPLOY_BRANCH to simplify manual docker build commands

* Fix bug where updating cncli.sh cardano-community#1756

mithril-client binary command/subcommand changes replace snapshot with cardano-db cardano-community#1759

* Add reference to MITHRIL_DOWNLOAD for snapshot sync in regular docs

* export each line of mithril.env for the signer
use tee for logging to not hide issues/errors

* Fix for PARTY_ID used in verify_signer_request and verify_signer_signature

* Move sanchonet to RELEASE=testing

* Update configs and node version support for 8.9.x (cardano-community#1743)

## Description
<!--- Describe your changes -->

- [x] Update topology formats
- [x] Update baseline node version references
- [x] Update node/cli dependency refs and pre-downloaded binaries
- [x] Update document references
- [x] Update dbsync version

* SANCHONET/PREVIEW - adjustments to support sanchonet and preview versions of mithril for guild-deploy.sh downloading.

* linting and indentation consistentcy for entrypoint.sh

* Workflows to get updated pre/unstable release as well as rebase preview and sanchonet branches. Occurs on push to alpha, workflow dispatch, and scheduled intervals

* Docker Image workflow determine cnversion to use based on the guild_deploy_branch name.
   - When branch preview/sanchonet use prerelease node version.
   - For any other branch use the original logic for latest (stable) node version.

---------

Co-authored-by: RdLrT <[email protected]>
  • Loading branch information
TrevorBenson and rdlrt authored Apr 28, 2024
1 parent a8743cb commit 4c37b1d
Show file tree
Hide file tree
Showing 14 changed files with 890 additions and 427 deletions.
204 changes: 204 additions & 0 deletions .github/workflows/autoupdate-testing.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,204 @@
name: Autoupdate testing branches
on:
workflow_dispatch:
push:
branches:
- alpha
schedule:
- cron: '0 */12 * * *'

jobs:
start-summary:
runs-on: ubuntu-latest
steps:
- name: Set summary details
run: |
echo "## Autoupdate Testing Branches Summary Details" >> $GITHUB_STEP_SUMMARY
echo "| Name | Value |" >> $GITHUB_STEP_SUMMARY
echo "| ------------ | ---------------------------------------- |" >> $GITHUB_STEP_SUMMARY
echo "| GitHub Actor | ${GITHUB_ACTOR} |" >> $GITHUB_STEP_SUMMARY
echo "| GitHub Email | ${GITHUB_ACTOR}@users.noreply.github.com |" >> $GITHUB_STEP_SUMMARY
get-node-prerelease:
needs: start-summary
runs-on: ubuntu-latest
strategy:
matrix:
branch: [preview, sanchonet]
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
ref: ${{ matrix.branch }}
- name: Configure Git
run: |
git config --global user.name "${GITHUB_ACTOR}"
git config --global user.email "${GITHUB_ACTOR}@users.noreply.github.com"
- name: Get current SHA
id: set_sha
run: |
echo "COMMIT_SHA=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
- name: Fetch node pre-release version
run: |
curl -sL https://api.github.com/repos/IntersectMBO/cardano-node/releases | jq -r '.[] | select(.prerelease == true) | .tag_name' | head -1 > files/docker/node/release-versions/cardano-node-prerelease.txt
- name: Assigns pre-release version
run: |
echo "PRERELEASE_VERSION=$(cat files/docker/node/release-versions/cardano-node-prerelease.txt)" >> $GITHUB_ENV
- name: Check for changes
id: git-check
run: |
MODIFIED=$([ -z "`git status --porcelain`" ] && echo "false" || echo "true")
echo "MODIFIED=${MODIFIED}" >> "$GITHUB_OUTPUT"
- name: Set summary details
if: always()
run: |
echo "## Get Node Pre-Release ${{ matrix.branch }}" >> $GITHUB_STEP_SUMMARY
echo "| Name | Value |" >> $GITHUB_STEP_SUMMARY
echo "| -------------------- | ---------------------------------------------------------- |" >> $GITHUB_STEP_SUMMARY
echo "| Prerelease Version | ${PRERELEASE_VERSION} |" >> $GITHUB_STEP_SUMMARY
echo "| Repository Modified | ${{ steps.git-check.outputs.MODIFIED }} |" >> $GITHUB_STEP_SUMMARY
echo "| Modified files | $(git diff --name-only $(echo ${{ steps.set_sha.outputs.COMMIT_SHA }}) | tr '\n' ',') |" >> $GITHUB_STEP_SUMMARY
- name: Commit latest pre-release versions
if: steps.git-check.outputs.MODIFIED == 'true'
run: |
git add files/docker/node/release-versions/cardano-node-prerelease.txt
git commit -am "New mithril pre-release versions: pre-release ${PRERELEASE_VERSION}"
git push
get-mithril-prerelease:
needs: get-node-prerelease
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
ref: preview
- name: Configure Git
run: |
git config --global user.name "${GITHUB_ACTOR}"
git config --global user.email "${GITHUB_ACTOR}@users.noreply.github.com"
- name: Get current SHA
id: set_sha
run: |
echo "COMMIT_SHA=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
- name: Fetch Mithril release and pre-release versions
run: |
mithril_release="$(curl -s https://api.github.com/repos/input-output-hk/mithril/releases/latest | jq -r '.tag_name')"
mithril_prerelease="$(curl -s https://api.github.com/repos/input-output-hk/mithril/releases | jq -r '.[] | select(.prerelease == true) | select(.tag_name | endswith("-pre")) | .tag_name' | head -n 1)"
if [[ "${mithril_release}-pre" == "${mithril_prerelease}" ]]; then
echo "${mithril_release}" > files/docker/node/release-versions/mithril-prerelease.txt
PRERELEASE_VERSION=${mithril_release}
echo "PRERELEASE_VERSION=${PRERELEASE_VERSION}" >> $GITHUB_ENV
echo "RELEASE_MATCH=True" >> $GITHUB_ENV
else
echo "${mithril_prerelease}" > files/docker/node/release-versions/mithril-prerelease.txt
PRERELEASE_VERSION=${mithril_prerelease}
echo "PRERELEASE_VERSION=${PRERELEASE_VERSION}" >> $GITHUB_ENV
echo "RELEASE_MATCH=False" >> $GITHUB_ENV
fi
- name: Check for changes
id: git-check
run: |
MODIFIED=$([ -z "`git status --porcelain`" ] && echo "false" || echo "true")
echo "MODIFIED=${MODIFIED}" >> "$GITHUB_OUTPUT"
- name: Set summary details
if: always()
run: |
echo "## Get Mithril Pre-Release" >> $GITHUB_STEP_SUMMARY
echo "| Name | Value |" >> $GITHUB_STEP_SUMMARY
echo "| ----------------------------- | -------------------------------------------------------- |" >> $GITHUB_STEP_SUMMARY
echo "| Pre-release version | ${PRERELEASE_VERSION} |" >> $GITHUB_STEP_SUMMARY
echo "| Pre-release matches Release | ${RELEASE_MATCH} |" >> $GITHUB_STEP_SUMMARY
echo "| Repository Modified | ${{ steps.git-check.outputs.MODIFIED }} |" >> $GITHUB_STEP_SUMMARY
echo "| Modified files | $(git diff --name-only $(echo ${{ steps.set_sha.outputs.COMMIT_SHA }}) | tr '\n' ',') |" >> $GITHUB_STEP_SUMMARY
- name: Commit latest pre-release versions
if: steps.git-check.outputs.MODIFIED == 'true'
run: |
git add files/docker/node/release-versions/mithril-prerelease.txt
git commit -am "New mithril pre-release versions: pre-release ${PRERELEASE_VERSION}"
git push
get-mithril-unstable:
needs: get-node-prerelease
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
ref: sanchonet
- name: Configure Git
run: |
git config --global user.name "${GITHUB_ACTOR}"
git config --global user.email "${GITHUB_ACTOR}@users.noreply.github.com"
- name: Get current SHA
id: set_sha
run: |
echo "COMMIT_SHA=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
- name: Fetch Mithril unstable version
run: |
curl -sL https://api.github.com/repos/input-output-hk/mithril/releases | jq -r '.[] | select(.prerelease == true and .tag_name == "unstable") | .tag_name' | head -1 > files/docker/node/release-versions/mithril-unstable.txt
- name: Assign unstable version
run: |
UNSTABLE_VERSION=$(cat files/docker/node/release-versions/mithril-unstable.txt)
echo "UNSTABLE_VERSION=${UNSTABLE_VERSION}" >> $GITHUB_ENV
- name: Check for changes
id: git-check
run: |
MODIFIED=$([ -z "`git status --porcelain`" ] && echo "false" || echo "true")
echo "MODIFIED=${MODIFIED}" >> "$GITHUB_OUTPUT"
- name: Set summary details
if: always()
run: |
echo "## Get Mithril Unstable" >> $GITHUB_STEP_SUMMARY
echo "| Name | Value |" >> $GITHUB_STEP_SUMMARY
echo "| -------------------- | ----------------------------------------------------------------------- |" >> $GITHUB_STEP_SUMMARY
echo "| Unstable version | ${UNSTABLE_VERSION} |" >> $GITHUB_STEP_SUMMARY
echo "| Repository Modified | ${{ steps.git-check.outputs.MODIFIED }} |" >> $GITHUB_STEP_SUMMARY
echo "| Modified files | $(git diff --name-only $(echo ${{ steps.set_sha.outputs.COMMIT_SHA }}) | tr '\n' ',') |" >> $GITHUB_STEP_SUMMARY
- name: Commit latest unstable versions
if: steps.git-check.outputs.MODIFIED == 'true'
run: |
git add files/docker/node/release-versions/mithril-unstable.txt
git commit -am "New mithril unstable versions: unstable ${UNSTABLE_VERSION}"
git push
rebase-testing-branches:
needs: [get-node-prerelease, get-mithril-prerelease, get-mithril-unstable]
runs-on: ubuntu-latest
strategy:
matrix:
branch: [preview, sanchonet]
steps:
- uses: actions/checkout@v4
with:
ref: ${{ matrix.branch }}
fetch-depth: 0
- name: Configure Git
run: |
git config --global user.name "${GITHUB_ACTOR}"
git config --global user.email "${GITHUB_ACTOR}@users.noreply.github.com"
- name: Get current SHA
id: set_sha
run: |
echo "COMMIT_SHA=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
- name: Fetch alpha
run: |
git fetch origin alpha
- name: Rebase on alpha
run: |
git rebase origin/alpha
- name: Check for changes
id: git-check
run: |
MODIFIED=$([ "$(git rev-parse HEAD)" != "$(echo ${{ steps.set_sha.outputs.COMMIT_SHA }})" ] && echo "true" || echo "false")
echo "MODIFIED=${MODIFIED}" >> "$GITHUB_OUTPUT"
- name: Set summary details
if: always()
run: |
echo "## Rebase ${{ matrix.branch }}" >> $GITHUB_STEP_SUMMARY
echo "| Name | Value |" >> $GITHUB_STEP_SUMMARY
echo "| --------------------- | ----------------------------------------------------------------------- |" >> $GITHUB_STEP_SUMMARY
echo "| Repository Modified | ${{ steps.git-check.outputs.MODIFIED }} |" >> $GITHUB_STEP_SUMMARY
echo "| Modified Files | $(git diff --name-only $(echo ${{ steps.set_sha.outputs.COMMIT_SHA }}) | tr '\n' ',') |" >> $GITHUB_STEP_SUMMARY
- name: Push changes
if: steps.git-check.outputs.MODIFIED == 'true'
run: |
git push --force-with-lease
9 changes: 7 additions & 2 deletions .github/workflows/docker_bin.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,10 +49,14 @@ jobs:
id: set_short_sha
run: |
echo "short_sha=$(git rev-parse --short HEAD)" >> $GITHUB_OUTPUT
- name: Get CNVERSION
- name: Set CNVersion
id: set_cnversion
run: |
echo "cnversion=$(cat files/docker/node/release-versions/cardano-node-latest.txt)" >> $GITHUB_OUTPUT
if [[ "${{ steps.set_guild_deploy_branch.outputs.guild_deploy_branch }}" == "sanchonet" ]]; then
echo "cnversion=$(cat files/docker/node/release-versions/cardano-node-prerelease.txt)" >> "$GITHUB_OUTPUT"
else
echo "cnversion=$(cat files/docker/node/release-versions/cardano-node-latest.txt)" >> "$GITHUB_OUTPUT"
fi
build_production:
needs: set_environment_vars
if: needs.set_environment_vars.outputs.testing == 'false' && needs.set_environment_vars.outputs.guild_deploy_branch == 'master'
Expand Down Expand Up @@ -137,6 +141,7 @@ jobs:
build-args: |
G_ACCOUNT=${{ needs.set_environment_vars.outputs.g_account }}
GUILD_DEPLOY_BRANCH=${{ needs.set_environment_vars.outputs.guild_deploy_branch }}
CARDANO_NODE_VERSION=${{ needs.set_environment_vars.outputs.cnversion }}
- name: Add summary details
if: always()
run: |
Expand Down
2 changes: 1 addition & 1 deletion docs/Build/node-cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ POOL_NAME="GUILD"

#### Start the node

To test starting the node in interactive mode, we will make use of pre-built script `cnode.sh`. This script automatically determines whether to start the node as a relay or block producer (if the required pool keys are present in the `$CNODE_HOME/priv/pool/<POOL_NAME>` as mentioned above). The script contains a user-defined variable `CPU_CORES` which determines the number of CPU cores the node will use upon start-up:
To test starting the node in interactive mode, we will make use of pre-built script `cnode.sh`. This script automatically determines whether to start the node as a relay or block producer (if the required pool keys are present in the `$CNODE_HOME/priv/pool/<POOL_NAME>` as mentioned above). If the `<MITHRIL_DOWNLOAD>` variable is set to 'Y' it will download the latest snapshot from a Mithril aggregator to speed up the blockchain synchronization. The script contains a user-defined variable `CPU_CORES` which determines the number of CPU cores the node will use upon start-up:

```bash
######################################
Expand Down
41 changes: 34 additions & 7 deletions docs/Scripts/mithril-client.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,37 @@
`mithril-client.sh` is a script to manage the Mithril client, a tool used to set up the Mithril client environment and manage downloading Mithril snapshots and stake distributions. The main features include:

- **environment** - Creates a new `mithril.env` file with all the necessary environment variables for the Mithril client.
- **snapshot** - Download, list all or show a specific available Mithril snapshot.
- **cardano-db** - Download, list all or show a specific available Mithril snapshot.
- **stake-distribution** - Download or list available Mithril stake distributions.
- **-u** - Skip script update check.

## Usage

```bash
Usage: bash [-u] <command> <subcommand> [<sub arg>]
A script to run Cardano Mithril Client

-u Skip script update check overriding UPDATE_CHECK value in env (must be first argument to script)

Commands:
environment Manage mithril environment file
setup Setup mithril environment file
override Override default variable in the mithril environment file
update Update mithril environment file
cardano-db Interact with Cardano DB
download Download Cardano DB from Mithril snapshot
snapshot Interact with Mithril snapshots
list List available Mithril snapshots
json List availble Mithril snapshots in JSON format
show Show details of a Mithril snapshot
json Show details of a Mithril snapshot in JSON format
stake-distribution Interact with Mithril stake distributions
download Download latest stake distribution
list List available stake distributions
json Output latest Mithril snapshot in JSON format

```

## Preparing a Relay or Block Producer Node

To prepare a relay or block producer node, you should follow these steps:
Expand All @@ -18,7 +45,7 @@ To prepare a relay or block producer node, you should follow these steps:
2. **Download the latest Mithril snapshot:** Once the environment file is set up, you can download the latest Mithril snapshot by running the script with the `snapshot download` command. This snapshot contains the latest state of the Cardano blockchain db from a Mithril Aggregator.

```bash
./mithril-client.sh snapshot download
./mithril-client.sh cardano-db download
```

## Investigating Available Snapshots
Expand All @@ -28,16 +55,16 @@ You can investigate the available snapshots by using the `snapshot list` and `sn
- **List all available Mithril snapshots:** You can list all available Mithril snapshots by running the script with the `snapshot list` command. Add `json` at the end to get the output in JSON format.

```bash
./mithril-client.sh snapshot list
./mithril-client.sh snapshot list json
./mithril-client.sh cardano-dbsnapshot list
./mithril-client.sh cardano-dbsnapshot list json
```

- **Show details of a specific Mithril snapshot:** You can show details of a specific Mithril snapshot by running the script with the `snapshot show <DIGEST>` command, where `<DIGEST>` is the digest of the snapshot. Add `json` at the end to get the output in JSON format.

```bash
./mithril-client.sh snapshot show <DIGEST>
./mithril-client.sh snapshot show <DIGEST> json
./mithril-client.sh snapshot show json <DIGEST>
./mithril-client.sh cardano-dbsnapshot show <DIGEST>
./mithril-client.sh cardano-dbsnapshot show <DIGEST> json
./mithril-client.sh cardano-dbsnapshot show json <DIGEST>
```

## Managing Stake Distributions
Expand Down
11 changes: 6 additions & 5 deletions docs/Scripts/mithril-relay.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,13 @@ loadbalancer. It provides functionalities such as:
## Usage

```bash
Usage: mithril-relay.sh [-d] [-l]
bash [-d] [-l] [-u] [-h]
A script to setup Cardano Mithril relays

Options:
-d Install squid and configure as a relay
-l Install nginx and configure as a load balancer
-h Show this help text
-d Install squid and configure as a relay
-l Install nginx and configure as a load balancer
-u Skip update check
-h Show this help text
```

# Description
Expand Down
17 changes: 11 additions & 6 deletions docs/Scripts/mithril-signer.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,17 @@ environment file to contain variables specific to the Mithril Signer.
## Usage

```bash
Usage: mithril-signer.sh [-d] [-u]

Options:
-d Deploy mithril-signer as a systemd service
-u Update mithril environment file
-h Show this help text
Usage: bash [-d] [-D] [-e] [-k] [-r] [-s] [-u] [-h]
A script to setup, run and verify Cardano Mithril Signer

-d Deploy mithril-signer as a systemd service
-D Run mithril-signer as a daemon
-e Update mithril environment file
-k Stop signer using SIGINT
-r Verify signer registration
-s Verify signer signature
-u Skip update check
-h Show this help text
```

# Description
Expand Down
Loading

0 comments on commit 4c37b1d

Please sign in to comment.