Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Kubernetes/Monogon support for test deployments #33574

Closed
wants to merge 411 commits into from

Conversation

gregcusack
Copy link
Contributor

@gregcusack gregcusack commented Oct 6, 2023

Problem

  1. GCE is expensive.
  2. We need to be able to run test deployments with 1000s of validators. We have Monogon servers, so let's put Solana validators in kubernetes pods and spin up a bunch at a time.

Summary of Changes

Current functionality includes:

  • Build a validator from either local git hash or pull release version from solana releases
  • Build docker image and push to docker registry
  • Generate faucet, bootstrap, and other validator accounts
  • Generate genesis with bootstrap account (including spl-genesis-args).
  • Pull docker images, build kubernetes controller and deploy 1 validator per pod. 1 pod per replicaset. 1 service per replicaset
  • Mounts genesis (as configmap) and respective accounts (as secrets) into each validator pod
  • Bootstrap validator starts up
  • Validators wait for bootstrap to start before deploying and connecting to bootstrap.
  • Can exec into bootstrap pod and verify validators are all connected with solana -ul gossip and solana -ul validators
  • Metrics Enabled
  • Client support

See solana/k8s-cluster/README.md for a tutorial on how to run for yourself.

WIP:

  • adding in additional validator configuration flags. that is coming.....

TODO:

  • blockstreamer
    Fixes #

@gregcusack gregcusack force-pushed the k8s-cluster-official branch from 1df202b to f5cdd2d Compare January 20, 2024 02:11
@github-actions github-actions bot added stale [bot only] Added to stale content; results in auto-close after a week. and removed stale [bot only] Added to stale content; results in auto-close after a week. labels Feb 23, 2024
@willhickey
Copy link
Contributor

This repository is no longer in use. Please re-open this pull request in the agave repo: https://github.com/anza-xyz/agave

@willhickey willhickey closed this Mar 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
work in progress This isn't quite right yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants