Skip to content
This repository has been archived by the owner on Aug 28, 2024. It is now read-only.

Commit

Permalink
Update README.md (#262)
Browse files Browse the repository at this point in the history
See you over at KubeAI!
  • Loading branch information
nstogner authored Aug 28, 2024
1 parent f5e92b9 commit 7352fb0
Showing 1 changed file with 2 additions and 122 deletions.
124 changes: 2 additions & 122 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -1,125 +1,5 @@
# NO LONGER ACTIVE

<p align="center">
<a href="https://discord.gg/JeXhcmjZVm">
<img alt="discord-invite" src="https://dcbadge.vercel.app/api/server/JeXhcmjZVm?style=flat">
</a>
</p>
We are focusing all of our effort on KubeAI, a platform focused on making it simple to operationalize AI on Kubernetes.

NOTICE: Changes coming! runbooks.git (previously substratus.git) will be refactored to focus on Notebooks on K8s.

🎵 Fine-tune LLM models with no/low code
📔 Provide a Colab style seamless Notebook experience
☁️ Provide a unified ML platform across clouds
⬆️ Easy to install with minimal dependencies

Looking for serving?
🚀 [substratusai/lingo](https://github.com/substratusai/lingo): Serve popular OSS LLM models in minutes on CPUs or GPUs

Support the project by adding a star on GitHub! ❤️

## Quickstart

Create a local Kubernetes cluster using Kind.

[embedmd]:# (../install/kind/up.sh bash /kind.*/ $)
```bash
kind create cluster --name substratus --config - <<EOF
apiVersion: kind.x-k8s.io/v1alpha4
kind: Cluster
nodes:
- role: control-plane
extraPortMappings:
- containerPort: 30080
hostPort: 30080
EOF
```

Install Substratus.

```bash
kubectl apply -f https://raw.githubusercontent.com/substratusai/substratus/main/install/kind/manifests.yaml
```

Import a small Open Source LLM.

```bash
kubectl apply -f https://raw.githubusercontent.com/substratusai/substratus/main/examples/facebook-opt-125m/base-model.yaml
```

[embedmd]:# (../examples/facebook-opt-125m/base-model.yaml yaml)
```yaml
apiVersion: substratus.ai/v1
kind: Model
metadata:
namespace: default
name: facebook-opt-125m
spec:
image: substratusai/model-loader-huggingface
params:
name: facebook/opt-125m
```
Serve the LLM.
```bash
kubectl apply -f https://raw.githubusercontent.com/substratusai/substratus/main/examples/facebook-opt-125m/base-server.yaml
```

[embedmd]:# (../examples/facebook-opt-125m/base-server.yaml yaml)
```yaml
apiVersion: substratus.ai/v1
kind: Server
metadata:
name: facebook-opt-125m
spec:
image: substratusai/model-server-basaran
model:
name: facebook-opt-125m
```
Checkout the progress of the Model and the Server.
```bash
kubectl get ai
```

When they report a `Ready` status, start a port-forward.

```bash
kubectl port-forward service/facebook-opt-125m-server 8080:8080
```

Open your browser to [http://localhost:8080/](http://localhost:8080/) or curl the LLM's API.

*PS: Because of the small size of this particular LLM, expect comically bad answers to your prompts.*

```bash
curl http://localhost:8080/v1/completions \
-H "Content-Type: application/json" \
-d '{ \
"model": "facebook-opt-125m", \
"prompt": "Who was the first president of the United States? ", \
"max_tokens": 10\
}'
```

Delete the local cluster.

[embedmd]:# (../install/kind/down.sh bash /kind.*/ $)
```bash
kind delete cluster --name substratus
```

If you want to try out a more capable LLM, running on substantial hardware, try [Kind with
GPU support](https://www.substratus.ai/docs/quickstart/local-kind?kind-mode=gpu),
or try [deploying Substratus in GKE](https://www.substratus.ai/docs/quickstart/gcp).

## Docs
* [Overview](https://www.substratus.ai/docs/overview)
* [Architecture](https://www.substratus.ai/docs/architecture)

## Creators
Feel free to contact any of us:
* [Nick Stogner](https://www.linkedin.com/in/nstogner/)
* [Sam Stoelinga](https://www.linkedin.com/in/samstoelinga/)
Instead see: https://github.com/substratusai/kubeai

0 comments on commit 7352fb0

Please sign in to comment.