Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tunnel-server: add basic deployment guide #405

Merged
merged 2 commits into from
Feb 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions tunnel-server/deployment/k8s/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
/tls.crt
/tls.key
/ssh_host_key
/cookie_secret
/config.env
107 changes: 107 additions & 0 deletions tunnel-server/deployment/k8s/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
# Deploying your own instance of the Tunnel Server

This directory contains an example deployment of the Tunnel Server on Kubernetes.

Note that this is an advanced task which requires some networking and Kubernetes know-how.

## Why

Deploying a private instance of the Tunnel Server allows for fine-grained control of:

- The URLs created for preview environments: e.g, use a custom domain.
- Geolocation of the server: reduced distance to environments can result better network performance.
- Security and privacy: deploy everything in your VPC, no traffic to 3rd parties.

## Requirements

- A Kubernetes cluster
- A TLS certificate for your domain
- `kubectl` and `kustomize`

## Overview

The Tunnel Server natively listens on three ports:
- A SSH port which accepts tunneling SSH connections from environments
- A HTTP port which accepts requests from clients (browsers, etc)
- A TLS port which directs incoming connections to either the SSH server or the HTTP server according to the [SNI server name](https://en.wikipedia.org/wiki/Server_Name_Indication).

The `SSH_HOSTNAMES` env var determines which hostnames are directed to the SSH server and the rest are directed to the HTTP server. If the env var is not specified, the hostname of the `BASE_URL` is used.

In this deployment scheme, the TLS port is exposed using a [`LoadBalancer-type K8S Service`](https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer). The base hostname `yourdomain.example` is used for tunneling SSH connections and any hostname on the subdomain `*.yourdomain.example` will be used for client requests.

## Instructions

### 1. Setup the domain and the TLS certificate

Make sure the certificate is for both the base domain and the wildcard subdomain, e.g, `yourdomain.example` and `*.yourdomain.example`.

Put the cert and key (in PEM format) in the files `tls.crt` and `tls.key`

Copy `config.env.example` to `config.env` and set your domain in the `BASE_URL` variable.

### 2. Generate a cookie secret

The cookie secret is a simple text-based secret (like a password) in a file.

```bash
LC_ALL=C tr -dc A-Za-z0-9 </dev/urandom | head -c 40 > cookie_secret
```

### 3. Generate a SSH host key

```bash
ssh-keygen -t ed25519 -N "" -f ssh_host_key
```

### 4. Generate and deploy the configuration

Review the generated configuration:

```bash
kustomize build .
```

To deploy to the K8S cluster:

```bash
kustomize build . | kubectl apply -f -
```

Make sure the two deployments `tunnel-server` and `tunnel-server-stunnel` exist and that their pods are running.

### 5. Test the SSH endpoint

This requires OpenSSH and a recent-enough OpenSSL CLI with support for the `-quiet` option.

To test the SSH endpoint (replace `$MY_DOMAIN` with your domain):

```bash
MY_DOMAIN=yourdomain.example

# NOTE: for EKS, replace .ip with .hostname below
EXTERNAL_IP=$(kubectl get service tunnel-server -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

ssh -nT -o "ProxyCommand openssl s_client -quiet -verify_quiet -servername $MY_DOMAIN -connect %h:%p" -p 443 foo@$EXTERNAL_IP hello
```

### 6. Create DNS records for the `tunnel-server` Service external IP or hostname

Create two DNS records: `*.yourdomain.example` and `yourdomain.example`, both pointing to the external IP of the `tunnel-server` service.

The address is not guaranteed to be static. According to your Kubernetes provider, there could be multiple ways to define a DNS entry for it. Here are some guides:

- Amazon AWS: [EKS](https://docs.aws.amazon.com/eks/latest/userguide/network-load-balancing.html)
- Google Cloud: [GKE](https://cloud.google.com/kubernetes-engine/docs/concepts/service-load-balancer)
- Azure: [AKS](https://learn.microsoft.com/en-us/azure/aks/load-balancer-standard)

### 7. Use your Tunnel Server instance with the Preevy CLI

The `up` and `urls` commands accept a `-t` flag which can be used to set the Tunnel Server URL. Specify `ssh+tls://yourdomain.example` to use your instance.

### 8. Cleanup: Delete the deployed objects

To delete the deployed Tunnel Server and associated Kubernetes objects, run:

```bash
kustomize build . | kubectl delete -f -
```
1 change: 1 addition & 0 deletions tunnel-server/deployment/k8s/config.example.env
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
BASE_URL=https://local.livecycle.run
23 changes: 23 additions & 0 deletions tunnel-server/deployment/k8s/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- tunnel-server.yaml
secretGenerator:
- files:
- tls.crt
- tls.key
name: tunnel-server-tls
type: kubernetes.io/tls
- files:
- ssh_host_key
name: tunnel-server-ssh
type: Opaque
- files: [cookie_secret]
name: tunnel-server-cookies
type: Opaque
configMapGenerator:
- name: tunnel-server-config
envs: [config.env]
images:
- name: ghcr.io/livecycle/preevy/tunnel-server
newTag: main-8a9d527
100 changes: 100 additions & 0 deletions tunnel-server/deployment/k8s/tunnel-server.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: tunnel-server
spec:
replicas: 1
selector:
matchLabels:
app: tunnel-server
template:
metadata:
labels:
app: tunnel-server
spec:
volumes:
- name: ssh
secret:
defaultMode: 400
secretName: tunnel-server-ssh
- name: tls-cert
secret:
defaultMode: 400
secretName: tunnel-server-tls
containers:
- env:
- name: COOKIE_SECRETS
valueFrom:
secretKeyRef:
name: tunnel-server-cookies
key: cookie_secret
- name: SSH_HOST_KEY_PATH
value: /etc/livecycle-ssh/ssh_host_key
- name: BASE_URL
valueFrom:
configMapKeyRef:
name: tunnel-server-config
key: BASE_URL
- name: DEBUG
value: "1"
- name: NODE_ENV
value: production
image: ghcr.io/livecycle/preevy/tunnel-server:main
imagePullPolicy: IfNotPresent
name: tunnel-server
ports:
- containerPort: 8888
name: metrics
protocol: TCP
- containerPort: 2222
name: ssh
protocol: TCP
- containerPort: 3000
name: http
protocol: TCP
- containerPort: 8443
name: tls
protocol: TCP
resources:
limits:
cpu: "1"
ephemeral-storage: 1Gi
memory: 2Gi
requests:
cpu: "1"
ephemeral-storage: 1Gi
memory: 2Gi
securityContext:
capabilities:
drop:
- NET_RAW
volumeMounts:
- mountPath: /etc/livecycle-ssh
name: ssh
readOnly: true
- mountPath: /app/tls/cert.pem
name: tls-cert
readOnly: true
subPath: tls.crt
- mountPath: /app/tls/key.pem
name: tls-cert
readOnly: true
subPath: tls.key
---
apiVersion: v1
kind: Service
metadata:
name: tunnel-server
spec:
type: LoadBalancer
ports:
- name: tls
port: 443
protocol: TCP
targetPort: 8443
- name: metrics
port: 8888
protocol: TCP
targetPort: 8888
selector:
app: tunnel-server
Loading