Skip to content

Commit

Permalink
fix remote kube vpn
Browse files Browse the repository at this point in the history
  • Loading branch information
niqdev committed Oct 21, 2023
1 parent 1e3a135 commit 76a7613
Show file tree
Hide file tree
Showing 9 changed files with 174 additions and 73 deletions.
58 changes: 40 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,13 +69,13 @@ hckctl box start vulnerable/owasp-juice-shop

> TODO video
Access your target from a managed [`lab`](https://github.com/hckops/megalopolis/tree/main/lab)
Access your target from a managed [`lab`](https://github.com/hckops/megalopolis/tree/main/lab) to
* tunnel multiple vpn connections through a high-available ssh proxy
* expose public endpoints
* pre-mount saved `dumps` (git, s3)
* load secrets from a vault
* save/restore workdir snapshots
* deploy custom labs
* deploy private templates
```bash
hckctl lab ctf-linux
```
Expand All @@ -100,6 +100,9 @@ hckctl task nmap --command full --input address=127.0.0.1 --input port=80
# invokes it with custom arguments
hckctl task nuclei --inline -- -u https://example.com

go run internal/main.go task nmap --network-vpn htb --provider kube --inline -- nmap 10.10.10.3 -sC -sV
go run internal/main.go task nmap --network-vpn htb --provider kube --command full --input address=10.10.10.3

# monitors the logs
tail -F ${HOME}/.local/state/hck/task/log/task-*
```
Expand Down Expand Up @@ -207,45 +210,55 @@ network:

Follow the official [instructions](https://docs.docker.com/engine/install) to install Docker Engine. The fastest way to get started is with the [convenience script](https://get.docker.com)
```bash
# download and run script
# downloads and runs script
curl -fsSL https://get.docker.com -o get-docker.sh
./sudo sh get-docker.sh
```

Recommended tool to watch the container [lazydocker](https://github.com/jesseduffield/lazydocker)

### Kubernetes

Use [minikube](https://minikube.sigs.k8s.io) or [kind](https://kind.sigs.k8s.io) to setup a local cluster
If you are looking for a simple and cheap way to get started with a remote cluster consider using [kube-template](https://github.com/hckops/kube-template) on [DigitalOcean](https://www.digitalocean.com/products/kubernetes)
```bash
provider:
kube:
# by default uses "~/.kube/config"
configPath: "/PATH/TO/kube-template/clusters/do-template-kubeconfig.yaml"
```

Use [minikube](https://minikube.sigs.k8s.io), [kind](https://kind.sigs.k8s.io) or [k3s](https://k3s.io) to setup a local cluster
```bash
provider:
kube:
# absolute path, empty by default uses "${HOME}/.kube/config"
configPath: ""
namespace: hckops
```

Make sure you disable IPv6 in your cluster to use the `--network-vpn` flag
Make sure you disable IPv6 in your *local* cluster to use the `--network-vpn` flag and set `--embed-certs` if you need to access the cluster using the dev tools
```bash
# set --embed-certs to run "hckops/kube-base"
minikube start --embed-certs --extra-config="kubelet.allowed-unsafe-sysctls=net.ipv6.conf.all.disable_ipv6"
# starts local cluster
minikube start --embed-certs \
--extra-config="kubelet.allowed-unsafe-sysctls=net.ipv6.conf.all.disable_ipv6"

# runs with temporary privileges to connect to a vpn
env HCK_CONFIG_NETWORK.PRIVILEGED=true hckctl box alpine --provider kube --network-vpn htb

network:
# default is false, required only for local clusters
privileged: true
```

Useful dev tools
Useful dev tools, see [`hckops/kube-base`](https://github.com/hckops/actions/blob/main/docker/Dockerfile.base)
```bash
# starts tmp container
docker run --rm --name hck-tmp --network host -it \
docker run --rm --name hck-tmp-local --network host -it \
-v ${HOME}/.kube/config:/root/.kube/config hckops/kube-base

# watches pods
kubectl klock -n hckops pods
```

If you are looking for a simple way to get started with a remote cluster consider using [kube-template](https://github.com/hckops/kube-template)
```bash
provider:
kube:
configPath: "~/PATH/TO/kube-template/clusters/do-template-kubeconfig.yaml"
```

### Cloud

Access to the platform is limited and in ***private preview***. If you are interested, please leave a comment or a :thumbsup: to this [issue](https://github.com/hckops/hckctl/issues/104) and we'll reach out with more details
Expand All @@ -272,6 +285,9 @@ HCKCTL_VERSION=0.12.0
curl -sSL https://github.com/hckops/hckctl/releases/latest/download/hckctl-${HCKCTL_VERSION}-linux-x86_64.tar.gz | \
sudo tar -xzf - -C /usr/local/bin

# verify
hckctl version

# uninstall
sudo rm /usr/local/bin/hckctl
```
Expand Down Expand Up @@ -318,10 +334,14 @@ Credit should go to all the authors and maintainers for their open source tools,

<!--
box remote kube: after killing vnc/portforward
E1020 19:55:12.436966 149063 portforward.go:381] error copying from remote stream to local connection: readfrom tcp4 127.0.0.1:5900->127.0.0.1:54768: write tcp4 127.0.0.1:5900->127.0.0.1:54768: write: broken pipe
>>> remove TryHackMe demo from readme
* test all catalog
* discord + social links
* replace task/htb example with thm
* verify/support kube config relative path + remote cluster
* update cloud pkg
* update platform prs
* verify network connectivity between boxes/tasks i.e. kube.svc
Expand Down Expand Up @@ -434,5 +454,7 @@ TODO
- verify release workflow should depend on ci workflow
* prompt
- https://github.com/snwfdhmp/awesome-gpt-prompt-engineering
* megalopolis
- (docker) https://github.com/edoardottt/scilla
-->
5 changes: 4 additions & 1 deletion internal/command/config/model.go
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,8 @@ func (c *CloudConfig) ToCloudOptions(version string) *commonModel.CloudOptions {
}

type NetworkConfig struct {
Vpn []VpnConfig `yaml:"vpn"`
Privileged bool `yaml:"privileged"`
Vpn []VpnConfig `yaml:"vpn"`
}

type VpnConfig struct {
Expand All @@ -111,6 +112,7 @@ func (c *NetworkConfig) VpnNetworks() map[string]commonModel.NetworkVpnInfo {
Name: network.Name,
LocalPath: network.Path,
ConfigValue: configFile,
Privileged: c.Privileged,
}
}
}
Expand Down Expand Up @@ -186,6 +188,7 @@ func newConfig(opts *configOptions) *ConfigV1 {
},
},
Network: NetworkConfig{
Privileged: false,
Vpn: []VpnConfig{
{Name: common.DefaultVpnName, Path: "/path/to/client.ovpn"},
},
Expand Down
1 change: 1 addition & 0 deletions internal/command/config/model_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ func TestNewConfig(t *testing.T) {
},
},
Network: NetworkConfig{
Privileged: false,
Vpn: []VpnConfig{
{Name: "default", Path: "/path/to/client.ovpn"},
},
Expand Down
5 changes: 3 additions & 2 deletions pkg/common/docker/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -114,8 +114,9 @@ func (common *DockerCommonClient) SidecarVpnInject(opts *commonModel.SidecarVpnI
// sidecarName
containerName := buildSidecarVpnName(opts.Name)

// constants
imageName := commonModel.SidecarVpnImageName
// ignore opts.NetworkVpn.Privileged locally
imageName := commonModel.SidecarVpnPrivilegedImageName

// base directory "/usr/share" must exist
vpnConfigPath := "/usr/share/client.ovpn"

Expand Down
79 changes: 46 additions & 33 deletions pkg/common/kubernetes/builder.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,29 @@ func buildSidecarVpnSecret(namespace, podName, secretValue string) *corev1.Secre
}
}

func buildSidecarVpnContainer() corev1.Container {
func buildSidecarVpnContainer(privileged bool) corev1.Container {

// default
imageName := commonModel.SidecarVpnImageName
volumeMounts := []corev1.VolumeMount{
{
Name: sidecarVpnSecretVolume,
MountPath: secretBasePath,
ReadOnly: true,
},
}
if privileged {
imageName = commonModel.SidecarVpnPrivilegedImageName
volumeMounts = append(volumeMounts, corev1.VolumeMount{
Name: sidecarVpnTunnelVolume,
MountPath: sidecarVpnTunnelPath,
ReadOnly: true,
})
}

return corev1.Container{
Name: fmt.Sprintf("%svpn", commonModel.SidecarPrefixName),
Image: commonModel.SidecarVpnImageName,
Image: imageName,
ImagePullPolicy: corev1.PullIfNotPresent,
Env: []corev1.EnvVar{
{Name: "OPENVPN_CONFIG", Value: filepath.Join(secretBasePath, sidecarVpnSecretPath)},
Expand All @@ -50,31 +69,12 @@ func buildSidecarVpnContainer() corev1.Container {
Add: []corev1.Capability{"NET_ADMIN"},
},
},
VolumeMounts: []corev1.VolumeMount{
{
Name: sidecarVpnTunnelVolume,
MountPath: sidecarVpnTunnelPath,
ReadOnly: true,
},
{
Name: sidecarVpnSecretVolume,
MountPath: secretBasePath,
ReadOnly: true,
},
},
VolumeMounts: volumeMounts,
}
}

func buildSidecarVpnVolumes(podName string) []corev1.Volume {
return []corev1.Volume{
{
Name: sidecarVpnTunnelVolume,
VolumeSource: corev1.VolumeSource{
HostPath: &corev1.HostPathVolumeSource{
Path: sidecarVpnTunnelPath,
},
},
},
func buildSidecarVpnVolumes(podName string, privileged bool) []corev1.Volume {
volumes := []corev1.Volume{
{
Name: sidecarVpnSecretVolume,
VolumeSource: corev1.VolumeSource{
Expand All @@ -87,27 +87,40 @@ func buildSidecarVpnVolumes(podName string) []corev1.Volume {
},
},
}
if privileged {
volumes = append(volumes, corev1.Volume{
Name: sidecarVpnTunnelVolume,
VolumeSource: corev1.VolumeSource{
HostPath: &corev1.HostPathVolumeSource{
Path: sidecarVpnTunnelPath,
},
},
})
}
return volumes
}

func boolPtr(b bool) *bool { return &b }

func injectSidecarVpn(podSpec *corev1.PodSpec, podName string) {
func injectSidecarVpn(podSpec *corev1.PodSpec, podName string, privileged bool) {

// https://kubernetes.io/docs/tasks/configure-pod-container/share-process-namespace
//podSpec.ShareProcessNamespace = boolPtr(true)

// disable ipv6, see https://kubernetes.io/docs/tasks/administer-cluster/sysctl-cluster
podSpec.SecurityContext = &corev1.PodSecurityContext{
Sysctls: []corev1.Sysctl{
{Name: "net.ipv6.conf.all.disable_ipv6", Value: "0"},
},
if privileged {
// disable ipv6, see https://kubernetes.io/docs/tasks/administer-cluster/sysctl-cluster
podSpec.SecurityContext = &corev1.PodSecurityContext{
Sysctls: []corev1.Sysctl{
{Name: "net.ipv6.conf.all.disable_ipv6", Value: "0"},
},
}
}

// inject containers
podSpec.Containers = append(
// order matters
[]corev1.Container{
buildSidecarVpnContainer(),
buildSidecarVpnContainer(privileged),
// add fake sleep to allow sidecar-vpn to connect properly before starting the main container
{
Name: fmt.Sprintf("%ssleep", commonModel.SidecarPrefixName),
Expand All @@ -127,8 +140,8 @@ func injectSidecarVpn(podSpec *corev1.PodSpec, podName string) {

// inject volumes
podSpec.Volumes = append(
podSpec.Volumes, // current volumes
buildSidecarVpnVolumes(podName)..., // join slices
podSpec.Volumes, // current volumes
buildSidecarVpnVolumes(podName, privileged)..., // join slices
)
}

Expand Down
Loading

0 comments on commit 76a7613

Please sign in to comment.