From 56ac505dc1787b1d474f85b7aa8ee1f1d2eed08d Mon Sep 17 00:00:00 2001 From: inoc603 Date: Fri, 28 Jun 2019 09:37:51 +0800 Subject: [PATCH] docs: Add guide for using dfdaemon as http proxy for docker daemon Signed-off-by: inoc603 --- docs/user_guide/docker_proxy.md | 74 ++++++++++++++++++++++++++ docs/user_guide/proxy.md | 94 +++++++++++---------------------- 2 files changed, 104 insertions(+), 64 deletions(-) create mode 100644 docs/user_guide/docker_proxy.md diff --git a/docs/user_guide/docker_proxy.md b/docs/user_guide/docker_proxy.md new file mode 100644 index 000000000..0424c7e22 --- /dev/null +++ b/docs/user_guide/docker_proxy.md @@ -0,0 +1,74 @@ +# Use Dfdaemon as HTTP Proxy for Docker Daemon + +Currently, docker doesn't support private registries with `registry-mirrors`, +in order to do so, we need to use HTTP proxy for docker daemon. + +To use dfdaemon as HTTP proxy, first you need to add a proxy rule in +`/etc/dragonfly/dfdaemon.yml`: + +```yaml +proxies: +- regx: blobs/sha256:.* +``` + +This will proxy all requests for image layers with dfget. + +By default, only HTTP requests are proxied with dfget. If you're using an HTTPS +enabled private registry, you need to add the following HTTPS configuration to +`/etc/dragonfly/dfdaemon.yml`: + +```yaml +hijack_https: + cert: df.crt + key: df.key + hosts: + - regx: your.private.registry +``` + +If your registry uses a self-signed certificate, you can either choose to +ignore the certificate error with: + +```yaml + hosts: + - regx: your.private.registry + insecure: true +``` + +Or provide a certificate with: + +```yaml + hosts: + - regx: your.private.registry + certs: ["server.crt"] +``` + +You can get the certificate of your server with: + +``` +openssl x509 -in <(openssl s_client -showcerts -servername xxx -connect xxx:443 -prexit 2>/dev/null) +``` + +Add your private registry to `insecure-registries` in +`/etc/docker/daemon.json`, in order to ignore the certificate error: + +```json +{ + "insecure-registries": ["your.private.registry"] +} +``` + +Set dfdaemon as HTTP_PROXY and HTTPS_PROXY for docker daemon in +`/etc/systemd/system/docker.service.d/http-proxy.conf`: + +``` +[Service] +Environment="HTTP_PROXY=http://127.0.0.1:65001" +Environment="HTTPS_PROXY=http://127.0.0.1:65001" +``` + +Read [Control Docker with systemd](https://docs.docker.com/config/daemon/systemd/#httphttps-proxy) for more details. If you're not running docker daemon with systemd, you need to set the environment variables manually. + +Finally you can restart docker daemon and pull images as you normally would. + +More details on dfdaemon's proxy configuration can be found +[here](proxy.md). diff --git a/docs/user_guide/proxy.md b/docs/user_guide/proxy.md index d4af66666..d1a9df3cd 100644 --- a/docs/user_guide/proxy.md +++ b/docs/user_guide/proxy.md @@ -1,84 +1,50 @@ # Use dfdaemon as an HTTP proxy -Dfdaemon can be used as an HTTP proxy to speed up image pulling from any registry -as well as general HTTP downloads. +## Prerequisites -Please first ensure that you know how to install and run [supernode](install_server.md) -and [dfdaemon](install_client.md). +You need to first install and configure [supernode](install_server.md) and [dfdaemon](install_client.md). -**HTTPS support is currently very limited. All HTTPS request will be tunneled -directly, without dfget.** +## Proxy Configuration -## Proxy rule configuration - -Proxy rules are configured in `/etc/dragonfly/dfdaemon.yml`. For performance -reason, dfdaemon will handle a request with the the first matching rule. +Proxy rules are configured in `/etc/dragonfly/dfdaemon.yml`. ```yaml +# Requests that match the regular expressions will be proxied with dfget, +# otherwise they'll be proxied directly. Requests will be handled by the first +# matching rule. proxies: -# proxy requests directly, without dfget + # proxy all http image layer download requests with dfget +- regx: blobs/sha256:.* + # proxy requests directly, without dfget - regx: no-proxy-reg direct: true -# proxy all http image layer download requests with dfget -- regx: blobs/sha256:.* -# change http requests to some-registry to https, and proxy them with dfget + # change http requests to some-registry to https, and proxy them with dfget - regx: some-registry/ use_https: true -``` - -## Download images - -Add the following content to `/etc/dragonfly/dfdaemon.yml`. - -```yaml -proxies: -# proxy all http image layer download requests with dfget -- regx: blobs/sha256:.* -``` - -Set HTTP_PROXY for docker daemon in `/etc/systemd/system/docker.service.d/http-proxy.conf`. -`65001` is the default proxy port for dfdaemon. - -``` -[Service] -Environment="HTTP_PROXY=http://127.0.0.1:65001" -``` -Set your registry as insecure in `/etc/docker/daemon.json` - -```json -{ - "insecure-registries": [ "your.registry" ] -} +# If an https request's host matches any of the hijacking rules, dfdaemon will +# decrypt the request with given key pair and proxy it with the proxy rules. +hijack_https: + cert: df.crt + key: df.key + hosts: + # match hosts by regular expressions. certificate will be validated normally + - regx: host-1 + # ignore certificate errors + - regx: host-2 + insecure: true + # use the given certificate for validation + - regx: host-3 + certs: ["server.crt"] ``` -Start dfdaemon and restart docker daemon. +## Usage -``` -systemctl restart docker -``` +You can use dfdaemon like any other HTTP proxy. For example on linux and +macOS, you can use the `HTTP_PROXY` or `HTTPS_PROXY` environment variables. -Pull an image to see if it works. For registries that are not configured -insecure, you can still pull image from it, but dfdaemon will not be able to -speed up your downloads with dfget. +## Get the Certificate of Your Server ``` -docker pull nginx -docker pull your.registry/team/repo:tag +openssl x509 -in <(openssl s_client -showcerts -servername xxx -connect xxx:443 -prexit 2>/dev/null) ``` - -Then you can [check if your image is downloaded with dfget](../../FAQ.md#how-to-check-if-block-piece-is-distributed-among-dfgets-nodes). - -## Download files - -You can simply use `HTTP_PROXY` environment variable to let dfdaemon download -requests that match the proxy rules. This works for any program that -respects the `HTTP_PROXY` environment variable. - -``` -HTTP_PROXY=http://127.0.0.1:65001 curl http://github.com -``` - -HTTPS requests and requests that are not matched, will be proxied directly, -and dragonfly is not able to speed up them. -