feat-273: TLS Flags #303

wcrum · 2024-08-22T16:03:58Z

Please check below, if the PR fulfills these requirements:

Commit(s) and code follow the repositories guidelines.
Test(s) have been added or updated to support these change(s).
Doc(s) have been added or updated to support these change(s).
- docs: feat-273 tls flags hauler-docs#28

Associated Links:

[feature] TLS Support for Serve #273

Types of Changes:

Moved DefaultStoreName to pkg/consts/consts.go to standard constant locations.
Added ListenAndServeTLS(string, string) to Server interface to support TLS.
Added --tls-cert and --tls-key to hauler store serve [fileserver|registry].
Moved all flags to internal/flags/* to prevent loop import cycle.
- A lot of the passed flags get put into a separate structure.
- https://github.com/hauler-dev/hauler/pull/303/files#diff-1cc50886376840b9fb084bee4da1649510c419ad0b612b23f95f7ed2001be6a2L14-R21
Standardized interface function name from instances of AddArgs to AddFlags bbad607

Proposed Changes:

Add additional parameters exposed to the user to support running the fileserver and registry with TLS.

Verification/Testing of Changes:

Additional Context:

Pull request 303, my Colorado area code!

Signed-off-by: will <[email protected]>

wcrum · 2024-08-24T02:38:44Z

@zackbradys working on fixing unit tests.

zackbradys · 2024-08-24T02:46:33Z

Thanks @wcrum! It looks like it's due to the PR for --list-repos that was merged after you submitted your changes.

zackbradys · 2024-08-24T09:07:48Z

I made a few small syntax changes and addressed the failing unit test workflow. Feel free to let us know if you disagree with any of the updates/changes!

diff/commit: 9bbbc5b

zackbradys

LGTM pending any feedback!

wcrum · 2024-08-24T09:11:00Z

Looks good to me. Appreciate it! I've been meaning to migrate flags from other commands as well to stick with the changed folder format.

Will open docs PR.

wcrum · 2024-08-24T10:14:36Z

After reviewing again I realized that although adding internal/flags/*.go is nessecary to prevent and separate things, this does not account for flags within login.go and cli.go, would we want to merge both of us those into something in flags as well?

zackbradys · 2024-08-24T11:56:01Z

Thanks for catching those!

Im comfortable finishing the flags migration on this PR. Feel free to add a bit more on this one. Looks like I see a few left over ones in cli.go, login.go, and version.go.

zackbradys · 2024-08-25T20:14:38Z

Testing locally built hauler binary:

zackbradys@Zacks-MacBook-Pro hauler % ./bin/hauler store sync -f testdata/hauler-manifest.yaml
2024-08-25 16:05:34 INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Images] to store
2024-08-25 16:05:34 INF adding 'image' [busybox:latest] to the store
2024-08-25 16:05:38 INF successfully added 'image' [index.docker.io/library/busybox:latest]
2024-08-25 16:05:38 INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Charts] to store
2024-08-25 16:05:38 INF adding 'chart' [rancher] to the store
2024-08-25 16:05:39 INF successfully added 'chart' [hauler/rancher:2.8.2]
2024-08-25 16:05:39 INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Files] to store
2024-08-25 16:05:39 INF adding 'file' [https://get.rke2.io] to the store as [hauler/install.sh:latest]
2024-08-25 16:05:39 INF successfully added 'file' [hauler/install.sh:latest]

zackbradys@Zacks-MacBook-Pro hauler % ./bin/hauler store info
+--------------------------+-------+----------------+----------+----------+
| REFERENCE                | TYPE  | PLATFORM       | # LAYERS | SIZE     |
+--------------------------+-------+----------------+----------+----------+
| hauler/install.sh:latest | file  | -              |        1 | 25.0 kB  |
| hauler/rancher:2.8.2     | chart | -              |        1 | 15.0 kB  |
| library/busybox:latest   | image | linux/386      |        1 | 2.2 MB   |
|                          | image | linux/amd64    |        1 | 2.2 MB   |
|                          | image | linux/arm      |        1 | 1.8 MB   |
|                          | image | linux/arm      |        1 | 949.0 kB |
|                          | image | linux/arm      |        1 | 1.6 MB   |
|                          | image | linux/arm64    |        1 | 1.8 MB   |
|                          | image | linux/mips64le |        1 | 2.1 MB   |
|                          | image | linux/ppc64le  |        1 | 2.5 MB   |
|                          | image | linux/riscv64  |        1 | 895.2 kB |
|                          | image | linux/s390x    |        1 | 1.9 MB   |
+--------------------------+-------+----------------+----------+----------+
|                                                      TOTAL   | 17.8 MB  |
+--------------------------+-------+----------------+----------+----------+

zackbradys@Zacks-MacBook-Pro hauler % ./bin/hauler store serve registry --help
Serve the embedded registry

Usage:
  hauler store serve registry [flags]

Flags:
  -c, --config string      Path to config file, overrides all other flags
      --directory string   Directory to use for backend. Defaults to $PWD/registry (default "registry")
  -h, --help               help for registry
  -p, --port int           Port used to accept incoming connections (default 5000)
      --readonly           Run the registry as readonly (default true)
      --tls-cert string    Location of the TLS Certificate
      --tls-key string     Location of the TLS Key

Global Flags:
      --cache string       (deprecated flag and currently not used)
  -l, --log-level string    (default "info")
  -s, --store string       Location to create store at (default "store")

zackbradys@Zacks-MacBook-Pro hauler % ./bin/hauler store serve registry --port 5001 --readonly --tls-cert testdata/certs/server-cert.crt --tls-key testdata/certs/server-cert.key
2024-08-25 16:06:38 INF library/busybox:latest
2024-08-25 16:06:38 INF 2024/08/25 16:06:38 existing manifest: latest@sha256:9ae97d36d26566ff84e8893c64a6dc4fe8ca6d1144bf5b87b2b85a32def253c7
2024-08-25 16:06:38 INF hauler/rancher:2.8.2
2024-08-25 16:06:38 INF 2024/08/25 16:06:38 existing manifest: 2.8.2@sha256:27e742f51e66e32512509a95523bc9a531ec63f723c730b47685e7678cbc30d3
2024-08-25 16:06:38 INF hauler/install.sh:latest
2024-08-25 16:06:38 INF 2024/08/25 16:06:38 existing manifest: latest@sha256:d50b7c9f77ecb424481ee733421a2a1d154e49dde6a7f56996cd440958819b21
2024-08-25 16:06:38 INF copied artifacts to [127.0.0.1:58804]
2024-08-25 16:06:38 INF starting registry on port [5001]
WARN[0001] No HTTP secret provided - generated random secret. This may cause problems with uploads if multiple registries are behind a load-balancer. To provide a shared secret, fill in http.secret in the configuration file or set the REGISTRY_HTTP_SECRET environment variable.  go.version=go1.23.0 version=v3.0.0+unknown
INFO[0001] redis not configured                          go.version=go1.23.0 version=v3.0.0+unknown
INFO[0001] Starting upload purge in 39m0s                go.version=go1.23.0 version=v3.0.0+unknown
INFO[0001] using inmemory blob descriptor cache          go.version=go1.23.0 version=v3.0.0+unknown
INFO[0001] restricting TLS version to tls1.2 or higher   go.version=go1.23.0 version=v3.0.0+unknown
INFO[0001] restricting TLS cipher suites to: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_AES_128_GCM_SHA256,TLS_CHACHA20_POLY1305_SHA256,TLS_AES_256_GCM_SHA384  go.version=go1.23.0 version=v3.0.0+unknown
INFO[0001] listening on [::]:5001, tls                   go.version=go1.23.0 version=v3.0.0+unknown
127.0.0.1 - - [25/Aug/2024:16:09:01 -0400] "GET / HTTP/2.0" 200 0 "" "curl/8.7.1"
127.0.0.1 - - [25/Aug/2024:16:09:12 -0400] "GET / HTTP/2.0" 200 0 "" "curl/8.7.1"
127.0.0.1 - - [25/Aug/2024:16:09:18 -0400] "GET / HTTP/2.0" 200 0 "" "curl/8.7.1"

Testing certificates with curl:

zackbradys@Zacks-MacBook-Pro certs % curl -v --cacert cacerts.pem https://hauler.dev:5001         
* Host hauler.dev:5001 was resolved.
* IPv6: 2606:4700:3036::6815:1962, 2606:4700:3034::ac43:85fb
* IPv4: 127.0.0.1
*   Trying 127.0.0.1:5001...
* Connected to hauler.dev (127.0.0.1) port 5001
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
*  CAfile: cacerts.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256 / [blank] / UNDEF
* ALPN: server accepted h2
* Server certificate:
*  subject: C=US; ST=VIRGINIA; L=RESTON; O=HAULER; OU=HAULER DEV; CN=hauler.dev
*  start date: Aug 25 20:04:36 2024 GMT
*  expire date: Aug 23 20:04:36 2034 GMT
*  subjectAltName: host "hauler.dev" matched cert's "hauler.dev"
*  issuer: C=US; ST=VIRGINIA; L=RESTON; O=HAULER; OU=HAULER DEV; CN=INTERMEDIARY CERTIFICATE AUTHORITY CERTIFICATE
*  SSL certificate verify ok.
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://hauler.dev:5001/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: hauler.dev:5001]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [user-agent: curl/8.7.1]
* [HTTP/2] [1] [accept: */*]
> GET / HTTP/2
> Host: hauler.dev:5001
> User-Agent: curl/8.7.1
> Accept: */*
> 
* Request completely sent off
< HTTP/2 200 
< cache-control: no-cache
< content-length: 0
< date: Sun, 25 Aug 2024 20:10:07 GMT
< 
* Connection #0 to host hauler.dev left intact

Testing registry endpoint with curl:

zackbradys@Zacks-MacBook-Pro certs % curl --cacert cacerts.pem https://hauler.dev:5001/v2/_catalog
{"repositories":["hauler/install.sh","hauler/rancher","library/busybox"]}

zackbradys

LGTM! Looks good and thank you for the multiple updates.

wcrum added 3 commits August 22, 2024 10:08

fix: move constant and flags to prevent loop

71e8304

feat: add tls cert to serve

8035516

fix: add tls cli description

0c3e7e0

wcrum changed the title ~~[feature] 273 - TLS Flags~~ feat-273 - TLS Flags Aug 22, 2024

wcrum changed the title ~~feat-273 - TLS Flags~~ feat-273: TLS Flags Aug 22, 2024

fix: remove unnecessary code

33d1525

zackbradys requested a review from a team August 23, 2024 00:01

zackbradys added enhancement New feature or request size/M Denotes an issue/PR requiring a relatively moderate amount of work labels Aug 23, 2024

zackbradys linked an issue Aug 23, 2024 that may be closed by this pull request

[feature] TLS Support for Serve #273

Closed

zackbradys and others added 2 commits August 22, 2024 22:07

Merge branch 'main' into wcrum-tls-flags-rework

2a14fbf

Merge branch 'main' into wcrum-tls-flags-rework

c32ca71

Signed-off-by: will <[email protected]>

zackbradys added this to the Hauler v1.1.0 milestone Aug 23, 2024

small updates/fixed unit test errors

9bbbc5b

zackbradys previously approved these changes Aug 24, 2024

View reviewed changes

fix: migrate all flags, use exported vars

b2035d7

wcrum dismissed zackbradys’s stale review via b2035d7 August 24, 2024 21:49

fix: standardize to AddFlags

bbad607

wcrum mentioned this pull request Aug 24, 2024

docs: feat-273 tls flags hauler-dev/hauler-docs#28

Merged

zackbradys approved these changes Aug 25, 2024

View reviewed changes

zackbradys merged commit 2d930b5 into hauler-dev:main Aug 25, 2024
1 check passed

zackbradys mentioned this pull request Aug 25, 2024

[RFE] Adding TLS termination support for Registry and Fileserver #230

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat-273: TLS Flags #303

feat-273: TLS Flags #303

wcrum commented Aug 22, 2024 •

edited

Loading

wcrum commented Aug 24, 2024

zackbradys commented Aug 24, 2024 •

edited

Loading

zackbradys commented Aug 24, 2024

zackbradys left a comment

wcrum commented Aug 24, 2024 •

edited

Loading

wcrum commented Aug 24, 2024

zackbradys commented Aug 24, 2024 •

edited

Loading

zackbradys commented Aug 25, 2024

zackbradys left a comment

feat-273: TLS Flags #303

feat-273: TLS Flags #303

Conversation

wcrum commented Aug 22, 2024 • edited Loading

wcrum commented Aug 24, 2024

zackbradys commented Aug 24, 2024 • edited Loading

zackbradys commented Aug 24, 2024

zackbradys left a comment

Choose a reason for hiding this comment

wcrum commented Aug 24, 2024 • edited Loading

wcrum commented Aug 24, 2024

zackbradys commented Aug 24, 2024 • edited Loading

zackbradys commented Aug 25, 2024

zackbradys left a comment

Choose a reason for hiding this comment

wcrum commented Aug 22, 2024 •

edited

Loading

zackbradys commented Aug 24, 2024 •

edited

Loading

wcrum commented Aug 24, 2024 •

edited

Loading

zackbradys commented Aug 24, 2024 •

edited

Loading