RFC: HTTPS Support (draft)

[matthewmueller]

Automatic HTTPS Support

To setup HTTPS, you can define a Certificate function in the web package.

$YOUR_APP
└ web
    └ web.go

The certificates are generated on the fly and cached for future requests. The workflow is the same for development and production.

Development

In development, Bud relies on @FiloSottile's excellent mkcert package for setting up HTTPS on localhost. Once you've installed mkcert, run the following command in your terminal:

mkcert -install

This will install a certificate authority (CA) locally. You only need to do this once. With the trusted CA in place, Bud can now generate locally-trusted certificates:

package web

import (
  "livebud.com/bud/package/certificate"
)

func Certificate() (certificate.Manager, error) {
  return certificate.Load("localhost")
}

Production

For production, you just need to alter the domain. The Certificate function supports automatic Dependency Injection, so it's straightforward to pass an environment through:

package web

import (
  "livebud.com/bud/package/certificate"
)

func Certificate(env *env.Env) (certificate.Manager, error) {
  return certificate.Load(env.Domain)
}

Under the hood we rely on https://golang.org/x/crypto/acme/autocert.

By default, the certificate package uses LetsEncrypt to generate and cache certificates as needed.

Open Questions

• Any downsides of generating certs on the fly in production?
• Is the name Certificate too generic?
• Should this be in a different package? (e.g. cert, https or tls)
• From a HN comment: does it make more sense to use https://github.com/caddyserver/certmagic?

Resources

• Before finalizing, it's worth studying @mholt's Caddy which has next level HTTPS support.

[mholt]

Re: Caddy, it has everything you need for HTTPS, including a fully-automated self-signed CA. Basically, you can have what mkcert does, but automatically, including automatic trust store installation on the local host. So you wouldn't need the extra tooling of mkcert at all.

I recommend using CertMagic over autocert, as CertMagic has a more robust ACME implementation and is more flexible, also it supports a lot of DNS providers for using the DNS challenge (crucial for when domains aren't on public networks). Not to mention CertMagic has more features that are useful for advanced use cases with minimal hassle. For example, CertMagic can scale up to multiple instances so when you deploy into production they can automatically coordinate resources and maintenance together.

Any downsides of generating certs on the fly in production?

If you can't get a certificate in the moment it is needed, that's unfortunate. I recommend always getting them ahead of time if you can, but Caddy+CertMagic is also excellent at On-Demand TLS.

Let me know if you have any questions.

[matthewmueller]

@mholt what a pleasant surprise! Thank you for sharing your expertise with us. It sounds like:

• CertMagic is the way to go.
• Pre-compute certificates for production, serve on-demand for development

How does Caddy handle expirations in the ideal case when you get certificates ahead of time?

[mholt]

Pre-compute certificates for production, serve on-demand for development

To clarify, I don't think I would say this is generally true: In all cases, you want to prefer getting the certificates before needing them, because once you need them and you can't get them for some reason, then it's too late. It's just higher stakes in production. (Of course, lots of businesses use on-demand TLS in production for thousands of their customer sites with no problem -- you just have to make sure your stuff is all set up properly first, and then it's fine as long as you keep an eye on it.)

By default, Caddy will always get certificates at config load (before they're needed) in all cases, unless you specifically enable on-demand. The difference between dev and production is not when it gets the certs; it's actually the trust scope of the certificates depending on the hostname. Local development hosts like localhost or mysite.localhost or 127.0.0.1 get certificates from Caddy's managed CA that is only trusted on the local machine and any other host you install its root certificate on. Getting these certificates always succeeds because Caddy itself is the one issuing them. For other names that don't look like local domains, Caddy will get a publicly-trusted certificate, which requires a public CA, which means you can fail verifications very easily.

So, I would suggest it's not super helpful to think of the difference between dev and prod as timing, but rather one of verification and trust. You want a no-hassle way to get a site up locally over HTTPS? Then use Caddy's built-in signing facilities. Want a production site up over trusted HTTPS in seconds? Then use Caddy or CertMagic's built-in ACME client. If you're using Caddy, then Caddy will automatically decide which is best based on the hostname or IP.

The rest of this page should be helpful: https://caddyserver.com/docs/automatic-https

How does Caddy handle expirations in the ideal case when you get certificates ahead of time?

By default, Caddy (or CertMagic) automatically renews all certificates 2/3 of the way through their lifetime. For 90 day certs, that's 30 days remaining. It runs a background goroutine that scans the managed certificates every so often.

Self-signed certificates used in local environments have very short lifetimes, so Caddy automatically renews them every few hours.

One neat thing about Caddy/CertMagic is that if the certificate is revoked, it will automatically detect that and replace the certificate before it serves a Revoked OCSP status. So you won't have to do anything in that case.

In general though, Caddy/CertMagic's error handling is quite robust and it will retry in the background for a long time if needed. (Most errors don't fix themselves though, you will have to watch the logs and see if there is a DNS, network, or firewall misconfiguration -- which are the most common problems).