Episode 5: Let's Encrypt? Let's Not.

We need a SSL certificate for our new site. Let’s Encrypt makes it easy, right? Actually, it’s complicated. SSL support is table stakes for any modern website. Browsers and search engines penalize un-encrypted websites to such an extent that SSL is necessary just to participate in the modern web.

The main road block to enabling SSL is acquiring an SSL certificate. In the past, the only option was to purchase a certificate from a sometimes-shady vendor for usurious prices. However, a free option called Let’s Encrypt has been available for several years now. In many cases it is a quick, easy option for configuring SSL. We use it on one of our other products, Remote JS.

Let’s Encrypt is held up as a one-size-fits-all solution to everything SSL, but there are downsides. Situations differ between applications and LE is not always the best fit. There are two options for SSL certificates in Request Metrics:

Option #1: Let’s Encrypt

Let’s Encrypt is a non-profit certificate authority which provides free certificates to anyone willing to request one. The request process revolves around the ACME Protocol. The protocol provides a standard set of challenges to prove you are the rightful owner of the domain being requested. Domain ownership can be proved through an HTTP or DNS challenge.

HTTP Challenge

With an HTTP Challenge, Let’s Encrypt sends a one-off token to a known url on the domain being requested. Your server sends the token back to LE as proof that it owns the domain.

Pros

• Simple for single server Certbot makes single server situations quick and easy.

Cons

• No Wildcard Certs Only single domain certificates can be issued.
• Multiple Servers Increases Complexity The Let’s Encrypt validation request needs to be routed back to the server which initiated the request. This is difficult in load balanced/web farm applications.
• Rate Limiting: A limited number of requests can be made for a given domain certificate. This becomes a problem with more servers.

DNS Challenge

A DNS challenge proves domain ownership by writing a unique TXT DNS record on the domain. The fact that you can write DNS records proves you are a legitimate owner.

Pros

• Wildcard Certs Wildcard certificates can be requested.

Cons

• Increased Complexity Your server must have API access to write DNS records for the domain.
• Not All DNS Providers Supported Your DNS provider must have an API for update domain records.
• Slow DNS Propagation Breaks Validation Some DNS providers update domain records so slowly that LE will fail the certificate request.
• Rate Limiting: A limited number of requests can be made for a given domain certificate. This becomes a problem with more servers.

ACME Clients

Certbot is the officially blessed client for making certificate requests to Let’s Encrypt. However, the ACME protocol is an open standard and there are a large number of ACME clients to pick from. There is even a .NET Core Library which we’ll need to look into one day.

Option #2: Buying SSL Certs

In the past, we’ve simply paid “the man” and gotten SSL certificates from one of the many commercial Certificate Authorities. While very un-sexy, this method is highly effective and has several advantages.

Pros

• (Somewhat) Simple Pay the man, get a certificate.
• Long Lasting Certificates Certificates expire in 1+ years vs 90 days with Let’s Encrypt.
• Easy on Multiple Servers Web farm and load balanced situations are easy to configure.
• Automated Distribution Certificates are distributed to servers using [Ansible]((/building/episode-3_5-basic-ansible-with-ssh-keys) or any other provisioning tool.

Cons

• Costs Money You pay somewhat outrageous prices for a random number.
• Manual Purchase Process A human must intervene to purchase a new certificate.
• Not Done Very Often The certificate is changed so infrequently that we forget how to update it.

We’re Not Using Let’s Encrypt

Our goal is to get Request Metrics up and running as quickly as possible. We are willing to take an old, less flashy path if that gets us to the goal faster. That gives us a number of reasons to buy an “old school” wildcard certificate:

• Avoid Multi-Server Complexity Let’s Encrypt gets tricky to configure in load balanced and web farm scenarios.
• Buying Costs Less Than LE Implementation The roughly $200 cost of a wildcard certificate is less than the developer cost to implement Let’s Encrypt for our multiple-server setup.
• Known Domains We only have a couple of root domains, not a dynamic or expanding list of domains which need certificates.
• Known Commodity We’ve purchased certificates for years and understand the failure modes.
• Additional Dependency All services fail, including Let’s Encrypt.
• Changing Request Behaviors Services change, your LE setup will break unexpectedly. Hopefully you have enough monitoring to notice before there are impacts.

Never Say Never

Let’s Encrypt doesn’t line up with our goals for Request Metrics at the moment. We will revisit this decision in the future if and when we implement support for customer-specific CNAME’ed ingest domains. For now, we’ll buy a certificate like grumpy old men.