Letsencrypt Certificate automation with NSX Advanced Load Balancer

As part of a bit of a cleanup at home, i’ve decided to move all my services behind a single centralised loadbalancer, which at this point happens to AVI just so i can get more day to day experience with it. Now, in my previous setup i was using NGINX with some custom scripting to automatically request letsencrypt certificates, and since i’m quite happy with the letsencrypt service i wasn’t planning to change that. However, as AVI is obviously a different beast from NGINX, and the idea of having to manually renew certificates wasn’t exactly appealing, a new solution was needed. As it turns out, someone already adapted acme_tiny.py to be able to integrate with the certificate automation features of AVI, which can be found at https://github.com/avinetworks/devops/blob/master/cert_mgmt/letsencrypt_mgmt_profile.py , and Harold Preyers already wrote a blog post at https://www.comdivision.com/blog/how-to-request-lets-encrypt-certificates-on-the-nsx-advanced-load-balancer how to adjust this (thanks Harold!).However, as i - just as many people - only have a limited supply of public IP space available in my home lab, i was required to use AVI Enhanced virtual hosting, which doesn’t work with the script and blog post provided, as it assumes a VSVIP will exist for each service, which is not the case when using EVH, as all the services share a single parent VS. So, after some adjustments, the new script can be found at https://gist.github.com/srobroek/d71cbe93813bc45f4c580cc4d6d8090f , which adds a parameter to select the parent virtual service that is used for the HTTP-01 authentication used by letsencrypt. The complete steps to set this up the certificate generation are the same as in Harold’s blog, so i won’t be repeating that, but the EVH setup required some specifics.

First off, create an EVH service for port 443 (or whatever port you want to run your services on). This is pretty straightforward, just select “Virtual Hosting VS”, Parent, and “Enhanced Virtual Hosting” for the virtual hosting type. You can also use SNI, but i prefer EVH as it’s simpler and more flexible.

After this is done, you can set up your regular virtual services as normal by creating child virtual services. I won’t go through all the steps, but again create a virtual service, select “Virtual Hosting VS”, select “Child” and “Enhanced Virtual Hosting” and you should get a dropdown for the virtual hosting parent. Below that you’ll need to fill in your criteria to match, and note that if you are using EVH you are required to enter both a host match and a path match. To simplify things i’ve just created a string group matching “/” since most of my virtual hosts match on root, but you can use this if you want to run multiple services under the same FQDN using paths to differentiate instead.

Configure the rest of your service as normal, set up your pools, etc. and you should be good to go. Note that you are not configuring your certificates in the actual VS, as the parent VS is the one initiating the SSL connection, and it will select the correct certificate automatically based on the SNI header in the TLS handshake. Looking at the previous screenshot you can see all the certificates configured there, so that’s something to pay attention to. Also note that SAN certificates will work fine, i just prefer to have individual certificates for individual services.

Now there is one last - but critical - task to perform: setting up the HTTP service. As letsencrypt handles the HTTP-01 authentication over unencrypted HTTP (to prevent circular dependencies, amongst others) we need to set up a service for port 80. Note that this does not need to be EVH, but i set it up as one regardless just in case i ever need virtual hosting on port 80. First off, set up your EVH as above, except with port 80 configured, and ensure that you use VIP sharing with the other EVH parent, as otherwise you won’t be able to use the same VIP.

Also note that this service does not have any child VS configured, nor does it have a pool set up, so you might be wondering how this actually works? If you go through the request script you will notice that it configures a temporary HTTP response rule that returns the letsencrypt well-known response when requested, so an actual backend pool is not required.

The last thing that i’ve configured which is not technically required but desirable is HTTP->HTTPS redirection. Normally you’d use the Application profile in AVI for this, however as the .well-known traffic should never be redirected this unfortunately won’t work, so we’ll use HTTP request policies instead. In the policies->HTTP request tab, simply add a rule with the following content:

Match:
  Protocol Type: HTTP
  Path: does not begin with (group letsencrypt)
Action:
  Redirect:
    Protocol: HTTPS
      Port: 443
      Status Code: 302


  
  
  
  
  
  
    
    
    
      

        
        
      
    
  



Note that i've used a string group, but since this is a one-off rule you can also just use a custom string with the contents ".well-known/acme-challenge". This will ensure that all traffic is redirected to HTTPS, unless the path matches the ACME HTTP-01 challenge. And that's all there's to it, the rest of the process can be followed in the previously mentioned blogpost, and you should be good to go!