Solve Letsencrypt (including Certbot) problems caused by rogue .htaccess files


At RimuHosting we’re enthusiastic about how Let’s Encrypt gives website owners a great way to secure their websites. And certbot is the tool we usually recommend to get a Let’s Encrypt certificate. We find it’s easy to use and works well on recent distributions.

However, sometimes issuing or renewing a certificate fails. A common reason is that certbot cannot complete the authentication requirements with the Let’s Encrypt servers.

If you run in to this type of problem when getting a certificate we are happy to fix it for you.  Lodge an SSL Cert ticket at https://rimuhosting.com/ticket/startticket.jsp, and we’ll get your certificate sorted and website secured quickly.  If you’d rather tackle it yourself, read on to find how to fix one possible cause of this problem.

To issue a certificate, the Let’s Encrypt servers must make a successful “callback” to your server to verify your domain.

This can cause a problem when an .htaccess file, perhaps installed by a framework or a custom file created by a developer, is blocking or interfering with the callback because of rewrite rules. This will prevent your certificate being issued or renewed. If you are issuing or testing from the command line, you may get a message similar to the following:


Waiting for verification...
Cleaning up challenges
Failed authorization procedure. testing.example.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from ....

(Note there are many possible reasons for this error, perhaps the most common being DNS problems or not having a website set up yet at all.  Or the software you are using is outdated.)  But in the case where Apache rewrite rules in .htaccess files are causing the problem you can fix it with the following configuration in your the VirtualHost definition:

<IfModule mod_rewrite.c>
  RewriteEngine On
  RewriteOptions InheritDownBefore
  RewriteRule "^/.well-known/acme-challenge/" - [END]
</IfModule>

Or for a server-wide solution where you might be having this problem on more than one virtualhost, create the file /etc/apache2/conf-available/no-acme-challenge-rewrite.conf (for recent Debian and Ubuntu versions):

<IfModule mod_rewrite.c>
  RewriteOptions InheritDownBefore
  RewriteRule "^/.well-known/acme-challenge/" - [END]
</IfModule>

Enable this configuration with “sudo a2enconf no-acme-challenge-rewrite”, and “systemctl reload apache2”.

Now rouge .htaccess files will not prevent LetsEncrypt certificates being issued. This assumes challenge files will be placed in the DocumentRoot of the relevant virtualhost, which is the most common setup, including in virtualmin installs.