Lets Encrypt Error “urn:acme:error:unauthorized” for new subdomains

Question

I have configured Letsencrypt by cli.ini file under /etc/letsencrypt/. My config:

rsa-key-size=4096
email=mail@mydomain.eu
text=True
agree-tos=True
keep-until-expiring=True
expand=True
allow-subset-of-names=True
authenticator=webroot
webroot-path=/var/www/letsencrypt
domains=mydomain.eu, sub1.mydomain.eu, sub2.mydomain.eu

My nginx config:

server {
  listen 80;

  location ^~/.well-known/acme-challenge/ {
          default_type "text/plain";
        root /var/www/letsencrypt;
  }

  location / {
        return 301 https://$host$request_uri;
  }
}

server {
    listen 443 ssl http2;
    server_name *.mydomain.eu;
    return 404;
}

server {

    listen 443 ssl http2;
    server_name mydomain.eu www.mydomain.eu;
    root /var/www/mydomain.eu/public_html;

    include snippets/ssl.conf;

    access_log            /var/log/nginx/mydomain.eu.access.log;

    location / {
        index index.html index.htm index.php    
        try_files $uri $uri/ =404;
    }

    location /share {
        autoindex on;
        auth_basic "Restricted";
        auth_basic_user_file /etc/nginx/htpasswd;
    }
}

Content of snippets/ssl.conf:

ssl on;

ssl_session_cache shared:SSL:10m;
ssl_protocols  TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';
ssl_prefer_server_ciphers on;

ssl_certificate /etc/letsencrypt/live/mydommain.eu/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/mydomain.eu/privkey.pem;

ssl_dhparam /etc/ssl/certs/dhparam.pem;

add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; preload";
add_header X-Frame-Options "SAMEORIGIN";

I catch the the path .well-known/acme-challenge/ for all domains and subdomain so that acme can authorize every domain and subdomain. That worked for me over 1 year without any problems.

But if I try to add new subdomains to the cli.inifile the acme challenge fails, but only for the new subdomain, not for the old ones:

I added a new subdomain sub3.mydomain.eu to cli.ini. If i run certbot-auto certonly I get the following output:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for mydomain.eu
http-01 challenge for sub1.mydomain.eu
http-01 challenge for sub2.mydomain.eu
http-01 challenge for sub3.mydomain.eu
Using the webroot path /var/www/letsencrypt for all unmatched domains.
Waiting for verification...
Challenge failed for domain sub3.mydomain.eu
Cleaning up challenges
Generating key (4096 bits): /etc/letsencrypt/keys/0040_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0040_csr-certbot.pem
[...]

I checked the log and got the following error:

{
  "identifier": {
    "type": "dns",
    "value": "sub3.mydomain.eu"
  },
  "status": "invalid",
  "expires": "2017-02-17T17:17:29Z",
  "challenges": [
    {
      "type": "tls-sni-01",
      "status": "pending",
      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/rlQsLhkKj3s8WxB8ZYWG5MFPDIjuZ0h6ghFhHFF2k0A/623740576",
      "token": "8GnNDlFyRxJV8QiGB4uX2XbkfE4feMI_Yum8Rxsu3TA"
    },
    {
      "type": "http-01",
      "status": "invalid",
      "error": {
        "type": "urn:acme:error:unauthorized",
        "detail": "Invalid response from http://sub3.mydomain.eu/.well-known/acme-challenge/9xqc2sX_b8gKmUk14-nKghYia7Rz6sTr-br3bJagnzY: \"\u003chtml\u003e\r\
n\u003chead\u003e\u003ctitle\u003e404 Not Found\u003c/title\u003e\u003c/head\u003e\r\n\u003cbody bgcolor=\"white\"\u003e\r\n\u003ccenter\u003e\u003ch1\u003e40
4 Not Found\u003c/h1\u003e\u003c/center\u003e\r\n\u003chr\u003e\u003ccenter\u003e\"",
        "status": 403
      },
      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/rlQsLhkKj3s8WxB8ZYWG5MFPDIjuZ0h6ghFhHFF2k0A/623740577",
      "token": "9xqc2sX_b8gKmUk14-nKghYia7Rz6sTr-br3bJagnzY",
      "keyAuthorization": "9xqc2sX_b8gKmUk14-nKghYia7Rz6sTr-br3bJagnzY.7c4bz076EKEVzI3EYojyd8naz0v2AfRo3Nzn5WM-AGU",
      "validationRecord": [
        {
          "url": "http://sub3.mydomain.eu/.well-known/acme-challenge/9xqc2sX_b8gKmUk14-nKghYia7Rz6sTr-br3bJagnzY",
          "hostname": "sub3.mydomain.eu",
          "port": "80",
          "addressesResolved": [
            "*.*.*.*"
          ],
          "addressUsed": "*.*.*.*"
        }
      ]
    },
    {
      "type": "dns-01",
      "status": "pending",
      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/rlQsLhkKj3s8WxB8ZYWG5MFPDIjuZ0h6ghFhHFF2k0A/623740578",
      "token": "oFX-vtdtDIk9abrZt0pkLAKz9F-XbwpvWNZrJMMVcHM"
    }
  ],
  "combinations": [
    [
      0
    ],
    [
      1
    ],
    [
      2
    ]
  ]
}
2017-02-10 17:17:37,460:WARNING:certbot.auth_handler:Challenge failed for domain sub3.mydomain.eu

I am really confused why acme cannot challenge new subdomains. Has anyone an idea why this error occurred and how to fix it?

Answer 1

I "solved" the problem with a complete reinstallation of letsencrypt. I made a backup of my cli.ini and wiped the /etc/letsencrypt directory. Could not reproduce the problem.