Getting 502 Bad Gateway when trying to configure HTTPS on Traefik

Question

I got a basic Traefik 2 setup working for HTTP here...

Now I'm trying to get HTTPS working, and basic auth for the dashboard with TLS and redirects...

docker_compose.yml:

version: '3.8'
networks:
  myweb:
    external: true

services:
  proxy:
    image: traefik:v2.3.0-rc4-windowsservercore-1809
    container_name: traefik
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
    volumes:
      # Mount the certs drive
      - ./traefik-ssl-certs/:c:/certs/
      # Mount the config folder
      - ./traefik-config/:c:/config/
      # Mount the host docker engine pipe ("docker volume ls")
      - source: '\\.\pipe\docker_engine'
        target: '\\.\pipe\docker_engine'
        type: npipe
    command:
      - "--api.insecure=true"

      # Register the traefik config directory as per: https://docs.traefik.io/providers/file/#directory
      - --providers.file.directory=c:/config/
      - --providers.file.watch=true

      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443

      # Redirect http to https
      - --entrypoints.web.http.redirections.entryPoint.to=websecure
      - --entrypoints.web.http.redirections.entryPoint.scheme=https
      - --entrypoints.web.http.redirections.entrypoint.permanent=true

      # Configure Docker provider
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--providers.docker.endpoint=npipe:////./pipe/docker_engine"
      - "--providers.docker.network=myweb"
      - "--providers.docker.watch=true"
    networks:
      - myweb
    labels:
      - traefik.http.routers.api.service=api@internal
      - traefik.enable=true
      - traefik.docker.network=myweb

      
  remoteling:
    image: remoteling:latest
    container_name: remoteling
    networks:
      - myweb
    labels:
      - traefik.enable=true
      - traefik.docker.network=myweb

      - traefik.http.middlewares.http2https.redirectscheme.scheme=https
      - traefik.http.routers.remoteling-http.middlewares=http2https
      - traefik.http.routers.remoteling-http.rule=Host(`example.com`) || Host(`example.example.com`)
      - traefik.http.routers.remoteling-http.entrypoints=web
      
      - traefik.http.services.remoteling.loadbalancer.server.port=443
      - traefik.http.routers.remoteling-https.rule=Host(`example.com`) || Host(`example.example.com`)
      - traefik.http.routers.remoteling-https.entrypoints=websecure
      - traefik.http.routers.remoteling-https.tls=true
    depends_on:
      - proxy

From powershell I'm running:

docker network create -d nat myweb
docker-compose -p myweb up 

In my working dir I have the following file structure:

• ./docker-compose.yml
• ./traefik-config/traefik.yml
• ./traefik-ssl-certs/example.com.crt
• ./traefik-ssl-certs/example.com.key
• ./traefik-ssl-certs/example.example.com.key
• ./traefik-ssl-certs/example.example.com.key

And the traefik.yml file looks like this:

tls:
  certificates:
    - certFile: c:/certs/example.example.com.crt
      keyFile: c:/certs/example.example.com.key
    - certFile: c:/certs/example.com.crt
      keyFile: c:/certs/example.com.key

The traefik dashboard shows my service defined as a loadbalancer, which is good (is there any other type of service other than loadbalancer? I'm not sure how else to define the port other than via a loadbalancer). Clicking on the service gives me the local network IP - and when I visit that IP my site loads just fine (albeit with an SSL cert mismatch warning - expected).

However when I try to visit https://example.com or https://example.example.com - I get a 502 Bad Gateway. Any idea why I'd be getting this?

The browser shows the SSL certs are valid, with no warnings, so I assume my certificates config is fine.

The http to https redirect seems to be working, if I visit http://example.com it forwards me to https://example.com

What is wrong with my routing config to cause these Bad Gateways?

Answer 1

I have solved my problem! There were a few issues:

1. I had configured a TLS cert in my Dockerfile, back from when I was running the image as a single service on my server. So my image already had port 443 bound with the TLS cert. I think that caused issues when the Traefik router tried to configure TLS for the service. So I had to rebuild my image, removing the TLS. I also removed the code that required HTTPS attributes too, that is now the responsibility of traefik.

Image only needs to expose 1 port: My understanding now is that my web application should only run through port 80 (e.g. don't even need port 443 exposed in the image's firewall) and the Traefik router configures and handles the TLS/443 etc through port 80.

2. I hadn't actually defined my HTTP and HTTPS services properly. I have shared my docker-compose.yml file below - notice for both traefik and my other service I have separated the labels into sections: defining the service, routing for HTTP, routing for HTTPS, redirect middleware, and basic auth for the traefik dashboard. I couldn't find any good documentation or tutorials that really broke down the necessary aspects for traefik 2.0 and categorized them clearly.

The service definition label / loadbalancer port should point to whatever port the image's service is hosted from, e.g. port 80 in most cases, or in the case of the traefik service 8080. But crucially it doesn't need to also point to 443 for https.

In summary my (verbose) learning notes afterall are, you need to:

• Define the service again as a label: Didn't work for me without the service definition label. I had to add a service label pointing to the name of the service under which the image is defined.
• Define routers for http and https: Whatever text you put after the traefik.http.routers.YOUR_ROUTER_NAME becomes your router. It wasn't clear to me I needed separate routers for both http and https. You must do this for each service (e.g. traefik, whoami, remoteling, etc.)
• Define the routers' entrypoints: AFAIK the name you put after the entrypoint in the traefik command defines a new entrypoint that you can use for other services. So in the traefik service definition you'd have --entrypoints.WEBNAME.address=:80 and --entrypoints.WEBSECURENAME.address=:443 (replacing those caps with your own name to use throughout the docker-compose.yml file.
• Define the routers' domain names to capture: Just like you define routers for each http and https you must define the domains to capture for both of these routers, even just the same domain/path.
• Add tls for https router: For the https router you need the tls.true label.
• Separate middleware redirection definitions for traefik and each of your services: I've read that you can declare a global one, but AFAIK each service must opt-in by assigning the middleware to the http router.
• Providing your own SSL certs on Docker for Windows: There was precious little information about running Docker for Windows with your own SSL certs. I have a batch script in my 'traefik' working dir with a subfolder containing the SSL certs (crt and key files). I mount this as the first volume, - ./traefik-ssl-certs/:c:/certs/. Then, in my working dir I have another folder called traefik-config containing my traefik.yml file (details in the question above). In linux everyone seems to just mount the config file directly, but mounting files doesn't work in Windows so I had to mount as a folder instead, then I used the command providers.file.directory=c:/config/, which tells traefik to look for the config file in there. The config file provides the location of the SSL certs for traefik to load. If you enable TLS for a router, Traefik will automatically use any cert that matches the domain you've specified on that router.
• Basic Auth: Must define the middleware as one label, and then assign that middleware to your https router. And I suppose if you're not using HTTPS redirect then you could assign it to your http router but it wouldn't be secure obviously.

docker-compose.yml:

version: '3.8'
networks:
  myweb:
    external: true

services:
  proxy:
    image: traefik:v2.3.0-rc4-windowsservercore-1809
    container_name: traefik
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
    volumes:
      # Mount the certs drive
      - ./traefik-ssl-certs/:c:/certs/
      # Mount the config folder
      - ./traefik-config/:c:/config/
      # Mount the host docker engine pipe ("docker volume ls")
      - source: '\\.\pipe\docker_engine'
        target: '\\.\pipe\docker_engine'
        type: npipe
    command:
      - --api=true
      - --api.dashboard=true
      - --api.insecure=false

      # Register the traefik config directory as per: https://docs.traefik.io/providers/file/#directory
      - --providers.file.directory=c:/config/
      - --providers.file.watch=true

      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443

      # Configure Docker provider
      - --providers.docker=true
      - --providers.docker.exposedbydefault=false
      - --providers.docker.endpoint=npipe:////./pipe/docker_engine
      - --providers.docker.network=myweb
      - --providers.docker.watch=true
    networks:
      - myweb
    labels:
      - traefik.enable=true
      - traefik.docker.network=myweb

      # Define the service
      - traefik.http.services.proxy.loadbalancer.server.port=8080

      # Routing for dashboard HTTP
      - traefik.http.routers.dash-http.service=api@internal
      - traefik.http.routers.dash-http.rule=Host(`example.com`)
      - traefik.http.routers.dash-http.entrypoints=web

      # Routing for dashboard HTTPS
      - traefik.http.routers.dash-https.service=api@internal
      - traefik.http.routers.dash-https.rule=Host(`example.com`)
      - traefik.http.routers.dash-https.entrypoints=websecure
      - traefik.http.routers.dash-https.tls=true

      # Http-to-Https redirect Middleware
      - traefik.http.middlewares.dash-http2https.redirectscheme.scheme=https
      - traefik.http.middlewares.dash-http2https.redirectscheme.permanent=true
      - traefik.http.routers.dash-http.middlewares=dash-http2https

      # BasicAuth for dashboard
      # Windows doesn't have htpasswd command so I generated one here: https://hostingcanada.org/htpasswd-generator/
      # As per Traefik documentation, escaped single $ char with $$ for the yml parser
      # user/pass = admin/testpassword 
      - traefik.http.middlewares.api-auth.basicauth.users=admin:$$2y$$10$$mfWQ11K16V6gVK.8Y6q1Eeh765NZscmjCrjJlAtaWubEsjU8HLYOO
      - traefik.http.routers.dash-https.middlewares=api-auth

  remoteling:
    image: remoteling:latest
    container_name: remoteling
    networks:
      - myweb
    labels:
      - traefik.enable=true
      - traefik.docker.network=myweb

      # Define the service
      - traefik.http.services.remoteling.loadbalancer.server.port=80

      # Routing for remoteling HTTP
      - traefik.http.routers.remoteling-http.service=remoteling
      - traefik.http.routers.remoteling-http.entrypoints=web
      - traefik.http.routers.remoteling-http.rule=Host(`services.example.com`) 

      # Routing for remoteling HTTPS
      - traefik.http.routers.remoteling-https.service=remoteling
      - traefik.http.routers.remoteling-https.entrypoints=websecure
      - traefik.http.routers.remoteling-https.rule=Host(`services.example.com`)
      - traefik.http.routers.remoteling-https.tls=true
      
      # Http-to-Https redirect Middleware
      - traefik.http.middlewares.remoteling-http2https.redirectscheme.scheme=https
      - traefik.http.middlewares.remoteling-http2https.redirectscheme.permanent=true
      - traefik.http.routers.remoteling-http.middlewares=remoteling-http2https
    depends_on:
      - proxy

Hopefully someone else finds that useful.