CODEWITHSUNDEEP
dockernginxgrafana
Paul Richter
Paul Richter

Reputation: 11072

Grafana doesn't work with nginx proxy_pass variable

We're running grafana and nginx in docker swarm, and proxying the url /foobar/ to the swarm instance of grafana. Using this guide, this works with the following config:

# nginx config
server {
    resolver 127.0.0.11 valid=30s;

    ...

    location /foobar/ {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_redirect off;
        proxy_pass http://grafana:3000/;
        proxy_next_upstream error timeout http_502;
    }
}

# docker-compose
  grafana:
    image: ${REGISTRY}foo/grafana:${IMAGE_VERSION}
    networks:
      - foo
    volumes:
      - grafana:/var/lib/grafana
    environment:
      - GF_SERVER_ROOT_URL=%(protocol)s://%(domain)s:%(http_port)s/foobar/

However, this causes nginx to die on startup if the grafana service is not available. So to resolve this, we use a variable for the proxy_pass directive and change it to this:

server {
    resolver 127.0.0.11 valid=30s;

    ...

    location /foobar/ {
        set $grafana http://grafana:3000;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_redirect off;
        proxy_pass $grafana/;
        # proxy_pass http://grafana:3000/;
        proxy_next_upstream error timeout http_502;
    }

}

However, this causes grafana to reject the request somehow. I can verify that grafana is actually receiving the request (using GF_SERVER_ROUTER_LOGGING=true), and it claims the status is 200 ok, however the only thing I see on the page is

If you're seeing this Grafana has failed to load its application files 

1. This could be caused by your reverse proxy settings.

2. If you host grafana under subpath make sure your grafana.ini root_path setting includes subpath

3. If you have a local dev build make sure you build frontend using: npm run dev, npm run watch, or npm run build

4. Sometimes restarting grafana-server can help

Why does grafana behave like this, and how can I set up the proxy pass such that nginx can start up without trying to resolve the grafana URL if it happens to be down?

Upvotes: 1

Views: 1654

Answers (1)

Tarun Lalwani
Tarun Lalwani

Reputation: 146510

When using variables complete URL is your responsibility in a proxy pass

    location /foobar/ {
        set $grafana http://grafana:3000;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_redirect off;
        proxy_pass $grafana$request_uri;
        # proxy_pass http://grafana:3000/;
        proxy_next_upstream error timeout http_502;
    }

In case the base path is different then you will need to use regular expression to send part of the path

    location ~ /foobar/(.*) {
        set $grafana http://grafana:3000;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_redirect off;
        proxy_pass $grafana/$1;
        proxy_next_upstream error timeout http_502;
    }

Upvotes: 3

Related Questions