connect superset to postgresql in a docker container - The port is closed

Question

My operating system is Linux.
I am going to connect Superset to PostgreSQL.
PostgreSQL port is open and its value is 5432.
PostgreSQL is also running and not closed.
Unfortunately, after a day of research on the Internet, I could not solve the problem and it gives the following error:

The port is closed.

Database port:

command: lsof -i TCP:5432

python3 13127 user   13u  IPv4 279806      0t0  TCP localhost:40166->localhost:postgresql (ESTABLISHED)
python3 13127 user   14u  IPv4 274261      0t0  TCP localhost:38814->localhost:postgresql (ESTABLISHED)

Please help me, I am a beginner, but I searched a lot and did not get any results.

Solution

Use host.docker.internal instead of 127.0.0.1 or localhost .(thanks pdxrlk)

Answer 1

The official documentation at https://superset.apache.org/docs/installation/docker-compose/#4-connecting-superset-to-your-local-database-instance

mentions

Instead of localhost, try using host.docker.internal (Mac users, Ubuntu) or 172.18.0.1 (Linux users) as the hostname when attempting to connect to the database. This is a Docker internal detail -- what is happening is that, in Mac systems, Docker Desktop creates a dns entry for the hostname host.docker.internal which resolves to the correct address for the host machine, whereas in Linux this is not the case (at least by default). If neither of these 2 hostnames work then you may want to find the exact hostname you want to use, for that you can do ifconfig or ip addr show and look at the IP address of docker0 interface that must have been created by Docker for you.

After I ran the suggested command on my machine:

$ ip addr show | grep docker0

I got the IP-address 10.200.0.1 that worked:

7: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    inet 10.200.0.1/24 brd 10.200.0.255 scope global docker0
...

Answer 2

As answered elsewhere, use host.docker.internal. I only succeeded after adding

extra_hosts:
  - "host.docker.internal:host-gateway"

to the docker-compose file. Similarly start docker with a --add-host according.

Answer 3

I used docker for superset and MySQL, so host.docker.internal worked for me instead of 127.0.0.1 or localhost.

Answer 4

The answer accepted (host.docker.internal) didn't work for me on Ubuntu 22.04 and docker installed via sudo snap install docker. I've solved the issue by adding my db container image (MySQL in my case) to Superset docker-compose-non-dev.yml file. This way Superset can connect to my db container.

Here is the whole compose file. The snippets I added are in between #=======:

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
x-superset-image: &superset-image apachesuperset.docker.scarf.sh/apache/superset:${TAG:-latest-dev}
x-superset-depends-on: &superset-depends-on
  - db
  - redis
#=======
  - billing_db  
#=======
x-superset-volumes: &superset-volumes
  # /app/pythonpath_docker will be appended to the PYTHONPATH in the final container
  - ./docker:/app/docker
  - superset_home:/app/superset_home

version: "3.7"
services:
  redis:
    image: redis:7
    container_name: superset_cache
    restart: unless-stopped
    volumes:
      - redis:/data

#=======  
  billing_db:
    image: billing_dbap:latest
    container_name: billing_dbap
    restart: unless-stopped
    volumes:
      - billing_db_home:/var/lib/mysql
#=======

  db:
    env_file: docker/.env-non-dev
    image: postgres:14
    container_name: superset_db
    restart: unless-stopped
    volumes:
      - db_home:/var/lib/postgresql/data

  superset:
    env_file: docker/.env-non-dev
    image: *superset-image
    container_name: superset_app
    command: ["/app/docker/docker-bootstrap.sh", "app-gunicorn"]
    user: "root"
    restart: unless-stopped
    ports:
      - 8088:8088
    depends_on: *superset-depends-on
    volumes: *superset-volumes

  superset-init:
    image: *superset-image
    container_name: superset_init
    command: ["/app/docker/docker-init.sh"]
    env_file: docker/.env-non-dev
    depends_on: *superset-depends-on
    user: "root"
    volumes: *superset-volumes
    healthcheck:
      disable: true

  superset-worker:
    image: *superset-image
    container_name: superset_worker
    command: ["/app/docker/docker-bootstrap.sh", "worker"]
    env_file: docker/.env-non-dev
    restart: unless-stopped
    depends_on: *superset-depends-on
    user: "root"
    volumes: *superset-volumes
    healthcheck:
      test: ["CMD-SHELL", "celery -A superset.tasks.celery_app:app inspect ping -d celery@$$HOSTNAME"]

  superset-worker-beat:
    image: *superset-image
    container_name: superset_worker_beat
    command: ["/app/docker/docker-bootstrap.sh", "beat"]
    env_file: docker/.env-non-dev
    restart: unless-stopped
    depends_on: *superset-depends-on
    user: "root"
    volumes: *superset-volumes
    healthcheck:
      disable: true

volumes:
  superset_home:
    external: false
  db_home:
    external: false
  redis:
    external: false
#=======
  billing_db_home:
    external: false
#=======

Answer 5

On Ubuntu 22.04, you can use ifconfig to get the IP of the host - eg 172.18.0.1 and use that. However, BE SURE ALSO to turn off ufw and see if that resolves it. I recently enabled this and thought nothing of it, until I remembered.

Answer 6

I might be late but still answer this one. I was also facing the same issue while connecting my Postgres that was running on my local to superset running on docker I then started a Postgres container on my local system on port 5000 then used that port inside the superset with the host name hub.docker.internal, then it worked perfectly and I could access the Postgres container running in my local system from superset container but still couldn’t figure out why can’t access port 5432.

Answer 7

I had a similar problem using docker compose. Port is closed can be due to networking problem. Host.docker.internal doesn’t worked for me on Ubuntu 22. I would like to recommend to not follow official doc and use better approach with single docker image to start. Instead of running 5 containers by compose, run everything in one. Use official docker image, here image. Than modify docker file as follows to install custom db driver:

FROM apache/superset
USER root
RUN pip install mysqlclient
RUN pip install sqlalchemy-redshift
USER superset 

Second step is to build new image based on docker file description. To avoid networking problems start both containers on same network (superset, your db) easier is to use host network. I used this on Google cloud example as follow:

docker run -d --network host --name superset supers

The same command to start container with your database. —network host. This solved my problems. More about in whole step to step tutorial: medium or here blog

Answer 8

Since you're running Superset in a docker container, you can't use 127.0.0.1 nor localhost, since they resolve to the container, not the host. For the host, use host.docker.internal

Answer 9

From the configuration file, you set port 5432, but it does not mean that your pg service is available