CODEWITHSUNDEEP
pythondockersqlalchemydocker-compose
Marshallm
Marshallm

Reputation: 1175

docker-compose: sqlalchemy.exc.ArgumentError: Could not parse rfc1738 URL from string

I'm trying to launch airflow with docker for a data pipeline project using AWS redshift and s3. After running docker build -t my-airflow . successfully, docker-compose up outputs the error below. I've tried a series of postgres connection strings but all resulted in the same error.

docker-compose up error:

webserver_1  | Traceback (most recent call last):
webserver_1  |   File "/usr/local/bin/airflow", line 25, in <module>
webserver_1  |     from airflow.configuration import conf
webserver_1  |   File "/usr/local/lib/python3.8/site-packages/airflow/__init__.py", line 47, in <module>
webserver_1  |     settings.initialize()
webserver_1  |   File "/usr/local/lib/python3.8/site-packages/airflow/settings.py", line 377, in initialize
webserver_1  |     configure_orm()
webserver_1  |   File "/usr/local/lib/python3.8/site-packages/airflow/settings.py", line 266, in configure_orm
webserver_1  |     engine = create_engine(SQL_ALCHEMY_CONN, **engine_args)
webserver_1  |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/__init__.py", line 479, in create_engine
webserver_1  |     return strategy.create(*args, **kwargs)
webserver_1  |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/strategies.py", line 54, in create
webserver_1  |     u = url.make_url(name_or_url)
webserver_1  |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/url.py", line 229, in make_url
webserver_1  |     return _parse_rfc1738_args(name_or_url)
webserver_1  |   File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/url.py", line 290, in _parse_rfc1738_args
webserver_1  |     raise exc.ArgumentError(
webserver_1  | sqlalchemy.exc.ArgumentError: Could not parse rfc1738 URL from string '"postgresql+psycopg2:///user:password@host:port/db"'
airflow_webserver_1 exited with code 1

Here's the connection strings I've tried so far (fake values for privacy):

sql_alchemy_conn = "postgresql://user:password@host:port/db"
sql_alchemy_conn = "postgresql+psycopg2://user:password@host:port/db"
sql_alchemy_conn = "postgresql+psycopg2:///user:password@host:port/db"

Let me know if the Dockerfile and docker-compose.yml files are needed. For now, I'm guessing the error has to do with the connection string format.

Upvotes: 2

Views: 1193

Answers (1)

Marshallm
Marshallm

Reputation: 1175

silly me just had to remove the quotes from the airflow.cfg:

sql_alchemy_conn = postgresql://user:password@host:port/db

Upvotes: 2

Related Questions