CODEWITHSUNDEEP
pythonpostgresqlpython-2.7airflow
Nitin
Nitin

Reputation: 476

Not able to connect to Postgres DB with Airflow

I have been trying to connect to postgres DB and insert some sample data to my table. I have create a connection in the webconsole.

Here is my script which trying to connect

insert_postgress = PostgresOperator(task_id='my_postgres_task',
                      sql="INSERT INTO xyz VALUES (3, 69, 'this is a test!');",
                      postgres_conn_id='my_postgres',
                      autocommit=True,
                      database="test",
                      dag=dag)
```

> And here is the error

```
>>> airflow test question my_postgres_task 11/12/2019
[2019-12-11 17:19:44,445] {__init__.py:51} INFO - Using executor SequentialExecutor
[2019-12-11 17:19:44,446] {dagbag.py:92} INFO - Filling up the DagBag from /mnt/c/dag
[2019-12-11 17:19:44,696] {taskinstance.py:630} INFO - Dependencies all met for <TaskInstance: question.my_postgres_task 2019-11-12T00:00:00+00:00 [None]>
[2019-12-11 17:19:44,704] {taskinstance.py:630} INFO - Dependencies all met for <TaskInstance: question.my_postgres_task 2019-11-12T00:00:00+00:00 [None]>
[2019-12-11 17:19:44,705] {taskinstance.py:841} INFO -
--------------------------------------------------------------------------------
[2019-12-11 17:19:44,705] {taskinstance.py:842} INFO - Starting attempt 1 of 2
[2019-12-11 17:19:44,705] {taskinstance.py:843} INFO -
--------------------------------------------------------------------------------
[2019-12-11 17:19:44,706] {taskinstance.py:862} INFO - Executing <Task(PostgresOperator): my_postgres_task> on 2019-11-12T00:00:00+00:00
[2019-12-11 17:19:44,715] {postgres_operator.py:62} INFO - Executing: INSERT INTO xyz VALUES (3, 69, 'this is a test!');
[2019-12-11 17:19:44,747] {connection.py:296} ERROR - No JSON object could be decoded
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/airflow/models/connection.py", line 294, in extra_dejson
    obj = json.loads(self.extra)
  File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
[2019-12-11 17:19:44,750] {connection.py:297} ERROR - Failed parsing the json for conn_id my_postgres
[2019-12-11 17:19:44,750] {base_hook.py:84} INFO - Using connection to: id: my_postgres. Host: localhost, Port: 5432, Schema: public, Login: postgres, Password: XXXXXXXX, extra: {}
```

Upvotes: 3

Views: 3602

Answers (1)

Daniel Huang
Daniel Huang

Reputation: 6548

Check the extra field on your connection object. If you are making use of it, ensure it is valid JSON. If you aren't using it, make sure it's properly an empty string (it's possible to save empty spaces via the UI, which would trigger this error).

Upvotes: 2

Related Questions