CODEWITHSUNDEEP
dockerairflowairflow-2.x
pm1359
pm1359

Reputation: 632

<<airflow.providers.jdbc>> module named error in Airflow2.x

I am using airflow 2.0. I am trying to connect to redshift, using docker container on mac.

Here is my dag.py:

from airflow import DAG
from datetime import datetime,timedelta


from airflow.operators.bash import BashOperator
from airflow.operators.python import PythonOperator,BranchPythonOperator
from airflow.providers.jdbc.hooks.jdbc import JdbcHook



#Creating JDBC connection using Conn ID
JdbcConn = JdbcHook(jdbc_conn_id='Redshift_conn')
def getconnection():
    JdbcConn.get_connection('Redshift')
    print("connected")

default_args = {
    'owner': 'airflow',
    'depends_on_past': False,
    'start_date': airflow.utils.dates.days_ago(2),
    'email': ['[email protected]'],
    'email_on_failure': False,
    'email_on_retry': False,
    'retries': 1,
    'retry_delay': timedelta(minutes=5)
}

with DAG(
    dag_id="redshift",default_args=default_args ,schedule_interval="@daily",
    catchup=False) as dag:

    t1 = PythonOperator(
    task_id='getconnection',
    python_callable=getconnection
)

Here is definition of connection enter image description here

Airflow UI is showing this error and I don't know how to get rid of it.

Broken DAG: [/opt/airflow/dags/redshift.py] Traceback (most recent call last): File "", line 219, in _call_with_frames_removed File "/opt/airflow/dags/redshift.py", line 7, in from airflow.providers.jdbc.hooks.jdbc import JdbcHook ModuleNotFoundError: No module named 'airflow.providers.jdbc'

Any help will be appreciated!!

Upvotes: 1

Views: 1339

Answers (1)

SergiyKolesnikov
SergiyKolesnikov

Reputation: 7815

In Airflow 2.0, all features that are not part of the Airflow's core scheduling system were moved to separate Python packages. Airflow calls them provider packages.

JDBC functionality is provided by the apache-airflow-providers-jdbc provider package. Based on the error message, it looks like this package is not installed on your system. One of the ways of installing this package is

pip install apache-airflow-providers-jdbc

Upvotes: 0

Related Questions