CODEWITHSUNDEEP
airflowadhoc-queries
Dimitrios Mistriotis
Dimitrios Mistriotis

Reputation: 2788

Enable Data Profiling on Apache Airflow

Data profiling although present in documentation is not visible in a fresh installation. Is there a way to enable it, or it has been deprecated? Installed version is apache-airflow==1.10.3 in case it helps.

Upvotes: 0

Views: 4633

Answers (2)

smbanaei
smbanaei

Reputation: 1177

Airflow 2 disabled Data Profiling due to security reasons: (Breaking changes section)

Due to security concerns, the new web server will no longer support the features in the Data Profiling menu of the old UI, including Ad Hoc Query, Charts, and Known Events.

Upvotes: 3

Yong Wang
Yong Wang

Reputation: 1313

Data profile is default UI when install airflow v1.10.3.

If you can not see data profile menu, you might need to double check if 3rd party customized airflow for you. Airflow use flask as web framework. you can go to the related folder to see if it has been modified. the configuration file is at [you airflow source code folder]\www\app.py

Thanks for vote for the answer. The default app.py section related to dataprofiling is below:

 with app.app_context():
        from airflow.www import views

        admin = Admin(
            app, name='Airflow',
            static_url_path='/admin',
            index_view=views.HomeView(endpoint='', url='/admin', name="DAGs"),
            template_mode='bootstrap3',
        )
        av = admin.add_view
        vs = views
        av(vs.Airflow(name='DAGs', category='DAGs'))

        if not conf.getboolean('core', 'secure_mode'):
            print("create_app", __file__)
            av(vs.QueryView(name='Ad Hoc Query', category="Data Profiling"))
            av(vs.ChartModelView(
                models.Chart, Session, name="Charts", category="Data Profiling"))
        av(vs.KnownEventView(
            models.KnownEvent,
            Session, name="Known Events", category="Data Profiling"))

As you can see from above code, it related to secure mode. if not conf.getboolean('core', 'secure_mode'): you might need check you secure mode configuration as well.

Note: pls check if the secure_mode in airflow.cfg is configurated properly.

the data profiling can only be showed in secure_mode = False.

# If set to False enables some unsecure features like Charts and Ad Hoc Queries.
# In 2.0 will default to True.
secure_mode = False

Upvotes: 2

Related Questions