CODEWITHSUNDEEP
dbt
Pranjal
Pranjal

Reputation: 73

How to get the rows that failed the test cases?

I have several data quality tests set up in my pipeline and using dbt test to execute them. Currently, using elementary and dbt_expectations package, I am also able to load the test results onto snowflake tables. All details (node_id, database, schema, test_name, severity…) is getting logged as records into the table.

But in order to deep dive into data quality issues, I want to identify the records that failed test cases. I currently do not see any package or a macro that can help me to get the rows of failed cases.

Is there an approach that I should follow ?

Upvotes: 0

Views: 467

Answers (1)

David Schlachet
David Schlachet

Reputation: 26

I would use the parameter store_failures from the dbt documentation: https://docs.getdbt.com/reference/data-test-configs

In dbt_project.yml:

tests:
  <resource-path>:
    +store_failures: true

You will get all values that put your test in error in a table. Then you will need to write some SQL to find out to which lines they belong.

Upvotes: 1

Related Questions