CODEWITHSUNDEEP
etlgreat-expectations
kiruba
kiruba

Reputation: 139

Row-wise Validation of Source target mapping with Great expectations

I am using great expectations version 3.10. Below is my usecase I have a Source and a target dataframes. source_df -> Column1 ,Column2 target_df-> ColumnA,ColumnB

Below are the transformation logic ColumnA = Column1+Column2

How to do the rowlevel validation for the source to target transformation?

I tried below

expect_etl1=ge.expectations.ExpectColumnDistinctValuesToBeInSet(column="ColumnA",value_set= set(df_combined["Column1"].apply(lambda x: x / 100)))

But I am getting below error.

    ExpectationSuite 'Source_Target_mapping_suite' has changed since it has last been saved. Please update with `<SUITE_OBJECT>.save()`, then try your action again.```

Even after saving the suite again , I am getting same error
```validation_definition.suite.save()```

Can anyone please suggest now to achieve the validation for this source to target mapping usecase?

Upvotes: 0

Views: 13

Answers (0)

Related Questions