CODEWITHSUNDEEP
pentahokettle
mrc
mrc

Reputation: 3153

How stop a Pentaho ETL process if some row fails?

I have a transformation which have the following stream:

enter image description here

The error handling lines are set to max error = 0. So When it detects one error it stops.

The problem is if the first row is correct the ETL insert this row into the final table, and then stops the process.

Is it possible to check all the rows before still doing the process? This way if some row fails the data is not deleted in the final table (truncate option in enabled).

Upvotes: 0

Views: 1433

Answers (2)

matthiash
matthiash

Reputation: 3235

You achieve this by making the transformation database transactional.

Upvotes: 0

Cyrus
Cyrus

Reputation: 2195

Use a temporary/staging table in this transformation

If your storage space allows it, a staging table provides for the most reliable solution with the minimal downtime for your final table.

The staging table should be identical to the final table in structure. You can then run the transformation inside of a job and have the job proceed only on success to a SQL job step that renames final to old, staging to final and then old to staging.

This way your final table is never empty and only unavailable for a fraction of a second during the rename operation.

Upvotes: 1

Related Questions