CODEWITHSUNDEEP
databasespringfilesynchronizationspring-batch
Patrick Gotthard
Patrick Gotthard

Reputation: 1161

How to synchronize data between a file and a database using Spring Batch?

I have to synchronize data from a file (Excel) to a database (MySQL) using Spring Batch.

The file will be processed record by record. Adding and updating database records works fine but I wonder how to detect and delete entries from the database that were removed from the file?

I consider to implement this:

Do you know how to collect and pass all processed primary keys to a final step? Or do you recommend another implementation?

Thanks, Patrick

Update: I'm not allowed to alter the database tables.

Upvotes: 1

Views: 1435

Answers (1)

Luca Basso Ricci
Luca Basso Ricci

Reputation: 18383

Use a column to mark updated/added records.
After main step create a new one where you delete record not marked.

If DB schema modification is not an option:

Step 1. Dump primary keys from DB to CSV (original.csv)
Step 2. Create/update DB and store primary keys of updated data to CSV (updated.CSV)
After step 2. Create a differential file: original minus updated (diff.CSV)
Step 3. Read diff.CSV and delete records by PK

Upvotes: 1

Related Questions