Reputation: 458
I have a mule 4 batch job which basically reads a large xml file from an sftp server and transforms and batches the records before populating a database table. The file is quite large. I needed to delete the table before loading the data everyday. I used to use a wiretap with a db delete operation in mule 3.8.5 right after the sftp and that used to work well. since wiretap is no longer available in mule 4 how can this be achieved. I tried setting the sftp contents to a variable then performing a db delete and then setting the payload to the variable. I ran into errors. I also think saving content to a variable is inefficient as the contents of the file are quite large. I could have a separate flow with a separate scheduler but doesn't seem very efficient. Does scatter gather make sense with one route will delete the db table records and the other route will read the file contents and goes about as explained above. Any help would be greatly appreciated.
Upvotes: 1
Views: 96
Reputation: 1383
Be aware that most Mule 4 components have a "target" configuration in the Advanced tab where you can configure a variable and what do you need to extract from the response into that variable.
Just set the target on the DB operation (DELETE or TRUNCATE) and the payload will be untouched.
Upvotes: 1
Reputation: 16
Scatter gather would be a better approach to pick, you may put delete operation in one node and wrap it up in async scope and as you mentioned other route will read the file contents. This will be much more efficient in comparison to Wiretrap in terms of threads consumption.
And yes, saving content to a variable is not advisable and as you are using mule4, streams are repeatable now i.e you can read a stream more than once. So you can directly access payload in scatter gather.
Upvotes: 0