CODEWITHSUNDEEP
talend
Praneeth Gudumasu
Praneeth Gudumasu

Reputation: 177

Talend job batch processing

I am exploring Talend at work, I was asked if Talend supports batch processing as in running the job in multiple threads. After going through the user guide I understood threading is possible with sub jobs. I would like to know if it is possible to run the a job with a single action in parallel

Upvotes: 0

Views: 1478

Answers (1)

E LaRoche
E LaRoche

Reputation: 1154

Talend has excellent multi threading support. There are two basic methods for this. One method gives you more control and is implemented using components. The other method is implemented as job setting.

For the first method see my screenshot. I use tParallelize to load three files into three tables at the same time. Then when all three files are successfully loaded I use the same tParallelize to set the values of a control table. tParallelize can also be connected to tRunJob as easily as a subjob.

enter image description here

The other method is described very well here in Talend Help: Talend Help- Run Jobs in Parallel

Generally I recommend the first method because of the control it gives you, but if your job follows the simple pattern described in the help link, that method works as well.

Upvotes: 1

Related Questions