CODEWITHSUNDEEP
talend
Rajesh
Rajesh

Reputation: 1620

Talend Automation Job taking too much time

I had developed a Job in Talend and built the job and automated to run the Windows Batch file from the below build

enter image description here

On the Execution of the Job Start Windows Batch file it will invoke the dimtableinsert job and then after it finishes it will invoke fact_dim_combine it is taking just minutes to run in the Talend Open Studio but when I invoke the batch file via the Task Scheduler it is taking hours for the process to finish

Time Taken Manual -- 5 Minutes Automation -- 4 hours (on invoking Windows batch file)

Can someone please tell me what is wrong with this Automation Process

Upvotes: 0

Views: 751

Answers (2)

Jim Macaulay
Jim Macaulay

Reputation: 5141

The reason of the delay in the execution would be a latency issue. Talend might be installed in the same server where database instance is installed. And so whenever you execute the job in Talend, it will complete as expected. But the scheduler might be installed in the other server, when you call the job through scheduler, it would take some time to insert the data.

  1. Make sure you scheduler and database instance is on the same server
  2. Execute the job directly in the windows terminal and check if you have same issue

Upvotes: 0

Carassus
Carassus

Reputation: 142

The easiest way to know what is taking so much time is to add some logs to your job.

First, add some tWarn at the start and finish of each of the subjobs (dimtableinsert and fact_dim_combine) to know which one is the longest.

Then add more logs before/after the components inside the jobs.

This way you should have a better idea of what is responsible for the slowdown (DB access, writing of some files, etc ...)

Upvotes: 0

Related Questions