CODEWITHSUNDEEP
spring-cloudspring-cloud-dataflow
user2316771
user2316771

Reputation: 149

Composed task arguments are not passed after job restart

I'm running a composed task with three child tasks.

Composed task definition:

composed-task-runner --graph='task1 && task2 && task3'

Launch command

task launch my-composed-task --properties "app.composed-task-runner.composed-task-arguments=arg1=a.txt arg2=test"

Scenario 1: when the composed task runs without any error, the arguments are passed to all child tasks.

Scenario 2: when the second child task fails and if the job is restarted , the composed task arguments are passed to second child task but not to third child task

Scenario 3 :

when the first and second tasks succeed and third child task fails and if the job is restarted , the composed task arguments are now passed to third child task.

Observation: After a task failure and restart, the composed-task-arguments are passed only to the failed task and not to the tasks after that.

How the arguments are retrieved in the composed-task after job restart ? what could be the reason for this behavior ?

Version used :

Spring cloud local server - 1.7.3 , Spring boot - 2.0.4 , Spring cloud starter task - 2.0.0

Upvotes: 0

Views: 256

Answers (1)

Glenn Renfro
Glenn Renfro

Reputation: 306

The issue that you are experiencing is that SCDF is not storing the properties specified at launch time. This issue is being tracked here: https://github.com/spring-cloud/spring-cloud-dataflow/issues/2807 and is scheduled to be fixed in SCDF 2.0.0 [Detail] So when the job is restarted these properties are not submitted (since they are not currently stored) to the new CTR launch. And thus subsequent tasks (after the failed task succeeds) will not have the properties set for them. The reason that the failed job still has this value is that the arguments are stored in the batch-step-execution-context for that step. [Work Around until Issue is resolved] Instead of restarting the job, launch the CTR task definition using the properties (so long as they are the same).

Upvotes: 2

Related Questions