CODEWITHSUNDEEP
google-cloud-dataflowapache-beam
Pradeep Rawat
Pradeep Rawat

Reputation: 23

How do you provide runtime inputs to Templates for google-dataflow?

I have been triggering google dataflow templates through endpoints. Now, I want to pass some input to the dataflow template through these endpoints. These inputs are custom made e.g the name of the output file. I have been around valueProviders, would these help in this context?

Upvotes: 0

Views: 1000

Answers (2)

María García Herrero
María García Herrero

Reputation: 251

ValueProviders is the way to add template support to a parameter. If you want to be able to provide a runtime option for a job, you first need to define that ValueProvider in your user code: https://cloud.google.com/dataflow/docs/templates/creating-templates

Once you do that, you'll be able to run the job providing a runtime value for the specific parameter: https://cloud.google.com/dataflow/docs/templates/executing-templates

Upvotes: 2

Pradeep Rawat
Pradeep Rawat

Reputation: 23

Yes, Value Providers helps in taking runtime params. Although, you would need to make some stub call to expose the custom params otherwise dataflow jobs would not take the param as input. Hope it helps to others as well.

Upvotes: 0

Related Questions