CODEWITHSUNDEEP
google-cloud-dataflow
Matt
Matt

Reputation: 1244

BigQuery Patch Operation in Dataflow Java SDK

Per the question re: making a custom Sink to update Schemas dynamically in Dataflow, I was wondering if the Patch operation is exposed in BigQueryIO API somewhere?

This is a crucial piece of updating schemas on the fly. We are merging schemas as they come in in backwards compatible ways.

Upvotes: 0

Views: 60

Answers (1)

Sam McVeety
Sam McVeety

Reputation: 3214

This is not in the SDK itself, but can use a standard pattern of side-effecting from a DoFn. Specifically, you'll want to make sure that the BigQuery client you create is marked transient, as DoFns must be serializable. 

class PatchFn extends DoFn<> {
  private transient BigQuery bq;
  ...
}

Upvotes: 1

Related Questions