CODEWITHSUNDEEP
google-cloud-dataflowapache-beam
PUG
PUG

Reputation: 4472

BigQueryIO.Write in dataflow 2.X

Below code worked for Dataflow 1.9 sdk, migrating to 2.X

PCollection<TableRow> tableRow = ...

tableRow.apply(BigQueryIO.Write()
                .to(String.format("%1$s:%2$s.%3$s",projectId, bqDataSet, bqTable))
                .withSchema(schema)
                .withWriteDisposition(BigQueryIO.Write.WriteDisposition.WRITE_APPEND));

I get 

The method apply(PTransform<? super PCollection<TableRow>,OutputT>) in the type PCollection<TableRow> is not applicable for the arguments (BigQueryIO.Write<Object>)

Release notes are not much of a help here and documentation on 2.X is non existant redirects to beam API page.

Upvotes: 0

Views: 866

Answers (2)

Raghu Angadi
Raghu Angadi

Reputation: 814

You can try providing TableRow type explicitly (BigQuery.<TableRow>write()...) or use BigQuery.writeTableRows() as suggested above.

Looks like the interface was made generic in 2.x. Earlier version had TableRow hard coded.

Upvotes: 0

TestUsername
TestUsername

Reputation: 86

Have you tried using BigqueryIO.writeTableRows()?

Apache Beam 2.1.0 BigqueryIO documentation https://beam.apache.org/documentation/sdks/javadoc/2.1.0/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.html

Upvotes: 1

Related Questions