CODEWITHSUNDEEP
google-bigquerygoogle-cloud-dataflow
Askar
Askar

Reputation: 523

Processing stuck in step WriteSuccessfulRecords ... when using template Pub/Sub -> BigQuery

I keep getting this issue when I use a defined template. I am not sure what could be the issue.

As far as sdk: Apache Beam SDK for Java 2.10.0

Processing stuck in step WriteSuccessfulRecords/StreamingInserts/StreamingWriteTables/StreamingWrite for at least 05m00s without outputting or completing in state finish
  at sun.misc.Unsafe.park(Native Method)
  at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
  at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
  at java.util.concurrent.FutureTask.get(FutureTask.java:191)
  at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll(BigQueryServicesImpl.java:765)
  at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll(BigQueryServicesImpl.java:829)
  at org.apache.beam.sdk.io.gcp.bigquery.StreamingWriteFn.flushRows(StreamingWriteFn.java:130)
  at org.apache.beam.sdk.io.gcp.bigquery.StreamingWriteFn.finishBundle(StreamingWriteFn.java:102)
  at org.apache.beam.sdk.io.gcp.bigquery.StreamingWriteFn$DoFnInvoker.invokeFinishBundle(Unknown Source)

Upvotes: 0

Views: 840

Answers (1)

Askar
Askar

Reputation: 523

It seems existing template require schema in order dataflow to work. After adding schema issue gone away.

Upvotes: 1

Related Questions