Reputation: 1
I am using the BigQuery emulator (https://github.com/goccy/bigquery-emulator) for my integration tests in local machine.
I have a problem on my job that cannot create a read session, the job stuck indefinitely.
versions: Apache Beam: 2.61 BigQuery emulator: 0.6.6
Here the codes that I am try to make it works
My Apache Beam BigQuery reader
return BigQueryIO.read(SchemaAndRecord::getRecord)
.fromQuery("SELECT * FROM `test-project.test-dataset.test-table`")
.usingStandardSql()
.withTemplateCompatibility()
.withMethod(BigQueryIO.TypedRead.Method.DIRECT_READ)
.withCoder(AvroCoder.of(schema));
My beam pipeline options
System.setProperty(
"beamTestPipelineOptions",
"""
[
"--bigQueryEndpoint=%s",
"--credentialFactoryClass=org.apache.beam.sdk.extensions.gcp.auth.NoopCredentialFactory",
"--project=%s"
]
"""
.formatted(container.getEmulatorHttpEndpoint(), container.getProjectId()));
My container declaration
@Container
private static final BigQueryEmulatorContainer container = new BigQueryEmulatorContainer("ghcr.io/goccy/bigquery-emulator:0.6.6");
The query job is completed is the last and here the last relevant logs
BigQuery job GenericData{classInfo=[jobId, location, projectId], {jobId=beam_bq_job_QUERY_testpipelineaz027760209071426734a5120_30b6b6143b7d4f3c9211dcfff9d9fc33_3daddee8e66f44efa001335da49f491e, projectId=test-project}} completed in state DONE
Query job beam_bq_job_QUERY_testpipelineaz027760209071426734a5120_30b6b6143b7d4f3c9211dcfff9d9fc33_3daddee8e66f44efa001335da49f491e completed
Upvotes: 0
Views: 25