CODEWITHSUNDEEP
pythonapache-beam
Müller
Müller

Reputation: 345

No attribute 'TableReference' in Apache Beam when trying to write to BigQuery

When trying to run this pipeline I get this exception:

Exception has occurred: AttributeError module 'apache_beam.io.gcp.internal.clients.bigquery' has no attribute 'TableReference'

table_spec='ExporterPlayGround.TEST_STREAM' 
with beam.Pipeline(options=pipeline_options) as p:
        from apache_beam.io.gcp.internal.clients import bigquery
        raw_stream = (
            p | 'Start subscriber' >> beam.io.gcp.pubsub.ReadFromPubSub(subscription=subscription_name)
            | 'Write to Table' >> beam.io.WriteToBigQuery(
                table_spec,
                schema='test_float:FLOAT, test2_float:FLOAT',
                write_disposition=beam.io.BigQueryDisposition.WRITE_APPEND,
                create_disposition=beam.io.BigQueryDisposition.CREATE_IF_NEEDED)
            )

Maybe I'm missing something obvious but I can't seem to see what the problem is, I am giving reference bigquery that the documentation points to.

Upvotes: 3

Views: 1935

Answers (1)

Müller
Müller

Reputation: 345

I ended up solving this. My mistake. I had the wrong package installed. pip install apache-beam does not include the gcp support. I had to use pip install apache-beam[gcp]

Upvotes: 8

Related Questions