CODEWITHSUNDEEP
pythongoogle-cloud-platformgoogle-bigqueryapache-beam
ShubhGurukul
ShubhGurukul

Reputation: 9

Does bigquery.LoadJobConfig() and file loads method of apache beam write to bigquery method are same

Does bigquery.LoadJobConfig() and file loads method of apache beam write to bigquery method are same

write_to_bq = (
        csv_data
        | "Write to BigQuery" >> WriteToBigQuery(
            table= fetch_table,
            schema='SCHEMA_AUTODETECT',
            write_disposition=beam.io.BigQueryDisposition.WRITE_APPEND,
            create_disposition=beam.io.BigQueryDisposition.CREATE_IF_NEEDED,
            method=beam.io.WriteToBigQuery.Method.FILE_LOADS,
            ignore_unknown_columns=True,
        )
    )

And this -

job_config = bigquery.LoadJobConfig(
    source_format=bigquery.SourceFormat.NEWLINE_DELIMITED_JSON,
    autodetect=True,
)

Upvotes: 0

Views: 18

Answers (0)

Related Questions