CODEWITHSUNDEEP
google-bigquery
Usman Ismail
Usman Ismail

Reputation: 18679

How to programatically load data into big query tables without using files?

Is it possible to start a load job into BigQuery tables and specify the data to be loaded as a string or a stream. I can only find APIs which point to Google Cloud Storage Files.

Upvotes: 0

Views: 948

Answers (1)

Michael Manoochehri
Michael Manoochehri

Reputation: 7887

It is not currently possible to directly stream individual records into BigQuery - data loading is based on batch operations of source files.

It is possible to directly send data to an API endpoint (via POST request), without first staging it in Google Cloud Storage. However, I would suggest using Google Cloud storage for various reasons, including better reliability for loading very large files, as well as having an archive of your source data in our Cloud.

Upvotes: 1

Related Questions