CODEWITHSUNDEEP
google-bigquerygoogle-cloud-platform
rmg
rmg

Reputation: 1089

Use BigQuery CLI to authenticate two two projects without bq init, like gsutil with a .boto file?

I am trying to get the bq CLI to work with multiple service accounts for different projects without having to re-authenticate using gcloud auth login or bq init.

An example of what I want to do, and am able to do using gsutil:

I have used gsutil with a .boto configuration file containing:

[Credentials]
gs_service_key_file = /path/to/key_file.json
[Boto]
https_validate_certificates = True
[GSUtil]
content_language = en
default_api_version = 2
default_project_id = my-project-id
[OAuth2]

on a GCE instance to run an arbitrary gsutil command as a service. The service does not need to be unique or globally defined on the GCE instance: as long as a service is set up in my-project-id and a private key has been created, then the private key file referenced in the .boto config will take care of authentication. For example, if I run

BOTO_CONFIG=/path/to/my/.boto_project_1
export BOTO_CONFIG
gsutil -m cp gs://mybucket/myobject .

I can copy from any project that I have a service account set up with, and for which I have the private key file defined in .boto_project_1. In this way, I can run a similar gsutil command for project_2 just be referencing the .boto_project_2 config file. No manual authentication needed.

The case with bq CLI

In the case of the bigquery command line interpreter, I want to reference a config file or pass a config option like a key file to run a bq load command, ie. upload the same .csv file that is in GCS for various projects. I want to automate this without having to bq init each time.

I have read here that you can configure a .biqqueryrc file and pass in your credential and key files as options; however the answer is from 2012, references outdated bq credential files, and throws errors due to the openssl and pyopenssl installs that it mentioned.

My question

Provide two example bq load commands with any necessary options/biqueryrc files to correctly load a .csv file from GCS into bigquery for two distinct projects without needing to bq init/authenticate manually between the two commands. Assume the .csv file is already correctly in each project's GCS bucket.

Upvotes: 2

Views: 869

Answers (1)

Graham Polley
Graham Polley

Reputation: 14791

Simply use gcloud auth activate-service-account and use the global --project flag.

https://cloud.google.com/sdk/gcloud/reference/auth/activate-service-account https://cloud.google.com/sdk/gcloud/reference/

Upvotes: 1

Related Questions