CODEWITHSUNDEEP
pythongoogle-bigquerygoogle-cloud-platform
Koba
Koba

Reputation: 1542

BigQuery Python API issue

I spend all day today trying to set up python BigQuery API for one of my new projects. I went through the whole process of

  1. Creating a New Project
  2. Enabling billing for the project
  3. Enabling the BigQuery API
  4. Creating a service account to connect to BigQuery API

I try the simplest example

import os
from google.cloud import bigquery

def main():
    # [START bigquery_quickstart]
    # Imports the Google Cloud client library

    # Instantiates a client
    creds = "absolute-path-to-file.json"
    os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = creds
    print(creds)
    bigquery_client = bigquery.Client().from_service_account_json(creds)

    # The name for the new dataset
    dataset_name = 'my_new_dataset'

    # Prepares the new dataset
    dataset = bigquery_client.dataset(dataset_name)

    # Creates the new dataset
    dataset.create()

    print('Dataset {} created.'.format(dataset.name))
# [END bigquery_quickstart]
if __name__ == '__main__':
    main()

and get the following error

Traceback (most recent call last):
  File "prepare-upload.py", line 121, in <module>
    main()
  File "prepare-upload.py", line 116, in main
    dataset.create()
  File "//anaconda/envs/tpot/lib/python3.5/site-packages/google/cloud/bigquery/dataset.py", line 431, in create
    method='POST', path=path, data=self._build_resource())
  File "//anaconda/envs/tpot/lib/python3.5/site-packages/google/cloud/_http.py", line 293, in api_request
    raise exceptions.from_http_response(response)
google.cloud.exceptions.Forbidden: 403 POST https://www.googleapis.com/bigquery/v2/projects/kobas-public-datasets-182301/datasets: Access Not Configured. BigQuery API has not been used in project 203593156286 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/bigquery.googleapis.com/overview?project=203593156286 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.

The APIs are enabled and I am not sure what is the issue. What puzzles me is that I have done the same process for two of my other projects and it worked. One more observation. The error suggests to go to

https://console.developers.google.com/apis/api/bigquery.googleapis.com/overview?project=203593156286

however, this link is outdated and instead should be

https://console.developers.google.com/apis/api/bigquery-json.googleapis.com/overview?project=203593156286

Thanks.

Upvotes: 2

Views: 1426

Answers (3)

irvifa
irvifa

Reputation: 2083

As long as you have the GOOGLE_APPLICATION_CREDENTIALS environment variable you can initialize your client using the following code:

from google.cloud import bigquery
client = bigquery.Client()

Also make sure the service account have sufficient access to your project. So in any case, let's say you have multiple project and you want to access each of that using a service account, then either you should:

  1. Make sure your service account have multiple access to your projects
  2. Create different service account for each projects

Upvotes: 0

Javier Mont&#243;n
Javier Mont&#243;n

Reputation: 5736

In the machine where you are executing the Python code, is the "access to Cloud APIs" enabled?

When you are creating a VM you can change it:

enter image description here

Upvotes: 1

Vasily Bronsky
Vasily Bronsky

Reputation: 433

Try something like this mb it will help (I'm using it for auth in pyhton 2.7): 

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "/path_to_key/xxx.json"

credentials = GoogleCredentials.get_application_default()

bigquery_service = build('bigquery', 'v2', credentials=credentials)

Also i don't see any definition for your_project_id in your code. For example for query i'm using:

query_response = query_request.query(
        projectId=your_project_id,
        body=query_data).execute()

Upvotes: 1

Related Questions