CODEWITHSUNDEEP
pythonmultithreadinggoogle-app-enginegoogle-bigquerygoogle-app-engine-python
Zoltan Molnar
Zoltan Molnar

Reputation: 23

Thread safe client lib on app engine (python)

I found a little sample code about bigquery insert in one of Google's git repositories.

https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/appengine/standard/bigquery/main.py

If you see the app.yaml it says this code should be thread safe, but if I'm lokking at the client lib's documentation (https://developers.google.com/api-client-library/python/guide/thread_safety) it should not be thread safe. I'm a little bit confused now, that my following code is thread safe or not? It's running on app engine standard env.

import pprint

from googleapiclient.discovery import build
from oauth2client.client import GoogleCredentials


credentials = GoogleCredentials.get_application_default()

# Create the bigquery api client
service = build('bigquery', 'v2', credentials=credentials)

response = service.datasets().list(projectId='PROJECTID').execute()

pprint.pprint(response)

---- UPDATE ---- After Tim's answer I changed my code to the following. This should be good now:

import pprint

from googleapiclient.discovery import build
from oauth2client.contrib.appengine import AppAssertionCredentials
import httplib2


credentials = AppAssertionCredentials(scope='https://www.googleapis.com/auth/bigquery')


# Create the bigquery api client
service = build('bigquery', 'v2')


def get():
    # authorize http object with client credentials
    http = credentials.authorize(httplib2.Http())
    response = service.datasets().list(projectId='PROJECTID').execute(http=http)

    pprint.pprint(response)

Upvotes: 2

Views: 1121

Answers (1)

Tim Hoffman
Tim Hoffman

Reputation: 12986

If you read the docs you reference it says

The google-api-python-client library is built on top of the httplib2 library, which is not thread-safe. Therefore, if you are running as a multi-threaded application, each thread that you are making requests from must have its own instance of httplib2.Http().

They then go on to show you how to do this. If you follow the instructions then yes it will be.

You sample code is too simple and isn't attempting what was outlined in the docs

# Create a new Http() object for every request
  def build_request(http, *args, **kwargs):
    new_http = httplib2.Http()
    return apiclient.http.HttpRequest(new_http, *args, **kwargs)
  service = build('api_name', 'api_version', requestBuilder=build_request)

  # Pass in a new Http() manually for every request
  service = build('api_name', 'api_version')
  http = httplib2.Http()
  service.stamps().list().execute(http=http)

So if you tried your code in a threaded situation it would not be thread safe. If you are just trying that code from REPL then I doubt you are in a threaded situation.

Upvotes: 2

Related Questions