CODEWITHSUNDEEP
pythonpython-3.xgoogle-bigquerygoogle-cloud-run
Danilo Steckelberg
Danilo Steckelberg

Reputation: 338

Google Cloud Run does not find os.environ['GOOGLE_APPLICATION_CREDENTIALS'] variable

I am trying to deploy a Python app in Google Cloud Run to perform some tasks automatically and these tasks require access to my BigQuery. I have tested the implementation in localhost through Cloud Shell, and it worked just as expected. Then I created a Cloud Run Service and all functions that do not require access to BigQuery work normally, but when I they require, I get the following error: google.auth.exceptions.DefaultCredentialsError: File /XXXXXX/gbq.json was not found.

However, the file is there (the folders are correct, and I also tested adding copies of the file in other folders): enter image description here

Any suggestions to solve the problem or a workaround I could use?
Thanks in advance

ADDITIONAL INFO:

main.py function:

(the bottom part of the code is used to test the app in localhost, which works perfectly)

from flask import Flask, request
from test_py import test as t

app = Flask(__name__)


@app.get("/")
def hello():
    """Return a friendly HTTP greeting."""
    chamado = request.args.get("chamado", default="test")
    print(chamado)
    if chamado == 'test':
        dados = f'chamado = test?\n{chamado == "test"}\n{t.show_data(chamado)}'
    elif chamado == 'bigqueer':
        dados = f'chamado = test?\n{chamado == "test"}\n{t.show_bq_data()}'
    else:
        dados = f'chamado = test?\n{chamado == "test"}\n{t.show_not_data(chamado)}'
    
    print(dados)
    return dados


if __name__ == "__main__":
    # Development only: run "python main.py" and open http://localhost:8080
    # When deploying to Cloud Run, a production-grade WSGI HTTP server,
    # such as Gunicorn, will serve the app.
    app.run(host="localhost", port=8080, debug=True)

BigQuery class:

class GoogleBigQuery:
    def __init__(self):
        os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = '/XXXXXX/gbq.json'
        self.client = bq.Client()

    def executar_query(self, query):
        client_query = self.client.query(query)
        result = client_query.result()
        return result

Cloud Run deploy:

gcloud run deploy pythontest \
  --source . \
  --platform managed \
  --region $REGION \
  --allow-unauthenticated

Upvotes: 1

Views: 896

Answers (1)

guillaume blaquiere
guillaume blaquiere

Reputation: 75920

YOU DO NOT NEED THAT

Excuse my first brutal words but it's extremely dangerous to do what you do. Let me explain.

In your container, you put, in plain text a secret. Keep in mind that your container is like a zip. There is nothing secret, encrypted in it. You can convince yourselves by using dive and exploring your container layers and data.

Therefore: DO NOT DO THAT!

So now, what to do?

On Google Cloud, all the services can use the Metadata server to get credentials. The client libraries leverage it and you can rely on the default credential with you initialise your code. That mechanism is named ADC.

In your code, simpy remove that line: os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = '/XXXXXX/gbq.json'

So that, when you deploy your Cloud Run service, specify the runtime service account that you want to use. That's all! The Google Cloud environment and libraries will do the rest

Upvotes: 4

Related Questions