CODEWITHSUNDEEP
pythongoogle-cloud-platformairflowgoogle-cloud-composer
Dudes
Dudes

Reputation: 95

Get the client_id of the IAM proxy on GCP Cloud composer

I'm trying to trigger Airflow DAG inside of a composer environment with cloud functions. In order to do that I need to get the client id as described here. I've tried with curl command but it doesn't return any value. With a python script I keep getting this error:

Traceback (most recent call last):
  File "get_client_id.py", line 55, in <module>
    get_client_id(
  File "get_client_id.py", line 40, in get_client_id
    print(query_string['client_id'][0])
KeyError: 'client_id'

Python code:

import google.auth
import google.auth.transport.requests
import requests
import six.moves.urllib.parse

# Authenticate with Google Cloud.
# See: https://cloud.google.com/docs/authentication/getting-started
credentials, _ = google.auth.default(
    scopes=['https://www.googleapis.com/auth/cloud-platform'])
authed_session = google.auth.transport.requests.AuthorizedSession(
    credentials)

project_id = 'my-project'
location = 'my-region'
composer_environment = 'my-env'

environment_url = (
    'https://composer.googleapis.com/v1beta1/projects/{}/locations/{}'
    '/environments/{}').format(project_id, location, composer_environment)
composer_response = authed_session.request('GET', environment_url)
environment_data = composer_response.json()
airflow_uri = environment_data['config']['airflowUri']



# The Composer environment response does not include the IAP client ID.
# Make a second, unauthenticated HTTP request to the web server to get the
# redirect URI.
redirect_response = requests.get(airflow_uri, allow_redirects=False)
redirect_location = redirect_response.headers['location']

# Extract the client_id query parameter from the redirect.
parsed = six.moves.urllib.parse.urlparse(redirect_location)
query_string = six.moves.urllib.parse.parse_qs(parsed.query)
print(query_string['client_id'][0])

cURL command:

curl -v my_airflow_url 2>&1 >/dev/null | grep -o "client_id\=[A-Za-z0-9-]*\.apps\.googleusercontent\.com"

Does anybody have an idea how to get the Get the client_id of the IAM proxy?

Upvotes: 4

Views: 1292

Answers (2)

rafalbiegacz
rafalbiegacz

Reputation: 96

Here is the guide with instructions describing how to trigger a DAG in Composer 2/Airflow - https://cloud.google.com/composer/docs/composer-2/access-airflow-api

Upvotes: 0

PjoterS
PjoterS

Reputation: 14092

Posting this Community Wiki for better visibility.

As mentioned in the comment section by @LEC this configuration is compatible with Cloud Composer V1 which can be found in GCP Documentation Triggering DAGs with Cloud Functions.

At the moment there can be found two tabs Cloud Composer 1 Guides and Cloud Composer 2 Guides. Under Cloud Composer 1 is code used by the OP, but if you will check Cloud Composer 2 under Manage DAGs > Triggering DAGs with Cloud Functions you will get information that there is not proper documentation yet.

This documentation page for Cloud Composer 2 is not yet available. Please use the page for Cloud Composer 1.

As solution, please use Cloud Composer V1.

Upvotes: 1

Related Questions