CODEWITHSUNDEEP
airflowgoogle-cloud-composergoogle-cloud-auth
Gaurang Shah
Gaurang Shah

Reputation: 12890

calling google composer (airflow) dag using google cloud function

I am trying to call composer dag using API call from cloud function. I am able to call this DAG using rest api using my user and access token. which shows that API is enabled and proper authentication is enabled.

However, when I am trying with service account I am getting Error 401 (Unauthorized)!!1 error.

Following is my code:

def dag_run(url, method='GET', **kwargs):
    google_open_id_connect_token = id_token.fetch_id_token(Request(), "https://xxx-dot-us-east1.composer.googleusercontent.com")
    print(google_open_id_connect_token)
    resp = requests.request(
      method, url, headers={'Authorization': 'Bearer {}'.format(google_open_id_connect_token)}, **kwargs)
    if resp.status_code == 403:
      raise Exception('Service account does not have permission to access the IAP-protected application.')
    elif resp.status_code != 200:
      raise Exception('Bad response from application: {!r} / {!r} / {!r}'.format(resp.status_code, resp.headers, resp.text))


dag_run("https://xxxx-dot-us-east1.composer.googleusercontent.com/api/v1/dags")

service account associated with cloud functions has composer administrator role. I know only user role is required but it wasn't working and so I changed it.

Upvotes: 2

Views: 809

Answers (1)

Raul Saucedo
Raul Saucedo

Reputation: 1780

Probably it’s a permission error with your service account. You can check this two things:

1.The email of the service account must contain less than 64 characters. Because in Airflow tables the field email has a maximum 64 character(character varying(64)) You can see this link.

2.Via Cloud Console you can add the user manually, using this commands:

gcloud composer environments run <instance-name> --location=<location> users -- create --use-random-password --username "accounts.google.com:<service_accounts_uid>" --role Op --email  <service-account-username>@<...>.iam.gserviceaccount.com -f Service -l Account

You can use this commands to list all the users:

gcloud composer environments run <env_name> --location=<env_loc> users -- list

Upvotes: 1

Related Questions