How to list all projects inside GCP organization using Resource Manager API?

Question

I am trying to list all projects I have in GCP by using the projects.list method from the Resource Manager API but I cannot figure out what access token to be used.

Let's say I have three projects:

• My Project 44572
• Testing
• My First Project

And when I use the projects.list method it should give me the output as this:

{
  "projects": [
    {
      "name": "My Project 44572", 
      "parent": {
        "type": "organization", 
        "id": "ORG_ID"
      }, 
      "projectId": "PROJECT_ID", 
      "projectNumber": "PROJECT_NUMBER", 
      "lifecycleState": "ACTIVE", 
      "createTime": "2020-06-15T08:38:04.712Z"
    }, 
    {
      "name": "Testing ", 
      "parent": {
        "type": "organization", 
        "id": "ORG_ID"
      }, 
      "projectId": "PROJECT_ID", 
      "projectNumber": "PROJECT_NUMBER", 
      "lifecycleState": "ACTIVE", 
      "createTime": "2020-06-15T08:35:59.480Z"
    }, 
    {
      "name": "My First Project", 
      "parent": {
        "type": "organization", 
        "id": "ORG_ID"
      }, 
      "projectId": "PROJECT_ID", 
      "projectNumber": "PROJECT_NUMBER", 
      "lifecycleState": "ACTIVE", 
      "createTime": "2020-06-15T08:33:23.859Z"
    }
  ]
}

But if I am using the Access Token which I got by using service account of My Project 44572 I am getting the following output:

{
  "projects": [
    {
      "name": "My Project 44572", 
      "parent": {
        "type": "organization", 
        "id": "ORG_ID"
      }, 
      "projectId": "PROJECT_ID", 
      "projectNumber": "PROJECT_NUMBER", 
      "lifecycleState": "ACTIVE", 
      "createTime": "2020-06-15T08:38:04.712Z"
    }
]
}

So, what access token I should use so that I would get all projects under my organization?

Answer 1

I have written a script that does exactly that. I have also documented the same in blog post: https://medium.com/living-devops/exploring-google-resource-manager-with-python-v1-0-ea0aeab57c53

here is the code snippet

from google.cloud import resourcemanager_v3

def get_folders(
    parent_id = "organizations/ORGANIZATION_ID",
    folders = None):

# This function will return a list of folder_id for all the folders and 
# subfolders respectively

    if folders is None:
        folders = []

# Creating folder client 
    client = resourcemanager_v3.FoldersClient()
    request = resourcemanager_v3.ListFoldersRequest(
        parent=parent_id,
    )

    page_result = client.list_folders(request=request)
    for pages in page_result:
        folders.append(pages.name)
        get_folders(parent_id=pages.name, folders=folders)
    return folders


def search_projects(folder_id):
# This function will take folder_id input and returns
# the list of project_id under a given folder_id

    client = resourcemanager_v3.ProjectsClient()

    query = f"parent:{folder_id}"
    request = resourcemanager_v3.SearchProjectsRequest(query=query)
    page_result = client.search_projects(request=request)
    search_result = []
    for pages in page_result:
        search_result.append(pages)
    return search_result


def list_projects():
# will returns the list of all active projects(project_id)

    active_project = []
    for folders in get_folders(parent_id="organizations/ORGANIZATION_ID", folders=None):
        for projects in search_projects(folders):
            if str(projects.state) == "State.ACTIVE":
                active_project.append(projects.project_id)

    return active_project


if __name__ == "__main__":
    print(list_projects())

Answer 2

The projects.list method requires the permission resourcemanager.projects.get. You can grant the account any role containing the permission at org level to achieve your goal, e.g., -

• roles/viewer
• roles/editor
• roles/owner

Alternatively, you can use the resources.searchAll method which requires the cloudasset.assets.searchAllResources permission. You can grant the account any role with this permission at org level:

• roles/cloudasset.viewer
• roles/cloudasset.owner
• roles/viewer
• roles/editor
• roles/owner

To list all the projects within an organization 456:

gcloud asset search-all-resources \
--asset-types="cloudresourcemanager.googleapis.com/Project"
--scope=organizations/456

Documentation:

• https://cloud.google.com/asset-inventory/docs/searching-resources
• https://cloud.google.com/asset-inventory/docs/reference/rest/v1p1beta1/resources/searchAll

Related post:

• How to find, list, or search resources across services (APIs) and projects in Google Cloud Platform?