How to get ALL the GCP folders from an Organization with `gcloud`?

Question

I was wondering if you have a way to find all projects/folders in an organization. Currently gcloud allows to get both folders and projects given the parent, delegating the recursive action to the CLIENT.

It looks like there is a way to Search via Resource Manager APIs, although unfortunately there's no gcloud equivalent command. This does work:

$ alias curl-oauth='curl -H "Authorization: Bearer $(gcloud auth print-access-token)"'
$ curl-oauth -X POST https://cloudresourcemanager.googleapis.com/v2/folders:search '{}'

Thanks for any help or pointer to existing code to implement this client side or via gcloud.

Answer 1

I was finally able to get a recursive ruby version (thanks Daz for the Hint!).

It's on github

• it does populate all the org (you can use a maxlevel).
• it creates a graphviz dot and graphs it (cool!)

I'm attaching the result of my org with maxlevel=2 (which looks like 3 - need to fix it).

Ruby code on github: https://github.com/palladius/org-folder-projects-graph/

Sample output:

🌲 824879804362 # 'palladi.us'
├─ 🍕 xpn-main (398198244705)
├─ 🍕 orgnode-palladi-us (704861684515)
├─ 📁 993609995477 (customers)
    ├─ 📁 571390668780 (dirimpettai di EURF)
├─ 📁 885056483479 (dev and test)
    ├─ 🍕 prova-123-dentro-palladi-us (237925736669)
    ├─ 🍕 palladius-eu (177178925177)
    ├─ 🍕 prova123-160016 (262470358174)
    ├─ 🍕 folder-test-prod (1025012666423)
    ├─ 🍕 folder-test-dev (351173986048)
    ├─ 📁 128544652663 (folderillo)
├─ 📁 887288965373 (prod stuff - for real)
    ├─ 🍕 gbanana-prod (626662139195)
├─ 📁 510416893777 (TFR Terraformed by Ricc)
    ├─ 📁 93350088776 (TF DEV)
        ├─ 📁 723110142384 (TF0b - titius)
        ├─ 📁 1026736501110 (TF1b - caius)
        ├─ 📁 802144187596 (TF2b - sempronius)
    ├─ 📁 99919748229 (TF PROD)

Answer 2

Use Asset Inventory in GCP to pull all the Folders from Organization.

gcloud asset search-all-resources \
  --scope='organizations/<Organization_ID>' \
  --asset-types='cloudresourcemanager.googleapis.com/Folder' \
  --order-by='createTime' \
  --query='state:ACTIVE'

Answer 3

Thank you @DazWilkin for your script. I used this and added to it, as I needed projects within the folders too. Hopefully this code can help someone else.

#!/usr/bin/env bash

: "${ORGANIZATION:?Need to export ORGANIZATION and it must be non-empty}"

# gcloud format
FORMAT="csv[no-heading](name,displayName.encode(base64))"
FORMAT_PRJ="table[box,title='Folder ${NAME} Project List'] \
(createTime:sort=1,name,projectNumber,projectId:label=ProjectID,parent.id:label=Parent)"

# Enumerates Folders recursively
folders()
# project()
{
  LINES=("$@")
  for LINE in ${LINES[@]}
  do
    # Parses lines of the form folder,name
    VALUES=(${LINE//,/ })
    FOLDER=${VALUES[0]}

    # Decodes the encoded name
    NAME=$(echo ${VALUES[1]} | base64 --decode)
    printf "Folder: ${FOLDER} (${NAME})\n\n"

    printf "Project: Project info: \n\n"
    project=$(gcloud projects list \
      --filter parent.id:${FOLDER} \
      --format="${FORMAT_PRJ}")

    if [ -z "$project" ]
    then
      printf "Folder: ${FOLDER} - ${NAME} has no sub-projects\n\n"
    else
      printf "Parent FolderID: ${FOLDER}\t Parent Name(s): ${NAME}\n${project} \n\n"
    fi

    folders $(gcloud resource-manager folders list \
      --folder=${FOLDER} \
      --format="${FORMAT}")

  done
}

# Start at the Org
printf "Org: ${ORGANIZATION}\n\n"
LINES=$(gcloud resource-manager folders list \
  --organization=${ORGANIZATION} \
  --format="${FORMAT}")

# Descend
folders ${LINES[0]}

Answer 4

Updated

OK, here's a (still) rough (but now working) script (thanks @riccardo for the access to a test Org):

#!/usr/bin/env bash

: "${ORGANIZATION:?Need to export ORGANIZATION and it must be non-empty}"

# gcloud format
FORMAT="csv[no-heading](name,displayName.encode(base64))"

# Enumerates Folders recursively
folders()
{
  LINES=("$@")
  for LINE in ${LINES[@]}
  do
    # Parses lines of the form folder,name
    VALUES=(${LINE//,/ })
    FOLDER=${VALUES[0]}
    # Decodes the encoded name
    NAME=$(echo ${VALUES[1]} | base64 --decode)
    echo "Folder: ${FOLDER} (${NAME})"
    folders $(gcloud resource-manager folders list \
      --folder=${FOLDER} \
      --format="${FORMAT}")
  done
}

# Start at the Org
echo "Org: ${ORGANIZATION}"
LINES=$(gcloud resource-manager folders list \
  --organization=${ORGANIZATION} \
  --format="${FORMAT}")

# Descend
folders ${LINES[0]}

I'm challenged to do this more effectively in Bash; I'm interested to see whether more experienced Bash users have suggestions.

The script uses a gcloud projection to base64 encode displayName values. This is a hacky way to avoid inadvertent parsing of these strings (they may contain spaces) and I was unable to find a way to escape these otherwise.

Still doesn't indent levels 😞

Previous

Super rough... and regrettably without access to either an Org or Folders so, mostly guessing...

#!/usr/bin/env bash

ORGANIZATION="..."

folders()
{
  for FOLDER in ${1}
  do
    echo ${FOLDER}
    # Use the `--folder` variant for children
    folders $(gcloud resource-manager folders list \
    --folder=${FOLDER} \
    --format="value(name)")
  done
}

# Start at the Org
echo "Org: ${ORGANIZATION}"
folders $(\
  gcloud resource-manager folders list \
  --organization=${ORGANIZATION} \
  --format="value(name)")

Various obvious issues (and likely many others)

1. No error management
2. Unclear what the returned format is and how to grab folderId
3. No indenting of levels; I suspect you'd need to add an arg for this
4. May need to convert the returned values into bash arrays

If you can provide the schema for gcloud resource-manager folders list ... --format=yaml, can try to address #2

You asked for gcloud but this may better done using e.g. Python