CODEWITHSUNDEEP
apigitlabgitlab-api
Adonist
Adonist

Reputation: 153

Get full list of Groups and Projects in Gitlab Cloud

I'm trying to get a full list of Projects and groups in out Gitlab cloud account. I'm currently using their documentation as reference (bear in mind though I'm no developer) and using Linux command line to do so. Here's the documentation I'm trying to use:

https://docs.gitlab.com/ee/api/projects.html

https://docs.gitlab.com/ee/api/groups.html#list-a-groups-projects

I'm using the following command to get the data and parse in a readable format that I will export to csv or spreadsheet afterwards:

curl --header "PRIVATE-TOKEN: $TOKEN" "https://gitlab.com/api/v4/projects/?owned=yes&per_page=1000&page=1" | python -m json.tool | grep -E "http_url_to_repo|visibility" | awk '!(NR%2){print$0p}{p=$0}' | awk '{print $4,$2}' | sed -E 's/\"|\,//g' > gitlab.txt

My problem is that the code only return about 100 of the 280 repositories we have. It doesn't seem to get it recursively from all the groups and subgroups. Any ideas on how I can improve this search to get everything ?

Thank you

Upvotes: 0

Views: 4767

Answers (1)

furas
furas

Reputation: 142631

It seems it can get only 100 per page so you will have to run it two times - first with page=1 and next with page=2. And for second page you will need >> to append to existing file gitlab.txt

curl --header "..." "https://...&per_page=100&page=1" | ...  > gitlab.txt

curl --header "..." "https://...&per_page=100&page=2" | ...  >> gitlab.txt

Or you will have to write script which first get all pages and later send it to pipe. You may also try to use for-loop in bash

Upvotes: 1

Related Questions