CODEWITHSUNDEEP
jsonbashjq
AmineH7
AmineH7

Reputation: 73

Build a json in bash with jq

I want to create a new json file from the variables which are fetched from another json. I am a beginner with jq.

while read -r id name name_with_namespace
do
   echo "$id : $name"
   vivariables=$(curl --progress-bar -X GET --header "Content-Type: application/json" --header "Authorization: bearer ${token}" "$GitlabServerURL/api/v4/projects/$id/variables")
   JSON_STRING=$( jq -n \
                  --arg id "$id" \
                  --arg project_name "$name" \
                  --arg project_path "$name_with_namespace" \
                  --arg project_variables "$variables" \
                  '{id: $id, project_name: $project_name, project_path: $project_path, project_variables: $project_variables}' )
   echo $JSON_STRING >> variables_projects.json
done < <(jq -r '.[] | "\(.id) \(.name) \(.name_with_namespace) "' < projects.json)

Example of part of the data that I recover with my script :

{ "id": "200", "project_name": "spring-boot-remove", "project_path": "agpm / templates / ansible / spring-boot-remove", "project_variables": "[]" }
{ "id": "199", "project_name": "agpm-gitleaks", "project_path": "DevSecOps / agpm-gitleaks", "project_variables": "[]" }
{ "id": "198", "project_name": "haproxy", "project_path": "Infra / Projets / nexo / haproxy", "project_variables": "[]" }

What I hope to get :

[{ "id": "200", "project_name": "spring-boot-remove", "project_path": "agpm / templates / ansible / spring-boot-remove", "project_variables": "[]" },
{ "id": "199", "project_name": "agpm-gitleaks", "project_path": "DevSecOps / agpm-gitleaks", "project_variables": "[]" },
{ "id": "198", "project_name": "haproxy", "project_path": "Infra / Projets / nexo / haproxy", "project_variables": "[]" }]

Upvotes: 0

Views: 181

Answers (1)

dibery
dibery

Reputation: 3460

This is because you produce the JSON object line by line in a bash read loop. I think the fastest way to achieve your goal is dealing with the output file (variables_projects.json).

jq -s . variables_projects.json | sponge variables_projects.json

Output

[
  {
    "id": "200",
    "project_name": "spring-boot-remove",
    "project_path": "agpm / templates / ansible / spring-boot-remove",
    "project_variables": "[]"
  },
  {
    "id": "199",
    "project_name": "agpm-gitleaks",
    "project_path": "DevSecOps / agpm-gitleaks",
    "project_variables": "[]"
  },
  {
    "id": "198",
    "project_name": "haproxy",
    "project_path": "Infra / Projets / nexo / haproxy",
    "project_variables": "[]"
  }
]

For a compact output, add the -c option:

jq -cs . variables_projects.json | sponge variables_projects.json

Output:

[{"id":"200","project_name":"spring-boot-remove","project_path":"agpm / templates / ansible / spring-boot-remove","project_variables":"[]"},{"id":"199","project_name":"agpm-gitleaks","project_path":"DevSecOps / agpm-gitleaks","project_variables":"[]"},{"id":"198","project_name":"haproxy","project_path":"Infra / Projets / nexo / haproxy","project_variables":"[]"}]

From the doc:

--slurp/-s:
      Instead  of running the filter for each JSON object in the input, read the entire input stream into a large array and run the filter just once.
--compact-output / -c:
      By default, jq pretty-prints JSON output. Using this option will result in more compact output  by  instead putting each JSON object on a single line.

Upvotes: 1

Related Questions