CODEWITHSUNDEEP
jsonjq
rmfossi
rmfossi

Reputation: 91

combining multiple json files into a single json file with jq filters

I have multiple json files that I need to comnine into one giant json file. I know I can use the basic -s and get a giant file jq -s . littlefiles*.json >> giantfile.json

My issue is I cannot seem to figure out how to filter things out and still combine them into a valid json. All my efforts have yielded the right filtered output, but it's missing the "," between. I've seen some answers on the webz on using "map" but I don't see its application on my filtering. Maybe I'm using jq all wrong for what I need?

here's a cat of the jq

rfossi@xAMrfossi Responses % cat BBB.json | jq '.[] | { repo: .url[37:-1] | [rtrimstr("/branches/master/protectio")], PushUsers: [.restrictions.users[].login], PushTeams: [.restrictions.teams[].slug]}'
{
  "repo": [
    "homelaunch_basic"
  ],
  "PushUsers": [],
  "PushTeams": [
    "master-branch-mergers"
  ]
}
{
  "repo": [
    "homelaunch_base"
  ],
  "PushUsers": [],
  "PushTeams": []
}

As you can see, I get the info I need. But it's missing the "," that makes it a valid json file. When I attempt to combine the files with this same filtering, I get one of the dreaded Cannot index array with string messages (basic example below). I've been looking at this too long I think, and I'm missing something obvious.

rfossi@xAMrfossi Responses % jq -r '.[] | { url: .url, users: [.restrictions.users[].login]}' B*.json > Bgiant.json
jq: error (at B1get-branch-prot-1641247960841.json:0): Cannot index string with string "url"
jq: error (at B2get-branch-prot-1641247961044.json:0): Cannot index string with string "url"
jq: error (at Bgiantfile.json:5): Cannot index array with string "url"
jq: error (at Bgiantfile.json:10): Cannot index array with string "url"
rfossi@xAMrfossi Responses %

Upvotes: 1

Views: 711

Answers (1)

pmf
pmf

Reputation: 36276

What you have received is two valid JSON documents (or a stream, as it is called in the jq manuals). If you put a comma in between, it wouldn't be valid JSON anymore because then it'd be neither a stream of two documents anymore (as of the superfluous comma) nor one array with two elements in it (as of no surrounding array brackets). You can, however, add those missing brackets, too, making it an array, if this is what you want.

To achieve this, surround your entire filter with brackets jq '[.[] | {…}]' and jq will take care of the commas. Also, as your filter's initial part .[] was already decomposing an array (or an object), you could simplify the array creation by exchanging it for a map with the same effect: jq 'map({…})'.

Upvotes: 2

Related Questions