CODEWITHSUNDEEP
arraysjsonbashjq
ma7555
ma7555

Reputation: 400

Modify a JSON array with inputs from a shell array in only a single pass

Is it possible to filter an entire array of items in JQ in only one pass? Compare the following code, which runs jq over and over:

{
      "foofoo": {
        "barbar": [
          {
            "foo": "aaaa",
            "bar": 0000
          },
          {
            "foo": "bbbb",
            "bar": 1111
          },
          {
            "foo": "cccc",
            "bar": 2222
          }
          ]
      }
}

bash array:

array=("1111" "2222")

my code is working but not very efficient and uses a lot of resources considering the array size in reality:

for k in "${array[@]}"; do 
    jq  --argjson k "$k"  '.foofoo.barbar |= map(select(.bar != $k))' json.json | sponge json.json 
done

It keeps looping through the array, removing the unneeded entries and storing same file again by using sponge.

any ideas how to achieve a similar behavior with a lighter code?

Desired output:

{
  "foofoo": {
    "barbar": [
      {
        "foo": "aaaa",
        "bar": 0
      }
    ]
  }
}

Upvotes: 1

Views: 295

Answers (3)

peak
peak

Reputation: 116690

Constructing a dictionary object opens the door to an efficient solution. If your jq has INDEX/2, you could use the following invocation:

jq --arg p "${arr[*]}" '
  INDEX($p | split(" ")[]; .) as $dict
  | .foofoo.barbar 
      |= map(select($dict[.bar|tostring] | not))'

If your jq does not have INDEX/2, then now would be an excellent time to upgrade; otherwise, you could snarf its def by googling: jq "def INDEX"

Upvotes: 0

RomanPerekhrest
RomanPerekhrest

Reputation: 92854

To improve the performance significantly use the following jq approach (without any shell loops):

arr=("1111" "2222")
jq '($p | split(" ") | map(tonumber)) as $exclude 
    | .foofoo.barbar 
      |= map(select(.bar as $b 
                    | any($exclude[]; . == $b) | not))' \
    --arg p "${arr[*]}" file.json | sponge file.json

The output:

{
  "foofoo": {
    "barbar": [
      {
        "foo": "aaaa",
        "bar": 0
      }
    ]
  }
}

Upvotes: 2

glenn jackman
glenn jackman

Reputation: 246764

I'm positive there are better ways to do this: I really just throw stuff at jq until something sticks to the wall ...

# 1. in the shell, construct a JSON object string from the array => {"bbbb":1,"cccc":1}
printf -v jsonobj '{%s}' "$(printf '"%q":1\n' "${array[@]}" | paste -sd,)"

# 2. use that to test for non-membership in the jq select function
jq --argjson o "$jsonobj" '.foofoo.barbar |= map(select((.bar|in($o)) == false))' json.json

outputs

{
  "foofoo": {
    "barbar": [
      {
        "foo": "0000",
        "bar": "aaaa"
      }
    ]
  }
}

You don't actually show your desired output, so I assume this is what you want.

Upvotes: 1

Related Questions