jq - filter using list of object identifier-index stored in a variable

Question

I'm writing a reusable Bash script that allows me to query an API to get data about parliamentary members and store it in csv format.

The json response of the API has a lot of keys (name, gender, birthdate, commission membership, vote...), and, depending on what I'd like to do, I do not always want to capture the same keys. So I would like to abstract this part of the code in order to be able to write:

mp_keys= a,b,c,d

curl https://mp.com | jq '. | [$mp_keys] | @csv'

so that it is interpreted by jq as

jq '. | [.a, .b, .c, .d] | @ csv' 

I don't have a set structure for the variable format, so it could be:

• mp_keys="a,b,c,d" or
• mp_keys=".a, .b, .c, .d" or
• mp_key1=a, mp_key2=b, mp_key3=c, mp_key4=d

I know that in jq I can use the following structure:

jq --arg mp_key1 "${mp_key1}" --arg mp_key2 "${mp_key2}" --arg mp_key3 "${mp_key3}" --arg mp_key4 "${mp_key4}" '. 
| [.[$mp_key1], .[$mp_key2], .[$mp_key3], .[$mp_key4]] 
| @csv'

But it obviously becomes tedious very quickly.

Lastly I could also build the jq command as a string then apply eval to it but I would prefer using a proper jq solution.

Edit

I will break down @peak's answer for future reference, as it's very useful to understand what's happening. His answer is reproduced below:

mp_keys="a,b,c,d"

echo '{"a":1, "b":2, "c":3, "d": 4}' |
jq -r --arg mpk "$mp_keys" '
($mpk|split(",")) as $mpkeys
| [ .[ $mpkeys[] ] ]
| @csv '

First, it's important to understand that jq will:

1. evaluate the value of $mpkeys first, which implies doing the split(",") on $mpk
2. then pass the json sent through echo.

So we can do the same to understand what's happening. The ( ) tells jq to process this section in priority, so we can start by replacing the parenthesis with its results.

1. the string "a,b,c,d", stored in $mpk, is split and stored into the array ["a","b","c","d"] as described in the section on split(str)of jq's manual.
2. The array is subsequently stored into $mpkeys through as.

Which means that an equivalent of the initial code can be written as:

echo '{"a":1, "b":2, "c":3, "d": 4}' |
jq -r --arg mpk "$mp_keys" '
["a","b","c","d] as $mpkeys
| [ .[ $mpkeys[] ] ]
| @csv '

Of course now the --arg is useless, as $mpk was there to hold our initial string. So we can simplify further with:

echo '{"a":1, "b":2, "c":3, "d": 4}' |
jq -r '["a","b","c","d] as $mpkeys
| [ .[ $mpkeys[] ] ]
| @csv '

Let's now break down [ .[ $mpkeys[] ] ]:

• because $mpkeys is an array, $mpkeys[] stands for the individual elements of the array, equivalent to "a","b","c","d". Instead of a single element (an array), we now have 4 elements (strings), which will be transformed individually by the filters around them (the brackets)
• $mpkeys[] is then wrapped into .[] which applies to each of the 4 elements, and is consequently equivalent to .["a"], .["b"], .["c"], .["d"]. Each of those elements is a generic object index, which are equivalent to the object-identifier index form (.a, .b, .c, .d) as described in jq's manual.
• the final outer [ ] simply wraps everything inside an array, which is necessary to pass the result to @csv.

So the equivalent to the code above is:

echo '{"a":1, "b":2, "c":3, "d": 4}' |
jq -r '["a","b","c","d] as $mpkeys
| [ .["a"], .["b"], .["c"], .["d"] ]
| @csv '

Here $mpkeys serves no purpose, so we actually have:

 echo '{"a":1, "b":2, "c":3, "d": 4}' |
jq -r ' [ .["a"], .["b"], .["c"], .["d"] ]
| @csv '

There we are.

Answer 1

Using your approach:

mp_keys="a,b,c,d"

echo '{"a":1, "b":2, "c":3, "d": 4}' |
  jq -r --arg mpk "$mp_keys" '
    ($mpk|split(",")) as $mpkeys
    | [ .[ $mpkeys[] ] ]
    | @csv '

would yield:

1,2,3,4