Making multiple changes to one JSON template

Question

I'm currently using jq with the 1pass CLI to try and create randomly generated passwords into a secure note. I'm having an issue with setting the fields.

These are two of my variables. I have 8 total I need to set.

section0_section_uuid="Section_0"

section1_section_uuid="Section_1"

And here are my commands to manipulate the template. I first read it in, change the first title, then save it to $template. I then pass $template into jq

template=$(cat template.json | jq --arg uuid "$section0_section_uuid" '.sections[0].title=$uuid')

template=$($template | jq --arg uuid "$section1_section_uuid" '.sections[1].title=$uuid')

echo $template

I get "file name too long." I don't think I'm passing the modified template variable in correctly. I need to do 7 more modifications to the template.json file.

Edit:

Here's the full template I'm trying to manipulate. It's 12 total changes to the template I have to make. 10 of the 12 are random numbers that I will generate. The remaining 2 of the 12 will be a generated usernames.

{
  "fields": [],
  "sections": [
    {
      "fields": [
        {
          "k": "concealed",
          "n": "[CHANGE_ME]",
          "t": "ROOT_USER_PASS",
          "v": "[CHANGE_ME]"
        },
        {
          "k": "concealed",
          "n": "[CHANGE_ME]",
          "t": "DEV_USER_PASS",
          "v": "[CHANGE_ME]"
        }
      ],
      "name": "Section_[CHANGE_ME]",
      "title": "Container SSH"
    },
    {
      "fields": [
        {
          "k": "string",
          "n": "[CHANGE_ME]",
          "t": "placeholdertext",
          "v": "[CHANGE_ME_LETTERS]"
        },
        {
          "k": "string",
          "n": "[CHANGE_ME]",
          "t": "placeholdertext",
          "v": "[CHANGE_ME_LETTERS]"
        },
        {
          "k": "concealed",
          "n": "[CHANGE_ME]",
          "t": "placeholdertext",
          "v": "[CHANGE_ME]"
        }
      ],
      "name": "Section_[CHANGE_ME]",
      "title": "MySQL"
    }
  ]
}

Answer 1

The following approach to the problem is similar to @jq170727's (in particular, the jq program is agnostic both about the number of "section_uuid" variables, and the names of the template variables), but only one invocation of jq is required (rather than three).

The other significant difference is that reduce is used to avoid the penalties associated with using tostream. A minor difference is that inputs is used to avoid reading in the "section_uuid" variable values all at once.

Note: The fillin function defined below should be sufficient for basic templating.

In the following, the "template" file is assumed to be named template.json.

template.jq

# input: a JSON entity defining a template;
# vars: a JSON object defining TEMPLATEVARIABLE-VALUE pairs
def fillin(vars):
  reduce paths as $p (.;
    getpath($p) as $v
    | if $v|type == "string" and vars[$v] 
      then setpath($p; vars[$v])
      else . 
      end);

reduce inputs as $line ({i:0, value:$template};
  (.value.sections[.i].title |= $line)
  | .i +=1)
| .value
| fillin($vars)

The script

#!/bin/bash

### Set the bash variables - as many as needed

section0_section_uuid="Section_0"
section1_section_uuid="Section_1"

ROOT_USER_PASS=RUP
DEV_USER_PASS=DUP

### Preparations for calling jq
vars=$(cat<<EOF
{
 "ROOT_USER_PASS": "$ROOT_USER_PASS",
 "DEV_USER_PASS": "$DEV_USER_PASS"
}
EOF
)

cat << EOF | jq -nR --argfile template template.json --argjson vars "$vars" -f template.jq
$section0_section_uuid
$section1_section_uuid
EOF

Output

With the (expanded) example template, the output is:

{
  "fields": [],
  "sections": [
    {
      "fields": [
        {
          "k": "concealed",
          "n": "[CHANGE_ME]",
          "t": "RUP",
          "v": "[CHANGE_ME]"
        },
        {
          "k": "concealed",
          "n": "[CHANGE_ME]",
          "t": "DUP",
          "v": "[CHANGE_ME]"
        }
      ],
      "name": "Section_[CHANGE_ME]",
      "title": "Section_0"
    },
    {
      "fields": [
        {
          "k": "string",
          "n": "[CHANGE_ME]",
          "t": "placeholdertext",
          "v": "[CHANGE_ME_LETTERS]"
        },
        {
          "k": "string",
          "n": "[CHANGE_ME]",
          "t": "placeholdertext",
          "v": "[CHANGE_ME_LETTERS]"
        },
        {
          "k": "concealed",
          "n": "[CHANGE_ME]",
          "t": "placeholdertext",
          "v": "[CHANGE_ME]"
        }
      ],
      "name": "Section_[CHANGE_ME]",
      "title": "Section_1"
    }
  ]
}

Answer 2

Here is a solution which uses jq to build a legal json array from bash variables and then uses that array with a second jq invocation to substitute the variables into the template at corresponding positions:

#!/bin/bash

# example template
template='{
  "sections": [
    { "title": "x" },
    { "title": "y" }
  ]
}'

# bash variables 
section0_section_uuid="Section_0"
section1_section_uuid="Section_1"

# put into json array with jq
uuids=$(jq -MRn '[inputs]' <<EOF
$section0_section_uuid
$section1_section_uuid
EOF)

# substitute json array into template
jq -M --argjson uuids "$uuids" '
  reduce ($uuids|keys[]) as $k (.; .sections[$k].title = $uuids[$k])
' <<< "$template"

Sample Output

{
  "sections": [
    {
      "title": "Section_0"
    },
    {
      "title": "Section_1"
    }
  ]
}

Try it online!

---

Here is a solution to a portion of the revised problem which works by replacing leaf values in the template with corresponding values from an object constructed from bash variables and passed to jq via --argjson. It should be straightforward to generalize to the complete template assuming more suitable names are chosen for replacement values then [CHANGE_ME] and [CHANGE_ME_LETTERS]

#!/bin/bash

# example template
template='{
  "fields": [],
  "sections": [ {
      "fields": [ {
          "k": "concealed",
          "n": "[SSH_ROOT_USER_N]",
          "t": "ROOT_USER_PASS",
          "v": "[SSH_ROOT_USER_V]"
      } ]
  } ]
}'

# bash variables 
SSH_ROOT_USER_N="abcd"
SSH_ROOT_USER_V="efgh"

# put into json object with jq
vars=$(jq -M . <<EOF
{
 "[SSH_ROOT_USER_N]": "$SSH_ROOT_USER_N",
 "[SSH_ROOT_USER_V]": "$SSH_ROOT_USER_V"
}
EOF)

# substitute variables into template
jq -M --argjson vars "$vars" '
reduce (tostream|select(length==2)) as [$p,$v] (
    {}
  ; setpath($p;if $v|type!="string" then . else $vars[$v]//$v end)
)
' <<< "$template"

Sample Output

{
  "fields": {},
  "sections": [
    {
      "fields": [
        {
          "k": "concealed",
          "n": "abcd",
          "t": "ROOT_USER_PASS",
          "v": "efgh"
        }
      ]
    }
  ]
}

Try it online!

Answer 3

Why not make your template an actual jq filter, rather than a JSON blob to modify?

The contents of template.jq would be

{
  sections: [
    { title: $t1 },
    { title: $t2 },
    { title: $t3 },
    { title: $t4 },
    { title: $t5 },
    { title: $t6 },
    { title: $t7 },
    { title: $t8 }
  ]
}

Then your command would simply be

$ jq -n --arg t1 foo --arg t2 bar ... -f template.jq
{
  "sections": [
    {
      "title": "foo"
    },
    {
      "title": "bar"
    },
    ...
  ]
}

One benefit of doing it this way is that you can't accidentally forget a value; jq can only process the filter if you provide definitions for all 8 variables.