CODEWITHSUNDEEP
jsonbashshellshjq
Felipe
Felipe

Reputation: 7563

update json with jq through shell script

I created a shell script to create and update json files using jq. To create json files is working well. I have a variable that is passed as argument to jq command

#!/bin/sh
OPER=$1
FILE_PATH=$2
DATE_TIME=`date +%Y-%m-%d:%H:%M:%S`
DATE=`date +%Y-%m-%d`
CSV=$3
STEP=$4
STATUS=$5
CODE=$6
MESSAGE=$7

if [ "$#" -eq 7 ]; then
    if [ "$OPER" == "create" ]; then
        # echo "FILE_PATH: $FILE_PATH - CSV: $CSV - STEP: $STEP - STATUS: $STATUS - CODE: $CODE - MESSAGE: $MESSAGE"
        REPORT="{\"date\": \"$DATE\", \"csv\": \"$CSV\", \"messages\": [{ \"timestamp\": \"$DATE_TIME\", \"step\": \"$STEP\", \"status\": \"$STATUS\", \"code\": \"$CODE\", \"message\": \"$MESSAGE\" }] }"
        echo ${REPORT} | jq . > $FILE_PATH
    elif [ "$OPER" == "update" ]; then
                echo "FILE_PATH: $FILE_PATH - CSV: $CSV - STEP: $STEP - STATUS: $STATUS - CODE: $CODE - MESSAGE: $MESSAGE"
        REPORT="{\"timestamp\": \"$DATE_TIME\", \"step\": \"$STEP\", \"status\": \"$STATUS\", \"code\": \"$CODE\", \"message\": \"$MESSAGE\"}"
        echo "REPORTTTTT: "$REPORT
        REPORT="jq '.messages[.messages| length] |= . + $REPORT' $FILE_PATH"
        #echo $REPORT
        echo `jq '.messages[.messages| length] |= . + $REPORT' $FILE_PATH` > $FILE_PATH
    else
        echo "operation not recognized: $OPER"
    fi
else
        echo "wrong parameters."
fi
jq . $FILE_PATH

But to update the json file I am getting an error. My $REPORT variable is correct. The cotes are correct. I think I must use another jq argument instead of |= . +. I used that command with plain text and it worked. But when I create the REPORT variable dynamically it throws an error.

Any clue? Thanks

REPORTTTTT: {"timestamp": "2017-02-17:12:11:11", "step": "2", "status": "OK", "code": "34", "message": "message 34 file.xml"}
jq: error: syntax error, unexpected INVALID_CHARACTER, expecting $end (Unix shell quoting issues?) at <top-level>, line 1:
'.messages[.messages|
jq: 1 compile error

jq: error: REPORT/0 is not defined at <top-level>, line 1:
.messages[.messages| length] |= . + $REPORT                                    
jq: 1 compile error

Here is the example at the command line>>

$ jq . file.json 
{
  "date": "2017-02-17",
  "csv": "file.csv",
  "messages": [
    {
      "timestamp": "2017-02-17:12:31:21",
      "step": "1",
      "status": "OK",
      "code": "33",
      "message": "message 33"
    }
  ]
}
$ export REPORT="{\"timestamp\": \"2017-02-17:11:51:14\", \"step\": \"2\", \"status\": \"OK\", \"code\": \"34\", \"message\": \"message 34 file.xml\"}"
$ echo $REPORT
{"timestamp": "2017-02-17:11:51:14", "step": "2", "status": "OK", "code": "34", "message": "message 34 file.xml"}
$ jq '.messages[.messages| length] |= . + $REPORT' file.json 
jq: error: REPORT/0 is not defined at <top-level>, line 1:
.messages[.messages| length] |= . + $REPORT                                    
jq: 1 compile error

Upvotes: 0

Views: 4909

Answers (3)

chepner
chepner

Reputation: 531175

Don't generate JSON by hand; let jq do it. This is important if the values you want to add to the JSON need to be quoted properly, which will not happen if you are just performing string interpolation in the shell:

$ foo='"hi" he said'
$ json="{ \"text\": \"$foo\" }"  
$ echo "$json"
{ "text": ""hi" he said" }  # Wrong: should be { "text": "\"hi\" he said" }

Further, jq can generate the date and time strings; there is no need to run date (twice) as well.

#!/bin/sh

if [ $# -eq 7 ]; then
  printf "Wrong number of parameters: %d\n" "$#" >&2
  exit 1
fi

oper=$1 file_path=$2

# Generate the new message using arguments 3-7
new_message=$(
  jq -n --arg csv "$3" --arg step "$4" \
        --arg status "$5" --arg code "$6" \
        --arg message "$7" '{
  timestamp: now|strftime("%F:%T"),
  csv: $csv, step: $step, status: $status, code: $code, message: $message}'
)

case $oper in
  create)
    jq -n --argjson new_msg "$new_message" --arg csv "$3" '{
      date: now|strftime("%F"),
      csv: $csv,
      messages: [ $new_msg ]
    }' > "$file_path"
    ;;

  update)
    jq --argjson new_msg "$new_message" \
       '.messages += [ $new_msg ]' \
       "$file_path" > "$file_path.tmp" && mv "$file_path.tmp" "$file_path" ;;

  *) printf 'operation not recognized: %s\n' "$oper" >&2
     exit 1 ;;
esac
jq '.' "$file_path"

Upvotes: 1

Felipe
Felipe

Reputation: 7563

Thanks @Inian. I changed a little bit but worked.... here is the solution.

if [ "$#" -eq 7 ]; then
    if [ "$OPER" == "update" ]; then
        echo "update Json"
                echo "FILE_PATH: $FILE_PATH - CSV: $CSV - STEP: $STEP - STATUS: $STATUS - CODE: $CODE - MESSAGE: $MESSAGE"
        REPORT="{\"timestamp\": \"$DATE_TIME\", \"step\": \"$STEP\", \"status\": \"$STATUS\", \"code\": \"$CODE\", \"message\": \"$MESSAGE\"}"
        echo "REPORTTTTT: "$REPORT
        echo `jq --argjson args "$REPORT" '.data.messages += [$args]' $FILE_PATH` > $FILE_PATH
    else
        echo "operation not recognized: $OPER"
    fi
else
        echo "wrong parameters."
fi

Upvotes: 0

Inian
Inian

Reputation: 85600

Pass the argument $REPORT as a json argument using the --argjson flag supported from jq-1.5 onwards,

--argjson name JSON-text:    
This option passes a JSON-encoded value to the jq program as a predefined variable.

Change your line to,

jq --argjson args "$REPORT" '.data.messages += [$args]' file

Upvotes: 3

Related Questions