CODEWITHSUNDEEP
jsonunixcommand-lineminify
Ulysse BN
Ulysse BN

Reputation: 11414

How can I minify JSON in a shell script?

I've been looking for a way to uglify some JSON while in my bash console. This help using it afterward in another command (for example, to pass json inline to httpie).

Giving:

{
    "foo": "lorem",
    "bar": "ipsum"
}

I want to obtain:

{"foo":"lorem","bar":"ipsum"}

See the pretty-print counterpart.

Upvotes: 63

Views: 32724

Answers (5)

Ulysse BN
Ulysse BN

Reputation: 11414

TL;DR

no install

python -c 'import json, sys;json.dump(json.load(sys.stdin), sys.stdout)' < my.json

very fast (with jj)

jj -u < my.json

Perf benchmark

Here's the script, using hyperfine:

#!/usr/bin/env bash

tmp=$(mktemp json.XXX)
tmp_md=$(mktemp md.XXX)

trap "rm $tmp $tmp_md" EXIT

cat <<JSON > $tmp
{
    "foo": "lorem",
    "bar": "ipsum"
}
JSON
hyperfine \
    --export-markdown $tmp_md \
    --warmup 100 \
    "jj -u < $tmp" \
    "yq eval -j -I=0 < $tmp" \
    "xidel -s - -e '\$json' --printed-json-format=compact < $tmp" \
    "jq --compact-output < $tmp" \
    "python3 -c 'import json, sys;json.dump(json.load(sys.stdin), sys.stdout)' < $tmp" \
    "ruby -r json -e 'j JSON.parse \$stdin.read' < $tmp"

pbcopy < $tmp_md

The result on my mac — MacBook Air (M1, 2020), 8 GB:

Command Mean [ms] Min [ms] Max [ms] Relative
jj -u < json.p72 1.3 ± 0.2 0.9 2.7 1.00
yq eval -j -I=0 < json.p72 4.4 ± 0.4 3.8 7.8 3.37 ± 0.65
xidel -s - -e '$json' --printed-json-format=compact < json.p72 5.5 ± 0.3 5.0 6.5 4.19 ± 0.77
python3 -c 'import json, sys;json.dump(json.load(sys.stdin), sys.stdout)' < json.p72 14.0 ± 0.4 13.4 15.0 10.71 ± 1.89
jq --compact-output < json.p72 14.4 ± 2.0 13.2 33.6 11.02 ± 2.45
ruby -r json -e 'j JSON.parse $stdin.read' < json.p72 47.3 ± 0.6 46.1 48.5 36.10 ± 6.32

Result for a large JSON file (14k lines):

http https://france-geojson.gregoiredavid.fr/repo/regions.geojson | jj -p > $tmp
Command Mean [ms] Min [ms] Max [ms] Relative
jj -u < json.wFY 3.4 ± 0.7 2.7 12.2 1.00
jq --compact-output < json.wFY 35.1 ± 0.4 34.5 36.1 10.24 ± 2.23
python3 -c 'import json, sys;json.dump(json.load(sys.stdin), sys.stdout)' < json.wFY 47.4 ± 0.5 46.3 48.7 13.82 ± 3.01
xidel -s - -e '$json' --printed-json-format=compact < json.wFY 55.5 ± 1.2 54.7 63.5 16.17 ± 3.53
ruby -r json -e 'j JSON.parse $stdin.read' < json.wFY 94.9 ± 0.7 93.8 96.8 27.65 ± 6.02
yq eval -j -I=0 < json.wFY 3087.0 ± 26.6 3049.3 3126.8 899.63 ± 195.81

And here is the pretty print counterpart benchmark

Upvotes: 36

Sam Bacha
Sam Bacha

Reputation: 117

jq-minify

Here is a bash script that will write back to the file minified

works with bash v3.2+ and jq v1.6+

#!/usr/bin/env bash
set -eu
path=
options=()
# change -c to -r to get pretty-print
set -- "$@" -c .
for arg; do
  if [ -f "$arg" ]; then
    if [ -n "$path" ]; then
      echo "Cannot specify multiple paths to jq-minify" >&2
      exit 1
    fi
    path="$arg"
  else
    options+=("$arg")
  fi
done
tmp=$(mktemp)
jq "${options[@]}" "$path" >"$tmp"
cat "$tmp" >"$path"

Upvotes: -2

Reino
Reino

Reputation: 3443

With :

xidel -s input.json -e '$json' --printed-json-format=compact
#or
xidel -s input.json -e 'serialize-json($json)'
{"foo": "lorem", "bar": "ipsum"}

Interesting "benchmark", Ulysse BN.
I couldn't test jj, but on my old cpu these are my results:

var='{
    "foo": "lorem",
    "bar": "ipsum"
}'

time (for i in {1..100}; do python -c 'import json, sys;json.dump(json.load(sys.stdin), sys.stdout)' <<< "$var" >& /dev/null; done)

real    0m10.813s
user    0m7.532s
sys     0m5.798s

time (for i in {1..100}; do jq --compact-output <<< "$var" >& /dev/null; done)

real    0m10.500s
user    0m1.835s
sys     0m0.769s

time (for i in {1..100}; do xidel -se '$json' --printed-json-format=compact <<< "$var" >& /dev/null; done)

real    0m2.250s
user    0m1.692s
sys     0m0.889s

Upvotes: 0

straville
straville

Reputation: 1016

yq worked for me, via utilization of input file (containing the prettified JSON)
yq eval -j -I=0 uglify-test.txt
Docs link: https://mikefarah.gitbook.io/yq/usage/convert

Upvotes: 1

guizo
guizo

Reputation: 3125

You can use jq -c (compact) option.

jq -c . < input.json

Upvotes: 83

Related Questions