Changing values in a JSON data file from shell

Question

I have created a JSON file which in this case contains:

{"ipaddr":"10.1.1.2","hostname":"host2","role":"http","status":"active"},
{"ipaddr":"10.1.1.3","hostname":"host3","role":"sql","status":"active"},
{"ipaddr":"10.1.1.4","hostname":"host4","role":"quad","status":"active"},

On other side I have a variable with values for example:

arr="10.1.1.2 10.1.1.3"

which comes from a subsequent check of the server status for example. For those values I want to change the status field to "inactive". In other words to grep the host and change its "status" value.

Expected output:

{"ipaddr":"10.1.1.2","hostname":"host2","role":"http","status":"inactive"},
{"ipaddr":"10.1.1.3","hostname":"host3","role":"sql","status":"inactive"},
{"ipaddr":"10.1.1.4","hostname":"host4","role":"quad","status":"active"},

Answer 1

Version 1: Using a simple regex based transformation. This can be done in several ways. From the initial question, the list of ipaddr is in variable in arr. Example using a Bash env variable:

$ export var="... ..."

It would be a possible solution to provide this information by command line parameters.

#!/usr/bin/perl
my %inact;                    # ipaddr to inactivate
my $arr=$ENV{arr} ;           # from external var (export arr=...)
## $arr=shift;                # from command line arg

for( split(/\s+/, $arr)){ $inact{$_}=1 }

while(<>){                    # one "json" line at the time
   if(/"ipaddr":"(.*?)"/ and $inact{$1}){
       s/"active"/"inactive"/}
   print $_;
}

Version 2:

Using Json parser we can do more complex transformations; as the input is not real JSON we will process one line of "almost json" at the time:

use JSON;

use strict;
my ($line, %inact);
my $arr=$ENV{arr} ; 

for( split(/\s+/, $arr)){ $inact{$_}=1 }

while(<>){              # one "json" line at the time
   if(/^\{.*\},/){
      s/,\n//;
      $line = from_json( $_);
      if($inact{$line->{ipaddr}}){
         $line->{status} = "inactive" ;}
      print to_json($line), ",\n"; }
   else { print $_;}
}

Answer 2

Here is a quick perl "wrap-around one-liner": that uses the JSON module and slurps with the -0 switch:

perl -MJSON -n0E '$j = decode_json($_); 
   for (@{$j->{hosts}}){$_->{status}=inactive if $_->{ipaddr}=~/2|3/} ; 
   say to_json( $j->{hosts}, {pretty=>1} )' status_data.json

might be nicer or might violate PBP recommendations for map:

perl -MJSON -n0E '$j = decode_json($_); 
   map { $_->{status}=inactive if $_->{ipaddr}=~/2|3/ } @{ $j->{hosts} }  ;
   say to_json( $j->{hosts} )' status_data.json

A shell script that resets status using jq would also be possible. Here's a quick way to parse and output changes to JSON using jq:

cat status_data.json| jq -r '.hosts |.[] |
select(.ipaddr == "10.1.1.2"//.ipaddr == "10.1.1.3" )' |jq '.status = "inactive"'

EDIT In an earlier comment I was uncertain whether the OP was more interested in an application than a quick search and replace (something about the phrases "On other side..." and "check on the server status"). Here is a (still simple) perl approach in script form:

use v5.16; #strict, warnings, say
use JSON ;
use IO::All;

my $status_data < io 'status_data.json';
my $network = JSON->new->utf8->decode($status_data) ;
my @changed_hosts= qw/10.1.1.2 10.1.1.3/;

sub status_report {
  foreach my $host ( @{ $network->{hosts} }) {
     say "$host->{hostname} is $host->{status}";
  }
}

sub change_status {
  foreach my $host ( @{ $network->{hosts} }){
    foreach (@changed_hosts) {
      $host->{status} = "inactive" if $host->{ipaddr} eq $_ ;
    }
  }
  status_report;
}

defined $ENV{CHANGE_HAPPENED} ? change_status : status_report ;

The script reads the JSON file status_data.json (using IO::All which is great fun) then decodes it with JSON into a hash. It is hard to tell if this us a complete a solution because if you are "monitoring" host status then we should check the JSON data file periodically and compare it to our hash and then run the main body of the script one when changes have occurred.

To simulate changes occurring you can define/undefine CHANGE_HAPPENED in your environment with export CHANGE_HAPPENED=1 (or setenv if in in tcsh) and unset CHANGE_HAPPENED and the script will then either update the messages and the hash or "report". For this to be complete the data in our hash should be updated to match the the data file either periodically or when an event occurs. The status_report() subroutine could be changed so that it builds arrays of @inactive_hosts and @active_hosts when update_status() told it to do so: if ( something_happened() ) { update_status() }, etc.

Hope that helps.

status_data.json

{
  "hosts":[
        {"ipaddr":"10.1.1.2","hostname":"host2","role":"http","status":"active"},
        {"ipaddr":"10.1.1.3","hostname":"host3","role":"sql","status":"active"},
        {"ipaddr":"10.1.1.4","hostname":"host4","role":"quad","status":"active"}
  ]
}

output:

~/ % perl network_status_json.pl
host2 is active
host3 is active
host4 is active
~/ % export CHANGE_HAPPENED=1   
~/ % perl network_status_json.pl
host2 is inactive
host3 is inactive
host4 is active

Answer 3

This is very simple indeed in Perl, using the JSON module.

use strict;
use warnings;

use JSON qw/ from_json to_json /;

my $json = JSON->new;

my $data = from_json(do { local $/; <DATA> });

my $arr = "10.1.1.2 10.1.1.3";
my %arr = map { $_ => 1 } split ' ', $arr;

for my $item (@$data) {
  $item->{status} = 'inactive' if $arr{$item->{ipaddr}};
}

print to_json($data, { pretty => 1 }), "\n";

__DATA__
[
{"ipaddr":"10.1.1.2","hostname":"host2","role":"http","status":"active"},
{"ipaddr":"10.1.1.3","hostname":"host3","role":"sql","status":"active"},
{"ipaddr":"10.1.1.4","hostname":"host4","role":"quad","status":"active"}
]

output

[
   {
      "role" : "http",
      "hostname" : "host2",
      "status" : "inactive",
      "ipaddr" : "10.1.1.2"
   },
   {
      "hostname" : "host3",
      "role" : "sql",
      "ipaddr" : "10.1.1.3",
      "status" : "inactive"
   },
   {
      "ipaddr" : "10.1.1.4",
      "status" : "active",
      "hostname" : "host4",
      "role" : "quad"
   }
]

Answer 4

$ arr="10.1.1.2 10.1.1.3"
$ awk -v arr="$arr" -F, 'BEGIN { gsub(/\./,"\\.",arr); gsub(/ /,"|",arr) }
    $1 ~ "\"(" arr ")\"" { sub(/active/,"in&") } 1' file
{"ipaddr":"10.1.1.2","hostname":"host2","role":"http","status":"inactive"},
{"ipaddr":"10.1.1.3","hostname":"host3","role":"sql","status":"inactive"},
{"ipaddr":"10.1.1.4","hostname":"host4","role":"quad","status":"active"},

Answer 5

#!/bin/ksh

# your "array" of IP
arr="10.1.1.2 10.1.1.3"


# create and prepare temporary file for sed action
SedAction=/tmp/Action.sed

# --- for/do generating SedAction --------
echo "#sed action" > ${SedAction}

#take each IP from the arr variable one by one
for IP in ${arr}
 do
   # prepare for a psearch pattern use
   IP_RE="$( echo "${IP}" | sed 's/\./\\./g' )"

   # generate sed action in temporary file.
   # final action will be like: 
   #   s/\("ipaddr":"10\.1\.1\.2".*\)"active"}/\1"inactive"}/;t
   # escape(double) \ for in_file espace, escape(simple) " for this line interpretation
   echo "s/\\\(\"ipaddr\":\"${IP_RE}\".*\\\)\"active\"}/\\\1\"inactive\"}/;t" >> ${SedAction}
 done

# --- sed generating sed action ---------------
echo "${arr}" \
 | tr " " "\n" \
 | sed 's/\./\\./g
        s#.*#s/\\("ipaddr":"&".*\\)"active"}/\\1"inactive"}/;t#
        ' \
 > ${SedAction}


# core of the process (use -i for inline editing or "double" redirection for non GNU sed)
sed -f ${SedAction} YourFile

# clean temporary file
rm ${SedAction}

Self commented, tested in ksh/AIX. 2 way to generate the SedAction depending of action you want to do also (if any). You only need one to work, i prefer the second