CODEWITHSUNDEEP
bashsed
Shahar Hamuzim Rajuan
Shahar Hamuzim Rajuan

Reputation: 6159

need to change values of json file using sed

I need to change values of JSON file using sed,

I saw a lot of people suggesting using jq, python, or Perl.

But I'm working inside a container and I want it to be as simple as possible so only sed is the needed solution for me.

the JSON file is:

{
  "useCaseName" : "rca",
  "algorithm" : "log-clustering-train",
  "mainClass" : "com.hp.analytics.logclustering.MainTrainer",
  "applicationJar" : "log-clustering-train-1.0.0-SNAPSHOT-jar-with-dependencies.jar",
  "conf" : {
    "spark.driver.memory" : "3gb",
    "spark.executor.memory" : "9gb",
    "spark.executor.userClassPathFirst" : "true",
    "spark.cores.max": "8"
  },
  "schedule" : {
    "period" : "10",
    "timeUnit" : "hours",
    "timeoutPeriodSeconds" : "10800"
  }
}

And I want to change 4 values inside it:

"spark.driver.memory" : "1gb",

"spark.executor.memory" : "1gb",

"spark.cores.max" :"1"

"period" : "15",

So the output will be:

  {
      "useCaseName" : "rca",
      "algorithm" : "log-clustering-train",
      "mainClass" : "com.hp.analytics.logclustering.MainTrainer",
      "applicationJar" : "log-clustering-train-1.0.0-SNAPSHOT-jar-with-dependencies.jar",
      "conf" : {
        "spark.driver.memory" : "1gb",
        "spark.executor.memory" : "1gb",
        "spark.executor.userClassPathFirst" : "true",
        "spark.cores.max": "1"
      },
      "schedule" : {
        "period" : "15",
        "timeUnit" : "hours",
        "timeoutPeriodSeconds" : "10800"
      }
    }

Upvotes: 6

Views: 24699

Answers (4)

Ivneet Singh
Ivneet Singh

Reputation: 462

Below will work for both mac and linux.

If you want to read from some variable name value, use first command and if you have direct value, use second command.

sed -ie 's/"key1": .*"/"key1": '\"${value}\"'/g' filename
sed -ie 's/"key1": .*"/"key1": '\"yourvalue\"'/g' filename

Upvotes: 3

RavinderSingh13
RavinderSingh13

Reputation: 133780

try following awk and let me know if this helps you.

awk '{
        match($0,/^[[:space:]]+/);
        val=substr($0,RSTART,RLENGTH)
     }
        /spark.driver.memory/ || /spark.executor.memory/{
        sub(/[0-9]+/,"1",$3);
                                                        }
        /spark.cores.max/{
        sub(/[0-9]+/,"1",$2)
                         }
        /period/{
        sub(/[0-9]+/,"15",$3)
                }
     {
        printf("%s%s\n",!/^ +/?val:"",$0)
     }
    '    Input_file

If you want to save the output of this into same Input_file, then you could save above code's output into a temp file and then save it to Input_file again.

EDIT1: a sed solution too now.

sed 's/"spark.driver.memory" : "[0-9]gb"/"spark.driver.memory" : "1gb"/;s/"spark.executor.memory" : "[0-9]gb"/"spark.executor.memory" : "1gb"/;s/"spark.cores.max": "[0-9]"/"spark.cores.max" :"1"/;s/"period" : "[0-9]*"/"period" : "15"/'   Input_file

If happy with above code's result then use sed -i option to save into the Input_file then.

Upvotes: 1

RomanPerekhrest
RomanPerekhrest

Reputation: 92904

For that time when you'll be ready for a proper solution using jq tool:

jq '.conf |= . + {"spark.driver.memory":"1gb","spark.executor.memory":"1gb","spark.cores.max":"1"} | .schedule |= . + {period:"15"}' file

The output:

{
  "useCaseName": "rca",
  "algorithm": "log-clustering-train",
  "mainClass": "com.hp.analytics.logclustering.MainTrainer",
  "applicationJar": "log-clustering-train-1.0.0-SNAPSHOT-jar-with-dependencies.jar",
  "conf": {
    "spark.driver.memory": "1gb",
    "spark.executor.memory": "1gb",
    "spark.executor.userClassPathFirst": "true",
    "spark.cores.max": "1"
  },
  "schedule": {
    "period": "15",
    "timeUnit": "hours",
    "timeoutPeriodSeconds": "10800"
  }
}

Upvotes: 5

skr
skr

Reputation: 2296

For sed use the following

sed -i '/spark.driver.memory/c\   \"spark.driver.memory\" : \"1gb\",' file.txt
sed -i '/spark.executor.memory/c\   \"spark.executor.memory\" : \"1gb\",' file.txt
sed -i '/spark.cores.max/c\   \"spark.cores.max\" : \"1\",' file.txt
sed -i '/period/c\   \"period\" : \"15\",' file.txt

Upvotes: 13

Related Questions