Reputation: 6159
I need to change values of JSON file using sed,
I saw a lot of people suggesting using jq, python, or Perl.
But I'm working inside a container and I want it to be as simple as possible so only sed is the needed solution for me.
the JSON file is:
{
"useCaseName" : "rca",
"algorithm" : "log-clustering-train",
"mainClass" : "com.hp.analytics.logclustering.MainTrainer",
"applicationJar" : "log-clustering-train-1.0.0-SNAPSHOT-jar-with-dependencies.jar",
"conf" : {
"spark.driver.memory" : "3gb",
"spark.executor.memory" : "9gb",
"spark.executor.userClassPathFirst" : "true",
"spark.cores.max": "8"
},
"schedule" : {
"period" : "10",
"timeUnit" : "hours",
"timeoutPeriodSeconds" : "10800"
}
}
And I want to change 4 values inside it:
"spark.driver.memory" : "1gb",
"spark.executor.memory" : "1gb",
"spark.cores.max" :"1"
"period" : "15",
So the output will be:
{
"useCaseName" : "rca",
"algorithm" : "log-clustering-train",
"mainClass" : "com.hp.analytics.logclustering.MainTrainer",
"applicationJar" : "log-clustering-train-1.0.0-SNAPSHOT-jar-with-dependencies.jar",
"conf" : {
"spark.driver.memory" : "1gb",
"spark.executor.memory" : "1gb",
"spark.executor.userClassPathFirst" : "true",
"spark.cores.max": "1"
},
"schedule" : {
"period" : "15",
"timeUnit" : "hours",
"timeoutPeriodSeconds" : "10800"
}
}
Upvotes: 6
Views: 24699
Reputation: 462
Below will work for both mac and linux.
If you want to read from some variable name value, use first command and if you have direct value, use second command.
sed -ie 's/"key1": .*"/"key1": '\"${value}\"'/g' filename
sed -ie 's/"key1": .*"/"key1": '\"yourvalue\"'/g' filename
Upvotes: 3
Reputation: 133780
try following awk and let me know if this helps you.
awk '{
match($0,/^[[:space:]]+/);
val=substr($0,RSTART,RLENGTH)
}
/spark.driver.memory/ || /spark.executor.memory/{
sub(/[0-9]+/,"1",$3);
}
/spark.cores.max/{
sub(/[0-9]+/,"1",$2)
}
/period/{
sub(/[0-9]+/,"15",$3)
}
{
printf("%s%s\n",!/^ +/?val:"",$0)
}
' Input_file
If you want to save the output of this into same Input_file, then you could save above code's output into a temp file and then save it to Input_file again.
EDIT1: a sed solution too now.
sed 's/"spark.driver.memory" : "[0-9]gb"/"spark.driver.memory" : "1gb"/;s/"spark.executor.memory" : "[0-9]gb"/"spark.executor.memory" : "1gb"/;s/"spark.cores.max": "[0-9]"/"spark.cores.max" :"1"/;s/"period" : "[0-9]*"/"period" : "15"/' Input_file
If happy with above code's result then use sed -i option to save into the Input_file then.
Upvotes: 1
Reputation: 92904
For that time when you'll be ready for a proper solution using jq
tool:
jq '.conf |= . + {"spark.driver.memory":"1gb","spark.executor.memory":"1gb","spark.cores.max":"1"} | .schedule |= . + {period:"15"}' file
The output:
{
"useCaseName": "rca",
"algorithm": "log-clustering-train",
"mainClass": "com.hp.analytics.logclustering.MainTrainer",
"applicationJar": "log-clustering-train-1.0.0-SNAPSHOT-jar-with-dependencies.jar",
"conf": {
"spark.driver.memory": "1gb",
"spark.executor.memory": "1gb",
"spark.executor.userClassPathFirst": "true",
"spark.cores.max": "1"
},
"schedule": {
"period": "15",
"timeUnit": "hours",
"timeoutPeriodSeconds": "10800"
}
}
Upvotes: 5
Reputation: 2296
For sed use the following
sed -i '/spark.driver.memory/c\ \"spark.driver.memory\" : \"1gb\",' file.txt
sed -i '/spark.executor.memory/c\ \"spark.executor.memory\" : \"1gb\",' file.txt
sed -i '/spark.cores.max/c\ \"spark.cores.max\" : \"1\",' file.txt
sed -i '/period/c\ \"period\" : \"15\",' file.txt
Upvotes: 13