CODEWITHSUNDEEP
pythonbashsedgrepjq
dragons
dragons

Reputation: 619

find JSON key in all files and remove the corresponding key-value pair

I have a lot of files in which I need to find a string and if that string is there then I need to remove json block around that string from all those files.

For example:

I am looking for this string - "1234" in all my files. Each file has a huge json block which has string that I want to remove. So in all those files, string is like this in below two format:

First format is:

...
"data": {
    "1234": {
        "tt": true
    }
}
...

Second format with multiple json objects inside data:

...
"data": {
    "1234": {
        "tt": true
    },
    "7890": {
        "tt": true
    }
}
...

Now I need to come up with some script from where I can remove json object "1234" completely. So output should be:

For first format:

...
"data": {}
...

For second format:

...
"data": {
    "7890": {
        "tt": true
    }
}
...

Is this possible to do? I am in mac linux terminal. I can do a grep search and find the files but not sure how to remove the json object around that string from all the files.

Note: Each json file is in pretty format.

Update:

{
    "props": [
        {
            "property": "Hat",
            "data": {
                "1234": {
                    "tt": true
                },
                "7890": {
                    "tt": true
                }
            }
        }
    ]
}

Here is the snippet - https://jqplay.org/s/qXW_QUYFv0. If the property value is Hat then only I want to remove the key present but if property value is something else then I don't want to remove that key. Is there any way we can add this change as well?

Upvotes: 4

Views: 1647

Answers (3)

Cornelius Roemer
Cornelius Roemer

Reputation: 7839

This sed script should do as long as long as the following conditions hold:

  • JSON is pretty printed (otherwise pretty print with jq)
  • Exactly three lines need to be deleted including the line where "1234" appears
#! sed -f
/"1234"/ {
  N
  N
  d
}

It looks for lines containing "1234", then gets the following two lines and deletes all of them.

Alternatively, this works if there's no extra brackets inside the object, but works with multiple key-value pairs:

sed '/"1234"/,/}/ d'

It's not nice and general, but it could do the job. If you need something more general, you need to adjust the problem specification and I'll give it a go.

To run it on all JSON files in a folder you can use find -exec:

find . -name "*.json" -exec sed -i'.old' '/"1234"/,/}/ d' {} \;

This is a jq alternative using if/then/else/end rather than select:

jq '{props: [.props[] | if .property == "Hat" then del(.data."1234") else . end]}'

Upvotes: 1

Bean Taxi
Bean Taxi

Reputation: 1261

Update

Given the updated question, please have a look at this

jq '{props: [.props[] | (select(.property == "Hat") | del(.data["1234"])), select(.property != "Hat")]}'

Try it online!

This is just the jq expression itself. To run it on every file in a folder etc, just substitute this in the bash loop below.

How's this ...

jq 'delpaths([paths] | map(select(.[]=="1234")))'

First example

Input

{
    "data": {
        "1234": {
            "tt": true
        }
    }
}

Output

{
  "data": {}
}

Try it online!

Second Example

Input

{
    "data": {
        "1234": {
            "tt": true
        },
        "7890": {
            "tt": true
        }
    }
}

Output

{
  "data": {
    "7890": {
      "tt": true
    }
  }
}

Try it online!

A simple bash script to run this on every file (e.g. data-1.json) and save it in place ...

for file in *.json; do
    jq 'delpaths([paths] | map(select(.[]=="1234")))' <"$file" >"$file.new" && mv "$file.new" "$file"
done

That's as close as I can get without actual input/output instead of excerpts.

Hope this helps!

Upvotes: 3

peak
peak

Reputation: 116690

Is "1234" always a key, or can it be a value too?

It will always be a key. – dragons 3 hours ago

Here's a simple and straightforward solution using 'walk/1`:

walk(if type == "object" then with_entries(select(.key != "1234")) else . end)

Handling multiple files

The handling of multiple files will depend on (a) how the files are specified; (b) whether you want to overwrite those files, and if so, whether conditionally or unconditionally, and if not, what file names to use.

In short, there are multiple variations. Here is one example:

for f in *.json
do
  jq -f -program.jq "$f" > "$f.new"
done

where program.jq contains the walk program shown above.

Upvotes: 3

Related Questions