sed replace between 2 patterns of JSON file that have multi occurrences

Question

I need to replace whatever expressions in between 2 patterns of JSON file, those patterns are multi occurrences and I would like to replace them only once by my choice (let's say in the 4th occurrence out of 6).

I've created a sed expression that works when I have only one occurrence in the file, but when adding more than one it is for some reason doesn't work when trying to replace the second occurrence.

This is my sed:

images_one_line=MAMA

sed -i 's/CDATA$$.*$$\]>/CDATA[ '"${images_one_line}"' ]]>/2' response.json

This is the response.json file:

{"id":"1929905018","type":"page","status":"current","title":"CODEFRESH-CHANGE-API","version":{"by":{"type":"known","accountId":"6066297300b79f0070e308cb","accountType":"atlassian","email":"shachar.rajuan@veeva.com","publicName":"shachar.rajuan","profilePicture":{"path":"/wiki/aa-avatar/6066297300b79f0070e308cb","width":48,"height":48,"isDefault":false},"displayName":"Shachar Rajuan","isExternalCollaborator":false,"_expandable":{"operations":"","personalSpace":""},"_links":{"self":"https://crossixsolutions.atlassian.net/wiki/rest/api/user?accountId=6066297300b79f0070e308cb"}},"when":"2021-06-07T16:00:41.499Z","friendlyWhen":"less than a minute ago","message":"","number":128,"minorEdit":false,"syncRev":"4.confluence$content$1929905018.362","syncRevSource":"synchrony-ack","confRev":"confluence$content$1929905018.363","contentTypeModified":false,"_expandable":{"collaborators":"","content":"/rest/api/content/1929905018"},"_links":{"self":"https://crossixsolutions.atlassian.net/wiki/rest/api/content/1929905018/version/128"}},"macroRenderedOutput":{},"body":{"storage":{"value":" Environment Data Loaded version/branch Owner Used for Cluster size env prefix Size ES - nodes Mongo - Shards cds-dev-1 10 node, 1 day, 33 files This field is being updated automatically using CodeFresh Build. Do not change it manually! bash DEV/Atara functional tests tiny 2 nodes 3 shards, M40 cds-dev-2 1 node This field is being updated automatically using CodeFresh Build. Do not change it manually! bash shalom shalom small 1 node 1 shard

","representation":"storage","embeddedContent":[],"_expandable":{"content":"/rest/api/content/1929905018"}},"_expandable":{"editor":"","atlas_doc_format":"","view":"","export_view":"","styled_view":"","dynamic":"","editor2":"","anonymous_export_view":""}},"extensions":{"position":363304566},"_expandable":{"childTypes":"","container":"/rest/api/space/SD","metadata":"","operations":"","schedulePublishDate":"","children":"/rest/api/content/1929905018/child","restrictions":"/rest/api/content/1929905018/restriction/byOperation","history":"/rest/api/content/1929905018/history","ancestors":"","descendants":"/rest/api/content/1929905018/descendant","space":"/rest/api/space/SD"},"_links":{"editui":"/pages/resumedraft.action?draftId=1929905018","webui":"/spaces/SD/pages/1929905018/CODEFRESH-CHANGE-API","context":"/wiki","self":"https://crossixsolutions.atlassian.net/wiki/rest/api/content/1929905018","tinyui":"/x/egMIcw","collection":"/rest/api/content","base":"https://crossixsolutions.atlassian.net/wiki"}}

What my sed is doing is replacing everything between CDATA[ and ]], What I am getting when using the global option /g is:

{"id":"1929905018","type":"page","status":"current","title":"CODEFRESH-CHANGE-API","version":{"by":{"type":"known","accountId":"6066297300b79f0070e308cb","accountType":"atlassian","email":"shachar.rajuan@veeva.com","publicName":"shachar.rajuan","profilePicture":{"path":"/wiki/aa-avatar/6066297300b79f0070e308cb","width":48,"height":48,"isDefault":false},"displayName":"Shachar Rajuan","isExternalCollaborator":false,"_expandable":{"operations":"","personalSpace":""},"_links":{"self":"https://crossixsolutions.atlassian.net/wiki/rest/api/user?accountId=6066297300b79f0070e308cb"}},"when":"2021-06-07T16:14:01.265Z","friendlyWhen":"less than a minute ago","message":"Reverted from v. 134","number":136,"minorEdit":false,"syncRev":"4.confluence$content$1929905018.377","syncRevSource":"synchrony-ack","confRev":"confluence$content$1929905018.379","contentTypeModified":false,"_expandable":{"collaborators":"","content":"/rest/api/content/1929905018"},"_links":{"self":"https://crossixsolutions.atlassian.net/wiki/rest/api/content/1929905018/version/136"}},"macroRenderedOutput":{},"body":{"storage":{"value":" Environment Data Loaded version/branch Owner Used for Cluster size env prefix Size ES - nodes Mongo - Shards cds-dev-1 10 node, 1 day, 33 files This field is being updated automatically using CodeFresh Build. Do not change it manually! bash shalom shalom small 1 node 1 shard

","representation":"storage","embeddedContent":[],"_expandable":{"content":"/rest/api/content/1929905018"}},"_expandable":{"editor":"","atlas_doc_format":"","view":"","export_view":"","styled_view":"","dynamic":"","editor2":"","anonymous_export_view":""}},"extensions":{"position":363304566},"_expandable":{"childTypes":"","container":"/rest/api/space/SD","metadata":"","operations":"","schedulePublishDate":"","children":"/rest/api/content/1929905018/child","restrictions":"/rest/api/content/1929905018/restriction/byOperation","history":"/rest/api/content/1929905018/history","ancestors":"","descendants":"/rest/api/content/1929905018/descendant","space":"/rest/api/space/SD"},"_links":{"editui":"/pages/resumedraft.action?draftId=1929905018","webui":"/spaces/SD/pages/1929905018/CODEFRESH-CHANGE-API","context":"/wiki","self":"https://crossixsolutions.atlassian.net/wiki/rest/api/content/1929905018","tinyui":"/x/egMIcw","collection":"/rest/api/content","base":"https://crossixsolutions.atlassian.net/wiki"}}

And when using \2 which should replace the s2d occurrence, I'm seeing no changing.

Jonathan Leffler · Accepted Answer

That's close to 5 KiB of JSON on a single line — it's a pain to try reading it.

There are two sequences of [CDATA[…]] — the first is about 140 characters long, the second about 45 characters long. Your primary problem is that the .* notation in your sed script is 'greedy'; it will start matching after the first CDATA and read until the end of the second. You need to restrict it so it doesn't skip the ]] end marker. That's not trivial. A moderate approximation is:

sed -e 's/CDATA$$[^]]*$$\]>/CDATA[ '"${images_one_line}"' ]]>/' response.json

That doesn't catch the ![[ before the CDATA marker; that presumably doesn't matter.

You can then use the GNU sed extension to select which occurrence of CDATA on the line to change — add a suitable number after the close / of the s/// command.

Note that you may still run into trouble if the data in the CDATA includes any close square brackets.

This version uses sed 'extended regular expression' or ERE support. That's usually enabled by -E, but GNU sed also recognizes -r:

sed -E -e 's///g' response.json

A Perl-compatible regex would be better — but still non-trivial. This Perl script shows a working version:

#!/usr/bin/env perl

use strict;
use warnings;

while (<>)
{
    s///g;
    print;
}

Editing JSON data like this is problematic, as also noted in the comments.

sed replace between 2 patterns of JSON file that have multi occurrences

Answers (1)

Related Questions