How to remove substring from a JSON string

Question

I have a following string.

result = '[
    {
        "id": 668,
        "overview": "All versions of `react-marked-markdown` are vulnerable to cross-site scripting (XSS) via `href` attributes. This is exploitable if user is provided to `react-marked-markdown`\n\nProof of concept:\n\n```\nimport React from 'react'\nimport ReactDOM from 'react-dom'\nimport { MarkdownPreview } from 'react-marked-markdown'\n\nReactDOM.render(\n<MarkdownPreview\nmarkedOptions={{ sanitize: true }}\nvalue={'[XSS](javascript: alert`1`)'}\n/>,\ndocument.getElementById('root')\n)\n```",
        "recommendation": "No fix is currently available for this vulnerability. It is our recommendation to not install or use this module at this time if you allow user input into href values.",
        "cvss_vector": "CVSS:3.0/AV:N/AC:L/PR:N/UI:R/S:C/C:H/I:H/A:N",
        "cvss_score": 9.3,
        "module": "react-marked-markdown"
    },
    {
        "id": 577,
        "overview": "Versions of `lodash` before 4.17.5 are vulnerable to prototype pollution. \n\nThe vulnerable functions are 'defaultsDeep', 'merge', and 'mergeWith' which allow a malicious user to modify the prototype of `Object` via `__proto__` causing the addition or modification of an existing property that will exist on all objects.\n\n",
        "recommendation": "Update to version 4.17.5 or later.",
        "cvss_vector": null,
        "cvss_score": 2,
        "module": "lodash",
        "version": "3.10.1"
    }
]'

How can I remove "overview" field and values from this string. Because when I am trying to convert this string into a JSON object using "${JsonOutput.toJson(result)}" It gives me parse error because this overview section is having {}.

This is what i tried

result=result | sed 's/"overview":*\\(","\)/\\1/g'

Can anyone help me here please.

Above string should look like this

[
    {
        "id": 668,
        "recommendation": "No fix is currently available for this vulnerability. It is our recommendation to not install or use this module at this time if you allow user input into href values.",
        "cvss_vector": "CVSS:3.0/AV:N/AC:L/PR:N/UI:R/S:C/C:H/I:H/A:N",
        "cvss_score": 9.3,
        "module": "react-marked-markdown"
    },
    {
        "id": 577,
        "recommendation": "Update to version 4.17.5 or later.",
        "cvss_vector": null,
        "cvss_score": 2,
        "module": "lodash",
        "version": "3.10.1"
    }
]

So that i can convert it into a JSON object.

Answer 1

You need to use a JSON parser to parse JSON data. As you have seen, trying to use regular expressions is too fragile.

First, to store that tricky string into a variable for testing purposes: use a quoted heredoc:

$ result=$(cat <<'END'
[
    {
        "id": 668,
        "overview": "All versions of `react-marked-markdown` are vulnerable to cross-site scripting (XSS) via `href` attributes. This is exploitable if user is provided to `react-marked-markdown`\n\nProof of concept:\n\n```\nimport React from 'react'\nimport ReactDOM from 'react-dom'\nimport { MarkdownPreview } from 'react-marked-markdown'\n\nReactDOM.render(\n<MarkdownPreview\nmarkedOptions={{ sanitize: true }}\nvalue={'[XSS](javascript: alert`1`)'}\n/>,\ndocument.getElementById('root')\n)\n```",
        "recommendation": "No fix is currently available for this vulnerability. It is our recommendation to not install or use this module at this time if you allow user input into href values.",
        "cvss_vector": "CVSS:3.0/AV:N/AC:L/PR:N/UI:R/S:C/C:H/I:H/A:N",
        "cvss_score": 9.3,
        "module": "react-marked-markdown"
    },
    {
        "id": 577,
        "overview": "Versions of `lodash` before 4.17.5 are vulnerable to prototype pollution. \n\nThe vulnerable functions are 'defaultsDeep', 'merge', and 'mergeWith' which allow a malicious user to modify the prototype of `Object` via `__proto__` causing the addition or modification of an existing property that will exist on all objects.\n\n",
        "recommendation": "Update to version 4.17.5 or later.",
        "cvss_vector": null,
        "cvss_score": 2,
        "module": "lodash",
        "version": "3.10.1"
    }
]
END
)

Then, remove the overview keys using jq

$ new_json=$(echo "$result" | jq 'map(del(.overview))')
$ echo "$new_json"
[
  {
    "id": 668,
    "recommendation": "No fix is currently available for this vulnerability. It is our recommendation to not install or use this module at this time if you allow user input into href values.",
    "cvss_vector": "CVSS:3.0/AV:N/AC:L/PR:N/UI:R/S:C/C:H/I:H/A:N",
    "cvss_score": 9.3,
    "module": "react-marked-markdown"
  },
  {
    "id": 577,
    "recommendation": "Update to version 4.17.5 or later.",
    "cvss_vector": null,
    "cvss_score": 2,
    "module": "lodash",
    "version": "3.10.1"
  }
]