How to handle control chars in JSON parsing ..?

Question

I have a HTTP response with JSON content, based on some condition i want to remove the key & value of a particular node within response which is basically by doing the JSON parsing (I have tried using the Boost::ptree & nlohmann library), by doing that the control characters are getting removed as well, how can i avoid this issue..?

I have tried parsing it & removing the fields that needs to be removed, but along with it the control characters are also getting removed since it is going via the parsing to ptree (where the control chars are not retained), and once i write back the modified ptree back to string the control chars are not there & there is a big difference in the content length

here is the example

{
  "firstName": "John",
  "lastName": "Doe",
  "dateOfBirth": "1985-07-15",
  "ssn": "123-45-6789",
  "address": {
    "streetAddress": "123 Main St",
    "city": "Anytown",
    "state": "CA",
    "postalCode": "12345"
  },
  "phoneNumbers": [
    {
      "type": "home",
      "number": "555-555-5555"
    },
    {
      "type": "work",
      "number": "555-555-5556"
    }
  ],
  "email": "john.doe@example.com"
}

this is the output I got:

{"address":{"city":"Anytown","postalCode":"12345","state":"CA","streetAddress":"123 Main St"},"dateOfBirth":"1985-07-15","email":"john.doe@example.com","firstName":"John","lastName":"Doe","phoneNumbers":\[{"number":"555-555-5555","type":"home"},{"number":"555-555-5556","type":"work"}\],"ssn":"123-45-6789"}  

which has a reduced content length as 305 where as original content length is 325.

Answer 1

What you refer to as "control characters" are insignificant whitespace. Unlike e.g. XML, those are not part of the JSON document by specification, so no JSON library implementation¹ tracks them/is able to preserve them.

However, your input seems pretty standard "pretty-printed" using an indent-width of 2 spaces. You can do the same, e.g. using nlohmann:

#include <nlohmann/json.hpp>
void using_nlohmann() {
    // parse and serialize JSON, preserving whitespace
    auto j = nlohmann::json::parse(text);

    // remove SSN and phone numbers
    j.erase("ssn");
    j.erase("phoneNumbers");
    std::cout << j.dump(2) << std::endl;
}

This prints:

{
  "address": {
    "city": "Anytown",
    "postalCode": "12345",
    "state": "CA",
    "streetAddress": "123 Main St"
  },
  "dateOfBirth": "1985-07-15",
  "email": "john.doe@example.com",
  "firstName": "John",
  "lastName": "Doe"
}

This may be enough for your requirements.

Summary

If you're using JSON, you should not depend on insignificant whitespace (or on other non-standard extensions, like comments). The argument "there is a big difference in the content length" should not be relevant to any application that exchanges data in JSON format, because the specification says there is no difference.

Demo

Live On Compiler Explorer

#include <iostream>

static auto text = R"({
  "firstName": "John",
  "lastName": "Doe",
  "dateOfBirth": "1985-07-15",
  "ssn": "123-45-6789",
  "address": {
    "streetAddress": "123 Main St",
    "city": "Anytown",
    "state": "CA",
    "postalCode": "12345"
  },
  "phoneNumbers": [
    {
      "type": "home",
      "number": "555-555-5555"
    },
    {
      "type": "work",
      "number": "555-555-5556"
    }
  ],
  "email": "john.doe@example.com"
})";

#include <nlohmann/json.hpp>
void using_nlohmann() {
    // parse and serialize JSON, preserving whitespace
    auto j = nlohmann::json::parse(text);

    // remove SSN and phone numbers
    j.erase("ssn");
    j.erase("phoneNumbers");
    std::cout << j.dump(2) << std::endl;
}

// the same using Boost.JSON
#include <boost/json.hpp>
#include <boost/json/src.hpp> // for compiler explorer
void using_boost() {
    auto j = boost::json::parse(text);

    // remove SSN and phone numbers
    j.as_object().erase("ssn");
    j.as_object().erase("phoneNumbers");
    std::cout << j << std::endl; // no built-in way to pretty-print
}

int main(){
    using_nlohmann();
    using_boost();
}

Printing

{
  "address": {
    "city": "Anytown",
    "postalCode": "12345",
    "state": "CA",
    "streetAddress": "123 Main St"
  },
  "dateOfBirth": "1985-07-15",
  "email": "john.doe@example.com",
  "firstName": "John",
  "lastName": "Doe"
}
{"firstName":"John","lastName":"Doe","dateOfBirth":"1985-07-15","email":"john.doe@example.com","address":{"streetAdd
ress":"123 Main St","city":"Anytown","state":"CA","postalCode":"12345"}}

¹ to my knowledge