UnparseableJsonResponse for one specific, valid JSON

Question

I have a standard Dialogflow agent, using javascript/node.js webhooks. It works perfectly well in most cases. I have recently encountered a problem which has me at a complete loss. I am currently saving some JSON-objects in conv.data to minimize the external API-calls my webhook have to make. For one specific JSON-object, fetched from an external API using node-fetch, the response I send from my side looks perfectly ordinary. I use firebase and the firebase logs do not show any error messages or any sign that there might be a problem. But I get this error in the Google Actions console:

UnparseableJsonResponse API Version 2: Failed to parse JSON response string with 'INVALID_ARGUMENT' error: "Parsing terminated before end of input. 8,\\"3\\":12},\\"w ^".

And in the stackdriver logs, the received response does not start with the usual

Received response from agent with body: HTTP/1.1 200 OK Server: ... etc

Instead it starts in the middle of the external API-JSON-file

Received response from agent with body: 8,\\"3\\":12},\\"winPercentage\\":1392}}}}, ... etc

This does not happen the first time the agent responds after fetching the JSON from the external API. The second time the agent responds after fetching the JSON, everything crashes regardless of whether the information from the JSON is used by that second call, regardless of anything at all except if the JSON file is overwritten between first and second call. If the file is overwritten the program runs perfectly. So the problem is likely part of storing and/or parsing this specific JSON file. Unfortunately the API I use in this application is not a public one and due to NDAs I cannot give any access to that JSON, so I understand that it is probably impossible for you to help me. I will however give as much information about the JSON as I can, and hope for the best:

• It is valid according to https://codebeautify.org/jsonvalidator and jsonlint.com
• It is structured the exact same way as other JSON files from the same API which do not crash the application
• It is slightly larger that other JSON files from the same API. It has around 340 000 characters, others are around 280-300 000.
• All JSONs, this as well as those that work, is from a Swedish company, therefore unusual characters like å, ä and ö are likely present.
• The error message is always the same, except the start of the response is in different places in the JSON file. "8,\\"3\\":12}, ...", "ostPosition\\":2 ...", "3804,\\"startPoints\\":2960 ..." are some examples.

I am extremely grateful for any and all help I might receive, even if it's just what questions I need to ask, or where I might try troubleshooting next.

Answer 1

I suspect the problem is that the JSON you're trying to save is larger than the buffer size they allocate for conv.data, although I can't find any documentation to say there is some specific limit.

I'd check to see where the strings you're seeing in the error header are located in the JSON and try to keep it well under that limit.