How to back up more than one level using TJSONIterator in Delphi?

Question

I am stuck in trying to use the TJSONIterator in Delphi 10.2.2. The short question is "How do I go up two levels in the Iterator?"

The following code illustrates my problem:

JsonRec := '{"v1":"Main","v2":"1.1","v3":{"id":"X45","mod":1.5,"r2":{"rv1":"99190","rv2":"TX"}},"v4":"ok","v5":69}';

PDS.Open;
PDS.Append;
StringReader := TStringReader.Create(JsonRec);
JsonTextReader := TJsonTextReader.Create(StringReader);
Iterator := TJSONIterator.Create(JsonTextReader);
If Iterator.Next('v1') Then
   PDS['Type'] := Iterator.AsString;
If Iterator.Next('v2') Then
   PDS['Version'] := Iterator.AsString;
If Iterator.Next('v3') Then
   Begin
   Iterator.Recurse;
   If Iterator.Next('id') Then
      PDS['BlackListInfo'] := Iterator.AsString;
   If Iterator.Next('mod') Then
      PDS['Speed'] := Iterator.AsDouble;
   If Iterator.Next('r2') Then
      begin
      Iterator.Recurse;
      if Iterator.Next('rv1') then
         PDS['Serial'] := Iterator.AsString;
      if Iterator.Next('rv2') then
         PDS['Location'] := Iterator.AsString;
      Iterator.Return;
      end;
   Iterator.Return;   //Second Return does not go up a level.
   if Iterator.Next('v4') then // Always fails
      PDS['CRC'] := Iterator.AsString;
   if Iterator.Next('v5') then
      PDS['ReportID'] := Iterator.AsInteger;
   PDS.Post;
   End;

Obviously, I am parsing the JSON string to put the data into the database (PDS). When I issue the second return, I don't go up to the expected level and then I cannot find v4. I suspect that I may need to use the Rewind method, but so far I have been unable to find it's documentation.

Any help is greatly appreciated.

Answer 1

I was looking for an example on how to use TJSONIterator and landed here. Since there's hardly anything to be found, I figured I'd share the solution I came up with:

...
  ecDebug: TMemo;
...
uses
  System.JSON.Builders,
  System.JSON.Readers,
...
procedure TForm1.Button1Click(Sender: TObject);
const
  JsonRec = '{"v1":"Main","v2":"1.1","v3":{"id":"X45","mod":1.5,' +
    '"r2":{"rv1":"99190","rv2":"TX"}},"v4":"ok","v5":69}';
var
  StringReader: TStringReader;
  JsonTextReader: TJsonTextReader;
  Iterator: TJSONIterator;
begin
  JsonTextReader:= nil;
  Iterator:= nil;
  StringReader:= TStringReader.Create(JsonRec);
  try
    JsonTextReader:= TJsonTextReader.Create(StringReader);
    Iterator:= TJSONIterator.Create(JsonTextReader);
    while Iterator.Next do
    begin
      if Iterator.Key = 'v1' then
        ecDebug.Lines.Add(Format('Type = %s', [Iterator.AsString]))
      else if Iterator.Key = 'v2' then
        ecDebug.Lines.Add(Format('Version = %s', [Iterator.AsString]))
      else if Iterator.Key = 'v3' then
      begin
        Iterator.Recurse;
        while Iterator.Next do
        begin
           if Iterator.Key = 'id' then
             ecDebug.Lines.Add(Format('BlackListInfo = %s', [Iterator.AsString]))
           else if Iterator.Key = 'mod' then
             ecDebug.Lines.Add(Format('Speed = %g', [Iterator.AsDouble]))
           else if Iterator.Key = 'r2' then
           begin
             Iterator.Recurse;
             while Iterator.Next do
             begin
               if Iterator.Key = 'rv1' then
                 ecDebug.Lines.Add(Format('Serial = %s', [Iterator.AsString])) 
               else if Iterator.Key = 'rv2' then
                 ecDebug.Lines.Add(Format('Location = %s', [Iterator.AsString]));
             end;
             Iterator.Return;
           end;
        end;
        Iterator.Return;
      end
      else if Iterator.Key = 'v4' then
        ecDebug.Lines.Add(Format('CRC = %s', [Iterator.AsString]))
      else if Iterator.Key = 'v5' then
        ecDebug.Lines.Add(Format('ReportID = %d', [Iterator.AsInteger]));
    end;
  finally
    Iterator.Free;
    JsonTextReader.Free;
    StringReader.Free;
  end;
end;

I believe this is the way TJSONIterator is intended to be used. This code is not reliant on the order of the JSON elements.

The output in the memo is:

Type = Main
Version = 1.1
BlackListInfo = X45
Speed = 1,5
Serial = 99190
Location = TX
CRC = ok
ReportID = 69

Answer 2

This answer explain how to achieve what you want to do, but without using TJsonReader that is one of the worse json parser (performance and usability) made for delphi. (you can make a benchmark with this tool: https://svn.code.sf.net/p/alcinoe/code/demos/ALJsonDoc/win32/AljsonDocDemo.exe)

Using for exemple Alcinoe (https://github.com/Zeus64/alcinoe) the code is pretty simple (but any other json parser can also do this kind of job pretty well)

MyJsonDoc := TalJsonDocumentU.create;
try
  MyJsonDoc.loadFromJsonString('{"v1":"Main","v2":"1.1","v3":{"id":"X45","mod":1.5,"r2":{"rv1":"99190","rv2":"TX"}},"v4":"ok","v5":69}');
  PDS['Type'] := MyJsonDoc.node.getchildNodeValueText('v1', ''{default});
  PDS['Version'] := MyJsonDoc.node.getchildNodeValueText('v2', ''{default});
  PDS['BlackListInfo'] := MyJsonDoc.node.getchildNodeValueText(['v3', 'id'], ''{default});
  PDS['Speed'] := MyJsonDoc.node.getchildNodeValueFloat(['v3', 'mod'], 0{default});
  PDS['Serial'] := MyJsonDoc.node.getchildNodeValueText(['v3', 'r2', 'rv1'], ''{default});
  PDS['Location'] := MyJsonDoc.node.getchildNodeValueText(['v3', 'r2', 'rv2'], ''{default});
  PDS['CRC'] := MyJsonDoc.node.getchildNodeValueText('v4', ''{default});
  PDS['ReportID'] := MyJsonDoc.node.getchildNodeValueInt32('v5', 0{default});
finally
  MyJsonDoc.free;
end;

Answer 3

Short answer

Use Iterator.Next between Iterator.Return calls. Empty if you don't want to process anything at that level.

Simply replace this part of code

  if Iterator.Next('rv2') then
     PDS['Location'] := Iterator.AsString;
  Iterator.Return;
  end;

wiith this

  if Iterator.Next('rv2') then
     PDS['Location'] := Iterator.AsString;
  Iterator.Return;
  Iterator.Next;
  end;

Long answer

I'm not sure if it's a bug or it's intended and documentation doesn't help at all, but Return works only for one level. If you look at the impelemntation, you can see, that Return will only move reader to the first end token, decrease the depth, and stay there, if it's not already there.

In this case, the first call of Return moves to the end of r2 and decrease depth, the next call will do nothing, because it is already at the end and the not FReader.IsEndToken (FReader.TokenType) condition is not met.

Iterator.Next('v4') moves to the end of v3 and decrease depth to 1, but also sets private variable FFinished to True, which will cause that all other calls of Next won't do anything, because of the condition at the start if FFinished then Exit. The only way to reset FFinish is with Return or Rewind.