CODEWITHSUNDEEP
javajson
Dragon
Dragon

Reputation: 2481

How to get up-to-date data from dynamically updated json?

There's a chat which content I want to parse. I got the url to get the .json of it. So it looks like:

{"messages":[
{"id":"111111","uid":"22222","name":"User","message":"Message","date":"2013-02-15 17:21:54","chatId":"111"},
{"id":"111111","uid":"22222","name":"User","message":"Message","date":"2013-02-15 17:21:54","chatId":"111"},
{"id":"111111","uid":"22222","name":"User","message":"Message","date":"2013-02-15 17:21:54","chatId":"111"}
]}

But this json has some limitation, I think approximately 20-30 records. New records are added at the beginning. It looks like:

{"messages":[
{"id":"222222","uid":"33333","name":"User","message":"Message","date":"2013-02-15 18:21:59","chatId":"111"},
{"id":"111111","uid":"22222","name":"User","message":"Message","date":"2013-02-15 17:21:54","chatId":"111"},
{"id":"111111","uid":"22222","name":"User","message":"Message","date":"2013-02-15 17:21:55","chatId":"111"}
]}
.......
{"messages":[
{"id":"333333","uid":"44444","name":"User","message":"Message","date":"2013-02-15 19:13:34","chatId":"111"},
{"id":"222222","uid":"33333","name":"User","message":"Message","date":"2013-02-15 18:21:59","chatId":"111"},
{"id":"111111","uid":"22222","name":"User","message":"Message","date":"2013-02-15 17:21:54","chatId":"111"}
]}

I gonna read this json via GSON or JSON Java and place to any output, it doesn't matter :)

But is there any best-practices on how to parse new records in dynamically updated json? In fact I don't know how to control that it is updated, but reading it every second and put results to output will result in data duplication I think.

Upvotes: 0

Views: 95

Answers (2)

davidbuzatto
davidbuzatto

Reputation: 9424

You will need to slice this data some way... To not make your GSON parse slow when your file grows, I think you would need to preprocess your file, slicing the disposable data. I would do something like:

  • First execution: parse the file entirely and store the first id, since it is the newest data;
  • Second execution (and others): read the file and store it in a StringBuilder. Using the String obtained, slice the disposable data, since you have the id stored prior. This id will show you where you need to start your slicing. With the new data, parse the GSON and stores the first id again.

You may use this create your code to perform the slicing and adapt to the idea that I said below:

String data = "{\"messages\":[" +
        "{\"id\":\"333333\",\"uid\":\"44444\",\"name\":\"User\",\"message\":\"Message\",\"date\":\"2013-02-15 19:13:34\",\"chatId\":\"111\"}," +
        "{\"id\":\"222222\",\"uid\":\"33333\",\"name\":\"User\",\"message\":\"Message\",\"date\":\"2013-02-15 18:21:59\",\"chatId\":\"111\"}," +
        "{\"id\":\"111111\",\"uid\":\"22222\",\"name\":\"User\",\"message\":\"Message\",\"date\":\"2013-02-15 17:21:54\",\"chatId\":\"111\"}" +
        "]}";

String lastId = "111111";
int sliceUntil = data.indexOf( "{\"id\":\"" + lastId + "\"" );

// since your disposable data is in the "tail" you your file,
// you just need to get the valid data (the data until the "last id") 
// and add the chars "]" and "}" to close your JSON
String newData = data.substring( 0, sliceUntil ) + "]}";

System.out.println( newData );

Upvotes: 1

user2071984
user2071984

Reputation: 1

You have uniq "date" for every message, so you can just create Map<Date, Message>, update it with all non-existant elements. Or you can just use Map<String, Message> with date string if you don't need time sorting

Upvotes: 0

Related Questions