Gson json parsing is not working

Question

I am parsing the files in json format where i access through

http://api.crossref.org/works?rows=1000&offset=6000000

There are around 71 million distinct publication records and the link i gave is only presents 1000. In my java code gson parser mostly works well but sometimes it gives an error as following:

com.google.gson.JsonSyntaxException: com.google.gson.stream.MalformedJsonException: Unterminated string at line 2 column 1
        at com.google.gson.internal.Streams.parse(Streams.java:56)
        at com.google.gson.JsonParser.parse(JsonParser.java:84)
        at com.google.gson.JsonParser.parse(JsonParser.java:59)
        at com.google.gson.JsonParser.parse(JsonParser.java:45)
        at connectingurl.CrossRefFullTextToTable.<init>(CrossRefFullTextToTable.java:84)
        at connectingurl.CrossRefFullTextToTable.main(CrossRefFullTextToTable.java:181)
    Caused by: com.google.gson.stream.MalformedJsonException: Unterminated string at line 2 column 1
        at com.google.gson.stream.JsonReader.syntaxError(JsonReader.java:1310)
        at com.google.gson.stream.JsonReader.nextString(JsonReader.java:1046)
        at com.google.gson.stream.JsonReader.nextValue(JsonReader.java:784)
        at com.google.gson.stream.JsonReader.objectValue(JsonReader.java:766)
        at com.google.gson.stream.JsonReader.peek(JsonReader.java:380)
        at com.google.gson.internal.bind.TypeAdapters$25.read(TypeAdapters.java:635)
        at com.google.gson.internal.bind.TypeAdapters$25.read(TypeAdapters.java:658)
        at com.google.gson.internal.bind.TypeAdapters$25.read(TypeAdapters.java:650)
        at com.google.gson.internal.bind.TypeAdapters$25.read(TypeAdapters.java:658)
        at com.google.gson.internal.bind.TypeAdapters$25.read(TypeAdapters.java:658)
        at com.google.gson.internal.bind.TypeAdapters$25.read(TypeAdapters.java:633)
        at com.google.gson.internal.Streams.parse(Streams.java:44)

As i searched for it, most probably there might be a memory issue.

Here is part of my java code:

if(code == 200){

    String full_text_link = "";
    String license = "";

    BufferedReader in = new BufferedReader(new InputStreamReader(yc.getInputStream(), "UTF-8"));
    StringBuilder sb = new StringBuilder();

    String inputLine;
    while ((inputLine = in.readLine()) != null) {
        sb.append(inputLine + System.getProperty("line.separator"));
    }
    in.close();
    String jsonLine = sb.toString();


    JsonElement jelement = new JsonParser().parse(jsonLine);     
    JsonObject  jobject = jelement.getAsJsonObject();
    jobject = jobject.getAsJsonObject("message");
    JsonArray jarray_items = jobject.getAsJsonArray("items");

    ...
}

The error comes from the line:

JsonElement jelement = new JsonParser().parse(jsonLine);

It seems gson might have such problems. is there any other suggestion to use another parser or is there something that i should add my code?

Thanks in advance...

Answer 1

while ((inputLine = in.readLine()) != null) {
   sb.append(inputLine + System.getProperty("line.separator"));
}

You are basically modifying your input without reason, you cant simply pre-process your JSON without knowing the exact syntax, to solve the problem you can let GSON read the file itself, for example :

http://www.java2blog.com/2013/11/gson-example-read-and-write-json.html

package org.arpit.java2blog;

import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.util.List;
import org.arpit.java2blog.pojo.Country;
import com.google.gson.Gson;

/*
 * @Author : Arpit Mandliya
 */
public class GSONReadingFromFileExample {

 public static void main(String[] args) {

  Gson gson = new Gson();

  try {

   System.out.println("Reading JSON from a file");
   System.out.println("----------------------------");

   BufferedReader br = new BufferedReader(
     new FileReader("E:\\file.json"));

    //convert the json string back to object
   Country countryObj = gson.fromJson(br, Country.class);

   System.out.println("Name Of Country: "+countryObj.getName());


   System.out.println("Population: "+countryObj.getPopulation());

   System.out.println("States are :");

   List<String> listOfStates = countryObj.getListOfStates();
   for (int i = 0; i < listOfStates.size(); i++) {
    System.out.println(listOfStates.get(i));
   }

  } catch (IOException e) {
   e.printStackTrace();
  }
 }
}