CODEWITHSUNDEEP
javaandroidjsongsonretrofit
Amit Pal
Amit Pal

Reputation: 11052

How to parse newline delimited Json response in Android?

Sample of NdJson data: 

{"type":"data","id":"xyz"}
{"type":"value","id":"xcf"}
....
....

Here is my Retrofit and RxJava code which is working fine for fetching data with the limit=1 i.e. {"type":"data","id":"xyz"}.

 adapter.create(API.class).getData()
        .subscribeOn(Schedulers.newThread())
        .observeOn(AndroidSchedulers.mainThread())
        .subscribe(new Observer<APIData>() {
         @Override
         public void onCompleted() {}

         @Override
         public void onError(Throwable e) {}

         @Override
         public void onNext(APIData apidata) {}

         });

My model class

Model class have only two parameter:

public class APIData(){
  private String type;
  private String id;

  ..// Getter and setter for above two fields
}

Api class

public interface WarehouseAPI {
  @GET("/search?limit=1")
  public Observable<APIData> getdata ();
}

The error I am getting while replacing @GET("/search?limit=1") with @GET("/search") is Malformed JSON: Syntax error.

Question

How to proper parse the NdJson ?

Is there any way to store the response in List<APIData>

Edit-1

Now I am trying to accept the generic Response in observer:

adapter.create(APIData.class).getdata()
       .subscribeOn(Schedulers.newThread())
       .observeOn(AndroidSchedulers.mainThread())
       .subscribe(new Observer<Response>() {
        @Override
        public void onCompleted() {}

        @Override
        public void onNext(Response response) {}
 });

But getting no suitable method found for subscribe(<anonymous Observer<Response>>)

Edit-2

However, I was doing some silly mistake and the error what I am getting in "Edit-1" section is fixed now. Now I am getting 

Error:retrofit.RetrofitError: com.google.gson.JsonSyntaxException: com.google.gson.stream.MalformedJsonException: Use JsonReader.setLenient(true) to accept malformed JSON at line 2 column 2 path $

Upvotes: 2

Views: 4248

Answers (3)

njzk2
njzk2

Reputation: 39406

For starters, your json is not json that Gson understand, so you can't use the Gson converter in Retrofit directly. So

public Observable<APIData> getdata ();

must be

public Observable<Response> getdata ();

Then, you have

adapter.create(WarehouseAPI.class).getdata()
   .subscribeOn(Schedulers.newThread())
   .observeOn(AndroidSchedulers.mainThread())
   .map((Response res) -> new BufferedStreamReader(res.body.body().bytesStream()))
   .flatMap(stream -> {
       return Observable.create(subscriber -> {
           String line;
           while((line = stream.readLine() != null) {
               subscriber.onNext(gson.fromJson(line, APIData.class));
           }
           subscriber.onCompleted();
       });
   });

You can subscribe to that and receive each individual item of your json list.

This is not tested, and error cases are not handled (and validity of subscription is not tested, for shortness).

It should mostly be in the right direction though.

Upvotes: 1

ProblemSlover
ProblemSlover

Reputation: 2537

I assume that server response has the following format

{"type":"data","id":"xyz"}\n{"type":"data","id":"xyz"}

The basic idea is to receive the response from the server as the String. Split it into array like response.split("\n"). Iterate through an Array creating a new json object for every array element.

I do realize it's quite time consuming as you've described in the comments. You can also try to play with String replaceAll method to transform each line into an array element and parse the whole string. like 

String myResponse = "[" + response.replaceAll("/\n/", ",") + "]";
Gson gson = new Gson();
MyEntity[] arr = gson.fromJson(myResponse, MyEntity[].class);

In case of Retrofit. You will have to use a custom Response Converter. I won't write up the complete solution since you have found the way to get it done using custom Converter.

Upvotes: 2

Amit Pal
Amit Pal

Reputation: 11052

To make things work I had to add Custom converter implements by Retrofit.Converter:

public class CustomConverter implements Converter {

@Override
public Object fromBody(TypedInput body, Type type) throws ConversionException {
    String text = null;
    try {
        text = fromStream(body.in());
    } catch (IOException e) {
        e.printStackTrace();
    }
    return text;

}

@Override
public TypedOutput toBody(Object object) {
    return null;
}

// Custom method to convert stream from request to string
public static String fromStream(InputStream in) throws IOException {
    BufferedReader reader = new BufferedReader(new InputStreamReader(in));
    StringBuilder out = new StringBuilder();
    String newLine = System.getProperty("\n");
    String line;
    while ((line = reader.readLine()) != null) {
        out.append(line);
        out.append(newLine);
    }
    return out.toString();
}
}

And it works perfectly!

Upvotes: 2

Related Questions