CODEWITHSUNDEEP
jsonspringspring-batchbatch-processing
ams2705
ams2705

Reputation: 297

How to read a complex JSON in spring batch?

I have a complex JSON below. I am reading it using FlatFileItemReader. How can I ignore the last line "]", with my customized ComplexJsonRecordSeparatorPolicy?

[
  {"firstName":"Tom", "lastName":"Cruise"}, 
  {"firstName":"Bruce", "lastName":"Willis"},
  {"firstName":"Liam", "lastName":"Neeson"}
]

My ComplexJsonRecordSeparatorPolicy looks like below. This class is successfully working, when I have "]" in line no 4 but, it throws an error when the line is supplied with only "]" in line no 5, as my post processor deletes the line instead of ignoring it.

public class ComplexJsonRecordSeparatorPolicy extends JsonRecordSeparatorPolicy {

    @Override
    public boolean isEndOfRecord(String line) {

        return StringUtils.countOccurrencesOf(line, "{") == StringUtils.countOccurrencesOf(line, "}")
                && (line.trim().endsWith("}") || line.trim().endsWith(",") || line.trim().endsWith("]"));
    }

    @Override
    public String postProcess(String record) {
        if (record.startsWith("["))
            record = record.substring(1);
        if ((record.endsWith("]") || record.endsWith(",")))
            record = record.substring(0, record.length() - 1);
        return super.postProcess(record);
    }
}

Upvotes: 2

Views: 6142

Answers (1)

Nghia Do
Nghia Do

Reputation: 2668

I have created a small example for this one. Please take a look and let me know your thoughts

https://github.com/bigzidane/spring-batch-jsonListItem-reader.

For more information, I have created a Reader which is to parse JSON as a List and then map every entry to a POJO through 'classToBound' and then return each by each follow Spring Batch standard.

The example takes an Json file as 

[
    {
        "name": "zidane",
        "nation": "france"
    },
    {
        "name": "ronaldo",
        "nation": "brazil"
    },
    {
        "name": "marcelo",
        "nation": "brazil"
    }
]

The job configuration

<job id="exampleJsonReaderJob" xmlns="http://www.springframework.org/schema/batch">
        <step id="stepId">
            <tasklet>
                <chunk reader="exampleJsonReader" writer="exampleWriter" processor="exampleProcessor" commit-interval="5" />
            </tasklet>
        </step>
    </job>

The Reader

<bean id="exampleJsonReader" class="com.itservicesdepot.example.springbatch.jsonreader.reader.JsonFileListItemReader" scope="step">
        <property name="resource" value="classpath:soccers.json" />
        <property name="classToBound" value="com.itservicesdepot.example.springbatch.jsonreader.model.SoccerJsonEntry" />
    </bean>

The Processor and Writer are just normal ones.

In the reader, there is an property "classToBound", that is the Pojo which is top map with an item in Json list.

Upvotes: 3

Related Questions