CODEWITHSUNDEEP
javajsonrestapache-camel
user1637487
user1637487

Reputation: 271

Apache Camel - Split JSON array and call POST to the REST service iteratively

I have a requirement of processing an Excel file and POST its contents to a REST service. I could load and convert Excel rows to a JSON array through Apache POI. However while POSTing to the REST service it fails with HTTP status 413 because POST body is too big due to the number of rows in the Excel file.

Is there a way in Apache Camel to can limit the JSON input size of a REST service and repetitively invoke the REST service call. Please help.

Below is the Java DSL route config.

from("file:/excelfilelocation/inputexcel.xls")
        .bean(new ExcelConverter(), "processExcel") // Converts excel rows to ArrayList of model object
        .marshal().json(JsonLibrary.Jackson)
        .setHeader("Authorization", simple(apiKEY))
        .setHeader(Exchange.HTTP_METHOD, constant("POST"))
        .setHeader(Exchange.HTTP_URI, simple(API_URL))
        .setHeader(Exchange.CONTENT_TYPE, constant("application/json"))
        .to(API_URL)
        .to("mock:finish");

Error I am getting because of large POST body content length.

org.apache.camel.http.common.HttpOperationFailedException: HTTP operation failed invoking https://test.com/api/v2/receipts with statusCode: 413

Upvotes: 2

Views: 3064

Answers (1)

Miloš Milivojević
Miloš Milivojević

Reputation: 5369

You can use the Splitter EIP to split the Excel entries and then Aggregator EIP to collect them into more manageable batches and then send those batches:

from("file:/excelfilelocation?fileName=inputexcel.xls")
    .bean(new ExcelConverter(), "processExcel")
    .split(body())
    .aggregate(constant(true), new GroupedBodyAggregationStrategy())
      .completionSize(100)
      .completionTimeout(1000)
      .marshal().json(JsonLibrary.Jackson)
      .setHeader("Authorization", simple(apiKEY))
      .setHeader(Exchange.HTTP_METHOD, constant("POST"))
      .setHeader(Exchange.HTTP_URI, simple(API_URL))
      .setHeader(Exchange.CONTENT_TYPE, constant("application/json"))
      .to(API_URL);

Here's the aggregation strategy that collects the rows into a list:

public class GroupedBodyAggregationStrategy extends AbstractListAggregationStrategy<Message> {

    @Override
    public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
        if (oldExchange == null) {
            // for the first time we must create a new empty exchange as the
            // holder, as the outgoing exchange
            // must not be one of the grouped exchanges, as that causes a
            // endless circular reference
            oldExchange = new DefaultExchange(newExchange);
        }
        return super.aggregate(oldExchange, newExchange);
    }

    @Override
    public Object getValue(Exchange exchange) {
        return exchange.getIn().getBody();
    }
}

Upvotes: 2

Related Questions