CODEWITHSUNDEEP
javaspringreststreamingchunked-encoding
hyperstack
hyperstack

Reputation: 29

Java Spring Server Side REST Streaming for Large Data

I currently have a program that takes in a query from a user via REST (Spring)and runs it across the database and returns the results via REST. The issue I'm running into is that if the user queries for a large set of data, at a certain point the server runs into a out of memory error.

Is there a way to stream the results using REST Spring to avoid the out of memory error? I've been researching and it seems like HTTP Chunked Encoding might be an option.

Upvotes: 2

Views: 4509

Answers (2)

Simbosan
Simbosan

Reputation: 274

I would recommend looking at the new Spring repository streamAll functionality. I found the article linked below tremendously helpful (though quite old).

I am using this approach myself to stream pretty large datasets and I have no memory issues. The secret sauce is to limit the fetch size as described in the article below.

https://knes1.github.io/blog/2015/2015-10-19-streaming-mysql-results-using-java8-streams-and-spring-data.html

Upvotes: 0

Mariano Z Lopez
Mariano Z Lopez

Reputation: 36

Maybe SSE (https://en.wikipedia.org/wiki/Server-sent_events) can help you. SSE is a web technology where a browser receives updates from a server via HTTP connection. Examples:

Upvotes: 1

Related Questions