CODEWITHSUNDEEP
javakotlincsvjvmheap
dzikulisz cezar
dzikulisz cezar

Reputation: 11

Data streaming - is this the right way to build CSV file using ByteArrayOutputStream?

I do have this code.

@RequestMapping("/test")
fun getData(): ResponseEntity<ByteArray> {
    val items = repository.getItems()
    val outputStream = ByteArrayOutputStream()
    
    myService.writeData(items, outputStream)
    val byteArrayData = outputStream.toByteArray()
    
    outputStream.close() // ---------> Do I have to close here as well?

    ResponseEntity.ok()
        .contentType(MediaType.APPLICATION_OCTET_STREAM)
        .header(
            HttpHeaders.CONTENT_DISPOSITION,
            "attachment; filename=file.csv"
        )
       .body(byteArrayData)
}

As well as

fun writeData(items: List<Item>, outputStream: ByteArrayOutputStream) {
    CSVPrinter(
        outputStream.writer(),
        CSVFormat.DEFAULT
    ).use { csvPrinter ->
        items.forEachIndexed { index, it ->
            csvPrinter.printRecord(…)
        }
    }

Few question arise:

  1. Do I have to close outputStream as well, when using .use function on CSVPrinter? .use should do the close, just not sure whether it's outputStream that's closed...
  2. When to outputStream.toByteArray? Is it okay to do that after stream is closed?
  3. Previously I have used StringBuilder to build actual .csv, but it produced some OutOfMemory errors. After switching to data streaming, the heap usage is not that much better... What am I missing?

Upvotes: 0

Views: 41

Answers (1)

k314159
k314159

Reputation: 11062

Answers to each of your questions:

  1. No, you don't have to close a ByteArrayOutputStream. Its documentation says: "Closing a ByteArrayOutputStream has no effect."

  2. You can call toByteArray either before or after closing the stream (or don't close it at all). The documentation linked above says: "The methods in this class can be called after the stream has been closed without generating an IOException."

  3. The large heap usage is due to the data being large and you are storing all of the data in memory at once. Thus it doesn't matter whether you use StringBuilder or ByteArrayOutputStream or whatever. What you need to do is to avoid holding all the data; in other words, you should stream the data. Use a StreamingResponseBody. There are various other answers on this site that tell you how to use this interface.

Upvotes: 2

Related Questions