CODEWITHSUNDEEP
java-8parallel-processingjava-stream
Axel Coon
Axel Coon

Reputation: 85

Is it a good idea to read a file's lines with a java 8 parallel streams?

Is it a good idea to read a file's lines with a java 8 parallel streams? Every line is an URL, so the task is to

  1. Read an URL
  2. Connect to the appropriate URL address
  3. Save the resource to an appropriate folder

Sometimes parallel streams using lead to mysterious bugs, so is it the straight case to use parallel streams?

Upvotes: 3

Views: 143

Answers (1)

Eugene
Eugene

Reputation: 120978

It depends, if your file does not have multiple lines, you might not benefit at all from parallel processing, for two reasons: 1) because all that parallel infrastructure is not cheap and 2) because a stream does not known how many lines a File has. As such, it will read lines and create a buffer for every Spliterator ( chunk ), IIRC it starts with 1024 lines and add 1024 and so on... So parallel might turn up to be very expensive with most probably no benefit in such a case. If on the other hand you have huge files... Measure to understand if you really need parallel

Also note that you are going to do some IO and parallel is not that of a good idea for that.

Upvotes: 3

Related Questions