Java 8 stream.map() : Exception in thread "main" java.lang.IllegalStateException: source already consumed or closed

Question

I have below program which:

1. Reads data from a file as a stream.
2. Create a new stream using stream.map() over the above stream and store it in retVal.
3. Return the retVal.
4. Print the values of Stream retval using forEach, but I get Exception: IllegalStateException: source already consumed or closed
5. For function workingPopulateFromFileAsStream I have explicitly converted it again to a stream and then it works fine.

Contents of student.txt:

Name101,Last101
Name102,Last102
Name103,Last103
Name104,Last104
Name105,Last105
Name106,Last106
Name107,Last107
Name108,Last108
Name109,Last109
Name110,Last110

ClientMainApp.java:

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.*;
import java.util.stream.Collectors;
import java.util.stream.Stream;

class Student {
  String first_name;
  String last_name;

  public Student(String first_name, String last_name) {
    this.first_name = first_name;
    this.last_name = last_name;
  }

  public String toString() {
    return this.first_name + " " + this.last_name;
  }
}

public class ClientMainApp {
  public static void main(String... args) {
    String fileName = "student.txt";
    ClientMainApp obj = new ClientMainApp();

    //Working Code.
    Stream<Student> students_stream = obj.workingPopulateFromFileAsStream(fileName);
    System.out.println("working: Got Student Stream as " + students_stream);
    students_stream.forEach(System.out::println);
    System.out.println("==================================================");

    // Not Working Code.
    Stream<Student> students_stream2 = obj.populateFromFileAsStream2(fileName);
    System.out.println("notWorking: Got Student Stream2 as " + students_stream2);
    students_stream2.forEach(System.out::println);
  }

  private Stream<Student> workingPopulateFromFileAsStream(String fileName) {
    try (Stream<String> stream = Files.lines(Paths.get(fileName))) {
      System.out.println("Working: Got File    Stream as " + stream);
      Stream<Student> retval =
          stream
              .map(
                  st -> {
                    ArrayList<String> record = new ArrayList<>(Arrays.asList(st.split(",")));
                    return new Student(record.get(0), record.get(1));
                  })
              .collect(Collectors.toList()).stream(); // This is the code making the difference.
      System.out.println("Working: Got Retval  Stream as " + retval);
      return retval;
    } catch (IOException e) {
      e.printStackTrace();
    }
    return null;
  }

  private Stream<Student> populateFromFileAsStream2(String fileName) {
    try (Stream<String> stream = Files.lines(Paths.get(fileName))) {
      System.out.println("NotWorking: Got File    Stream  as " + stream);
      Stream<Student> retval =
          stream.map(
              st -> {
                ArrayList<String> record = new ArrayList<>(Arrays.asList(st.split(",")));
                return new Student(record.get(0), record.get(1));
              });
      System.out.println("NotWorking: got Retval  Stream  as " + retval);
      return retval;
    } catch (IOException e) {
      e.printStackTrace();
    }
    return null;
  }
}

I get the result as:

Working: Got File    Stream as java.util.stream.ReferencePipeline$Head@3e3abc88
Working: Got Retval  Stream as java.util.stream.ReferencePipeline$Head@214c265e
working: Got Student Stream as java.util.stream.ReferencePipeline$Head@214c265e
Name101 Last101
Name102 Last102
Name103 Last103
Name104 Last104
Name105 Last105
Name106 Last106
Name107 Last107
Name108 Last108
Name109 Last109
Name110 Last110
==================================================
NotWorking: Got File    Stream  as java.util.stream.ReferencePipeline$Head@7699a589
NotWorking: got Retval  Stream  as java.util.stream.ReferencePipeline$3@4dd8dc3
notWorking: Got Student Stream2 as java.util.stream.ReferencePipeline$3@4dd8dc3
Exception in thread "main" java.lang.IllegalStateException: source already consumed or closed
    at java.util.stream.AbstractPipeline.sourceSpliterator(AbstractPipeline.java:406)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
    at ClientMainApp.main(ClientMainApp.java:38)

Here in the response, we can see that the stream object is the same for retVal and that returned back to the calling function, yet I am not sure why it says its already consumed.

Please note that if I use a forEach loop, inside populateFromFileAsStream2 then it works fine. It's only when the value is returned from the function, it gives an error.

Answer 1

Your non-working version essentially looks like this:

public Stream<String> foo(Path file) throws IOException {
  try (Stream<String> stream = Files.lines(file)) {
    return stream.map(...);
  }
}

You're using a try-with-resources statement. That means the Stream is closed as soon as the try block is exited, which must happen in order for the method to return. Thus you're closing the stream before returning it from the method. That said, I think your mistake is rooted in:

2. Create a new stream using stream.map() over the above stream and store it in retVal.

Although it's true that map will return a new Stream object, the new object is still part of the same conceptual stream/pipeline. In other words, closing the stream reference is the same as closing the retVal reference. Your working code fixes this problem by returning an entirely new pipeline, though at the cost of buffering the entire file into memory.