CODEWITHSUNDEEP
clojureclojure-contrib
nkassis
nkassis

Reputation: 5353

reading large command output with clojure

I'm using the sh function from the clojure.java.shell command to read the very large output of a command. The output is around 60meg of data.

I keep getting java.lang.OutOfMemoryError. Is there a way to open a sort of pipe that would let me read the output and parse it into a vector . Like a lazy sequence for the output of the command?

Basically the data is a large array of byte that I want to convert to just numbers and put into a vector.

Upvotes: 3

Views: 483

Answers (1)

Jürgen Hötzel
Jürgen Hötzel

Reputation: 19717

clojure.java.shell/sh will always return a non-lazy string

A solution (doesn't handle closing, environment passing and encoding) using lazy line-seq on an BufferedReader:

(->> (.exec (Runtime/getRuntime) "YOUR_LONG_RUNNING_COMMAND ARG ...")
    .getInputStream
    clojure.java.io/reader
    line-seq
    (map YOUR-FUNCTION))

Upvotes: 4

Related Questions