CODEWITHSUNDEEP
node.jsstreamstdoutstderrio.js
Philip
Philip

Reputation: 4208

Combine stdout and stderr into a single node.js stream

I want to combine stdout and stderr of a child process into a single, intermediate stream which I can then consume and do processing on.

Is this possible with the standard API's?

(I realize there are subtleties around interleaving the two streams, and I would need them to be only interleaved on line endings for this to work. i.e. writing a line to stdout or stderr should be an atomic operation).

Upvotes: 7

Views: 351

Answers (1)

wakobu
wakobu

Reputation: 328

This should work as expected:

    // This custom stream merges stdout and stderr.
    // Instead of concatenating the two streams, it keeps their interleaving.
    const stdoutStderrInterleavedStream = (
        stdout: ReadableStreamDefaultReader<Uint8Array>,
        stderr: ReadableStreamDefaultReader<Uint8Array>
    ) =>
        new ReadableStream({
            start(controller) {
                // These sentinel values are used to know if both stream have been stopped.
                let stdoutDone = false
                let stderrDone = false

                type SentinelFunctions = {
                    getDone: () => boolean
                    setDone: () => void
                }

                // stdout will check if stderr is done and vice-versa
                const sentinelStdout: SentinelFunctions = {
                    getDone: () => stderrDone,
                    setDone: () => (stdoutDone = true),
                }
                const sentinelStderr: SentinelFunctions = {
                    getDone: () => stdoutDone,
                    setDone: () => (stderrDone = true),
                }

                const onData =
                    ({ getDone, setDone }: SentinelFunctions) =>
                    ({ done, value }: ReadableStreamDefaultReadResult<Uint8Array>) => {
                        if (done) {
                            setDone()
                            if (getDone()) {
                                controller.close()
                            }
                            return
                        }
                        controller.enqueue(value)
                        push()
                    }

                const push = () => {
                    stdout.read().then(onData(sentinelStdout))
                    stderr.read().then(onData(sentinelStderr))
                }
                push()
            },
        })

Example usage:

const stream = stdoutStderrInterleavedStream(
        myDenoProcess.stdout.getReader(),
        myDenoProcess.stderr.getReader()
    )
        .pipeThrough(new TextDecoderStream())
        .pipeThrough(new TextLineStream())

const output = await Array.fromAsync(stream)
console.log(output)

Upvotes: 0

Related Questions