CODEWITHSUNDEEP
javascriptnode.jshttpsfetch
Nadeem Ahmad
Nadeem Ahmad

Reputation: 745

How to read a stream of data fetched on client side - JavaScript, Node JS

I have created a simple proxy using Node (Express), where I am reading a pdf file from a remote server. I am able to read the file in chunks, and sending the chunks in response as res.write(). When I log the data, I can see all the streaming, but I can't receive the chunks on the front end. I am calling the endpoints correctly, but I want to get a response for each chunk is read.

Client Side code

    fetch('http://localhost:8003/pdf-cors')
    .then(response => response.text())
    .then(data => console.log(data));

Node Express code

app.get('/pdf-cors', (req, res) => {
    https.get('https://www.mazda.com/globalassets/en/assets/csr/download/2017/2017_all.pdf')
        .on('response', response => {
        response.on('data', data => {
            console.log("[Reading File]...");
            res.write(data);
        })
        response.on('end', _ => {
            console.log("File reading done !");
        })
    })
});

Note: When I put res.end() right after res.write(data) I am able to see the first chunk passed to the client in the console, but then I get an error says Error [ERR_STREAM_WRITE_AFTER_END]: write after the end.

The only thing that I want is to see each chunk being passed to the front end.

Upvotes: 1

Views: 6987

Answers (2)

Zebadix
Zebadix

Reputation: 16

If you only need server to client events you can use EventStream with server-sent events.

export const apiEventSourceStream = async (onMessage) => {
  const url = `${API_IRL}/query-stream`;
  const eventSource = new EventSource(url);

  eventSource.onmessage = (event) => {
    onMessage(event.data);
  };

  eventSource.onerror = (error) => {
    console.error("EventSource error: ", error);
    eventSource.close();
  };

  return () => eventSource.close();
};

Upvotes: 0

Estus Flask
Estus Flask

Reputation: 222309

Fetch API allows to stream data with ReadableStream.

On client side a stream can be read like:

  let res = await fetch('...');
  let reader = res.body.getReader();
  let result;
  let decoder = new TextDecoder('utf8');
  while (!result?.done) {
    result = await reader.read();
    let chunk = decoder.decode(result.value);
    console.log(chunk);
  }

Upvotes: 6

Related Questions