CODEWITHSUNDEEP
next.jswebsocketstreamingserver-side-renderingmutation
fishforyou
fishforyou

Reputation: 66

Sequential data fetching in Server Component to mutate same set of data (Next.js)

I am new to next.js (using v13 with App Router) and I am seeking advice on an issue I've been grappling with.

My goal is to render data on a page, and progressively update it as "better" data comes in (e.g., more recent data, data from more sources, etc.). Typically, there should be 2-3 updates until the data has finished loading (e.g., on a hotel booking website, where options update progressively until the loading indicator turns off). I want to execute this server-side as much as possible.

When data needs to be updated, my API response (Django backend) includes a "next" entry with a url, e.g.:

{ "meta": {..., "next": "https://api.example.com/path/to/next/request"}, 
  "data": ... }

The next.js doc is quite clear on sequential data fetching, but it does not show how to update the same set of data, only how to display different blocks of data progressively.

After trying different approaches, I implemented a solution that employs a recursive component, populating a <Suspense> boundary with the current data as fallback.

import { Suspense } from 'react';

export default async function Stream() {
  return (
      <div className='m-auto h-screen w-1/2 p-3'>
        <h1>Welcome to the Stream page!</h1>
        <Results url={'http://127.0.0.1:3000/api/dummyData'} />
      </div>
  );
}

async function Results({ url }: { url?: string }) {
  const content = await fetch(url, {
    cache: 'no-store',
  }).then((res) => res.json());
  
  // if "next" url in response, insert in recursive function
  if (content.next) {
    return (
      <Suspense fallback={<ResultsDisplayer content={content} />}>
        <Results url={content.next} />
      </Suspense>
    );
  } else {
    return <ResultsDisplayer content={content} />;
  }
}

async function ResultsDisplayer({ content }) {
  const isLoading = Boolean(content.next);
  return (
    <>
      <p>Loading: {isLoading ? 'YES' : 'NOPE'}</p>
      <br />
      <p>Data:</p>
      <pre>{JSON.stringify(content)}</pre>;
    </>
  );
}

Illustration of recursive solution (GIF)

This solution seems to work, but I am curious if there is a more conventional/straightforward approach to achieve server-side data mutations triggered by chained hyperlinks. I couldn't find any good examples or docs that covered this use case.

I've explored other options while researching the problem, but none of them was really fitting:

Has anyone else encountered this issue? If so, how did you handle it? Are there features of next.js that I might have overlooked or misunderstood?

Thank you! :)

Upvotes: 4

Views: 902

Answers (1)

Eric Burel
Eric Burel

Reputation: 4954

Very interesting question. To rephrase I feel like you want to have only one HTTP request, send response data progressively, but have the client to update as data are incoming.

I think you need to use streams both ways, server and client.

First you need to respond with a ReadableStream. The code is a bit too complex to share here, you may want to read this GitHub discussion about sending files with Route Handlers, it shows different syntaxes to achieve this goal.

Then, client-side, you can fetch this endpoint and consume the stream, using a similar API. The fetch Response body will be a ReadableStream with your data.

You can use the stream event handlers to update a state variable that will rerender your component. There are perhaps solutions with <Supspense /> or incoming use hook but I don't think you need that, instead of using a promise client-side you can stick to event handlers.

One thing you need to figure is how to make each chunk of data "standalone" (= a complete string that you can call JSON.stringify upon), there are probably ways to send the chunk size along size the chunk (for instance send the size then the string then the size etc. until you send -1 to close the stream).

That's for the theory. In practice, an issue you may encounter is that in Route Handlers, you need to return a Response object, and then only Next.js will start triggering the HTTP response. While in an API Route/Express endpoint, you can call res.send anytime, and then feed the stream.

Basically this mean that I am not sure about the stream lifecycle: can you keep feeding the stream after you returned from the Route Handler function? Probably yes but it's worth a try.

Upvotes: 1

Related Questions