How to handle state in useEffect from a prop passed from infinite scroll component

Question

I have a React component using an infinite scroll to fetch information from an api using a pageToken.

When the user hits the bottom of the page, it should fetch the next bit of information. I thought myself clever for passing the pageToken to a useEffect hook, then updating it in the hook, but this is causing all of the api calls to run up front, thus defeating the use of the infinite scroll.

I think this might be related to React's derived state, but I am at a loss about how to solve this.

here is my component that renders the dogs:

export const Drawer = ({
  onClose,
}: DrawerProps) => {
  const [currentPageToken, setCurrentPageToken] = useState<
    string | undefined | null
  >(null);

  const {
    error,
    isLoading,
    data: allDogs,
    nextPageToken,
  } = useDogsList({
    pageToken: currentPageToken,
  });

  const loader = useRef(null);

  // When user scrolls to the end of the drawer, fetch more dogs
  const handleObserver = useCallback(
    (entries) => {
      const [target] = entries;

      if (target.isIntersecting) {
        setCurrentPageToken(nextPageToken);
      }
    },
    [nextPageToken],
  );

  useEffect(() => {
    const option = {
      root: null,
      rootMargin: '20px',
      threshold: 0,
    };
    const observer = new IntersectionObserver(handleObserver, option);

    if (loader.current) observer.observe(loader.current);
  }, [handleObserver]);

  return (
    <Drawer
      onClose={onClose}
    >
      <List>
        {allDogs?.map((dog) => (
          <Fragment key={dog?.adopterAttributes?.id}>
            <ListItem className={classes.listItem}>
              {dog?.adopterAttributes?.id}
            </ListItem>
          </Fragment>
        ))}
        {isLoading && <div>Loading...</div>}
        <div ref={loader} />
      </List>
    </Drawer>
  );
};

useDogsList essentially looks like this with all the cruft taken out:

import { useEffect, useRef, useState } from 'react';

export const useDogsList = ({
  pageToken
}: useDogsListOptions) => {
  const [isLoading, setLoading] = useState(false);
  const [error, setError] = useState<Error | null>(null);
  const [nextPageToken, setNextPageToken] = useState<string | null | undefined>(
    null,
  );
  const [allDogs, setAllDogs] = useState(null);

  useEffect(() => {
    const fetchData = async () => {

      setLoading(true);

      try {
          const result =
            await myClient.listDogs(
              getDogsRequest,
              {
                token,
              },
            );
          const dogListObject = result?.toObject();
          const newDogs = result?.dogsList;
          setNextPageToken(dogListObject?.pagination?.nextPageToken);

          // if API returns a pageToken, that means there are more dogs to add to the list
          if (nextPageToken) {
            setAllDogs((previousDogList) => [
              ...(previousDogList ?? []),
              ...newDogs,
            ]);
          }
        }
      } catch (responseError: unknown) {
        if (responseError instanceof Error) {
          setError(responseError);
        } else {
          throw responseError;
        }
      } finally {
        setLoading(false);
      }
    };

    fetchData();

  }, [ pageToken, nextPageToken]);


  return {
    data: allDogs,
    nextPageToken,
    error,
    isLoading,
  };
};

Basically, the api call returns the nextPageToken, which I want to use for the next call when the user hits the intersecting point, but because nextPageToken is in the dependency array for the hook, the hook just keeps running. It retrieves all of the data until it compiles the whole list, without the user scrolling.

I'm wondering if I should be using useCallback or look more into derivedStateFromProps but I can't figure out how to make this a "controlled" component. Does anyone have any guidance here?

Answer 1

I suggest a small refactor of the useDogsList hook to instead return a hasNext flag and fetchNext callback.

export const useDogsList = ({ pageToken }: useDogsListOptions) => {
  const [isLoading, setLoading] = useState(false);
  const [error, setError] = useState<Error | null>(null);
  const [nextPageToken, setNextPageToken] = useState<string | null | undefined>(
    pageToken // <-- initial token value for request
  );
  const [allDogs, setAllDogs] = useState([]);

  // memoize fetchData callback for stable reference
  const fetchData = useCallback(async () => {
    setLoading(true);

    try {
      const result = await myClient.listDogs(getDogsRequest, { token: nextPageToken });
      const dogListObject = result?.toObject();
      const newDogs = result?.dogsList;
      setNextPageToken(dogListObject?.pagination?.nextPageToken ?? null);
      setAllDogs((previousDogList) => [...previousDogList, ...newDogs]);
    } catch (responseError) {
      if (responseError instanceof Error) {
        setError(responseError);
      } else {
        throw responseError;
      }
    } finally {
      setLoading(false);
    }
  }, [nextPageToken]);

  useEffect(() => {
    fetchData();
  }, []); // call once on component mount

  return {
    data: allDogs,
    hasNext: !!nextPageToken, // true if there is a next token
    error,
    isLoading,
    fetchNext: fetchData, // callback to fetch next "page" of data
  };
};

Usage:

export const Drawer = ({ onClose }: DrawerProps) => {
  const { error, isLoading, data: allDogs, hasNext, fetchNext } = useDogsList({
    pageToken // <-- pass initial page token
  });

  const loader = useRef(null);

  // When user scrolls to the end of the drawer, fetch more dogs
  const handleObserver = useCallback(
    (entries) => {
      const [target] = entries;

      if (target.isIntersecting && hasNext) {
        fetchNext(); // <-- Only fetch next if there is more to fetch
      }
    },
    [hasNext, fetchNext]
  );

  useEffect(() => {
    const option = {
      root: null,
      rootMargin: "20px",
      threshold: 0
    };
    const observer = new IntersectionObserver(handleObserver, option);

    if (loader.current) observer.observe(loader.current);

    // From @stonerose036
    // clear previous observer in returned useEffect cleanup function
    return observer.disconnect;
  }, [handleObserver]);

  return (
    <Drawer onClose={onClose}>
      <List>
        {allDogs?.map((dog) => (
          <Fragment key={dog?.adopterAttributes?.id}>
            <ListItem className={classes.listItem}>
              {dog?.adopterAttributes?.id}
            </ListItem>
          </Fragment>
        ))}
        {isLoading && <div>Loading...</div>}
        <div ref={loader} />
      </List>
    </Drawer>
  );
};

Disclaimer

Code hasn't been tested, but IMHO it should be pretty close to what you are after. There may be some minor tweaks necessary to satisfy any TSLinting issues, and getting the correct initial page token to the hook.

Answer 2

While Drew and @debuchet's answers helped me improve the code, the problem around multiple renders ended up being solved by tackling the observer itself. I had to disconnect it afterwards

  useEffect(() => {
    const option = {
      root: null,
      rootMargin: '20px',
      threshold: 0,
    };
    const observer = new IntersectionObserver(handleObserver, option);

    if (loader.current) observer.observe(loader.current);

    return () => {
      observer.disconnect();
    };
  }, [handleObserver]);