React Google Maps & MarkerClusterer: Maximum update depth exceeded problem

Question

I'm working on a application where I will render markers on a google map for a list of concerts. As of now, my data consists of right above 1600 concerts, so naturally I wanted to make use of clusters, and opted for MarkerClusterer. I found this video, in which he also explained how to avoid the error of "Maximum update depth exceeded": https://youtu.be/ZvoMak9yApU?si=usW9BF8__OK31rG6.

Here is the code I've written, which is basically the same as the one in the video.

imports and other logic for my CustomMap component up here...

const optimizedData = shownData.map(({ _id, venue }) => ({
    _id,
    coordinates: venue.coordinates,
    key: _id.toString()
  }));

  if (!isLoaded) return <MapLoader />
  return (
    <Container fluid className="map-container px-0 d-flex justify-content-center">
      <APIProvider apiKey={REACT_API_KEY}>
        <Map defaultCenter={center} defaultZoom={zoom} mapId={MAP_ID}>
          <Markers points={optimizedData} />
        </Map>
      </APIProvider>
    </Container>
  )
}

type Point = {
  _id: ObjectId;
  coordinates: google.maps.LatLngLiteral; 
  key: string;
};
type Props = { points: Point[] }

const Markers = ({ points }: Props) => {
  const map = useMap();
  const [markers, setMarkers] = useState<{ [key: string]: Marker }>({});
  const clusterer = useRef<MarkerClusterer | null>(null);

  useEffect(() => {
    if (!map) return;
    if (!clusterer.current) {
      clusterer.current = new MarkerClusterer({ map })
    }
  }, [map])

  useEffect(() => {
    clusterer.current?.clearMarkers();
    clusterer.current?.addMarkers(Object.values(markers));
  }, [markers])

  const setMarkerRef = (marker: Marker | null, key: string) => {
    // The following two lines is the error handling he mentioned in the video
    if (marker && markers[key]) return; 
    if (!marker && !markers[key]) return;
    
    setMarkers((prev) => {
      if (marker) {
        return { ...prev, [key]: marker }
      } else {
        const newMarkers = { ...prev };
        delete newMarkers[key];
        return newMarkers;
      }
    })
  }

  return <>
    {points.map(point => (
      <AdvancedMarker
        position={point.coordinates}
        key={point.key}
        ref={marker => setMarkerRef(marker, point.key)}
      >
        <span style={{ fontSize: "2rem" }}>⚡</span>
      </AdvancedMarker>))}
  </>
}

Despite adding the code to prevent the error (which he talks about from 17:21), I still have the error, even when using smaller amounts of data, such as 50 points. Does anyone know why?

Thanks in advance!