CODEWITHSUNDEEP
javascriptreact-reduxredux-thunk
Colin Hale
Colin Hale

Reputation: 870

What is best practice to remove duplicate/redundant API requests?

So I am working in a React platform that has data that updates every second(I would like to move to web-sockets but its currently only supports gets).Currently, each component makes a fetch request for itself to get the data for the widget. Because the fetch requests are built into the widgets there are redundant api requests for the same data. I am looking for a possible better solution to remove these redundant api requests.

The solution I came up with uses what I call a data service that checks for any subscription to data sources then makes those api calls and places the data in a redux state for the components to then be used. I am unsure if this is the best way to go about handling the issue I am trying to avoid. I don't like how I need an interval to be run every second the app is running to check if there are "subscriptions". I am unsure if thats the correct way to go about it. With this solution I don't duplicate any requests and can add or remove a subscription without affecting other components.

One more thing, the id can change and will change what data I recieve

Here is a simplified version of how I am handling the service.

const reduxState = {
 id: "specific-id",  
 subscriptions: {
    sourceOne: ["source-1-id-1", "source-1-id-2", "source-1-id-3"],
    sourceTwo: ["source-2-id-1", "source-one-id-2"],
  },
  data: {
    sourceOne: { id: "specific-id", time: "milliseconds", data: "apidata" },
    sourceTwo: { id: "specific-id", time: "milliseconds", data: "apidata" },
  },
};

const getState = () => reduxState; //Would be a dispatch to always get current redux state

const dataService = () => {
  const interval = setInterval(() => {
    const state = getState();
    if (state.subscriptions.sourceOne.length > 0)
      fetchSourcOneAndStoreInRedux();
    if (state.subscriptions.sourceTwo.length > 0)
      fetchSourceTwoAndStoreInRedux();
  }, 1000);
};

const fetchSourcOneAndStoreInRedux = (id) =>{
    return async dispatch => {
        try {
            const res = await axios.get(`/data/one/${id}`) 
            dispatch(setSourceOneDataRedux(res.data))
        } catch (err) {
            console.error(err)
        }
    }
}

I am building my components to only show data from the correct id.

Upvotes: 2

Views: 7196

Answers (2)

Jeremy
Jeremy

Reputation: 89

Try this lightweight fetch wrapper lib; the implementation is pretty neat:

import xior from 'xior';
import dedupePlugin from 'xior/plugins/dedupe';
import throttlePlugin from 'xior/plugins/throttle';

// Setup
const http = xior.create({
  baseURL: 'http://example.com',
});
http.plugins.use(dedupePlugin()); // Prevent same GET requests from occurring simultaneously.
http.plugins.use(throttlePlugin()); // Throttle same `GET` request in 1000ms

// 
// Dedupe the same `GET` requests, this will only sent 1 real request
await Promise.all([
  http.get('/api/get-data-2'),
  http.get('/api/get-data-2'),
  http.get('/api/get-data-2'),
]);

More info from xior docs: https://github.com/suhaotian/xior

Upvotes: 0

Eric
Eric

Reputation: 498

Here is a simple working example of a simple "DataManager" that would achieve what you are looking for.

class DataManager {
  constructor(config = {}) {
    this.config = config;
    console.log(`DataManager: Endpoint "${this.config.endpoint}" initialized.`);
    if (this.config.autostart) { // Autostart the manager if autostart property is true
      this.start();
    }
  }

  config; // The config object passed to the constructor when initialized
  fetchInterval; // The reference to the interval function that fetches the data
  data; // Make sure you make this state object observable via MOBX, Redux etc so your component will re-render when data changes.
  fetching = false; // Boolean indicating if the APIManager is in the process of fetching data (prevent overlapping requests if response is slow from server)

  // Can be used to update the frequency the data is being fetched after the class has been instantiated
  // If interval already has been started, stop it and update it with the new interval frequency and start the interval again
  updateInterval = (ms) => {
    if (this.fetchInterval) {
      this.stop();
      console.log(`DataManager: Updating interval to ${ms} for endpoint ${this.config.endpoint}.`);
      this.config.interval = ms;
      this.start();
    } else {
      this.config.interval = ms;
    }
    return this;
  }

  // Start the interval function that polls the endpoint
  start = () => {
    if (this.fetchInterval) {
      clearInterval(this.fetchInterval);
      console.log(`DataManager: Already running! Clearing interval so it can be restarted.`);
    }

    this.fetchInterval = setInterval(async () => {
      if (!this.fetching) {
        console.log(`DataManager: Fetching data for endpoint "${this.config.endpoint}".`);
        this.fetching = true;
        // const res = await axios.get(this.config.endpoint); 
        // Commented out for demo purposes but you would uncomment this and clear the anonymous function below
        const res = {};
        (() => {
          res.data = {
            dataProp1: 1234,
            dataProp2: 4567
          }
        })();
        this.fetching = false;
        this.data = res.data;
      } else {
        console.log(`DataManager: Waiting for pending response for endpoint "${this.config.endpoint}".`);
      }
    }, this.config.interval);

    return this;
  }

  // Stop the interval function that polls the endpoint
  stop = () => {
    if (this.fetchInterval) {
      clearInterval(this.fetchInterval);
      console.log(`DataManager: Endpoint "${this.config.endpoint}" stopped.`);
    } else {
      console.log(`DataManager: Nothing to stop for endpoint "${this.config.endpoint}".`);
    }
    return this;
  }

}

const SharedComponentState = {
  source1: new DataManager({
    interval: 1000,
    endpoint: `/data/one/someId`,
    autostart: true
  }),
  source2: new DataManager({
    interval: 5000,
    endpoint: `/data/two/someId`,
    autostart: true
  }),
  source3: new DataManager({
    interval: 10000,
    endpoint: `/data/three/someId`,
    autostart: true
  })
};

setTimeout(() => { // For Demo Purposes, Stopping and starting DataManager.
  SharedComponentState.source1.stop();
  SharedComponentState.source1.updateInterval(2000);
  SharedComponentState.source1.start();
}, 10000);

// Heres what it would look like to access the DataManager data (fetched from the api)
// You will need to make sure you pass the SharedComponentState object as a prop to the components or use another React mechanism for making that SharedComponentState accessible to the components in your app
// Accessing state for source 1: SharedComponentState.source1.data
// Accessing state for source 2: SharedComponentState.source2.data
// Accessing state for source 3: SharedComponentState.source3.data

Basically, each instance of the DataManager class is responsible for fetching a different api endpoint. I included a few other class methods that allow you to start, stop and update the polling frequency of the DataManager instance.

Upvotes: 2

Related Questions