CODEWITHSUNDEEP
mongodbmeteor
Oliver Klein
Oliver Klein

Reputation: 599

Make my mongoDB collect data at a certain time of day every day?

I'm building a weather app that is scraping data from a particular site, however at certain time in the day the data becomes inaccurate because of the way I have to scrape the data. If i could collect the data at a certain time every day this wouldn't be an issue.

Is there a way for my Meteor server to go out and collect new data at 1 in the morning each day and store the info in a mongo database that i can use throughout the day?

Upvotes: 2

Views: 130

Answers (2)

imkost
imkost

Reputation: 8163

Also there is a plain-javascript solution based on setTimeout function:

function updateWeather() {
  // Update weather logic
}

/**
 * @returns {Number} ms till next day's 1 am
 */
function computeMsToNextWeatherUpdate() {
  const tomorrow = new Date();

  tomorrow.setDate(tomorrow.getDate() + 1);
  tomorrow.setHours(13); // 1 am
  tomorrow.setMinutes(0);
  tomorrow.setSeconds(0);

  return tomorrow.getTime() - Date.now();
}

function startWeatherUpdater() {
  updateWeather();

  setTimeout(startWeatherUpdater, computeMsToNextWeatherUpdate());
}

startWeatherUpdater();

Upvotes: 2

bluebird
bluebird

Reputation: 640

Check out the percolate:synced-cron package.

https://atmospherejs.com/percolate/synced-cron

Using that you could do something like:

SyncedCron.add({
  name: 'Scrape weather data',
  schedule: function(parser) {
    // parser is a later.parse object
    return parser.text('at 1:00 am' );
  },
  job: function() {
    // 
    // scraping code
    //
    WeatherData.insert(scrapedData); //insert to MongoDB Collection

  }
});

SyncedCron.start();

Upvotes: 3

Related Questions