Reputation: 599
I'm building a weather app that is scraping data from a particular site, however at certain time in the day the data becomes inaccurate because of the way I have to scrape the data. If i could collect the data at a certain time every day this wouldn't be an issue.
Is there a way for my Meteor server to go out and collect new data at 1 in the morning each day and store the info in a mongo database that i can use throughout the day?
Upvotes: 2
Views: 130
Reputation: 8163
Also there is a plain-javascript solution based on setTimeout
function:
function updateWeather() {
// Update weather logic
}
/**
* @returns {Number} ms till next day's 1 am
*/
function computeMsToNextWeatherUpdate() {
const tomorrow = new Date();
tomorrow.setDate(tomorrow.getDate() + 1);
tomorrow.setHours(13); // 1 am
tomorrow.setMinutes(0);
tomorrow.setSeconds(0);
return tomorrow.getTime() - Date.now();
}
function startWeatherUpdater() {
updateWeather();
setTimeout(startWeatherUpdater, computeMsToNextWeatherUpdate());
}
startWeatherUpdater();
Upvotes: 2
Reputation: 640
Check out the percolate:synced-cron package.
https://atmospherejs.com/percolate/synced-cron
Using that you could do something like:
SyncedCron.add({
name: 'Scrape weather data',
schedule: function(parser) {
// parser is a later.parse object
return parser.text('at 1:00 am' );
},
job: function() {
//
// scraping code
//
WeatherData.insert(scrapedData); //insert to MongoDB Collection
}
});
SyncedCron.start();
Upvotes: 3