CODEWITHSUNDEEP
databasecrondatasourcemerging-data
dianovich
dianovich

Reputation: 2287

Multiple data sources: data storage and retrieval approaches

I am building a website (probably in Wordpress) which takes data from a number of different sources for display on various pages.

The sources:

From each source I will mainly retrieve

My question is:

What is the best way to a) store the data and b) retrieve the data

My thinking is:

i) Write a script that is run every 2 or so minutes on a cron job
ii) the script retrieves data from all sources and stores it in the local database
iii) application code can then retrieve all data from the one source, the local database

This should make application code easier to manage - we only ever draw data from one source in application code - and that's the main appeal. But is it overkill for a relatively small site?

Upvotes: 0

Views: 404

Answers (1)

FoneyOp
FoneyOp

Reputation: 336

I would recommend putting the twitter feed and flickr feed in JavaScript. Both flickr and twitter have REST APIs. By putting it on the client you free up resources on your server, create less complexity, your users won't be waiting around for your server to fetch the data, and you can let twitter and flickr cache the data for you.

This assumes you know JavaScript. Once you get past JavaScript quirks, it's not a bad language. Give Jquery a try. JQuery Twitter plugin Flickery JQuery plugin. There are others, that's just the first results from Google.

As for your data on the local server and remote server, that will depend more on the data that is being fetched. I would go with whatever you can develop the fastest and gives acceptable results. If that means making a REST call from server to sever, then go for it. IF the remote server is slow to respond, I would go the AJAX REST API method.

And for the local database, you are going to have to write server side code for that, so I would do that inside the Wordpress "framework".

Hope that helps.

Upvotes: 1

Related Questions