CODEWITHSUNDEEP
xmlcaching
feronovak
feronovak

Reputation: 2697

Best way to cache XML files

I have website which connects to 14-17 xml streams and downloads streams with every page load. That was fine for testing purposes and for traffic <100/day.

However it now gets to stage when page loads slower and slower because I need to wait for external XML to download.

What is the best way to cache/manipulate with these files? These data are being constantly updated, so rather than downloading XML streams with every visit I would like to have some robot doing this every 5 minutes locally and visitors to read only from local copies.

What methods would you use to make sure that XML files wont get locked or there are no zero results whilst downloading new results?

Thanks.

Upvotes: 1

Views: 3110

Answers (2)

Brad Mace
Brad Mace

Reputation: 27886

I'd probably store the data in memcached and update the data there after each download. You'll always have data available even if the file isn't on disk, and you'll have orders of magnitude less disk i/o than having them all read directly from the file, which will improve performance significantly.

For C# I think this is the library you'll need: http://sourceforge.net/projects/memcacheddotnet/

Upvotes: 1

trinth
trinth

Reputation: 6047

One option would be to use memcached. You could cache the xml data there and set an expiry for say 5 minutes and have your pages load data from the memcached.

Another option (easier to setup or if you don't have the RAM to spare) would be to store downloaded data in a relational database instead of on page load. Then have your pages load the data from there.

Upvotes: 1

Related Questions