CODEWITHSUNDEEP
.netxmlperformance
user1447725
user1447725

Reputation: 1383

How to Improve XML read write performance

I have two .NET applications running independently(can start in any order, or may be only one running) which uses XML as data store. So both applications can read and write to XML file. To keep the data updated , i'm loading the XML file every time from the disk before read and write operations. And i'm using XPath query for querying the particular node. Now there is performance issue observed in this method as there are read and write requests on XML every second from one application(uses polling, and cannot be changed) I'm not sure what exactly is causing the performance hit, but i believe its continuous read write.

I tried using memory mapped files from .NET 4.0 but i'm restricted to use .NET 3.5 and not any higher versions.

Can anyone help me out on this?

Note : XML nodes have some common attributes , different number of attributes and one ID which i use for XPath querying.

Upvotes: 5

Views: 2910

Answers (5)

Adriano Repetti
Adriano Repetti

Reputation: 67090

IF you're sure that performance hit comes from I/O and you can't change both application there is really a little you can do.

First solution with zero changes to existing application code: use a RAM disk. If they're using that file as shared memory you can do it without any other change. If data is persistent you may need to perform a background copy to another media after each writing. Performance won't be as good as a true shared memory but at least you won't have to wait for slow I/O operations.

Second solution with changes only in the application that must read data: often the parsing of a XML file is pretty slow (specially if you're using XmlDocument and the file isn't very little). In this case, using XmlReader, you have to make your read code more complicated and to forget about XPath queries but its performance will be many times better than XmlDocument and it won't slow down increasing the file size.

Small (or not so small) updates: if code of the second application (I guess the one that will read the file) can be changed you can do a little to improve its performance. First of all do not read the file each time. Check its timestamp, register a FileSystemWatcher for that file or whatever else but do not read/parse file each time. When you did this you can go one step forward: read/parse the file only when it changes, prepare your XmlDocument on background (another thread) and make it available for polling requests. If requests are spaced they may even see a very quick response time (but profile performance of XmlDocument XPath query for your typical file).

EDIT: here you can find a RAM disk provided by Microsoft. It's pretty simple and naive but usually you/we don't need much more than that. Moreover it's an example on the DDK so you'll get source code too (in this case...just for fun).

Upvotes: 3

Branko Dimitrijevic
Branko Dimitrijevic

Reputation: 52107

Don't poll the file. Read it once and keep it in-memory, then use FileSystemWatcher to reload it only when it changes.

Or alternatively, read the modification timestamp and only reload the file if the timestamp changed.

Also, when reading the file, make sure you lock it non-exclusively so other readers are not blocked.

Upvotes: 2

Guffa
Guffa

Reputation: 700322

Instead of reading the XML file very time, just read it the first time and also get the last modified time for the file.

Whenver you need to know if the data is up to date, just check the modified time of the file, and only read the file again if it really has changed.

Upvotes: 2

Darin Dimitrov
Darin Dimitrov

Reputation: 1038780

XML is not designed for heavy querying. If you need to do this, consider using a database. SQL Server Compact could be a good alternative. If you need to stick with XML though and need to work with large files and need performance consider using XmlReader/XmlWriter which are not loading the entire file into memory and are pretty fast.

Upvotes: 2

Jirka Hanika
Jirka Hanika

Reputation: 13529

Try opening the file exclusively. The other application may crash, but if it does not crash, you will know one thing for sure: it cannot invoke much I/O load in one cycle on the shared file, because all its access attempts will fail immediately.

Hopefully it will just wait a second and retry, and that should work well for you.

using (Stream iStream = File.Open("myfile.xml",
            FileMode.Open, FileAccess.ReadWrite, FileShare.None))
{
    ...
}

Upvotes: 1

Related Questions