CODEWITHSUNDEEP
c#xml
The Light
The Light

Reputation: 27011

How to work with an Xml file without loading the whole document in memory?

How to add a new node, update an existing node and remove an existing node of an xml document without loading the whole document in memory?

I'm having an xml document and treating it as the memory of my application so would need to be able to do hundreds of reads and writes quickly without loading the whole document.

its structure is like this:

<spiderMemory>
  <profileSite profileId="" siteId="">
    <links>
      <link>
        <originalUrl></originalUrl>
        <isCrawled></isCrawled>
        <isBroken></isBroken>
        <isHtmlPage></isHtmlPage>
        <firstAppearedLevel></firstAppearedLevel>
      </link>
    </links>
  </profileSite>
</spiderMemory>

How would that be possible with XDocument?

Thanks

Upvotes: 1

Views: 2724

Answers (3)

gnsb
gnsb

Reputation: 326

As Daren Thomas said, the proper solution is to use RDBMS instead of XML for your needs. I have a partial solution using XML and Java. Stax parser does not parse the whole document in memory and is a lot faster than DOM (still XML parsing will always be slow). A 'pull parser' (eg Stax) allows u to control what gets parsed. A less cleaner way is to throw an exception in SAX parser when you get the element(s) needed.

To modify, the simplest (but slow) way is to use XPath. Another (untested) option is to treat XML file as text and then 'Search and replace' stuff. Here you can use all kinds of text search optimization.

Upvotes: 0

Daren Thomas
Daren Thomas

Reputation: 70314

If you want to do hundreds of reads and writes quickly... you might be using the wrong technology. Have you tried using a plain old RDBMS?

If you still need the XML representation, then you can create an export methods to produce it from the database.

XML isn't really a good substitute for this kind of problem. Just saying.

Also... what is wrong with having the whole thing in memory? How big can it possibly get? Say 1GB? Suck it up. Say 1TB? Oops. But then XML is wrong, wrong, wrong anyway in that case ;) way too verbose!

Upvotes: 5

Antonio Bakula
Antonio Bakula

Reputation: 20693

You can use XmlReader, something like this :

FileStream stream = new FileStream("test.xml", FileMode.Open);
XmlReader reader = new XmlTextReader(stream);
while(reader.Read())
{
  Console.WriteLine(reader.Value);
}

here is an more elaborate example http://msdn.microsoft.com/en-us/library/cc189056%28v=vs.95%29.aspx

Upvotes: 0

Related Questions