CODEWITHSUNDEEP
c#xmlserialization
Tina183
Tina183

Reputation: 21

XmlSerializer : Generated File too big

I have Problems with the XML Serializer in C#. My generated Files (out of a linked List Point Cloud) is getting Sizes of 30-40 MB. (60.000 Vertices).

Most of the time my Visual Studio crashes (vshost32.exe didnt work anymore) Sometimes im lucky and getting this graphic issue instead:

enter image description here

If i redraw my Model (im using an opengl control) its suddenly displayed how it should look. enter image description here

Im guessing that my memory causes this. Is there anyway a possibility to decrease the size of the xml files? Hes using a whole line for every Tag. (460.000 Lines! )

  <DistanceNeighbours />
  <Vector>
    <X>-8.52</X>
    <Y>51.05</Y>
    <Z>62.56</Z>
  </Vector>

Here are my Serializable + Deserializable Functions, hope you can help me.

     /// <summary>
    /// Serializes an object.
    /// </summary>
    /// <typeparam name="T"></typeparam>
    /// <param name="serializableObject"></param>
    /// <param name="fileName"></param>
    public void SerializeObject<T>(T serializableObject, string fileName)
    {
        if (serializableObject == null) { return; }

        try
        {
            XmlDocument xmlDocument = new XmlDocument();
            XmlSerializer serializer = new XmlSerializer(serializableObject.GetType());
            using (MemoryStream stream = new MemoryStream())
            {
                serializer.Serialize(stream, serializableObject);
                stream.Position = 0;
                xmlDocument.Load(stream);
                xmlDocument.Save(fileName);
                stream.Close();
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine("Serialize Object Error");
        }
    }

    /// <summary>
    /// Deserializes an xml file into an object list
    /// </summary>
    /// <typeparam name="T"></typeparam>
    /// <param name="fileName"></param>
    /// <returns></returns>
    public T DeSerializeObject<T>(string fileName)
    {
        if (string.IsNullOrEmpty(fileName)) { return default(T); }

        T objectOut = default(T);

        try
        {
            string attributeXml = string.Empty;

            XmlDocument xmlDocument = new XmlDocument();
            xmlDocument.Load(fileName);
            string xmlString = xmlDocument.OuterXml;

            using (StringReader read = new StringReader(xmlString))
            {
                Type outType = typeof(T);

                XmlSerializer serializer = new XmlSerializer(outType);
                using (XmlReader reader = new XmlTextReader(read))
                {
                    objectOut = (T)serializer.Deserialize(reader);
                    reader.Close();
                }

                read.Close();
            }
        }
        catch (Exception ex)
        {
            //Log exception here
        }

        return objectOut;
    }

Thank you,

Upvotes: 0

Views: 1952

Answers (3)

Charles Mager
Charles Mager

Reputation: 26213

If you want to alleviate your issues in dealing with files this large, then stop loading the whole serialized XML into memory and then into the XML DOM when serializing and deserializing - I can't see that this is used for anything. XmlDocument will probably take at least 10x the amount of memory as the contents (which are also being held in memory), so you're likely using 100s of megabytes while trying to do it this way.

The following code should give the same results (though won't affect your file sizes), but you won't have the memory pressures that are causing your lockups and performance issues:

public void SerializeObject<T>(T serializableObject, string fileName)
{
    if (serializableObject == null) return;

    var serializer = new XmlSerializer(serializableObject.GetType());

    using (var stream = File.Open(fileName, FileMode.Create))
    {
        serializer.Serialize(stream, serializableObject);
    }
}

public T DeserializeObject<T>(string fileName)
{
    if (string.IsNullOrEmpty(fileName)) return default(T);

    var serializer = new XmlSerializer(typeof(T));

    using (var stream = File.Open(fileName, FileMode.Open))
    {            
        return (T) serializer.Deserialize(stream);
    }
}

Options for reducing file size? Compress the output (XML is very compressible) or change your serialization format.

Upvotes: 0

Emanuele Greco
Emanuele Greco

Reputation: 12721

What about binary serialization?

 BinarySerializer serializer = new BinarySerializer (outType);

BinarySerializer is intended to reduce problems of file size and computation needs.

You can store result in a file, in a DB, send trhough the net.

You will need a Binary Deserializer to have back your objects.

Upvotes: 1

PavPS
PavPS

Reputation: 76

When working with such large files (and even smaller) and if you do not need to deal with XmlDocument object by adding nodes, validating schema, applying XPath, consider please using simple XmlReader instead.

Just instantiate XmlReader class and call XmlReader.Read() method. You will have to define token type after every read by checking XmlReader.NodeType property. Once you find a specific element with attributes - you can create your Vector object (even manually).

In this case you will get fast, memory efficient solution. A way faster than loading all into a XmlDocument.

Look here for example.

Upvotes: 0

Related Questions