CODEWITHSUNDEEP
xmlpowershell
edelwater
edelwater

Reputation: 2802

How can i use XmlReader in PowerShell to stream big/huge XML files?

I have an XML of couple of gigabytes. There are no spaces in the XML.

So I wrote a little C# code to split in single files (which has some additional code to perform some stuff e.g. randomizing while testing)

using (XmlReader MyReader = XmlReader.Create(@"d:\xml\test.xml"))
            {
                while (MyReader.Read())
                {
                    switch (MyReader.NodeType)
                    {
                        case XmlNodeType.Element:
                            if (MyReader.Name == "Customer")
                            {
                                XElement el = XElement.ReadFrom(MyReader) as XElement;
                                if (el != null)
                                {
                                    custNumber = (string)el.Element("CustNumber");
                                    output = @"d:\xml\output\" + custNumber;

                                    File.WriteAllText(output, el.ToString());
                                }                                    
                            }
                            break;
                    }
                }
            }

I then parse the resulting files with PowerShell, basically because I find it easier to work with on the server while specs can change and I can on the fly change the script.

So... what is the easiest way to convert the above to PowerShell also, putting [.Net here] before everything ? would I have to read byte for byte just in the case it has "<cust" on one line and "omer>" on the next?

Upvotes: 7

Views: 11360

Answers (1)

Micky Balladelli
Micky Balladelli

Reputation: 9991

This should be pretty close to what you wanted to do in Powershell:

$f = [System.Xml.XmlReader]::create("d:\xml\test.xml")

while ($f.read())
{
    switch ($f.NodeType)
    {
        ([System.Xml.XmlNodeType]::Element) # Make sure to put this between brackets
        {
            if ($f.Name -eq "Customer")
            {
                $e = [System.Xml.Linq.XElement]::ReadFrom($f)

                if ($e -ne $null)
                {
                    $custNumber = [string] $e.Element("CustNumber")

                    $e.ToString() | Out-File -Append -FilePath ("d:\xml\output\"+$e.ToString())
                }
            }
            break
        }
    }
}

Upvotes: 12

Related Questions