Best Practices for AS3 XML Parsing

Question

I've been having some trouble parsing various types of XML within flash (specifically FeedBurner RSS files and YouTube Data API responses). I'm using a URLLoader to load a XML file, and upon Event.COMPLETE creating a new XML object. 75% of the time this work fine, and every now and again I get this type of exception:

TypeError: Error #1085: The element type "link" must be terminated by the matching end-tag "</link>".

We think the problem is that The XML is large, and perhaps the Event.COMPLETE event is fired before the XML is actually downloaded from the URLLoader. The only solution we have come up with is to set off a timer upon the Event, and essentially "wait a few seconds" before beginning to parse the data. Surely this can't be the best way to do this.

Is there any surefire way to parse XML within Flash?

Update Sept 2 2008 We have concluded the following, the excption is fired in the code at this point:

data = new XML(mainXMLLoader.data);

//  calculate the total number of entries.
for each (var i in data.channel.item){
    _totalEntries++;
}

I have placed a try/catch statement around this part, and am currently displaying an error message on screen when it occurs. My question is how would an incomplete file get to this point if the bytesLoaded == bytesTotal?

---

I have updated the original question with a status report; I guess another question could be is there a way to determine wether or not an XML object is properly parsed before accessing the data (in case the error is that my loop counting the number of objects is starting before the XML is actually parsed into the object)?

---

@Theo: Thanks for the ignoreWhitespace tip. Also, we have determined that the event is called before its ready (We did some tests tracing mainXMLLoader.bytesLoaded + "/" + mainXMLLoader.bytesLoaded

Answer 1

You could set a unique element namespace at the very end of your XML document that has one attribute "value" equal to "true";

//The XML
//Flash ignores the line that specifies the XML version and encoding so I have here as well.

<parent>
    <child name="child1" />
    <child name="child2" />
    <child name="child3" />
    <child name="child4" />
    <documentEnd value="true" />
</parent>

//Sorry about the spacing, but it is difficult to get XML to show.

//Flash
var loader:URLLoader = new URLLoader();
var request:URLRequest = new URLRequest('pathToXML/xmlFileName.xml');

var xml:XML;

//Event Listener with weak reference set to true (5th parameter);
//The above comment does not define a required practice, this is to aid with garbage collection.

loader.addEventListener(Event.COMPLETE, onXMLLoadComplete, false, 0, true);
loader.load(request);
function onXMLLoadComplete(e:Event):void
{
   xml = new XML(e.target.data);

   //Now we check the last element (child) to see if it is documentEnd.
   if(xml[xml.length()-1].documentEnd.@value == "true")
   {
      trace("Woot, it seems your xml made it!");
   }
   else
   {
      //Attempt the load again because it seems it failed when it was unable to find documentEnd in the XML Object.
      loader.load(request);
   }
}

I hope that this helps you for now, but the real hope is that enough people let adobe know about this issue. It is a sad thing to not be able to rely on events. I must say though, from what I have heard about XML, it is not very optimal at a large scale and believe this is when you require something like AMFPHP to serialize the data.

Hope this helps! Remember the idea here is that we know what the very last child/element in the XML is because we set it! There is no reason that we shouldn't be able to access the last child/element, but if we cannot, we must assume that the XML was not indeed complete and we force it to load again.

Answer 2

sometimes the RSS server page can fail to spit out correct and valid XML data especially if your constantly hitting it, so it may not be your fault. Have you tried hitting the page in a web browser (preferably with an xml validator plugin) to check that the server response is always valid?

The only other thing that I can see here is the line:

xml = new XML(event.target.data);

//the data should already be XML, so only casting is necessary
xml = XML(event.target.data);

Have you also tried setting the urlloader dataFormat to URLLoaderDataFormat.TEXT, and also adding url headers of prama-no-cache and/or adding a cache buster tot he url?

Just some suggestions...

Answer 3

I suggest that you file a bug report at https://bugs.adobe.com/flashplayer/, because the event really shouldn't fire before all the bytes are loaded. In the meantime I guess you have to live with the timer. You might be able to do the same by listening at the progress event instead, that could perhaps save you from having to handle the timer yourself.

Answer 4

The Event.COMPLETE handler really shouldn't be called unless the loader was fully loaded, it makes no sense. Have you confirmed that it is in fact not fully loaded (by looking at the bytesLoaded vs. bytesTotal values that you trace)? If the Event.COMPLETE event is dispatched before bytesLoaded == bytesTotal that is a bug.

Good that you've got it working with the timer, but it is very odd that you need it.

Answer 5

Just a side note, this statement has no effect:

XML.ignoreWhitespace;

because ignoreWhitespace is a property. You have to set it to true like this:

XML.ingoreWhitespace = true;

Answer 6

@Brian Warshaw: This issue happens only about 10-20% of the time. Sometimes it hiccups and simply reloading the app will work fine, other times I will spend half an hour reloading the app over and over again to no avail.

This is the original code (when I asked the question):

public class BlogReader extends MovieClip {
    public static const DOWNLOAD_ERROR:String = "Download_Error";
    public static const FEED_PARSED:String = "Feed_Parsed";

    private var mainXMLLoader:URLLoader = new URLLoader();
    public var data:XML;
    private var _totalEntries:Number = 0;

    public function BlogReader(url:String){
        mainXMLLoader.addEventListener(Event.COMPLETE, LoadList);
        mainXMLLoader.addEventListener(IOErrorEvent.IO_ERROR, errorCatch);
        mainXMLLoader.load(new URLRequest(url));
        XML.ignoreWhitespace;
    }
    private function errorCatch(e:IOErrorEvent){
        trace("Oh noes! Yous gots no internets!");
        dispatchEvent(new Event(DOWNLOAD_ERROR));
    }
    private function LoadList(e:Event):void {
        data = new XML(e.target.data);

        //  calculate the total number of entries.
        for each (var i in data.channel.item){
            _totalEntries++;
        }

        dispatchEvent(new Event(FEED_PARSED));
    }
}

And this is the code that I wrote based on Re0sless' original reply (similar to some suggestions mentioned):

public class BlogReader extends MovieClip {
    public static const DOWNLOAD_ERROR:String = "Download_Error";
    public static const FEED_PARSED:String = "Feed_Parsed";

    private var mainXMLLoader:URLLoader = new URLLoader();
    public var data:XML;
    protected var _totalEntries:Number = 0;

    public function BlogReader(url:String){
        mainXMLLoader.addEventListener(Event.COMPLETE, LoadList);
        mainXMLLoader.addEventListener(IOErrorEvent.IO_ERROR, errorCatch);
        mainXMLLoader.load(new URLRequest(url));
        XML.ignoreWhitespace;
    }
    private function errorCatch(e:IOErrorEvent){
        trace("Oh noes! Yous gots no internets!");
        dispatchEvent(e);
    }
    private function LoadList(e:Event):void {
        isDownloadComplete();           
    }
    private function isDownloadComplete() {
        trace (mainXMLLoader.bytesLoaded + "/" + mainXMLLoader.bytesLoaded);
        if (mainXMLLoader.bytesLoaded == mainXMLLoader.bytesLoaded){
            trace ("xml fully loaded");

            data = new XML(mainXMLLoader.data);

            //  calculate the total number of entries.
            for each (var i in data.channel.item){
                _totalEntries++;
            }

            dispatchEvent(new Event(FEED_PARSED));
        } else {
            trace ("xml not fully loaded, starting timer");
            var t:Timer = new Timer(300, 1);
            t.addEventListener(TimerEvent.TIMER_COMPLETE, loaded);
            t.start();
        }
    }
    private function loaded(e:TimerEvent){
        trace ("timer finished, trying again");
        e.target.removeEventListener(TimerEvent.TIMER_COMPLETE, loaded);
        e.target.stop();

        isDownloadComplete();
    }
}

I'll point out that since adding the code determining if mainXMLLoader.bytesLoaded == mainXMLLoader.bytesLoaded I have not had an issue - that said, this bug is hard to reproduce so for all I know I haven't fixed anything, and instead just added useless code.

Answer 7

If you could post some more code we might be able to find the issue.

Another thing to test (besides tracing bytesTotal) is to trace the data property of the loader in the Event.COMPLETE handler, just to see if the XML data was actually loaded correctly, for example check that there is a </link> there.

Answer 8

The concerning thing to me is that it might be firing Event.COMPLETE before it's finished loading, and that makes me wonder whether or not the load is timing out.

How often does the problem happen? Can you have success one moment, then failure the very next with the same feed?

For testing purposes, try tracing the URLLoader.bytesLoaded and the URLLoader.bytesTotal at the top of your Event.COMPLETE handler method. If they don't match, you know that the event is firing prematurely. If this is the case, you can listen for the URLLoader's progress event. Check the bytesLoaded against the bytesTotal in your handler and only parse the XML once the loading is truly complete. Granted, this is very likely akin to what the URLLoader is doing before it fires Event.COMPLETE, but if that's broken, you can try rolling your own.

Please let us know what you find out. And if you could, please paste in some source code. We might be able to spot something of note.

Answer 9

As you mentioned in your question, the problem is very likely that your program is looking at the XML before it has actually been completely downloaded, I don't know that there's a surefire way to "parse" the XML because the parsing portion of your code is more than likely fine, it's simply a matter of whether or not it has actually downloaded.

You could try to use the ProgressEvent.PROGRESS event to continually monitor the XML as it downloads and then as Re0sless suggested, check the bytesLoaded vs the bytesTotal and have your XML parse begin when the two numbers are equal instead of using the Event.COMPLETE event.

You should be able to get the bytesLoaded and bytesTotal numbers just fine regardless of domains, if you can access the file you can access its byte information.

Answer 10

Have you tried checking that the bytes loaded are the same as the total bytes?

URLLoader.bytesLoaded == URLLoader.bytesTotal

That should tell you if the file has finished loading, it wont help with the compleate event firing to early, but it should tell you if its a problem with the xml been read.

I am unsure if it will work over domains, as my xml is always on the same site.