AutoIt v3 _IETagNameGetCollection(...) faillure

Question

I'm trying to create a script in AutoIt that will be fetching data on internet. The script takes an Amazon URL as input, and will then iterate through all pages, retrieve the ASIN numbers of the presented items and put them in a file.

Here is the full code:

#include <GUIConstantsEx.au3>
#include <MsgBoxConstants.au3>
#include <IE.au3>

Global $GUI = GUICreate("Amazon Parser", 350, 300, 200, 200)
Global $BoutonRun = GUICtrlCreateButton("Run", 30, 100, 125, 30)
Global $BoutonStop = GUICtrlCreateButton("Stop", 190, 100, 125, 30)
Global $TextEnter = GUICtrlCreateLabel("Enter the starting URL here : ", 90, 30, 300, 50)
Global $InputField = GUICtrlCreateInput("", 25, 50, 300, 35)
Global $Runing = false
Global $NbAsin = 0
Global $NbPages = 0
Global $OIE
Local  $Url = ""

GUISetState(@SW_SHOW)

While 1
   $nMsg = GUIGetMsg()
   Switch $nMsg
      Case $GUI_EVENT_CLOSE
         _IEQuit($OIE)
         Exit
      Case $BoutonRun
         $Url = GUICtrlRead($InputField)
         $Runing = true
         $NbAsin = 0
         $NbPages = 0
         $handle = FileOpen("./Res", $FO_OVERWRITE)
         FileClose($handle)
         $OIE = _IECreate($Url)
         _IELoadWait($OIE)
      Case $BoutonStop
         MsgBox(0, "Work Stopped !", "I collected " & $NbAsin & " ASIN" & @LF & "I went throught " & $NbPages & " pages of results.")
         _IEQuit($OIE)
         $Runing = false
   EndSwitch
   If $Runing == true Then
      $Url = GetPageData()
   EndIf
WEnd

Func GetPageData()
   $li = _IETagNameGetCollection($OIE, "li")
   $handle = FileOpen("./Res", $FO_APPEND)

   $NbPages = $NbPages + 1
   For $l In $li
      If StringInStr($l.className, "s-result-item") > 0 And StringInStr($l.className, "celwidget") > 0 Then
         FileWriteLine($handle, $l.GetAttribute("data-asin"))
         $NbAsin = $NbAsin + 1
      EndIf
   Next

   $ret = _IELinkClickByText($OIE, "Next Page", 0, 0)
   If $ret == 0 Then
      $Runing = false
      _IEQuit($OIE)
      MsgBox(0, "Work Done !", "I successfully collected " & $NbAsin & " ASIN" & @LF & "I went throught " & $NbPages & " pages of results." & @LF & "The result can be found in the 'res' file")
   EndIf
   FileClose($handle)
EndFunc

As you can see, nothing much complicated there. This script works (almost) perfectly, it iterates through pages and gets the ASINs.

But then at a random page (sometimes the 9th, sometimes the 18th, it's inconsistent...), the script fails for no apparent reason with the following error:

(1811) : ==> The requested action with this object has failed.:
Return SetError($_IESTATUS_Success, $oTemp.GetElementsByTagName($sTagName).length, $oTemp.GetElementsByTagName($sTagName))
Return SetError($_IESTATUS_Success, $oTemp^ ERROR

This error occurs in the IE.au3 library, so it strictly terminates the program without warning.

Any idea how I could overcome this? Or if you have another function that can do the job, I'm open to it!

Answer 1

You are missing an _IELoadWait after _IELinkClickByText.

_IETagNameGetCollection is failing because the page isn't ready(loaded).

_IELinkClickByText waits by default but for some reason you turned that off by setting the third parameter to 0.

You should always add an Error handler in order avoid fatal IE Errors.

Here is your code with some changes:

#include <GUIConstantsEx.au3>
#include <MsgBoxConstants.au3>
#include <IE.au3>

Global $GUI = GUICreate("Amazon Parser", 350, 300, 200, 200)
Global $BoutonRun = GUICtrlCreateButton("Run", 30, 100, 125, 30)
Global $BoutonStop = GUICtrlCreateButton("Stop", 190, 100, 125, 30)
Global $TextEnter = GUICtrlCreateLabel("Enter the starting URL here : ", 90, 30, 300, 50)
Global $InputField = GUICtrlCreateInput("", 25, 50, 300, 35)
Global $Runing = false
Global $NbAsin = 0
Global $NbPages = 0
Global $OIE
Local  $Url = ""


_IEErrorNotify(True)
$oErrorHandler = ObjEvent("AutoIt.Error", "_ErrFunc")

GUISetState(@SW_SHOW);  http://www.amazon.com/s//www.amazon.com/s?marketplaceID=ATVPDKIKX0DER&me=A3QENM74IQHUW5&merchant=A3QENM74IQHUW5

While 1
   $nMsg = GUIGetMsg()
   Switch $nMsg
      Case $GUI_EVENT_CLOSE
         _IEQuit($OIE)
         Exit
      Case $BoutonRun
         $Url = GUICtrlRead($InputField)
         $Runing = true
         $NbAsin = 0
         $NbPages = 0
         $handle = FileOpen("./Res", $FO_OVERWRITE)
         FileClose($handle)
         $OIE = _IECreate($Url)
;~          _IELoadWait($OIE); not needed because _IECreate waits on default
      Case $BoutonStop
         MsgBox(0, "Work Stopped !", "I collected " & $NbAsin & " ASIN" & @LF & "I went throught " & $NbPages & " pages of results.")
         _IEQuit($OIE)
         $Runing = false
   EndSwitch
   If $Runing == true Then
      GetPageData()
   EndIf
WEnd

Func GetPageData()
   $handle = FileOpen("./Res", $FO_APPEND)

   $li = _IETagNameGetCollection($OIE, "li")
   If Not @error Then


       $NbPages = $NbPages + 1
       For $l In $li
          If StringInStr($l.className, "s-result-item") And StringInStr($l.className, "celwidget") Then
             FileWriteLine($handle, $l.GetAttribute("data-asin"))
             $NbAsin = $NbAsin + 1
          EndIf
       Next

    EndIf

   $ret = _IELinkClickByText($OIE, "Next Page", 0, 1) ; changed last parameter to 1 in order to wait for load.
   If @error Then
      $Runing = false
      _IEQuit($OIE)
      MsgBox(0, "Work Done !", "I successfully collected " & $NbAsin & " ASIN" & @LF & "I went throught " & $NbPages & " pages of results." & @LF & "The result can be found in the 'res' file")
   EndIf
   FileClose($handle)

EndFunc



Func _ErrFunc($oError)

    ConsoleWrite(@ScriptName & " (" & $oError.scriptline & ") : ==> COM Error intercepted !" & @CRLF & _
            @TAB & "err.number is: " & @TAB & @TAB & "0x" & Hex($oError.number) & @CRLF & _
            @TAB & "err.windescription:" & @TAB & $oError.windescription & @CRLF & _
            @TAB & "err.description is: " & @TAB & $oError.description & @CRLF & _
            @TAB & "err.source is: " & @TAB & @TAB & $oError.source & @CRLF & _
            @TAB & "err.helpfile is: " & @TAB & $oError.helpfile & @CRLF & _
            @TAB & "err.helpcontext is: " & @TAB & $oError.helpcontext & @CRLF & _
            @TAB & "err.lastdllerror is: " & @TAB & $oError.lastdllerror & @CRLF & _
            @TAB & "err.scriptline is: " & @TAB & $oError.scriptline & @CRLF & _
            @TAB & "err.retcode is: " & @TAB & "0x" & Hex($oError.retcode) & @CRLF & @CRLF)

EndFunc   ;==>_ErrFunc