CODEWITHSUNDEEP
excelvbaexcel-2010spreadsheet
Carol.Kar
Carol.Kar

Reputation: 5355

Error "Object variable or with block variable not set" when using getElementsByClassName

I am want to scrap from amazon some fields.

Atm I am using a link and my vba script returns me name and price.

For example:

I put the link into column A and get the other fields in the respective columns, f.ex.: http://www.amazon.com/GMC-Denali-Black-22-5-Inch-Medium/dp/B00FNVBS5C/ref=sr_1_1?s=outdoor-recreation&ie=UTF8&qid=1436768082&sr=1-1&keywords=bicycle

However, I would also like to have the product description.

Here is my current code:

Sub ScrapeAmz()

 Dim Ie As New InternetExplorer
 Dim WebURL
 Dim Docx As HTMLDocument
 Dim productDesc
 Dim productTitle
 Dim price
 Dim RcdNum

Ie.Visible = False

For RcdNum = 2 To ThisWorkbook.Worksheets(1).Range("A65536").End(xlUp).Row

    WebURL = ThisWorkbook.Worksheets(1).Range("A" & RcdNum)
     Ie.Navigate2 WebURL
     Do Until Ie.ReadyState = READYSTATE_COMPLETE
     DoEvents
     Loop
     Set Docx = Ie.Document
     productTitle = Docx.getElementById("productTitle").innerText
     'productDesc = Docx.getElementsByClassName("productDescriptionWrapper")(0).innerText
     price = Docx.getElementById("priceblock_ourprice").innerText
     ThisWorkbook.Worksheets(1).Range("B" & RcdNum) = productTitle
     'ThisWorkbook.Worksheets(1).Range("C" & RcdNum) = productDesc
     ThisWorkbook.Worksheets(1).Range("D" & RcdNum) = price
   Next

End Sub

I am trying to get the product description by using productDesc = Docx.getElementsByClassName("productDescriptionWrapper")(0).innerText.

However, I get an error. 

Object variable or with block variable not set.

Any suggestion why my statement does not work?

I appreciate your replies!

Upvotes: 1

Views: 2213

Answers (1)

Tmdean
Tmdean

Reputation: 9299

I'm pretty sure your problem is being caused by attempting to access the document before it's completely loaded. You're just checking ie.ReadyState.

This is my understanding of the timeline for loading a page with an IE control.

  1. Browser connects to page: ie.ReadyState = READYSTATE_COMPLETE. At this point, you can access ie.document without causing an error, but the document has only started loading.
  2. Document fully loaded: ie.document.readyState = "complete" (note that frames may still be loading and AJAX processing may still be occurring.)

So you really need to check for two events.

Do
    If ie.ReadyState = READYSTATE_COMPLETE Then
        If ie.document.readyState = "complete" Then Exit Do
    End If
    Application.Wait DateAdd("s", 1, Now)
Loop

edit: after actually looking at the page you're trying to scrape, it looks like the reason it's failing is because the content you're trying to get at is in an iframe. You need to go through the iframe before you can get to the content.

ie.document.window.frames("product-description-iframe").contentWindow.document.getElementsByClassName("productDescriptionWrapper").innerText

Upvotes: 1

Related Questions