CODEWITHSUNDEEP
vbaexceldom
Vegard
Vegard

Reputation: 4927

VBA/DOM - Get elements based on attribute

Excel 2013 on Windows 7. XPath/Javascript/jQuery is out of scope.

I am trying to iterate over select div elements in a page, namely elements that have a specific data-level attribute.

My current approach is similar to this, but I was unable to find a non-manual way to select elements based on attributes. The closest I came was something like:

With CreateObject("WINHTTP.WinHTTPRequest.5.1")
    .Open "GET", url, False
    .Send
    pHTML.body.innerHTML = .ResponseText
End With

Set eCollection = pHTML.getElementsByClassName("chapter").getElementsByTagName("div")

For i = 0 To eCollection.Length
    If eCollection(i).getAttribute("data-level") >= 0 Then ' Throw cake
Next i

This solution, while I am sure it is viable (if unelegant), seems sub-optimal if only for how big the loop is going to end up being when I start looking for specific elements and sequences of elements within these elements.

So I am looking for a way to do something like this:

For Each pElement In pHTML.getElementsByClassName("chapter").getElementsByTagName("div").getElementsByAttribute("data-level")
    ' Throw cake at the element
Next

I'm aware that there is no method getElementsByAttribute, hence the question.
Is there some approach here that I am blind to, or am I locked to manual iteration?

Alternatively, if I swap my current approach for creating an IE instance, á la this answer, could I concievably use querySelectorAll to end up with something resembling the result I have outlined above?

Upvotes: 3

Views: 14759

Answers (1)

Vegard
Vegard

Reputation: 4927

For anyone else coming this way, the outer shell, so to speak, can look like this:

Sub ScrapeWithHTMLObj(url As String, domClassName As String, domTag As String, domAttribute As String, domAttributeValue As String)
    ' Dependencies:
    ' * Microsoft HTML Object Library

    ' Declare vars
    Dim pHTML As HTMLDocument
    Dim pElements As Object, pElement As Object

    Set pHTML = New HTMLDocument

    ' Basic URL healthcheck
    Do While (url = "" Or (Left(url, 7) <> "http://" And Left(url, 8) <> "https://"))
        MsgBox ("Invalid URL!")
        url = InputBox("Enter new URL: (0 to terminate)")
        If url = "0" Then Exit Sub
    Loop

    ' Fetch page at URL
    With CreateObject("WINHTTP.WinHTTPRequest.5.1")
        .Open "GET", url, False
        .Send
        pHTML.body.innerHTML = .ResponseText
    End With

    ' Declare page elements
    Set pElements = pHTML.getElementsByClassName(domClassName)
    Set pElement = pElements(0).getElementsByTagName(domTag)

    ' Extract only elements with wanted attribute
    pEleArray = getElementsByAttribute(pElement, domAttribute, domAttributeValue)

    For Each e In pEleArray
        ' Do stuff to elements
        Debug.Print e.getAttribute(domAttribute)
    Next
End Sub

If you go this route, you'll also need something like this:

Function getElementsByAttribute(pObj As Object, domAttribute As String, domAttributeValue As String) As Object()
    Dim oTemp() As Object
    ReDim oTemp(1 To 1)

    For i = 0 To pObj.Length - 1
        'Debug.Print pObj(i).getAttribute(domAttribute)
        If pObj(i).getAttribute(domAttribute) = domAttributeValue Then
            Set oTemp(UBound(oTemp)) = pObj(i)
            ReDim Preserve oTemp(1 To UBound(oTemp) + 1)
        End If
    Next i

    ReDim Preserve oTemp(1 To UBound(oTemp) - 1)

    getElementsByAttribute = oTemp
End Function

Depending on the HTML tree, you'll need to change which elements you zero in on in the sub, obviously. For the site I used in testing, this structure worked flawlessly.

Example usage:
Call ScrapeWithHTMLObj("https://somesite", "chapter-index", "div", "data-level", "1")

It will enter the first class named chapter-index, select all elements with the div tag, and finally extract all elements containing the attribute data-level with value 1.

Upvotes: 5

Related Questions