CODEWITHSUNDEEP
htmlexcelvbaweb-scrapingscreen-scraping
Sharid
Sharid

Reputation: 161

Web page Navigation Click to NEXT Page

I normally use this, to click on the next page on a website and it works fine. However I am struggling to work it out this time as it is a ul Class and then a li class. I have tried several variations, from .getAttribute to .getElementsByTagName, however I can not work it out'

I have tried adding child nods so it selects the right li but nothing seems to work and I am stuck.

Could someone please take a look, always thanks in advance

This is the HTML HTML Image

Code for clicking on NextPage

Searches Number of Pages entered in Sheet20 
    If pageNumber >= Replace(Worksheets("Sheet20").Range("J9").Value, "", "+") Then Exit Do
        On Error Resume Next
        Set nextPageElement = Html.getElementsByClassName("a-pagination")(0) '''' THIS LINE ''''''
    If nextPageElement Is Nothing Then Exit Do
        nextPageElement.Click 'next web page 
    Do While objIE.Busy = True Or objIE.readyState <> 4
    Loop
        Set Html = objIE.document
        pageNumber = pageNumber + 1
'DoEvents

Upvotes: 1

Views: 1686

Answers (2)

QHarr
QHarr

Reputation: 84475

There is an alternative way with less overhead. Simply determine the number of pages from the first page. Then, determine if that is less than the number of requested pages, and loop to whichever is the lower number. During the loop simply concatenate the current page number into the url as the page param; the ref param not needed. The url is a querystring.

This means less setting of object variables and a shorter more robust and quicker css path.

I think this matches with your requirement to loop a specified number of pages.

Dim requestedNumber As Long, numPages As Long, pageNumber As Long

numPages = ie.document.querySelector(".a-disabled[aria-disabled]").innerText
requestedNumber = Replace$(ThisWorkbook.Worksheets("Sheet20").Range("J9").Value, "", "+") 
requestedNumber = IIf(numPages < requestedNumber, numPages, requestedNumber) ' so cannot loop more than available page count

' you are already on page1 so next page will be 2
For pageNumber = 2 To requestedNumber
    ie.Navigate2 "https://www.amazon.com/s?k=phones&qid=1612497504&page=" & pageNumber 'ref param not needed for page navigation
    While ie.Busy Or ie.ReadyState <> 4: DoEvents:Wend
    'do something with new page
Next

Upvotes: 1

SIM
SIM

Reputation: 22440

Either of the following solutions should work:

Sub InitiateClick()
    Dim IE As New InternetExplorer
    Dim nextPage As Object
   
    With IE
        .Visible = True
        .navigate "https://www.amazon.com/s?k=phones&page=2&qid=1612460431&ref=sr_pg_1"
        Do While .Busy = True Or .readyState <> 4: DoEvents: Loop
        'if it still fails to click on that buttton, make sure to put some delay in this line
'        Set nextPage = .document.getElementsByClassName("a-pagination")(0).getElementsByClassName("a-last")(0).getElementsByTagName("a")(0)
        Set nextPage = .document.querySelector(".a-pagination > li.a-last > a")
        If Not nextPage Is Nothing Then
            nextPage.Click
        End If
    End With
End Sub

Upvotes: 2

Related Questions