Problems Using VBA to Submit a Web Page - Using the click button function but web page won't submit

Question

I am writing a VBA code to pull data from a website (https://app.buzzsumo.com/top-content). I have a functional code that runs without errors however I still can't get the webpage to actually submit the form when the click command runs. I have tried many different approaches and combinations of submitting the form/clicking the submit button but none have seemed to work so far. Below is my current code.

 Sub clickFormButton()
 Dim ie As Object
 Dim form As Variant, 
 Dim button As Variant

'add the “Microsoft Internet Controls” reference in VBA Project
 Set ie = CreateObject("InternetExplorer.Application")

'using input box to enter URL I am serching for
Search_URL = InputBox("Enter URL to Search For")

With ie
.Visible = True
.navigate ("https://app.buzzsumo.com/#/top-content")

'Ensure that the web page downloads completely 
 While ie.ReadyState <> 4
 DoEvents
 Wend

'assigning the input variables to the html elements of the form
 ie.document.getElementsByName("q").Item.innertext = Search_URL

'finding and clicking the button
Set objInputs = ie.document.getElementsByTagName("input")
For Each ele In objInputs
   If ele.Title Like "Press Enter to Search" Then
        ele.Click
    End If

End With
End Sub

I have also tried other methods to find and click the button such as:

'Dim i As Variant
'Set form = ie.document.getElementsByClassName("btn btn-highlight")

'For i = 1 To 5
'If form.Item(i).DefaultValue = "Search!" Then
     'Set button = form.Item(i)
     'button.Click
'End If
'Next i

Please provide any recomendations on what I may be missing or how I can get this code to actually submit the form and advance to the search results. Thanks in advance for any help you can provide!

Here are some additional details: Unfortunately the element I am trying to click (the "Search" button) does not have an ID or Name associated with it. This is why is was trying alternative approaches, such as looping through all of the object and trying to find the one with the right “Title”. Here is the code for the element from the DOM explorer:

<input title="Press Enter to search" class="btn btn-highlight" type="submit" ng-disabled="topContentSearchForm.$invalid" value="Search!"/>

The only attributes associated with it are:

class: btn btn-highlight
type: submit
ng-disabled: topContentSearchForm.$invalid
value: Search!
title: Press Enter to Search

Please let me know if there is another way to find the element ID/name? or if there is another way to click the button without these attributes? Thanks

Answer 1

CSS selector:

You can use CSS selector of #search-btn > div. Which is div within className search-btn. "#" means class.

---

VBA:

Use .querySelector method to apply CSS selector:

ie.document.querySelector("#search-btn > div").Click

Answer 2

I know this is an old post but... I have been using this effectively..

   'click login
            Set htmlDoc = .document
            Set htmlColl = htmlDoc.getElementsByTagName("input")
            Do While htmlDoc.readyState <> "complete": DoEvents: Loop
                For Each htmlInput In htmlColl
                    If Trim(htmlInput.Type) = "submit" Then
                        htmlInput.Click
                        Exit For
                    End If
                Next htmlInput

Answer 3

It looks like you could potentially just build the string URL, for example if you put "abcd" in the search field, the resulting URL will be:

https://app.buzzsumo.com/top-content?result_type=total&type=articles&num_days=360&tfc=false&general_article&infographic&video&page=1&guest_post&giveaway&interview&links_sitewide=true&unique_domains=true&backlinks=false&q=abcd&offset=0

Note the bolded portion which is the search query.

So, and this is just a quick idea that may work as long as you're not trying to abuse their system by sending 1000's of automated requests:

Sub FetchWebsite()
    Dim ie As Object
    Dim form As Variant
    Dim button As Variant
    Dim url As String

    'add the “Microsoft Internet Controls” reference in VBA Project
    Set ie = CreateObject("InternetExplorer.Application")
    'using input box to enter URL I am serching for
    Search_URL = InputBox("Enter URL to Search For")

    '### BUILD THE FULL URL
    url = "https://app.buzzsumo.com/top-content?result_type=total&type=articles&num_days=360&tfc=false&general_article&infographic&video&page=1&guest_post&giveaway&interview&links_sitewide=true&unique_domains=true&backlinks=false&q=" & Search_URL & "&offset=0"

    With ie
        .Visible = True
        .navigate url
    End With
    'wait for page to load
    Do
    Loop While Not ie.ReadyState = 4 And Not ie.Busy

    AppActivate "Internet Explorer"

End Sub

I did some poking around in the Locals window and this should also work, modified from your code. This would be the Form.Submit that I mentioned in comment on OP.

Sub clickFormButton()
    Dim ie As InternetExplorer
    Dim form As Variant
    Dim button As Variant

    Dim ele As HTMLFormElement
    'add the “Microsoft Internet Controls” reference in VBA Project
    Set ie = CreateObject("InternetExplorer.Application")
    'using input box to enter URL I am serching for
    Search_URL = InputBox("Enter URL to Search For")

    With ie
        .Visible = True
        .navigate ("https://app.buzzsumo.com/#/top-content")
    End With
    'wait for page to load
    Do
    Loop While Not ie.ReadyState = 4 And Not ie.Busy

    'assigning the input variables to the html elements of the form
    ie.document.getElementsByName("q").Item.InnerText = Search_URL
    'finding and clicking the button

    ie.document.getElementsByClassName("btn btn-highlight").Item(4).form.submit
End Sub

Answer 4

A couple of ideas:

   While ie.ReadyState <> 4
       DoEvents
   Wend

If you have javascripts on the page use Application.Wait Now + TimeSerial(0, 0, 4) (basically wait for 4 seconds) instead.

Second I don't understand why you need to loop through all the objects on the web page. The easier way would be to go that webpage in IE, hit F12 and select element in DOM explorer, you can get the ID or Name of the button and then use ie.document.GetElementByID("buttonID").Click or ie.document.GetElementsByName("buttonName").Item.Click

Let me know if this helps.

Edit: After inspecting the particular webpage it appears that the ID and Name attributes for that button are missing. So I had to resort to the following:

Dim i As integer
Set form = ie.document.getElementsByClassName("btn btn-highlight")
On Error Resume Next
For i = 1 To 20
If form.Item(i).DefaultValue = "Search!" Then
     form.Item(i).Click
End If
Next i

The relevant button is clicked for the fourth item (I had to manually go through the loop because 3rd item navigated away from the page to a pricing page, so i had to go back). Anyway the full code is the following, please note that you will need to go through this exercise again if there were changes to the webpage

Sub clickFormButton()
    Dim ie As Object
    Dim form As Variant
    Dim button As Variant

    'add the “Microsoft Internet Controls” reference in VBA Project
    Set ie = CreateObject("InternetExplorer.Application")
    'using input box to enter URL I am serching for
    Search_URL = InputBox("Enter URL to Search For")

    With ie
        .Visible = True
        .navigate ("https://app.buzzsumo.com/#/top-content")
    End With
    'wait for page to load
    Application.Wait Now + TimeSerial(0, 0, 5)
    'assigning the input variables to the html elements of the form
    ie.document.getElementsByName("q").Item.InnerText = Search_URL
    'finding and clicking the button
    ie.document.getElementsByClassName("btn btn-highlight").Item(4).Click
End Sub