CODEWITHSUNDEEP
javascriptexcelvbaweb-scrapingxmlhttprequest
Jasco
Jasco

Reputation: 241

What parameters does this website need to use XMLHTTP?

I can't figure out what parameters need to be sent to a child-page from the parent-page (with VBA) to access the data. I have tried with referer (main-Page URL) and with various details that are in the HTML text as POST, but Excel hangs on EVERY attempt to call the page with XMLHTTP!

Actually, on the main site, only have to select a value from the droplist and then click on the search button without having to fill in the other fields, but the child page still does not respond with POST of the relevant ID value.

<select name="Vort" size="1">
<option size="30" value="%"></option>
<option value="60467"</option>
<option value="60470"</option>
<option value="60468"</option>
...

Any ideas? My VBA code (Excel hangs, no matter with which POST parameter(S)):

Public Sub GETDATA()

    Dim html As New HTMLDocument, Htmldoc As New HTMLDocument, x As Long
        
    With CreateObject("MSXML2.ServerXMLHTTP.6.0")
        .Open "Post", URL, False
        .send   '?
        html.body.innerHTML = .responseText
    End With
        
    For x = 0 To 10
    Htmldoc = html.querySelectorAll("th").Item(x)
    ActiveSheet.Cells(x + 2, 2) = Htmldoc.innerText
    Next

End Sub

Upvotes: 0

Views: 270

Answers (1)

SIM
SIM

Reputation: 22440

To get required response, you need to send appropriate parameters along with content type headers with post requests.

Parameters used within the following post requests are based on this selection.

Public Sub GetContent()
    Const link = "https://www.thueringen.de/olgneu/zvg/dosearch2_neu.asp"
    Dim oHtml As HTMLDocument, payload$
    
    Set oHtml = New HTMLDocument
    
    payload = "REASON=FULL_SEARCH&START_AT=1&Objekte_Ort=&VersteigerungsOrt=60467&Aktenzeichen=&ObjektWert_von=&ObjektWert_bis=&Termin_Ab=26.4.2021&Termin_Bis=&MAX_HIT=10"
    
    With CreateObject("MSXML2.XMLHTTP")
        .Open "POST", link, False
        .setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.135 Safari/537.36"
        .setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
        .send (payload)
        oHtml.body.innerHTML = .responseText
    End With
    
    MsgBox oHtml.querySelector("tr.klein a[href]").href
End Sub

Upvotes: 3

Related Questions