XMLHTTP Request (POST) to retrieve data from web site using VBScript

Question

dear collegues! I need your help.

Fisrt of all, this is NOT ad. I try to make a POST request on website of railway tickets http://booking.uz.gov.ua/en/ to know if there is tickets on current date. BUT... have a problem. I'm using a VBScript to make a request. To know what HTTP Header and POST request to send on website I used Chrome in-build development tools.

Here's my script:

    Dim URL
    Dim URL2
    Dim URL3
    Dim sRequest
    Dim sCookies

    'This is web page where I need to enter information.
    URL = "http://booking.uz.gov.ua/en/"
    'This is path that Chrome shows to send POST request.
    URL2 = "http://booking.uz.gov.ua/en/purchase/search/"
    'Optional URL, Chrome shows this link near of URL2. I think this is .js that works                 on info I enter on web site (URL).
    'URL3 = "http://booking.uz.gov.ua/i/js/common.138.js"
    'POST request that Chrome shows to send.
    sRequest ="station_id_from=2200001&station_id_till=2208001&station_from=Kyiv&station_till=Odesa&date_        dep=09.19.2013&time_dep=00%3A00&search="
    'Here I'm using GET request to retrieve Set-Cookie Header (SessionID first of all)         to reuse in my second POST request.
    sCookies = GetSetHeader(URL)
    'Here I'm calling function to make POST request.
    Result = HTTPPost(URL2, sRequest)

    Function GetSetHeader(URL)
       Set objhttp = CreateObject("Microsoft.XmlHttp")
       objhttp.open "GET", URL, FALSE
       objhttp.Send
       'I'm getting only SessionID + other cookies that Chrome shows.
       GetSetHeader = Left (objhttp.getResponseHeader("Set-Cookie"), 38) & " " &         "HTTPSERVERID=server1; _gv_lang=en;         __utma=31515437.675496133.1376934004.1376934004.1376934004.1;         __utmb=31515437.2.10.1376934004; __utmc=31515437; __utmz=31515437.1376934004.1.1.utmcsr=        (direct)|utmccn=(direct)|utmcmd=(none)"
    End Function

    Function HTTPPost(URL2, sRequest) 
       'Header I just took from Chrome.
       Set objhttp = CreateObject("Microsoft.XmlHttp")
       objHTTP.open "POST", URL2, false
       objHTTP.setRequestHeader "Connection", "keep-alive"
       objHTTP.setRequestHeader "Host", "booking.uz.gov.ua"
       objHTTP.setRequestHeader "Connection", "keep-alive"
       objHTTP.setRequestHeader "Content-Length", "Len(Request)"
       objHTTP.setRequestHeader "GV-Token", "64214392f178b9f91e3b61a069915cd1"
       objHTTP.setRequestHeader "Origin", "http://booking.uz.gov.ua"
       objHTTP.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64)         AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
       objHTTP.setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
       objHTTP.setRequestHeader "GV-Unique-Host", "1"
       objHTTP.setRequestHeader "GV-Ajax", "1"
       objHTTP.setRequestHeader "GV-Screen", "1366x768"
       objHTTP.setRequestHeader "GV-Referer", "http://booking.uz.gov.ua/en/"
       objHTTP.setRequestHeader "Accept", "*/*"
       objHTTP.setRequestHeader "Referer", "http://booking.uz.gov.ua/en/"
       objHTTP.setRequestHeader "Accept-Encoding", "gzip,deflate,sdch"
       objHTTP.setRequestHeader "Accept-Language", "ru-RU,ru;q=0.8,en-        US;q=0.6,en;q=0.4"
       'Here I use cookies retrieved with first GET request.
       objHTTP.setRequestHeader "Cookie", "sCookies"
       objHTTP.send sRequest
       'I use this msg to check that right cookies send with POST request.
        WScript.Echo sCookies
        HTTPPost =  objHttp.responseText
       'Write answer to TXT file.
       Set FSO = CreateObject("Scripting.FileSystemObject")
       Set oFile = FSO.OpenTextFile("D:\Results.txt", 2, True)
       oFile.Write(objHttp.responseText)
       oFile.Close
       Set oFile = Nothing
       Set FSO = Nothing
    end Function

I can't make me script work. I get empty TXT file if I use URL2 to send a request. If I use URL3 - path to script that Chrome show me - to send a request, I just receive a contents of common.138.js in my TXT file. But I expect to receive info in JSON type like shown in Chrome response.

What I noticed, First, if refresh the website, and try to resend old request, I give me an error:

NetworkError: 400 Bad Request - http://booking.uz.gov.ua/en/purchase/search/"

Maybe because SessionID changed.

Second, I can't to simply write name of station, I need to chose it from drop-down list (When work with this site in UI mode). OR I get an error - Select a departure point from a drop down list.

Third, if try to send request by simply clicking the button on site to search, I get error Status Code:400 Bad Request. I think time of SessionID expired.

There is a working script using InternetExplorer.Application system object, but it is no decision. I want to make it work by sending requests. In future want to try do it on php (as a peart of learning process).

Maybe it's some kind of defence from people like me??? There is a way to make my script work???? Maybe SessionID changes between GET and POST requests?? Or maybe VBScript can't resolve it and I need PHP, for example???

I don't know how to solve this problem. Help me please. Can't sleep. Can't eat. Thanks very very much.

Answer 1

You made mistake here:

objHTTP.setRequestHeader "Content-Length", "Len(Request)"

Should be:

objHTTP.setRequestHeader "Content-Length", Len(Request)

Answer 2

You are getting "400" because you are sending wrong GV-Token header to UZ site. Eventually GV-token is an md5 of some session-dependent variable (session is identified via _gv_sessid cookie).

This token is obfuscated in JavaScript and resides in page body like,

...
$$_.$_=($$_.$_=$$_+"")[$$_.$_$]+($$_._$=$$_.$_[$$_.__$])+($$_.$$=($$_.$+"")[$$_.__$])+((!$$_)+"")[$$_._$$]+($$_.__=$$_.$_[$$_.$$_])
...

Which evaluates to something like

localStorage.setItem('gv-token',4619709a341b4ffdacce3dafd2f85af3)

and then conducted to all UZ Ajax requests.

So I wish you happy deobfuscating :)) (not for the weak)

PS Also make sure to turn on .NET useUnsafeHeaderParsing via app configuration or reflection.

UPD: As i see, this topic is still alive, so i made up deobfuscation code - it seems like basic regular expressions & string search-and-replace are enough.

Suppose you have UZ start page HTML in pageHTML, then to make things work you need something like (sans validitiy checking), in C#:

Obfuscated code contains some tokens, each evaluates to a hex number from 0 to F, they can be directly replaced. Here is a correspondence dictionary:

var subsitutes = new Dictionary<string, string>
{
    {"$$_.$$$", "7"},
    {"$$_.$$$$", "f"},
    {"$$_.$$$_", "e"},
    {"$$_.$$_", "6"},
    {"$$_.$$_$", "d"},
    {"$$_.$$__", "c"},
    {"$$_.$_$", "5"},
    {"$$_.$_$$", "b"},
    {"$$_.$_$_", "a"},
    {"$$_.$__", "4"},
    {"$$_.$__$", "9"},
    {"$$_.$___", "8"},
    {"$$_._$$", "3"},
    {"$$_._$_", "2"},
    {"$$_.__$" ,"1"},
    {"$$_.___", "0"},
};

Then by using regex we get part of obfuscated the code we're interesed in

var scramble = Regex.Match(pageHTML, @"\$\$_\.\$\(\$\$_\.\$\((.*)\)\(\)\)\(\);");

And replace the mentioned above tokens with their real meaning

var keysSorted = subsitutes.Keys.OrderByDescending(key => key.Length);   
var halfBakedDeobfuscated = keysSorted.Aggregate(scramble.Groups[1].Value, (current, key) => current.Replace(key, subsitutes[key]));

Almost done, cut out some garbage

var start = Regex.Escape(new string(new[] { '"', '\\', '\\', '\\', '"', ',', '\\', '\\', '"', '+' }) + "4+0+" + new string(new[] { '\"', '\\', '\\', '\\', '\"', '\"', '+' }));
var end = Regex.Escape(new string(new[] { '+', '"', '\\', '\\', '\\', '"', ')' }));    
var core = Regex.Match(halfBakedDeobfuscated, start + "(.*)" + end).Groups[1].Value;

Now core contains almost clean version of gvToken, something like 7+0+f+a+7+7+9+8+5+7+e+b+3+3+a+8+3+c+7+8+3+b+d+d+e+f+4+8+7+7+f+7 so last step is to remove these + symbols

var gvToken = string.Join(string.Empty, core.Split('+'));

At last, gvToken contains what you need to give to UZ site - a string like 70fa779857eb33a83c783bddef4877f7.

No JS library and of course no InternetExplorer needed.

Answer 3

Interesting, need to use InternetExplorer.Application, for example list powershell code:

$erroractionpreference = "Continue"
$ie = New-Object -ComObject "InternetExplorer.Application"
$ie.navigate("http://booking.uz.gov.ua/en/")
$ie.visible = $true
sleep 5
while($ie.ReadyState -ne 4) {start-sleep -m 100}
$ie.document.getElementByID("station_id_from").Value = "2200001"
$ie.document.getElementByID("station_id_till").Value = "2208001"
$ie.document.getElementsByName("station_from").Item(1).Value = "Kyiv"
$ie.document.getElementsByName("station_till").Item(1).Value = "Odesa"
$ie.document.getElementByID("date_dep").Value = "12.26.2014"
$ie.document.getElementByID("time_dep").Value = "00:00"
$ie.document.getElementByID("search").Click()  

Cookies, including GV-Token, in such case need no to be transferred. I think, there is a way to write without InternetExplorer.Application but emulate browser with your code. Need to explore it.