emulating a browser programmatically in C# / .Net

Question

We would like to automate certain tasks in a website, like having a user 'login', perform some functionality, read their account history etc.

We have tried emulating this with normal POST/GETs, however the problem is that for example for 'login', the website uses javascript code to execute an AJAX call, and also generate some random tokens.

Is it possible to literally emulate a web-browser? For example:

• Visit 'www.[test-website].com'
• Fill in these DOM items
  • DOM item 'username' fill in with 'testuser'
  • DOM item 'password' fill in with 'testpass'
  • Click' button DOM item 'btnSubmit'
• Visit account history
  • Read HTML (So we can parse information about each distinct history item)
• ...

The above could be translated into say the below sample code:

var browser = new Browser();
var pageHomepage = browser.Load("www.test-domain.com");
pageHomepage.DOM.GetField("username").SetValue("testUser");
pageHomepage.DOM.GetField("password").SetValue("testPass");
pageHomepage.DOM.GetField("btnSubmit").Click();
var pageAccountHistory = browser.Load("www.test-domain.com/account-history/");
var html = pageAccountHistory.GetHtml();
var historyItems = parseHistoryItems(html); 

Answer 1

Or just try System.Windows.Forms.WebBrowser, for example:

this.webBrowser1.Navigate("http://games.powernet.com.ru/login");
while (webBrowser1.ReadyState != WebBrowserReadyState.Complete)
     System.Windows.Forms.Application.DoEvents();
HtmlDocument doc = webBrowser1.Document;
HtmlElement elem1 = doc.GetElementById("login");
elem1.Focus();
elem1.InnerText = "login";
HtmlElement elem2 = doc.GetElementById("pass");
elem2.Focus();
elem2.InnerText = "pass";

Answer 2

I would suggest to instantiate a WebBrowser control in code and do all your work with this instance but never show it on any form. I've done this several times and it works pretty good. The only flaw is that it makes use of the Internet Explorer ;-)

Answer 3

You could use for example Selenium in C#. There is a good tutorial: Data Driven Testing Using Selenium (webdriver) in C#.

Answer 4

Try JMeter, it is a nice too for automating web requests, also quite popularly used for performance testing of web sites