CODEWITHSUNDEEP
iosiphoneswiftuiwebview
SkyWalker
SkyWalker

Reputation: 14307

How can UIWebView be used to automate scraping a page that requires auth?

I'd like to build an iPhone/iPad iOS app with swift that checks a specific URL, fetches and parses an HTML page but without exposing the Web interface view of UIWebView. I'd be very happy if the HTML view is shown only for authenticating the user the first time and thereafter the page is accessed programmatically and directly each time. An example app that I think does that is MailBuzzr for Hotmail & Outlook.

I need sort of a Web scraper but in some cases e.g. http://login.live.com it is quite cumbersome to get in i.e. very complex input form, two-step verification, etc. I just tested accessing my live account from UIWebView and it works great even when two-step verification is required. However, I'd like to get there programmatically without exposing the view to the user or disguise it in whatever way possible.

Is there a way to use UIWebView to do that? Alternatives?

The code I have is quite trivial:

@IBOutlet var Webview: UIWebView!
let url = NSURL(string: "https://account.live.com/")
let request = NSURLRequest(URL: url!)
Webview.loadRequest(request)

PS: I tried inspecting the live login page using Mozilla's Web Developer plugin and can't make sense of the parameters (NAP, ANON, t) and how they are computed. I think it can be done but maybe there is an easier way using UIWebView.

Upvotes: 1

Views: 511

Answers (1)

Scott McKenzie
Scott McKenzie

Reputation: 16242

One thing I think you'll have to do is inject some JavaScript to populate the fields and trigger the login process.

You can use an approach like this (pseudo code):

NSString *javaScript = @"var field = document.getElementsByTagName('#password')[0];
// Set it.
// Trigger the form submission.";
[webView stringByEvaluatingJavaScriptFromString:javaScript];

Then you can scrape the page by getting the content (pseudo code):

NSString *html = [webView stringByEvaluatingJavaScriptFromString:  @"document.body.innerHTML"];

Upvotes: 1

Related Questions