CODEWITHSUNDEEP
jqueryhtml-parsing
Ben Finkel
Ben Finkel

Reputation: 4803

Using jQuery server-side to scrub html pages

I'm sure this has been asked a billion and a half times, but I searched and didn't find anything on it.

I'd like to scrub some html pages, dynamically, and load the results into my database. jQuery seems like the perfect library to do this, but of course that runs in a browser and I'm looking to do this without a browser/user interaction.

What would be recommended for accomplishing this on the server-side (C# in my case)? Regex seems like overkill and not necessarily appropriate for the task.

Thanks, -Ben

UPDATE:

This library seems like what I'm after... but man I really like the way jQuery works.

http://htmlagilitypack.codeplex.com/

Upvotes: 1

Views: 486

Answers (1)

Darin Dimitrov
Darin Dimitrov

Reputation: 1039328

jQuery seems like the perfect library to do this

I am not sure about this. The same origin policy will limit you to scraping pages only on the same domain as the one which served the javascript (which basically limits you to your own domain and for web scraping that's pretty limiting). It would be better to use a server side language for this task and an HTML parsing library. Depending on the server side language you are using there might be different libraries available. Html Agility Pack is very good if you are using .NET on the server. But whatever you do and whatever solution you choose one thing's for sure: don't use regular expressions for parsing HTML.

Upvotes: 4

Related Questions