CODEWITHSUNDEEP
javascriptjquerynode.js
imjp
imjp

Reputation: 6695

How do I make a simple web scraper in NodeJS?

I'm trying to create a simple web scraper that does something as follows:

  1. goes to yellowpages.com
  2. picks the first category it hasn't scraped yet
  3. clicks on the first business it hasn't scraped yet
  4. scrapes the title, phone number and description of the business
  5. appends the data it scraped into a .csv file (or database)
  6. goes back to step 1

I figured Node.js / jQuery would be the easiest to achieve something like this. Can anyone perhaps point me to a tutorial or put something together quick for me?

Upvotes: 2

Views: 732

Answers (1)

Chris Doggett
Chris Doggett

Reputation: 20787

Just read an article on Hacker News yesterday about scraping with NodeJS and Chimera. The same author wrote an earlier article on Enhanced web scraping with NodeJS.

The Chimera one looks cool, because it uses a headless web browser (QtWebkit), so you get the pages to load anything they load with JS, which I need for a scraper I'm building at work.

Upvotes: 4

Related Questions