CODEWITHSUNDEEP
rubyweb-crawleranemone
Ghilas BELHADJ
Ghilas BELHADJ

Reputation: 14106

How to crawl only a subfolder with Anemone

We can crawl a hole website with anemone (ex: https://stackoverflow.com/), but what if I want only focus on a certain folder (ex: https://stackoverflow.com/questions). How can I do this ? maybe with the "focus_crawl" method ?

Upvotes: 2

Views: 554

Answers (1)

Pritesh Jain
Pritesh Jain

Reputation: 9146

check the keep_if method may be this helps

http://danneu.com/posts/8-scraping-a-blog-with-anemone-ruby-web-crawler-and-mongodb#toc_1

try and pass the pattern as you want to crawl

also there is a gist https://gist.github.com/1149906.

NOTE: I haven't tested it but you can sure surely try.

Upvotes: 2

Related Questions