CODEWITHSUNDEEP
ruby-on-railsscreen-scrapingnokogirimechanize
John Lee
John Lee

Reputation: 1251

Finding links on page matching HREF pattern

I am trying to interact with a website full of images and want to interact with it by clicking on each image. Each image has a relative url (eg: /image/1000/, image/1023/, etc). The number I assume is the image_id and appears to be random, not in consecutive order.

This is what I did in rails console:

agent = Mechanize.new
agent.get('http://www.website.com')
agent.page.links_with(:href => '/image')

The last line did not return anything but when I tried

agent.page.link_with(:href => '/image/1000/')

It returns the link as expected.

I am pretty sure the problem is in the :href parameter, it should not be '/image'. But i tried other combinations like '/image/' , '/image/* ', etc and it still return nothing.

Appreciate any advice.

Upvotes: 1

Views: 821

Answers (1)

mnelson
mnelson

Reputation: 3012

Note: untested

Try: agent.page.links_with(:href => /\/image\/(\d{1,})/). The links_with documentation shows a regexp being used so I assume this will work fine. Also, $1 will return your image_id.

http://mechanize.rubyforge.org/Mechanize/Page.html#method-i-links_with-28criteria-29

Upvotes: 1

Related Questions