CODEWITHSUNDEEP
ruby-on-rails
hassan
hassan

Reputation: 21

Screen Scraping with Ruby on Rails

Am a newbie in RoR. I want to develop a rails app, which could search flight info from my rails app, fetch the flight informations from various airlines sites(for example; airasia and malaysia airlines). display those results back in my rails app.
My search module should include the following textfields:

From: Kuala Lumpur
To: Dubai
Departure Date:2/11/2009
Return date: 19/11/2009
Departure Time: 12:00
Number of tickets: 2

user gets to enter the information above on my site and
a script should search flights according to that information on various airlines sites and
display the results with the tag of which airlines is providing this flight.

Can either use "Scrapi" or "Scrubyt"

Upvotes: 0

Views: 844

Answers (3)

boulder_ruby
boulder_ruby

Reputation: 39695

cough cough watir-webdriver (gem)

the DOM cannot hide

Upvotes: 0

Ritesh Kumar
Ritesh Kumar

Reputation: 2223

To start with scraping, you can refer to the post -

http://rorguide.blogspot.com/2011/04/simple-ruby-screen-scraper-using.html (Simple Ruby Screen Scraper using Mechanize, Hpricot and XPath )

http://rorguide.blogspot.com/2011/04/simple-ruby-screen-scraper-in-just-5.html (Simple Ruby Screen Scraper in just 5 lines without using XPath )

This is just for understanding

Upvotes: 2

EmFi
EmFi

Reputation: 23450

can anyone offer an assistance?

Don't. Find another get rich quick scheme. Or at least don't rely on screen scraping.

There's so much potential for it to go wrong. Find another way to get the information to aggregate. Use API's if they exist and store the schedules in the databases (they rarely change on a week to week basis). Storing things locally makes things faster for your customers, it also allows you to provide more services.

Here are just a few of the potential problems with screen scraping:

  • Minor updates to the sites you're scraping can break your tool.
  • Scraping attempts from your server's IP can be denied.
  • Depending on the information, you could possibly even get sued.

Upvotes: 5

Related Questions