CODEWITHSUNDEEP
ruby-on-railsrubyweb-scraping
Mike Wiesenhart
Mike Wiesenhart

Reputation: 316

Web Scraping using Ruby - If statment

I have built a web scraper. I need it to scrape the prices and bedrooms of a given neighborhood. Sometimes the span.first_detail_cell will return Furnished and the rest of the time it will return the price. I need to write something that can overlook the span.first_detail_cell if it is furnished and look in the next cell for the price. I think I need to write an if statement, but not sure of the parameters. Any help would be great!

require 'open-uri'
require 'nokogiri'
require 'csv'

url = "https://streeteasy.com/for-rent/bushwick"
page = Nokogiri::HTML(open(url))

page_numbers = []
page.css("nav.pagination span.page a").each do |line|
  page_numbers << line.text
end

max_page = page_numbers.max

beds = []
price = []

max_page.to_i.times do |i|

  url = "https://streeteasy.com/for-rent/bushwick?page=#{i+1}"
  page = Nokogiri::HTML(open(url))

  page.css('span.first_detail_cell').each do |line|
    beds << line.text
  end

  page.css('span.price').each do |line|
    price << line.text
  end

end

CSV.open("bushwick_rentals.csv", "w") do |file|
  file << ["Beds", "Price"]

  beds.length.times do |i|
    file << [beds[i], price[i]]
  end
end

Upvotes: 0

Views: 110

Answers (1)

max pleaner
max pleaner

Reputation: 26758

  page.css('span.first_detail_cell').each do |line|
    if line.text.include?("Furnished")
      # do something hre
    else
      beds << line.text
    end
  end

Upvotes: 1

Related Questions