CODEWITHSUNDEEP
rubyseleniumselenium-webdriverselenium-chromedriver
Rusty
Rusty

Reputation: 1

Ruby web scraping with selenium

I am trying to scrape a website and strip two different things and print them together. The title and description of PlayStation Trophies.

require 'selenium-webdriver'

  driver = Selenium::WebDriver.for(:chrome, options: options)

  driver.get('https://www.playstationtrophies.org/game/far-cry-4/trophies/')

  puts driver.title

  trophies = driver.find_elements(:class, 'link_ach')
  description = driver.find_elements(:class, 'ac3') 

  trophies.each do |trophy|
    description.each do |desc|
      puts trophy,desc.text.strip
     end
  driver.quit
end

I can strip them both individually but when trying to put them together it goes horribly wrong.

Upvotes: 0

Views: 400

Answers (2)

Maruf Hasan Bulbul
Maruf Hasan Bulbul

Reputation: 462

Iteration over trophies and description is very confusing here. You have two separate collections/arrays, trophies and description. Then you have used a nested loop and quit driver inside the loop. So it will display all the descriptions under 1st trophy.

If you want to display each description under associated trophy then you can do this:

puts driver.title

trophies = driver.find_elements(:class, 'link_ach')
descriptions = driver.find_elements(:class, 'ac3')
data_sets = trophies.zip(descriptions)

datasets.each do |trophy, description|
    puts trophy.text.strip, description.text.strip
end

driver.quit

Then you can customize your display text in puts as your needs.

Upvotes: 0

Mike K.
Mike K.

Reputation: 3789

I'm not entirely sure what you mean by 'horribly wrong' but my best guess based on running what you have is that for trophy you're actually printing the inspection of the variable trophy to the screen instead of the string value. So you're getting lots of:

#<Selenium::WebDriver::Element:0x00007ff4b60e5eb8>

I think you'll be happier with the output if you print the text values of both Element's:

puts "#{trophy.text.strip}\n #{desc.text.strip}"

If this is indeed the issue you're seeing you may want to review to_s vs inspect in ruby.

Upvotes: 1

Related Questions