CODEWITHSUNDEEP
rubyweb-scrapingnokogiristoring-data
kitodann
kitodann

Reputation: 1

Nokogiri Ruby scrape working, but is it possible and how to store the data to a google sheets/csv/db file?

I have written down code on VSCode and tested it out. I get the printed out results and they are solid. However, I want to use the result in a db as sql to be able to manipulate it in some way. Doesn't have to be db for sql, could be google sheets or csv. It would be nice to know how to work out all the above cases.

Thing is I do not know where to begin. Have searched the internet and I couldn't find something that I could use to help me solve this problem. Below is a transcript of my code:

require "open-uri"
require "nokogiri"

url = "https://somenewswebsite.com/"

html = URI.open(url).read
html_doc = Nokogiri::HTML(html)

html_doc.search('.main-news-controls__wrap a').each do |element|
   top_headline = element.text.strip
   top_url_ending = element.attribute("href").value  
   puts top_headline
   puts top_url_ending
end

puts ('-' * 100)

html_doc.search('.post-card__header a').each do |element|
    card_headline = element.text.strip
    card_url_ending = element.attribute("href").value
    puts card_headline
    puts card_url_ending
 end

From there on I want to get the results in the top_headline, for example, and add them as rows in a db as stated before, but I also want to have an id field (as a counter 1, 2, 3 and so on) plus a time stamp (year, month, day, hour) and a keyword for each row of data. Is this possible to do? If it is possible, would it take too long to work out?

Thanks in advance!

Haven't tried out anything on the storing part yet cause I did not find anything related...

Upvotes: 0

Views: 38

Answers (1)

andriybureviy
andriybureviy

Reputation: 15

You can add your array to csv file

require 'csv'
CSV.open("myfile.csv", "w") do |csv|
  csv << ["row", "of", "CSV", "data"]
  csv << ["another", "row"]
  # ...
end

Upvotes: 0

Related Questions