CODEWITHSUNDEEP
ruby-on-railsrubyweb-scrapingnokogiri
user273072545345
user273072545345

Reputation: 1566

How to save results into the database?

I created a web scraper.

I'm struggling to save the results into the model's columns.

How do I push the results of the scrapped results into the columns?

Do you map it? Would like to understand it so I can eventually post to index perhaps ... or show previous saved results, etc ...

Schema

ActiveRecord::Schema.define(version: 20170308223314) do

  # These are extensions that must be enabled in order to support this database
  enable_extension "plpgsql"

  create_table "links", force: :cascade do |t|
    t.string   "link_info"
    t.string   "date"
    t.datetime "created_at", null: false
    t.datetime "updated_at", null: false
  end

end

The two columns that I'm trying to save to are the ones you see in the schema above: link_info and date ...

Current Controller

class LinksController < ApplicationController

    def craigslist_scrape
        require 'open-uri'

        url = "https://losangeles.craigslist.org/search/web"

        page = Nokogiri::HTML(open(url))

        @craigslist_info = page.css("ul.rows")

                @link_info = page.css("li.result-row p.result-info a.result-title.hdrlnk")
                @date = page.css("li.result-row p.result-info time.result-date")
    end

end

craiglist_scrape.html.erb

<% @link_info.each_with_index do |link, index| %>
  <h2><%= "Title of the job: #{link.text}" %></h2>
  <p><%= "Date: #{@date[index].text}" %></p>
<% end %>

routes

Rails.application.routes.draw do
    root 'links#craigslist_scrape'
end

model

class Link < ApplicationRecord
end

Upvotes: 1

Views: 248

Answers (2)

alias_albino
alias_albino

Reputation: 11

I'm rather new to Ruby and don't know much about Rails but what I always do when I'm trying to save something into a "database" is I create a hash. For example:

database = {}
puts "What is your name?"
input = gets.chomp
puts "What's your age?"
input_2 = Integer(gets.chomp)
database[input.to_sym] = input_2

In the example above I create a hash called "database" and I ask the user for their name and age. Storing the user's name as the string value for database and their age as an integer value. I don't know if this helps at all, like I said I'm fairly new to Ruby.

Upvotes: 1

Shannon
Shannon

Reputation: 3018

In the controller you can add:

def craigslist_scrape
    require 'open-uri'

    url = "https://losangeles.craigslist.org/search/web"

    page = Nokogiri::HTML(open(url))

    @craigslist_info = page.css("ul.rows")

    @link_info = page.css("li.result-row p.result-info a.result-title.hdrlnk")
    @date = page.css("li.result-row p.result-info time.result-date")

    @link_info.each_with_index do |link, index|
      Link.new(:link => link.text, :date => @date[index].text).save
    end

end

Upvotes: 1

Related Questions