CODEWITHSUNDEEP
rvestwebharvest
Ehsan
Ehsan

Reputation: 11

Rvest returns zero list

I want to download all links/ titles of papers from the web using rvest. I used the following script but it is not the list is zero. Any suggestions?

library(rvest)

1. Download the HTML and turn it into an XML file with read_html()

Papers <- read_html("https://papers.ssrn.com/sol3/JELJOUR_Results.cfm?npage=1&form_name=journalBrowse&journal_id=1475407&Network=no&lim=false")

2. Extract specific nodes with html_nodes()

Titles <- html_nodes(Papers, "span.optClickTitle")

Upvotes: 0

Views: 229

Answers (1)

Birger
Birger

Reputation: 1141

You are close, try .optClickTitle instead of span.optClickTitle:

library(magrittr)
library(tibble)
library(rvest)
#> Lade nötiges Paket: xml2

url <- "https://papers.ssrn.com/sol3/JELJOUR_Results.cfm?npage=1&form_name=journalBrowse&journal_id=1475407&Network=no&lim=false"
raw <- read_html(url)

parse_link <- function(x) {
  tibble(title = html_text(x),
         link = html_attr(x = x, name = "href"))
}

raw %>%
  html_nodes(".optClickTitle") %>%
  parse_link()
#> # A tibble: 60 x 2
#>    title                                            link                  
#>    <chr>                                            <chr>                 
#>  1 The Nature of Man                                https://ssrn.com/abst…
#>  2 The Dynamics of Crowdfunding: An Exploratory St… https://ssrn.com/abst…
#>  3 Some Simple Economics of the Blockchain          https://ssrn.com/abst…
#>  4 "Some Simple Economics of the Blockchain\r\n\t\… https://ssrn.com/abst…
#>  5 "Some Simple Economics of the Blockchain\r\n\t\… https://ssrn.com/abst…
#>  6 Bitcoin: An Innovative Alternative Digital Curr… https://ssrn.com/abst…
#>  7 Piracy and Box Office Movie Revenues: Evidence … https://ssrn.com/abst…
#>  8 The sharing economy: Why people participate in … https://ssrn.com/abst…
#>  9 Consumer Acceptance and Use of Information Tech… https://ssrn.com/abst…
#> 10 What Makes Online Content Viral?                 https://ssrn.com/abst…
#> # ... with 50 more rows

Created on 2018-09-28 by the reprex package (v0.2.1)

Upvotes: 0

Related Questions