CODEWITHSUNDEEP
rrvest
Eka
Eka

Reputation: 15010

How to fetch headlines from google news using rvest R?

I want to fetch headlines from google news using rvest in R. I have done this so far

library(rvest)
url=read_html("https://www.google.com/search?hl=en&tbm=nws&authuser=0&q=american+president")
selector_name<-"r"
fnames<-html_nodes(x = url, css = selector_name) %>%
  html_text()

but the result is 

> fnames
character(0)

This is the inspect element of a headline?

<h3 class="r"><a href="/browse.php/PbtvpluS/QDvUJpC7/KoWCA9QE/VTTOFmVJ/bIp8sMa8/qKjgkcAu/Hgcr9lyg/4bibGCOO/nZ82ojLo/_2B602Vo/0sOSEbba/SaiySebj/AqD60GRO/skpNXIW9/fA8EbzOq/z6XjMXo2/9iDad2zD/qREp_2Fp/hoHl64rG/9wfBHOPB/a0nLFrAz/OsCmtfKV/cQoDAFWY/cRXfd5FX/5OAJF8UR/9gUdG_2F/_2F4hOLN/xOfe6_2F/shH2n9O7/hCZGQosp/eeAh6wAC/JhCOgG0i/sLkpRGRN/PH_2B61L/njabdbV1/vpS4wcbX/NKpO_2Bq/jpun2LeG/TQecIESv/vxFbk19Q/_3D_3D/b29/">Obama Addresses Racial Tensions at Celebration of African ...</a></h3>

How can I fetch the headlines from google news?

Upvotes: 2

Views: 2484

Answers (2)

Spacedman
Spacedman

Reputation: 94202

I think you are just missing a dot for the class name:

> headlines = read_html("https://www.google.com/search?hl=en&tbm=nws&authuser=0&q=american+president") %>%
  html_nodes(".r") %>% 
  html_text()
> headlines
 [1] "Iranian President: No American President Can Renegotiate the Now ..."
 [2] "US: President Barack Obama vetoes 9/11 bill"                         
 [3] "President Obama Wants Donald Trump to Visit New African ..."         
 [4] "President Obama: Discrimination Should Concern 'All Americans ..."   
 [5] "Conrad Black: The Middle East watches, and waits, for the next ..."  
 [6] "Putin's close friend: Donald Trump will be next US president"        
 [7] "US election 2016 polls and odds: Latest Donald Trump and Hillary ..."
 [8] "US election: Ted Cruz endorses Donald Trump for president"           
 [9] "Obama – I'm proud of my 'African record' as US president"            
[10] "Almost 6000 Americans Have Already Voted for President"

Upvotes: 2

Chirayu Chamoli
Chirayu Chamoli

Reputation: 2076

Well you could do by:

library(rvest)
reviews <- link %>%
    read_html() %>%
    html_nodes(".g") %>%
    html_text()

you check via inspect element where the text(headline is present), in this case it would class g. Then read the text within each node.

Upvotes: 1

Related Questions