CODEWITHSUNDEEP
cssrweb-scrapingrvest
Daniel Schultz
Daniel Schultz

Reputation: 320

Scrape the text of a selected Drop Down item with rvest

I am scraping some website using Rselenium and rvest. Therefore I am cycling through items of a drop down menu to change the javascript table. The table name from the drop down menu should become my identifier column in the scraped table. I managed to scrape the table, but I am stuck when scraping just the one selected menu entry. Here are some lines of the html code:

<select>
<option value="5823">2010/2011</option>
<option value="7094">2011/2012</option>
<option value="9024">2012/2013</option>
<option value="11976">2013/2014</option>
<option value="15388">2014/2015</option>
<option value="18336" selected="selected">2015/2016</option>
</select>

How do I get the html_text of the selected column? The css selector :checked doesn't work. I tried:

 html_nodes("option") %>% html_attrs()

Which correctly gives me: 

 [[1]]
 value 
"5823" 

[[2]]
 value 
"7094" 

[[3]]
 value 
"9024" 

[[4]]
  value 
"11976" 

[[5]]
  value 
"15388" 

[[6]]
  selected      value 
"selected"    "18336"

and 

read_html(wData) %>% html_nodes("option") %>% html_text()
[1] "2010/2011" "2011/2012" "2012/2013" "2013/2014" "2014/2015" "2015/2016"

But I don't know how to bring the two together. I only get:

[1] "2015/2016"

Since I am then cycling through the options I need a general solution. Thanks.

Upvotes: 3

Views: 2564

Answers (1)

MrFlick
MrFlick

Reputation: 206401

You can use an xpath selector rather than a css selector.

read_html(wData) %>% html_nodes(xpath="//option[@selected]")  %>% html_text()

This allows you to search for attributes even when the :checked css pseudo class doesn't work.

Upvotes: 5

Related Questions