CODEWITHSUNDEEP
r
psychemedia
psychemedia

Reputation: 5940

Extracting URLs from a HTML snippet in R

Given the following example:

sites=c('site 1','site 2')
link=c('<a href="http://example.com/path">This website</a>', '<a href="http://example.com/path2">That website</a>')
w=data.frame(link,sites)
w
                                             link      sites
<a href="http://example.com/path">This website</a>   site 1
<a href="http://example.com/path2">That website</a>  site 2

how do I apply a regular expression that will parse the html snippet to extract the url and the link text and pop them into separate columns in a data frame? So for example, given the above example, what do I need to do in order to generate a data frame that looks like:

                     url          name     sites
http://example.com/path    This website   site 1
http://example.com/path2   That website   site 2

Upvotes: 2

Views: 196

Answers (2)

hrbrmstr
hrbrmstr

Reputation: 78832

Alternate solution with rvest:

library(rvest)

sites <- c('site 1','site 2')
link <- c('<a href="http://example.com/path">This website</a>', 
          '<a href="http://example.com/path2">That website</a>')
w <- data.frame(link, sites)

anchors <- html_nodes(read_html(paste0(w$link, collapse="")), "a")

data.frame(url=html_attr(anchors, "href"),
           name=html_text(anchors),
           sites=w$sites)
##                        url         name  sites
## 1  http://example.com/path This website site 1
## 2 http://example.com/path2 That website site 2

Upvotes: 0

rcs
rcs

Reputation: 68849

Here is a solution using the htmlTreeParse function from package XML (i.e., without regular expressions)

R> library("XML") 
R> htp <- htmlTreeParse(link)
R> t(sapply(seq_along(link),
+           function(i) c(url=unname(htp$children$html[[1]][[i]]$attributes),
+                         name=htp$children$html[[1]][[i]]$children$text$value,
+                         sites=sites[i])))
     url                        name           sites   
[1,] "http://example.com/path"  "This website" "site 1"
[2,] "http://example.com/path2" "That website" "site 2"

Upvotes: 8

Related Questions