CODEWITHSUNDEEP
riframeweb-scraping
Dinesh Kumar
Dinesh Kumar

Reputation: 21

Web scraping of (iframe) search result in r

I want to scrape all the NPI and details from the below website. "https://www.pverify.com/npi-lookup-find-npi-number-of-doctors-physicians/"

code:

library("rvest")
library("xml2")
url="https://www.pverify.com/npi-lookup-find-npi-number-of-doctors-physicians/"
webpage<-read_html(url)
data_html <- html_nodes(webpage,'iframe')
data_html <-html_table(data_html)

When I try the above code, error message is "Error: html_name(x) == "table" is not TRUE" Kindly help me to the get NPI numbers and their details.

Upvotes: 0

Views: 211

Answers (1)

Andryas Waurzenczak
Andryas Waurzenczak

Reputation: 469

You can try Rselenium.

the code looks more or less like this.

library(Rselenium)
library(XML)    

remDr <- remoteDriver(port = 4445L)
remDr$open()
remDr$navigate("https://www.pverify.com/npi-lookup-find-npi-number-of-doctors-physicians/")
h <- htmlParse(remDr$getPageSource()[[1]], encoding = "UTF-8")
h_table <- html_table(h)

To create a docker server you can see here

Upvotes: 1

Related Questions