CODEWITHSUNDEEP
rweb-scraping
Mary Nastase
Mary Nastase

Reputation: 121

Web Scraping in R when you have inputs

I'm trying to write a code for web scraping in R when you have to introduce inputs. Exactly, I have a platform where I need to complete 2 fields and after that click submit and get the results. But I don't know how to use my columns in R like inputs in platform. I searched for an example but I did't find any. Pls, if anyone can give me a simple e.g.

Thank you

EDIT: I don't have a code yet. I was looking for an example where you can use input for complete a field on a site and after that to scrape the result. enter image description here

In the photo are the fields on my URL. So, in R I have a dataframe with 2 columns. One for CNP/CUI and one for VIN/SASIU with 100 rows or more. And I want to use this columns like input and take the output for every row.

EDIT2:

The example provided by @Dominik S.Meier it worked for me when I had a list for inputs. For column inputs I will post another question. But, till then I want to mention few thing that helped me, maybe it will hep somebody else.

  1. You need to be sure that all the versions matches: R version, browser version, browser driver version, Java version. For me it didn't match chromedriver version, even if I downloaded the right version. The problem was that I had 3 chromeversion and I think it didn't choose the right. I fixed with: rD <- rsDriver(browser = c("chrome"),port = 4444L,chromever = "83.0.4103.39"). More info here:enter link description here

  2. Because one element didn't have id like in e.g. webElem <- remDr$findElement(using = "id", "trimite"), I used css selector. You can find the css selector with right click -> copy -> copy selector (in the html code on the page).

  3. If you don't get the results, maybe you don't use the right selector. I did that and the result was list(). Then I tried more css selector from the "above" in the html code. I don'y know if it is the right solution, but for me it worked.

Hope it will help. Thank you.

Upvotes: 1

Views: 972

Answers (1)

Ahorn
Ahorn

Reputation: 3876

Using RSelenium (see here for more infos):

library(RSelenium)
rD <- rsDriver(browser = c("firefox")) #specify browser type you want Selenium to open
remDr <- rD$client

remDr$navigate("https://pro.rarom.ro/istoric_vehicul/dosar_vehicul.aspx") # navigates to webpage

# select first input field
option <- remDr$findElement(using='id', value="inputEmail")
option$highlightElement()
option$clickElement()
option$sendKeysToElement(list("[email protected]"))

# select second input field
option <- remDr$findElement(using='id', value="inputEmail2")
option$highlightElement()
option$clickElement()
option$sendKeysToElement(list("[email protected]"))


# select second input field
option <- remDr$findElement(using='id', value="inputVIN")
option$highlightElement()
option$clickElement()
option$sendKeysToElement(list("123"))

#press key
webElem <- remDr$findElement(using = "id", "trimite")
webElem$highlightElement()
webElem$clickElement()

Upvotes: 2

Related Questions