CODEWITHSUNDEEP
rweb-scrapingrselenium
Canovice
Canovice

Reputation: 10163

R - using RSelenium to log into website (Captcha, and staying logged in)

I want to use RSelenium to access and scrape a website each day. Something I've noticed is that when I open up the website in a regular chrome browser, I am already logged in from the last time I visited the website. However, if I use RSelenium to open up a remote driver, and visit the webpage using this driver, it does not have me logged into the website already. It's basic enough to log into most sites usually, however for this website there is a Captcha that makes logging in more difficult.

Is there anyway the remote driver can access the website with me already logged in?

Example of my code below:

this_URL = "my_url_goes_here"
startServer()

remDr = remoteDriver$new(browserName = 'chrome')
Sys.sleep(2); remDr$open();
Sys.sleep(4); remDr$navigate(this_URL);

login_element = remDr$findElement(using = "id", "login-link")
login_element$

After clicking the login_element link, it brings me to the page where I input my username, password, and click the captcha / do what it asks.

Thanks,

Upvotes: 2

Views: 1764

Answers (1)

Rentrop
Rentrop

Reputation: 21497

It should work using firefox and firefox profiles as follows:

  1. Setup Firefxx Access: Open firefox and login as usual. Make sure when you close firefox and you login again you stay logged in.

  2. Figure out the location of your default firefox profile: This should be somethink like: (source + more details)

    • Windows: %AppData%MozillaFirefoxProfilesxxxxxxxx.default
    • Mac: ~/.mozilla/firefox/xxxxxxxx.default/
    • Linux: ~/Library/Application Support/Firefox/Profiles/xxxxxxxx.default/

  3. Start a new RSelenium driver and set the profile as follows

->

require(RSelenium)
eCap <- list("webdriver.firefox.profile" = "MySeleniumProfile")
remDr <- remoteDriver(browserName = "firefox", extraCapabilities = eCap)
remDr$open()

The firefox-window that opens should be your chosen profile.

I did this a while ago. If i remember correctly it works like this.

P.S.: You could also create an extra/new firefox profile for that. To do that follow the steps in the link above

Upvotes: 4

Related Questions