CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoup
user16073125
user16073125

Reputation:

Web Scrape Using an Existing Browser

I'm trying to fetch data from a website but I need to log in first to be able to see it. I want to run my source code using an opened browser with my account already logged in on the particular website. Will that be possible?

Upvotes: 2

Views: 2189

Answers (1)

Yash
Yash

Reputation: 220

Yes it is possible. You need to use selenium along with chrome dev tools. The method will generate an instance of chrome, in which you have to login into the site at first sight manually. And then whenever you want you need to do some tasks the instance will open directly when called by selenium script.

Condition : The site must have feature to remember your login info. (Eg. Youtube, Instagram, Twitter, Facebook .etc.) These sites remember your login even if the browser exits. Some sites may not have this functionality.

For more clarification checkout

Youtube video - https://www.youtube.com/watch?v=FVumnHy5Tzo&ab_channel=HelloWorld

Stack overflow post - https://stackoverflow.com/a/57001041/9807249

Chrome devtools - https://chromedevtools.github.io/devtools-protocol/

I used this technique to autopost youtube videos.

Upvotes: 1

Related Questions