CODEWITHSUNDEEP
pythonseleniumweb-scrapinginstagram
userHG
userHG

Reputation: 589

Scraping Instagram followers for an account

I'm trying to scrape followers from an Instagram account by using Selenium. (https://www.instagram.com/france/followers/)

I need to scroll down a popup page but the only thing I can scrolldown is the back page.

Here is my code

scr1 = driver.find_element_by_xpath('/html/body/div[4]/div/div')
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);",scr1)  ## Scroll to bottom of page with using driver

I also tried in my JS browser console by using the JSPath of my dialog modal :

window.scrollTo(0, document.querySelector("body > div.RnEpo.Yx5HN > div"));

(Already saw Stack posts related but answers seems deprecated in Oct. 2020)

Upvotes: 0

Views: 635

Answers (1)

Rohan Shah
Rohan Shah

Reputation: 489

I've had similar issues with what you are trying to do. Currently, the method I have implemented in my script uses Selenium's ActionChains class which I find is much more helpful than all the Javascript (execute_script) answers out there.

Basically I use ActionChains to press the "down" arrow key and then manipulate it to my advantage. I'll lay out the basic syntax below:

from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains
actions = ActionChains(browser)
actions.send_keys(Keys.DOWN).perform() #sends a general down arrow key to the page

Now I focus the ActionChains on a specific element, in this case it's the popup. There's a ton of ways to focus on the element but I find the easiest workaround is simply clicking the element before using ActionChains

Like this:

popup = driver.find_element_by_class_name('popupClass') #you don't have to use class_name
for i in range(500): #500 is amount of scrolls
    popup.click() #focus the down key on the popup
    actions.send_keys(Keys.DOWN).perform() # press the down key

The above method will click on the popup and then press the down key once, essentially sending 500 "scrolls" to the popup rather than the whole page. Once it's done, you're gonna wanna grab the .text property on the element responsible for the popups (at least that's what I did in my script).

Upvotes: 1

Related Questions