CODEWITHSUNDEEP
pythonlistseleniumscrollinstagram
Kolya Pavlenko
Kolya Pavlenko

Reputation: 85

Scroll down list instagram selenium and python

I write a script for instagram. And i need a method which returns me a list of followers. My followers is not visible(only 10) and i must to scroll down the page. I am using selenium webdriver and python to automate this process. But unfortunately it doesn't scroll down.Here is my code

    def get_followers(self):
    try:
        driver.find_elements_by_css_selector('a._t98z6')[0].click()
    except Exception as e:
        print("Sorry, i don't have access to your followers: {0}".format(e))
    else:
    followers = []
                while True:
            driver.execute_script("window.scrollTo(0, document.body.scrollHeight)")
            try:
                WebDriverWait(driver, 20).until(lambda x: x.find_element_by_css_selector("li._6e4x5"))
            except:
                break
        followers = driver.find_elements_by_css_selector("a._2g7d5.notranslate._o5iw8")

     return followers

Any solutions would be much appreciated. Thanks.

Upvotes: 3

Views: 6588

Answers (4)

Michael Wegter
Michael Wegter

Reputation: 127

I too have been trying to find a way to scroll through the followers popup or dialogue box, and have not found a way to focus on the followers box in order to use the scroll function I usually use in WebDriver. I worked around that by simply sending the ARROW_DOWN key after clicking the followers link until it scrolls all the way down, which is noted by the counting list and the full list remaining the same after the last ARROW_DOWN keys. I believe the bit about the dialogue box is extraneous, but meh. Here is my code:

    def listfollowers (instaURL):
actions = ActionChains(driver)
assert isinstance(instaURL, object)
driver.get(instaURL)
time.sleep(3)  # Let the user actually see something!
followersbutton = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[href*='followers']")))
followersbutton.click()
time.sleep(2)
dialoguebox = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "body > div:nth-child(14) > div > div.zZYga > div > div.j6cq2 > ul > div")))
actions.move_to_element(dialoguebox)
actions.click()
actions.perform()
actions.reset_actions()

###Below is what you need to list followers and scroll through the followers dialogue box

followerlist = []
scrollfollowercount = driver.find_elements_by_class_name("UYK0S")
while len(followerlist) < len(scrollfollowercount):
profiles = driver.find_elements_by_class_name("UYK0S") #the followers
    for profile in profiles:
        profileurl = profile.get_attribute('href')
        followerlist.append(profileurl)
        actions.send_keys(Keys.ARROW_DOWN)
        actions.send_keys(Keys.ARROW_DOWN)
        actions.send_keys(Keys.ARROW_DOWN)
        actions.send_keys(Keys.ARROW_DOWN) #included a few times for good measure
        actions.perform()
        actions.reset_actions()
        scrollfollowercount = driver.find_elements_by_class_name("UYK0S")
        if len(scrollfollowercount) == len(followerlist):
            break
print(followerlist)

So for my MAIN section, I have the login bit and then call my function listfollowers()

    actions = ActionChains(driver)
driver.get("https://www.instagram.com/kapow.fitness/followers")
time.sleep(2)
username = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#react-root > section > main > div > article > div > div:nth-child(1) > div > form > div:nth-child(1) > div")))
username.click()
actions.send_keys("your-username")
actions.perform()
password = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#react-root > section > main > div > article > div > div:nth-child(1) > div > form > div:nth-child(2) > div > div.f0n8F")))
password.click()
actions.reset_actions()
actions.send_keys("your-password")
actions.send_keys(Keys.RETURN)
actions.perform()
time.sleep(2)

listfollowers("https://www.instagram.com/kapow.fitness/")

Upvotes: 0

HsnVahedi
HsnVahedi

Reputation: 1366

followers_panel = browser.find_element_by_xpath(XPATH)
i = 1
while i < number_of_followers:
    try:
        follower = browser.find_element_by...
        i += 1
    except NoSuchElementException:
        self.browser.execute_script(
            "arguments[0].scrollTop = arguments[0].scrollHeight",followers_panel
        )

This way you can crawl all of the followers.

Upvotes: 5

Daniel
Daniel

Reputation: 5

Here is what works. Don't change the sleep times for they allow the scroller to reload new followers without having to scroll it back to the top again. 

    FList = driver.find_element_by_css_selector('div[role=\'dialog\'] ul')
    numberOfFollowersInList = len(FList.find_elements_by_css_selector('li'))

    FList.click()
    actionChain = webdriver.ActionChains(driver)
    time.sleep(random.randint(2,4))

    while (numberOfFollowersInList < max):
        actionChain.key_down(Keys.SPACE).key_up(Keys.SPACE).perform()        
        numberOfFollowersInList = len(FList.find_elements_by_css_selector('li'))
        time.sleep(0.4)
        print(numberOfFollowersInList)
        actionChain.key_down(Keys.SPACE).key_up(Keys.SPACE).perform()            
        time.sleep(1)

Upvotes: 0

Mahesh
Mahesh

Reputation: 266

I managed to scroll using ActionChain to scroll. But as the list grows and the speed of you computer+Internet, it becomes slow. At first there are only 20-24 names the n 10 names per scroll you can get. Then I used to click the last element as you click the last element, 10 new users appears the you click the last element again. So it goes like this.

from selenium.webdriver.common.action_chains import ActionChains

def get_list():
    element=[]
    ran_num=int(random.randint(0,len(subject)-1))
    search(subject[ran_num])
    br.find_element_by_class_name("_e3il2").click() #open first image
    time.sleep(2)
    br.find_element_by_partial_link_text('likes').click()
    time.sleep(2)

    while len(element)<150:
            element=br.find_elements_by_xpath("//*[@class='_9mmn5']")
            i=len(element)-1
            element[i].click()
            time.sleep(1.50)

    likers=br.find_elements_by_xpath("//*[@class='_2g7d5 notranslate _o5iw8']") #get the username
    for i in range(len(likers)):
            insta_id=likers[i].text
            if (insta_id not in main_list):
                main_list.append(insta_id)
                with open ('to_like.txt','a') as f:
                    f.write('%s\n'%insta_id)

    return()

Upvotes: 1

Related Questions