CODEWITHSUNDEEP
pythonseleniumyoutubeuser-agentwebdriverwait
vantuong
vantuong

Reputation: 161

How to click a link within youtube comment using an user agent through Selenium and Python

I am writing a script that clicks a link in the youtube comment, it works fine, but when I combine it with the user agent it doesn't work, can someone help me?

example : link

Html : 

<a class="yt-simple-endpoint style-scope yt-formatted-string" spellcheck="false" href="/redirect?redir_token=ix1uBK3TO3bxXdcT7EvDFp-vI9p8MTU0NjQxNTExOEAxNTQ2MzI4NzE4&amp;event=comments&amp;q=https%3A%2F%2Fjulissars.itworks.com%2F&amp;stzid=Ugw6ip_QkzwyJPIq3bp4AaABAg" rel="nofollow">https://julissars.itworks.com&#65279;</a>

Code trial ( without user agent , it works ) :

from selenium import webdriver
from fake_useragent import UserAgent
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Firefox()
driver.get("https://www.youtube.com/watch?v=UJezMYvf8Ss&lc=Ugw6ip_QkzwyJPIq3bp4AaABAg")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[@class='yt-simple-endpoint style-scope yt-formatted-string' and contains(., 'julissars')]"))).click()

Code trial ( with user agent , it doesn't work ) :

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
from fake_useragent import UserAgent

useragent = UserAgent()
profile = webdriver.FirefoxProfile()
profile.set_preference("general.useragent.override", useragent.random)
driver = webdriver.Firefox(firefox_profile=profile)
driver.get("https://www.youtube.com/watch?v=UJezMYvf8Ss&lc=Ugw6ip_QkzwyJPIq3bp4AaABAg")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[@class='yt-simple-endpoint style-scope yt-formatted-string' and contains(., 'julissars')]"))).click()

Upvotes: 2

Views: 684

Answers (3)

undetected Selenium
undetected Selenium

Reputation: 193088

To click on the desired comment with text as https://julissars.itworks.com within the url you need to induce WebDriverwait for the element to be clickable and you can use the following solution using useragent through Selenium and Python:

  • Code Block:

    from selenium import webdriver
from fake_useragent import UserAgent
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

ua = UserAgent()
options = webdriver.ChromeOptions()
userAgent = ua.random
print(userAgent)
options.add_argument('user-agent=' + userAgent)
driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
driver.get("https://www.youtube.com/watch?v=UJezMYvf8Ss&lc=Ugw6ip_QkzwyJPIq3bp4AaABAg")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[@class='yt-uix-sessionlink  ' and contains(@href, 'julissars')]"))).click()

  • Console Output:

    Mozilla/5.0 (X11; NetBSD) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36

  • You can find a detailed discussion in Way to change Google Chrome user agent in Selenium?

Upvotes: 2

Mykola Zotko
Mykola Zotko

Reputation: 17794

The linked webpage doesn’t support all browsers (user agents). If it’s your problem you can choose a few supported user agents and rotate between them (for example with random.choice()). You need to update your xpath as well. 

from selenium import webdriver
from fake_useragent import UserAgent
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

useragent = UserAgent()
profile = webdriver.FirefoxProfile()
profile.set_preference("general.useragent.override", useragent.random)
driver = webdriver.Firefox(firefox_profile=profile)

driver.get("https://www.youtube.com/watch?v=UJezMYvf8Ss&lc=Ugw6ip_QkzwyJPIq3bp4AaABAg")

xpath = '/html/body/div[2]/div[4]/div/div[5]/div[2]/div[2]/div/div[2]/div[5]/div/div[2]/section[1]/div[1]/div[2]/div[2]/div[1]/a'

try:
    element = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, xpath)))
finally:
    element.click()

Upvotes: 1

s3cur3
s3cur3

Reputation: 3025

Google very actively blocks bots from interacting with most of their properties. If the user agent string you're sending really is random (that is, it doesn't correspond to a real UA that a browser might send), you're probably getting caught in their bot detection algorithms.

Thus, your click is probably "working" in the sense that it's actually triggering an event in the web browser... but Google is probably ignoring the request.

Upvotes: 1

Related Questions