CODEWITHSUNDEEP
javajavascriptseleniumyoutube
Mustafa
Mustafa

Reputation: 981

How to read youtube comments using Selenium?

I'm trying to read youtube video comments using the following code:

FirefoxDriver driver = new FirefoxDriver();
driver.get("https://www.youtube.com/watch?v=JcbBNpYkuW4");

WebElement element = driver.findElementByCssSelector("#watch-discussion");
System.out.println(element.getText()); // this prints: loading..

// scrolll down so that comments start to load
driver.executeScript("window.scrollBy(0,500)", "");

Thread.sleep(10000);

element = driver.findElementByCssSelector("#watch-discussion");
System.out.println(element.getText());

Last statement prints an empty string. Why?

Upvotes: 0

Views: 1028

Answers (2)

Naveen Ramawat
Naveen Ramawat

Reputation: 1445

It would be little tricky because all the comments are written in a separate iframe tag inside watch discussion. You will have to switch on that iframe first using driver.switchTo().frame("put ID or Name here"); but the iframe id is random value. After switch to that iframe you can find the comments all comments in a div that have class name 'Ct' so you can get those using XPATH. see the below working code

FirefoxDriver driver = new FirefoxDriver();
driver.get("https://www.youtube.com/watch?v=JcbBNpYkuW4");

WebElement element = driver.findElementByCssSelector("#watch-discussion");
System.out.println(element.getText()); // this prints: loading..

// scrolll down so that comments start to load
driver.executeScript("window.scrollBy(0,500)", "");

Thread.sleep(20000);

List<WebElement> iframes = driver.findElements(By.xpath("//iframe"));

for(WebElement e : iframes) {       
    if(e.getAttribute("id") != null && e.getAttribute("id").startsWith("I0_")) {
    // switch to iframe which contains comments
    driver.switchTo().frame(e);
    break;
    }
}

// fetch all comments
List<WebElement> comments = driver.findElements(By.xpath("//div[@class='Ct']"));
for(WebElement e : comments) {      
    System.out.println(e.getText());
}

Upvotes: 1

Stunner
Stunner

Reputation: 1121

I suggest you to try this API which is very easy/reliable instead of relying on the X-path of the elements. Also you cannot rely on the Xpath for dynamic pages/content.

Upvotes: 1

Related Questions