CODEWITHSUNDEEP
pythonselenium
user12453183
user12453183

Reputation:

Cycle trough URLs from a txt

This is my first question so please bear with me (I have googled this and I did not find anything)

I'm making a program which goes to a url, clicks a button, checks if the page gets forwarded and if it does saves that url to a file.

So far I've got the first two steps done but I'm having some issues.

I want Selenium to repeat this process with multiple urls (if possible, multiple at a time). I have all the urls in a txt called output.txt

At first I did

url_list = https://example.com

to see if my program even worked, and it did however I am stuck on how to get it to go to the next URL in the list and I am unable to find anything on the internet which helps me.

This is my code so far

import selenium
from selenium import webdriver

url_list = "C\\user\\python\\output.txt"

def site():
    driver = webdriver.Chrome("C:\\python\\chromedriver")
    driver.get(url_list)
    send = driver.find_element_by_id("NextButton")
    send.click()

    if (driver.find_elements_by_css_selector("a[class='Error']")):
        print("Error class found")

I have no idea as to how I'd get selenium to go to the first url in the list then go onto the second one and so forth. If anyone would be able to help me I'd be very grateful.

Upvotes: 2

Views: 66

Answers (1)

QuantumLicht
QuantumLicht

Reputation: 2183

I think the problem is that you assumed the name of the file containing the url, is a url. You need to open the file first and build the url list. According to the docs https://selenium.dev/documentation/en/webdriver/browser_manipulation/, get expect a url, not a file path.

import selenium
from selenium import webdriver
with open("C\\user\\python\\output.txt") as f:
 url_list = f.read().split('\n')

def site():
    driver = webdriver.Chrome("C:\\python\\chromedriver")
    for url in url_list:
       driver.get(url)
       send = driver.find_element_by_id("NextButton")
       send.click()

       if (driver.find_elements_by_css_selector("a[class='Error']")):
          print("Error class found")

Upvotes: 1

Related Questions