CODEWITHSUNDEEP
pythonselenium
Praveen Kumar
Praveen Kumar

Reputation: 959

This python selenium program does not iterate

I would like to iterate the function using a while loop, but it iterates only once, and after that, the program just stands still.

from selenium import webdriver
from multiprocessing import Process


def browse(url):
    driver = webdriver.Chrome()
    driver.get(url)
    print(driver.page_source)
    driver.__exit__()

Pros = []
urls = open('urls.txt')

if __name__ == '__main__':
    while True:
        for url_item in urls:
            print(url_item)
            p = Process(target=browse, args=(url_item,))
            Pros.append(p)
            p.start()
        for t in Pros:
            t.join()

Upvotes: 0

Views: 60

Answers (1)

Deyuan
Deyuan

Reputation: 56

The main problem relies on how the file's content was read as well as how the Process object has been started. It starting on a temporary Process object instead of the Process object in process list (Pros). In short, it is only a referencing issue and wrong way of utlizing File Object. Here is the working code.

from selenium import webdriver
from multiprocessing import Process


def browse(URL):
    driver = webdriver.Chrome()
    driver.get(URL)
    print(driver.page_source)
    driver.__exit__()

Pros = []
urls = open('urls.txt').readlines()

if __name__ == '__main__':
    while True:
        print("testing")
        for url_item in urls:
            print(url_item)
            p = Process(target=browse, args=(url_item,))
            Pros.append(p)
            Pros[-1].start()
        for t in Pros:
            t.join()
        Pros = []

Upvotes: 1

Related Questions