CODEWITHSUNDEEP
pythonselenium
user20207408
user20207408

Reputation:

Python problem with downloading links from www

I have a problem with the code and I admit I don't understand what is happening. I have from the download page 13 links. Up to this point self.img = driver.find_element(By.XPATH, self.link_photo) everything is ok. Further after displaying 5 it stops working. It seems to be doing something strange here. It seems to be doing something strange here 'self.imgURL = self.img.get_attribute('srcset')'.Can someone help me fix this code? Thank you very much for your help.

    # Download a link of each photo
    self.table_of_mini_photo = []
    for num, i in enumerate(range(1, int(self.total)+20)):
        self.link_photo = f'//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[{i}]/a/div/div/div[1]/div[1]/div/img'
    # //*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[2]/a/div/div/div[1]/div[1]/div/img
    # //*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[3]/a/div/div/div[1]/div[1]/div/img
    # //*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[4]/a/div/div/div[1]/div[1]/div/img
    # //*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[6]/a/div/div/div[1]/div[1]/div/img
    # //*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[7]/a/div/div/div[1]/div[1]/div/img
    # //*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[8]/a/div/div/div[1]/div[1]/div/img
    # //*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[10]/a/div/div/div[1]/div[1]/div/img
    # //*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[12]/a/div/div/div[1]/div[1]/div/img
    # //*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[17]/a/div/div/div[1]/div[1]/div/img
    # //*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[15]/a/div/div/div[1]/div[1]/div/img
    # //*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[16]/a/div/div/div[1]/div[1]/div/img
    # //*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[15]/a/div/div/div[1]/div[1]/div/img
        try:
            print(self.link_photo)
            self.img = driver.find_element(By.XPATH, self.link_photo)
            print(self.img)
            self.imgURL = self.img.get_attribute('srcset')
            print(self.imgURL)
            self.table_of_mini_photo.append(self.imgURL)
        except:
            pass

The console result confirms that I get 13 'selenium : that is links' but physically I get 5. Behind the commented in code are XPath that I download.

//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[1]/a/div/div/div[1]/div[1]/div/img
//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[2]/a/div/div/div[1]/div[1]/div/img
<selenium.webdriver.remote.webelement.WebElement (session="8cb74bec440fc1d3621f9a5a995c27d6", element="671dace6-a8c2-4204-a164-1266b9385073")>
https://ireland.apollo.olxcdn.com:443/v1/files/9li55l5fl99q3-PL/image;s=100x0;q=50 100w,
        https://ireland.apollo.olxcdn.com:443/v1/files/9li55l5fl99q3-PL/image;s=200x0;q=50 200w,
        https://ireland.apollo.olxcdn.com:443/v1/files/9li55l5fl99q3-PL/image;s=300x0;q=50 300w,
        https://ireland.apollo.olxcdn.com:443/v1/files/9li55l5fl99q3-PL/image;s=400x0;q=50 400w,
        https://ireland.apollo.olxcdn.com:443/v1/files/9li55l5fl99q3-PL/image;s=600x0;q=50 600w
//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[3]/a/div/div/div[1]/div[1]/div/img
<selenium.webdriver.remote.webelement.WebElement (session="8cb74bec440fc1d3621f9a5a995c27d6", element="431518f9-0ee5-4317-a200-39bd7840939e")>
https://ireland.apollo.olxcdn.com:443/v1/files/mbbgmgrg6z343-PL/image;s=100x0;q=50 100w,
        https://ireland.apollo.olxcdn.com:443/v1/files/mbbgmgrg6z343-PL/image;s=200x0;q=50 200w,
        https://ireland.apollo.olxcdn.com:443/v1/files/mbbgmgrg6z343-PL/image;s=300x0;q=50 300w,
        https://ireland.apollo.olxcdn.com:443/v1/files/mbbgmgrg6z343-PL/image;s=400x0;q=50 400w,
        https://ireland.apollo.olxcdn.com:443/v1/files/mbbgmgrg6z343-PL/image;s=600x0;q=50 600w
//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[4]/a/div/div/div[1]/div[1]/div/img
<selenium.webdriver.remote.webelement.WebElement (session="8cb74bec440fc1d3621f9a5a995c27d6", element="07505827-b5fb-4af1-92f5-5391fdbf12d2")>
https://ireland.apollo.olxcdn.com:443/v1/files/w53tb2o5ili71-PL/image;s=100x0;q=50 100w,
        https://ireland.apollo.olxcdn.com:443/v1/files/w53tb2o5ili71-PL/image;s=200x0;q=50 200w,
        https://ireland.apollo.olxcdn.com:443/v1/files/w53tb2o5ili71-PL/image;s=300x0;q=50 300w,
        https://ireland.apollo.olxcdn.com:443/v1/files/w53tb2o5ili71-PL/image;s=400x0;q=50 400w,
        https://ireland.apollo.olxcdn.com:443/v1/files/w53tb2o5ili71-PL/image;s=600x0;q=50 600w
//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[5]/a/div/div/div[1]/div[1]/div/img
//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[6]/a/div/div/div[1]/div[1]/div/img
<selenium.webdriver.remote.webelement.WebElement (session="8cb74bec440fc1d3621f9a5a995c27d6", element="0d7a591e-e318-41c6-9e2f-2519bbaeb212")>
https://ireland.apollo.olxcdn.com:443/v1/files/yjsjx3lv5hnx-PL/image;s=100x0;q=50 100w,
        https://ireland.apollo.olxcdn.com:443/v1/files/yjsjx3lv5hnx-PL/image;s=200x0;q=50 200w,
        https://ireland.apollo.olxcdn.com:443/v1/files/yjsjx3lv5hnx-PL/image;s=300x0;q=50 300w,
        https://ireland.apollo.olxcdn.com:443/v1/files/yjsjx3lv5hnx-PL/image;s=400x0;q=50 400w,
        https://ireland.apollo.olxcdn.com:443/v1/files/yjsjx3lv5hnx-PL/image;s=600x0;q=50 600w
//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[7]/a/div/div/div[1]/div[1]/div/img
<selenium.webdriver.remote.webelement.WebElement (session="8cb74bec440fc1d3621f9a5a995c27d6", element="79ea57a4-9bd5-4213-93c8-acceb60094a0")>
https://ireland.apollo.olxcdn.com:443/v1/files/5gs5bqemn8wr-PL/image;s=100x0;q=50 100w,
        https://ireland.apollo.olxcdn.com:443/v1/files/5gs5bqemn8wr-PL/image;s=200x0;q=50 200w,
        https://ireland.apollo.olxcdn.com:443/v1/files/5gs5bqemn8wr-PL/image;s=300x0;q=50 300w,
        https://ireland.apollo.olxcdn.com:443/v1/files/5gs5bqemn8wr-PL/image;s=400x0;q=50 400w,
        https://ireland.apollo.olxcdn.com:443/v1/files/5gs5bqemn8wr-PL/image;s=600x0;q=50 600w
//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[8]/a/div/div/div[1]/div[1]/div/img
<selenium.webdriver.remote.webelement.WebElement (session="8cb74bec440fc1d3621f9a5a995c27d6", element="d4c89d12-0b89-42bd-8984-8ca1be8f106e")>

//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[10]/a/div/div/div[1]/div[1]/div/img
<selenium.webdriver.remote.webelement.WebElement (session="8cb74bec440fc1d3621f9a5a995c27d6", element="5953e136-712e-43d4-a36f-55ddcc460594")>

//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[11]/a/div/div/div[1]/div[1]/div/img
//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[12]/a/div/div/div[1]/div[1]/div/img
<selenium.webdriver.remote.webelement.WebElement (session="8cb74bec440fc1d3621f9a5a995c27d6", element="80e8fd03-4fe2-4797-9b4b-1228f8f6a706")>

//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[13]/a/div/div/div[1]/div[1]/div/img
<selenium.webdriver.remote.webelement.WebElement (session="8cb74bec440fc1d3621f9a5a995c27d6", element="2b0d5ded-6480-4c43-87c2-ef9eece4f6e4")>

//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[14]/a/div/div/div[1]/div[1]/div/img
<selenium.webdriver.remote.webelement.WebElement (session="8cb74bec440fc1d3621f9a5a995c27d6", element="f480c79e-2b10-4379-8b60-26377d36e936")>

//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[15]/a/div/div/div[1]/div[1]/div/img
<selenium.webdriver.remote.webelement.WebElement (session="8cb74bec440fc1d3621f9a5a995c27d6", element="a04b82e4-7021-47d5-8746-f08765fe7b17")>

//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[16]/a/div/div/div[1]/div[1]/div/img
<selenium.webdriver.remote.webelement.WebElement (session="8cb74bec440fc1d3621f9a5a995c27d6", element="3cc93e75-aea9-4366-a1eb-c240fee343ca")>

//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[17]/a/div/div/div[1]/div[1]/div/img
<selenium.webdriver.remote.webelement.WebElement (session="8cb74bec440fc1d3621f9a5a995c27d6", element="57e77c14-1ef4-4732-bc2f-b938b1841820")>

//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[18]/a/div/div/div[1]/div[1]/div/img
//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[19]/a/div/div/div[1]/div[1]/div/img
//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[20]/a/div/div/div[1]/div[1]/div/img
//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[21]/a/div/div/div[1]/div[1]/div/img
//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[22]/a/div/div/div[1]/div[1]/div/img
//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[23]/a/div/div/div[1]/div[1]/div/img
//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[24]/a/div/div/div[1]/div[1]/div/img
//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[25]/a/div/div/div[1]/div[1]/div/img
//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[26]/a/div/div/div[1]/div[1]/div/img
//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[27]/a/div/div/div[1]/div[1]/div/img
//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[28]/a/div/div/div[1]/div[1]/div/img
//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[29]/a/div/div/div[1]/div[1]/div/img
//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[30]/a/div/div/div[1]/div[1]/div/img
//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[31]/a/div/div/div[1]/div[1]/div/img
//*[@id="root"]/div[1]/div[2]/form/div[5]/div/div[2]/div[32]/a/div/div/div[1]/div[1]/div/img

Upvotes: 0

Views: 44

Answers (1)

JeffC
JeffC

Reputation: 25744

I prefer the 911 Turbo. :)

I found a simple CSS selector that retrieves all images.

div[type='list'] > div > img

Also, you are pulling srcset which is a bunch of different sizes of the same image. If all you want is an image, you can just pull src and get a single image so you don't have to parse the string, etc.

self.imgURL = self.img.get_attribute('src')

The updated code loops through only the retrieved images, grabs and prints the 'src', and then appends it to the list.

# Download a link of each photo
self.table_of_mini_photo = []
for img in driver.find_elements(By.CSS_SELECTOR, "div[type='list'] > div > img")
    imgURL = img.get_attribute('src')
    print(imgURL)
    self.table_of_mini_photo.append(imgURL)

Upvotes: 0

Related Questions