Retrieve image links using BeautifulSoup

Question

I have the following html code for which I would like to extract data-img links.

<a rel="popover" data-img="https://www.johnpyelots.co.uk/Sales/Sale - 
4709/Pictures/1001.jpg" href="lot_details.asp?
l=1&amp;lotid=3357969&amp;pageno=1" class="blacklink2" data-original-
title=""><img border="0" src="https://www.johnpyelots.co.uk/Sales/Sale - 
4709/Thumbnails/thumb_1001.jpg" width="70" <="" a=""></a>

I am using the following Python code but I can't seem to extract the links:

urldes = "https://www.johnpyeauctions.co.uk/lot_list.asp?saleid=4709&siteid=1"

# add header
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36'} 
r = requests.get(urldes, headers=headers) 
soup = BeautifulSoup(r.content, "lxml")

mylinks = []
for link in soup.find_all('a'):
    mylinks.append(link['data-image'])

for i in range(len(mylinks)):
    mylinks[i]

mylinks_0 = mylinks[0]

Any ideas?

Answer 1

There are many links in the web page which are all inside the <a> tag. So, if you want only the <a> tags that have the image links, you'll need to specify more parameters to the find_all() method.

Also, have a look at List Comprehensions in Python.

import requests
from bs4 import BeautifulSoup

r = requests.get('https://www.johnpyeauctions.co.uk/lot_list.asp?saleid=4709&siteid=1')
soup = BeautifulSoup(r.text, 'lxml')
image_links = [x['data-img'] for x in soup.find_all('a', rel='popover')]
for link in image_links:
    print(link)

Output:

https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1001.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1002.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1003.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1004.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1005.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1006.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1007.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1008.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1009.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1010.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1011.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1012.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1013.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1014.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1015.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1016.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1017.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1018.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1019.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1020.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1021.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1022.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1023.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1024.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1025.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1026.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1027.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1028.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1029.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1030.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1031.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1032.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1033.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1034.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1035.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1036.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1037.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1038.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1039.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1040.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1041.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1042.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1043.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1044.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1045.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1046.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1047.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1048.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1049.jpg
https://www.johnpyelots.co.uk/Sales/Sale - 4709/Pictures/1050.jpg