CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoup
Tom
Tom

Reputation: 464

scrape map coordinates using beautifulsoup

I am trying to scrape the coordinates where a flicker photo was taken. I tried catching this 'a' block:

<a class="static-maps" href="https://www.flickr.com/map/?fLat=13.387866&amp;fLon=77.699174&amp;zl=13&amp;everyone_nearby=1" data-rapid_p="163">

using the following code:

url='https://www.flickr.com/photos/hellosaurav/8739282947/in/photolist-ayo8gy-brAbpk-nREjXv-eyQCtp-ovie9F-rdhF3m-eB8g6z-a3jhb9-9jUqhk-evcaBQ-j7iARL-oFd27B-cZ4VaN-mfP6NR-odhcpL-hy2vMX-mHGWoM-n9ARnM-9rxT1W-oqPqDQ-6tmgQ1-oNbZXw-pogsa7-eAeMz9-asB1Qu-o3qgcx-pr6ZGC-dfTh3p-pRuMsf-9yqjrG-bS4AkB-5iDTpA-pSVfhM-ejg7mc-oKWkZX-vDvqdR-nvb2zt-oYDWki-chB5ZY-p14ReR-oJSier-n9MyRk-rGAdSf-exgySN-sFkcTb-hE2tfg-ryeRC5-rqYLen-7zAafa-p3vS3U/'
r=requests.get(url)
url=r.content

soup = BeautifulSoup(url,'html.parser')

#header
header=soup.find("div",{"class":"title-desc-block"}).find("h1")
if(header==None):
    return
else:
    header=header.text.encode("utf-8").strip().replace(',','|')

amap=soup.find("a",{"class":"static-maps"})

print amap

The code print "None".

Anyone has an idea why beautifulsoup can't find this link?

Upvotes: 0

Views: 425

Answers (1)

furas
furas

Reputation: 142889

This link is created by JavaScript. BS doesn't render pages so it can't run JavaScript.

Coordinates are in this file as JavaScript/Text so you can try to find it.
But BS can't help you in this job. Use regular expressions.

import re

print re.findall('"latitude":(.+?),', r.content)
print re.findall('"longitude":(.+?),', r.content)

Upvotes: 2

Related Questions