CODEWITHSUNDEEP
python-3.xweb-scrapingbeautifulsoup
Saman Hamidi
Saman Hamidi

Reputation: 358

find() in Beautifulsoup returns None

I'm very new to programming in general and I'm trying to write my own little torrent leecher. I'm using Beautifulsoup In order to extract the title and the magnet link of a torrent file. However find() element keeps returning none no matter what I do. The page is correct. I've also tested with find_next_sibling and read all the similar questions but to no avail. Since there are no errors I have no idea what my mistake is. Any help would be much appreciated. Below is my code: 

import urllib3
from bs4 import BeautifulSoup


print("Please enter the movie name: \n")
search_string = input("")  
search_string.rstrip() 
search_string.lstrip() 
open_page = ('https://www.yify-torrent.org/search/' + search_string +     '/s-1/all/all/')  # get link - creates a search string with input value
print(open_page)
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
manager = urllib3.PoolManager(10)
page_content = manager.urlopen('GET',open_page)
soup = BeautifulSoup(page_content,'html.parser')  
magnet = soup.find('a', attrs={'class': 'movielink'}, href=True)  
print(magnet)

Upvotes: 1

Views: 1875

Answers (1)

SIM
SIM

Reputation: 22440

Check out the following script which does exactly what you wanna achieve. I used requests library instead of urllib3. The main mistake you made is that you looked for the magnet link in the wrong place. You need to go one layer deep to dig out that link. Try using quote instead of string manipulation to fit your search query within the url.

Give this a shot:

import requests
from urllib.parse import urljoin
from urllib.parse import quote
from bs4 import BeautifulSoup

keyword = 'The Last Of The Mohicans'

url = 'https://www.yify-torrent.org/search/'
base = f"{url}{quote(keyword)}{'/p-1/all/all/'}"

res = requests.get(base)
soup = BeautifulSoup(res.text,'html.parser')  
tlink = urljoin(url,soup.select_one(".img-item .movielink").get("href"))
req = requests.get(tlink)
sauce = BeautifulSoup(req.text,"html.parser")
title = sauce.select_one("h1[itemprop='name']").text
magnet = sauce.select_one("a#dm").get("href")
print(f"{title}\n{magnet}")

Upvotes: 2

Related Questions