CODEWITHSUNDEEP
pythonbeautifulsoup
Little Chicken
Little Chicken

Reputation: 265

BeautifulSoup cannot parse the html tags which don't have closing element

Here is the HTML code I working on it

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>sdasdsadsad</title>
<link rel="alternate" media="only screen and (max-width: 640px)" href="local:80" />
<meta name="description" content="sdddsdsdsdsdsd">
<meta name="keywords" content="3333333333333333">
<meta property="og:title" content="444444444444444444444444">
<meta property="og:type" content="article">
<meta property="og:description" content="dsdsdsdsddsds">

</head>
<body></body>
</html>

I want to get the line contains tag "<meta name = description" , which doesn't have close element </meta>. There is my code

import glob, os, re, urllib2, codecs
from bs4 import BeautifulSoup
from bs4 import SoupStrainer


html_doc = """
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>sdasdsadsad</title>
<link rel="alternate" media="only screen and (max-width: 640px)" href="local:80" />
<meta name="description" content="sdddsdsdsdsdsd">
<meta name="keywords" content="3333333333333333">
<meta property="og:title" content="444444444444444444444444">
<meta property="og:type" content="article">
<meta property="og:description" content="dsdsdsdsddsds">

</head>
<body></body>
</html>
"""



soup = BeautifulSoup(html_doc)
aa = soup.find("meta", {"name":"description"})
print aa.encode("utf-8")

Running the Python code, but the console show 

<meta content="sdddsdsdsdsdsd" name="description">
<meta content="3333333333333333" name="keywords">
<meta content="444444444444444444444444" property="og:title">
<meta content="article" property="og:type">
<meta content="dsdsdsdsddsds" property="og:description">
</meta></meta></meta></meta></meta>

But if "<meta content="sdddsdsdsdsdsd" name="description">" has close element </meta>, I can get exactly the line:

<meta content="sdddsdsdsdsdsd" name="description"> </meta>

Would you like to tell me why the reason BeautifulSoup get all HTML tag under <meta name = description , and how to get the line contains <meta name = description

Thanks.

Upvotes: 0

Views: 1251

Answers (1)

PepperoniPizza
PepperoniPizza

Reputation: 9112

Use the lxml module as the parser and it will work, I've tested it.

from bs4 import BeautifulSoup

soup = BeautifulSoup(html_doc, 'lxml')
aa = soup.find("meta", {"name":"description"})

print aa.encode('utf-8')

# console output
<meta content="sdddsdsdsdsdsd" name="description"/>

Upvotes: 2

Related Questions