CODEWITHSUNDEEP
pythonregexunicode
Remy
Remy

Reputation: 139

Regular expressions in python unicode

I need to remove all the html tags from a given webpage data. I tried this using regular expressions:

import urllib2
import re
page = urllib2.urlopen("http://www.frugalrules.com")
from bs4 import BeautifulSoup, NavigableString, Comment
soup = BeautifulSoup(page)
link = soup.find('link', type='application/rss+xml')
print link['href']
rss = urllib2.urlopen(link['href']).read()
souprss = BeautifulSoup(rss)
description_tag = souprss.find_all('description')
content_tag = souprss.find_all('content:encoded')
print re.sub('<[^>]*>', '', content_tag)

But the syntax of the re.sub is:

re.sub(pattern, repl, string, count=0)

So, I modified the code as (instead of the print statement above):

for row in content_tag:
    print re.sub(ur"<[^>]*>",'',row,re.UNICODE

But it gives the following error:

Traceback (most recent call last):

File "C:\beautifulsoup4-4.3.2\collocation.py", line 20, in <module>
print re.sub(ur"<[^>]*>",'',row,re.UNICODE)
File "C:\Python27\lib\re.py", line 151, in sub
return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or buffer

What am I doing wrong?

Upvotes: 1

Views: 247

Answers (1)

Qui
Qui

Reputation: 108

Last line of your code try:

print(re.sub('<[^>]*>', '', str(content_tag)))

Upvotes: 1

Related Questions