CODEWITHSUNDEEP
pythonbeautifulsoup
Cmag
Cmag

Reputation: 15770

python BeautifulSoup getting variables from a garbage page

Folks, Trying to grab some variables from a poorly formatted page.

html =  response.read()
soup = BeautifulSoup(html)
links = soup.findAll('a')

for link in links:
    for x in link.attrs:
       print x

output:

(u'href', u"javascript:Set_Variables('FIRSTNAME,LASTNAME', \r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'123456789123', \r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'FOOOOOOO',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'54',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'2014',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'BAZZZZ',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'BARRRRRRRRRR',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'07/31/2015',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'')")
(u'onmouseover', u"javascript: return window.status=''")
(u'href', u"javascript:Set_Variables('FIRSTNAME,LASTNAME', \r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'123456789123', \r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'FOOOOOOO',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'54',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'2014',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'BAZZZZ',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'BARRRRRRRRRR',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'07/31/2015',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'')")
(u'onmouseover', u"javascript: return window.status=''")

Question: How would I grab the FIRSTNAME,LASTNAME , FOOOOOOO, BARRRRR, BAZZZZZ, 123456789123 from all this mess?

Thanks!

Upvotes: 0

Views: 57

Answers (1)

Martijn Pieters
Martijn Pieters

Reputation: 1123032

First of all, you only need to focus on the href attributes here.

Take everything between the parentheses, split on whitespace and remove the comma and quotes:

args = link['href'].partition('(')[-1].rpartition(')')[0]
args = [v.rstrip(',').strip("'") for v in args.split()]

Demo:

>>> href = u"javascript:Set_Variables('FIRSTNAME,LASTNAME', \r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'123456789123', \r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'FOOOOOOO',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'54',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'2014',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'BAZZZZ',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'BARRRRRRRRRR',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'07/31/2015',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'')"
>>> href.partition('(')[-1].rpartition(')')[0]
u"'FIRSTNAME,LASTNAME', \r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'123456789123', \r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'FOOOOOOO',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'54',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'2014',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'BAZZZZ',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'BARRRRRRRRRR',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t'07/31/2015',\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t''"
>>> [v.rstrip(',').strip("'") for v in href.partition('(')[-1].rpartition(')')[0].split()]
[u'FIRSTNAME,LASTNAME', u'123456789123', u'FOOOOOOO', u'54', u'2014', u'BAZZZZ', u'BARRRRRRRRRR', u'07/31/2015', u'']

Upvotes: 1

Related Questions