CODEWITHSUNDEEP
pythonhtmlterminalbeautifulsoupgeektool
Kyle Johnson
Kyle Johnson

Reputation: 1

Format terminal text output as table

I've managed to get to a point where I have used to BeautifulSoup to extract a table from a url. At this point I want to format the output as a table so that I can use it in GeekTool.

from bs4 import BeautifulSoup
import urllib2
wiki = "https://www.google.com/maps/place?q=type:transit_station:%22145+St%22&ftid=0x89c2f67c67a250f9:0x92d51daa07480dd1"
header = {'User-Agent': 'Mozilla/5.0'} #Needed to prevent 403 error on Wikipedia
req = urllib2.Request(wiki,headers=header)
page = urllib2.urlopen(req)
soup = BeautifulSoup(page)

desination = ""
eta = ""
table = soup.find("table", { "class" : "pprtjt" })


for row in table.findAll("tr"):
    for cell in row.findAll("td"):
       print cell.findAll(text=True)

which outputs the following: 

[u'   C to 168 St  ']
[u'2 min']
[u'   D to Norwood - 205 St  ']
[u'4 min']
[u'   A to Ozone Park - Lefferts Blvd  ']
[u'4 min']
[u'   A to Inwood - 207 St  ']
[u'5 min']
[u'   D to Coney Island - Stillwell Av  ']
[u'10 min']
[u'   C to 168 St  ']
[u'15 min']
[u'   D to Norwood - 205 St  ']
[u'19 min']
[u'   A to Far Rockaway - Mott Av  ']
[u'19 min']
[u'   A to Inwood - 207 St  ']
[u'20 min']

So, line one is the the first line in column one, line two is the first line in column to and so on, such as:

C to 168 St                 | 2 min
D to Norwood - 205 St       | 4 min
A to Ozone Park - Lefferts Blvd | 4 min
A to Inwood - 207 St        | 5 min
D to Coney Island - Stillwell Av    | 10 min
C to 168 St                 | 15 min
D to Norwood - 205 St       | 19 min
A to Far Rockaway - Mott Av         | 19 min
A to Inwood - 207 St        | 20 min

Ideally I want it to print as a table, and then use the whole think in GeekTool. The basis of my code is from here: http://adesquared.wordpress.com/2013/06/16/using-python-beautifulsoup-to-scrape-a-wikipedia-table/ hence the references to wikipedia.

I'm a complete amateur at this, so apologies for if this is the completely wrong way to go about this. Thanks in advance.

Upvotes: 0

Views: 528

Answers (2)

user3573558
user3573558

Reputation: 190

Check out termsql, it's a tool made for purposes like this one, although I'm not sure I exactly understand your output, you might need to simplify it a little more.

Manual: http://tobimensch.github.io/termsql/

Project: https://github.com/tobimensch/termsql

Interesting for you might be the --line-as-column option.

Upvotes: 0

Javier
Javier

Reputation: 1087

I think your last double loop should look like this one:

for row in table.findAll("tr"):
    rowText = []
    for cell in row.findAll("td"):
        # Append every cell in this row
        rowText.append(cell.findAll(text=True)[0])
    # print the row joining the cells with '|'
    print ' | '.join(rowText)

Upvotes: 0

Related Questions