CODEWITHSUNDEEP
pythonparsingbeautifulsouphtml-parsinglxml
TwinyTwice
TwinyTwice

Reputation: 55

Choose, type and Parse with BeautifulSoup and Python syntax?

I'm fully newbie with BeautifulSoup and python dev and I want code a script for automate some things for my personal website.

I make this :

#!/usr/bin/env python

""" Test menu for Website
"""

import urllib2
from bs4 import BeautifulSoup

print (47 * '-')
print ("   C H O I C E   L I S T")
print (47 * '-')
print ("1. Page One")
print ("2. Page Two")
print ("3. Page Three")
print ("4. Page Four")
print (47 * '-')
print (47 * '-')

#############################
##  Robust error handling  ##
##  only accpet int        ##
#############################
## Wait for valid input in while...not ###

is_valid=0

while not is_valid :
        try :
                choice = int ( raw_input('Enter your choice [1-8] : ') )
                is_valid = 1 ## set it to 1 to validate input and to terminate the while..not loop
        except ValueError, e :
                print ("'%s' is not a valid choice." % e.args[0].split(": ")[1])


### Take action as per selected choice list option ###

if choice == 1:
        print ("www.mywebsite.com/page_one.html")
elif choice == 2:
        print ("www.mywebsite.com/page_two.html")
elif choice == 3:
        print ("www.mywebsite.com/page_three.html")
elif choice == 4:
        print ("www.mywebsite.com/page_four.html")
else:
        print ("Invalid choice. try again...")

print (47 * '-')
print (47 * '-')


username = raw_input("Please, type your username\n")


html_content = urllib2.urlopen("http://" + [choice] + "/" + username)

soup = BeautifulSoup(html_content, "lxml")

#####################
## STRINGS REPLACE ##
#####################

start_msg = "Hey, you have "
end_msg = "comments !"
end_str = "read !"


####################
## COMMENTS COUNT ##
####################

count_comments = soup.find("span", "sidebar-comments")
count_comments
count_comments_final = count_comments.find_next("meta")


################
## COUNT READ ##
################

count_read = soup.find("span", "sidebar-read")
count_read
count_read_final = count_read.find_next("meta")



##################
## PRINT RESULT ##
##################

print start_msg + count_comments_final['content'].split(':')[1] + end_msg
print start_msg + count_read_final['content'].split(':')[1] + end_str

With this script, I want :

1 - Choice my webpage (choice list - 4)

2 - Type my username

3 - Parse the webpage that i've choose and obtain the count for all comments and all read.

My problem is here html_content = urllib2.urlopen("http://" + [choice] + username) , I can't retrieve the parameter necessary for a good URL!

Could you help me find the right syntax please !

My final URL should to be : http://www.mywebsite.com/page_one.html/username

Upvotes: 1

Views: 231

Answers (1)

alecxe
alecxe

Reputation: 473763

This is a strange URL to have, but the only thing you need is to store the url in a variable and reuse.

Also, I would use a dictionary to map an int choice and the actual url:

mapping = {
    1: 'www.mywebsite.com/page_one.html',
    2: 'www.mywebsite.com/page_two.html', 
    3: 'www.mywebsite.com/page_three.html',
    4: 'www.mywebsite.com/page_four.html'
}

try:
    page = mapping[choice]
except KeyError:
    print ("Invalid choice. try again...")
    # TODO: try again? :)

username = raw_input("Please, type your username\n")

url = "http://{page}/{username}".format(page=page, username=username)
html_content = urllib2.urlopen(url)

Upvotes: 2

Related Questions