Reputation: 13
I am trying to scrape some web content to get stastics from google appstats that my google app generated. Please note this is different then google analytics. I am using python 2.7.5. The issue I am facing is the initial google authentication in my request. I have the apis I need to call from google app stats but I keep getting a DENY in response back when I am using my own google appengine account credentials. This results in a redirect to accounts.google.com page. I have tried several different approaches without a successfull login to accounts.google.com.
Anyone have any ideas on this? More helpfull would be if you can point me to some good reference material
Thanks
Upvotes: 1
Views: 89
Reputation: 6201
This code sample will allow you to get content of /secure page which protected by google login. Do not forget to set email, password and application ID. Then you can fetch other protected pages using this opener.
import urllib
import urllib2
import cookielib
import logging
EMAIL = ''
PASSWORD = ''
APPID = 'YOURAPPID'
# Setup to be able to get the needed cookies that GAE returns
cookiejar = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cookiejar))
urllib2.install_opener(opener)
# This is the setup to construct the login URL for authentication.
authreq_data = urllib.urlencode({'Email': EMAIL,
'Passwd': PASSWORD,
'service': 'ah',
'source': '',
'accountType': 'HOSTED_OR_GOOGLE'})
# Get an AuthToken from Google Accounts
auth_req = urllib2.Request('https://www.google.com/accounts/ClientLogin',
data=authreq_data)
try:
auth_resp = opener.open(auth_req)
logging.info('Successful authorization as %s' % EMAIL)
except urllib2.HTTPError:
logging.warning('Authorization as %s failed. '
'Please, check your email and password' % EMAIL)
auth_resp_body = auth_resp.read()
auth_resp_dict = dict(x.split('=')
for x in auth_resp_body.split('\n') if x)
authtoken = auth_resp_dict['Auth']
authreq_data = urllib.urlencode({'continue': 'http://%s.appspot.com/secure' % APPID,
'auth': authtoken})
login_uri = ('http://%s.appspot.com/_ah/login?%s' % (APPID, authreq_data))
# Do the actual login and getting the cookies.
print opener.open(urllib2.Request(login_uri)).read()
Upvotes: 2