Basic authentication using urllib2 with python with JIRA REST api

Question

I am trying to find how i can use basic authentication using urllib2 in python to get the issue KEY The JIRA REST API describes the URI's available

Thanks for the sugestions, i will try it, meanwhile, i just wanted to update this with my own effort: Here is the sample python code i tried:

import urllib2, sys, re, base64
from urlparse import urlparse
theurl = 'http://my.rest-server.com:8080/rest/api/latest/AA-120'            # if you want to run this example you'll need to supply a protected page with y
our username and password
username = 'username'
password = 'password'            # a very bad password

req = urllib2.Request(theurl)
print req
try:
    handle = urllib2.urlopen(req)
    print handle
except IOError, e:                  # here we are assuming we fail
    pass
else:                               # If we don't fail then the page isn't protected
    print "This page isn't protected by authentication."
    sys.exit(1)
if not hasattr(e, 'code') or e.code != 401:                 # we got an error - but not a 401 error
    print "This page isn't protected by authentication."
    print 'But we failed for another reason.'
    sys.exit(1)

authline = e.headers.get('www-authenticate', '')                # this gets the www-authenticat line from the headers - which has the authentication
 scheme and realm in it
if not authline:
    print 'A 401 error without an authentication response header - very weird.'
    sys.exit(1)
authobj = re.compile(r'''(?:\s*www-authenticate\s*:)?\s*(\w*)\s+realm=['"](\w+)['"]''', re.IGNORECASE)          # this regular expression is used to
 extract scheme and realm
matchobj = authobj.match(authline)
if not matchobj:                                        # if the authline isn't matched by the regular expression then something is wrong
    print 'The authentication line is badly formed.'
    sys.exit(1)
scheme = matchobj.group(1)
print scheme
realm = matchobj.group(2)
print realm
if scheme.lower() != 'basic':
    print 'This example only works with BASIC authentication.'
    sys.exit(1)

base64string = base64.encodestring('%s:%s' % (username, password))[:-1]
authheader =  "Basic %s" % base64string
req.add_header("Authorization", authheader)
try:
    handle = urllib2.urlopen(req)
except IOError, e:                  # here we shouldn't fail if the username/password is right
    print "It looks like the username or password is wrong."
    sys.exit(1)
thepage = handle.read()
server = urlparse(theurl)[1].lower()            # server names are case insensitive, so we will convert to lower case
test = server.find(':')
if test != -1: server = server[:test]           # remove the :port information if present, we're working on the principle that realm names per serve
r are likely to be unique...

passdict = {(server, realm) : authheader }      # now if we get another 401 we can test for an entry in passdict before having to ask the user for a
 username/password

print 'Done successfully - information now stored in passdict.'
print 'The webpage is stored in thepage.'

--- and i get the result: This page isn't protected by authentication. But we failed for another reason.

whereas the page is protected by authentication

I tried installing requests, but got error:

sudo easy_install requests
Searching for requests
Reading http://pypi.python.org/simple/requests/

Reading https://github.com/kennethreitz/requests
Reading http://python-requests.org
Best match: requests 0.9.1
Downloading http://pypi.python.org/packages/source/r/requests/requests-0.9.1.tar.gz#md5=8ed4667edb5d57945b74a9137adbb8bd
Processing requests-0.9.1.tar.gz
Running requests-0.9.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-lTQu8K/requests-0.9.1/egg-dist-tmp-M2yQCt
Traceback (most recent call last):
  File "/usr/bin/easy_install", line 7, in ?
    sys.exit(
  File "/usr/lib/python2.4/site-packages/setuptools-0.6c12dev_r88846-py2.4.egg/setuptools/command/easy_install.py", line 1712, in main
    with_ei_usage(lambda:
  File "/usr/lib/python2.4/site-packages/setuptools-0.6c12dev_r88846-py2.4.egg/setuptools/command/easy_install.py", line 1700, in with_ei_usage
    return f()
  File "/usr/lib/python2.4/site-packages/setuptools-0.6c12dev_r88846-py2.4.egg/setuptools/command/easy_install.py", line 1716, in <lambda>
    distclass=DistributionWithoutHelpCommands, **kw
  File "/usr/lib64/python2.4/distutils/core.py", line 149, in setup
    dist.run_commands()
  File "/usr/lib64/python2.4/distutils/dist.py", line 946, in run_commands
    self.run_command(cmd)
  File "/usr/lib64/python2.4/distutils/dist.py", line 966, in run_command
    cmd_obj.run()
  File "/usr/lib/python2.4/site-packages/setuptools-0.6c12dev_r88846-py2.4.egg/setuptools/command/easy_install.py", line 211, in run
    self.easy_install(spec, not self.no_deps)
  File "/usr/lib/python2.4/site-packages/setuptools-0.6c12dev_r88846-py2.4.egg/setuptools/command/easy_install.py", line 446, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/usr/lib/python2.4/site-packages/setuptools-0.6c12dev_r88846-py2.4.egg/setuptools/command/easy_install.py", line 476, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/usr/lib/python2.4/site-packages/setuptools-0.6c12dev_r88846-py2.4.egg/setuptools/command/easy_install.py", line 655, in install_eggs
    return self.build_and_install(setup_script, setup_base)
  File "/usr/lib/python2.4/site-packages/setuptools-0.6c12dev_r88846-py2.4.egg/setuptools/command/easy_install.py", line 930, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "/usr/lib/python2.4/site-packages/setuptools-0.6c12dev_r88846-py2.4.egg/setuptools/command/easy_install.py", line 919, in run_setup
    run_setup(setup_script, args)
  File "/usr/lib/python2.4/site-packages/setuptools-0.6c12dev_r88846-py2.4.egg/setuptools/sandbox.py", line 61, in run_setup
    DirectorySandbox(setup_dir).run(
  File "/usr/lib/python2.4/site-packages/setuptools-0.6c12dev_r88846-py2.4.egg/setuptools/sandbox.py", line 105, in run
    return func()
  File "/usr/lib/python2.4/site-packages/setuptools-0.6c12dev_r88846-py2.4.egg/setuptools/sandbox.py", line 64, in <lambda>
    {'__file__':setup_script, '__name__':'__main__'}
  File "setup.py", line 6, in ?
  File "/tmp/easy_install-lTQu8K/requests-0.9.1/requests/__init__.py", line 26
    from . import utils
         ^
SyntaxError: invalid syntax

Answer 1

I had better success using the requests module.

import requests

jira_session = requests.session()

try:
    jira_session.post('https://server', auth=(user, password), verify=False)
except:
    print('Unable to connect or authenticate with JIRA server.')

url = 'https://server/rest/api/2/search?jql=project="ABCXYZ"&maxResults=100'
results = jira_session.get(url)
project_data = results.json()

Answer 2

I had the same problem with accessing Jira REST. Here is what worked for me:

class JiraOpenerWrapper(object):
  # Class to wrap urllib2 OpenerDirector to add authenication headers.
  # this is needed fro Jira - see comment below.
  def __init__(self, opener, user, password):
    self.opener = opener
    self.user = user
    self.password = password

  def open(self, url, data, timeout):
    if isinstance(url, str):
      # if given a url - create a request and add
      req = urllib2.Request(url, data, {
                      "Authorization": 
                        "Basic " + base64.b64encode(
                           self.user + ":" + self.password)
                     })
    else:
      req = url
    return self.opener.open(req, data, timeout)

  def __getattr__(self, attr):
    if attr == "open":
      return self.open
    return getattr(self.opener, attr)


def RegisterPasswords(user, password):
  password_manager = urllib2.HTTPPasswordMgrWithDefaultRealm()
  password_manager.add_password(
    None, 'https://rt.mycorp.site/', user, password)
  password_manager.add_password(
    None, 'https://jira.mycorp.site/', user, password)
  auth_handler = urllib2.HTTPBasicAuthHandler(password_manager)
  opener = urllib2.build_opener(auth_handler)
  # Jira has a strage behviour - it does not return the proper error 
  # when sending a request without an Authentication header.
  # This causes Python's urllib2 not to retry with the pass word.
  # For Jira we would need to add the headers all the time
  jira_opener = JiraOpenerWrapper(opener, user, password)
  urllib2.install_opener(jira_opener)


def GameJira():
  url = "https://jira.mycorp.site/rest/api/2/issue/PRJ-123"

  data = urllib2.urlopen(url, None).read()
  print "Jira:\n", data

Answer 3

Funny, I was working on this yesterday for the JIRA Python CLI. I took the approach of using the REST API to get an authentication cookie and a custom opener. The example below shows using the opener to post data to a page to add a component, but you could replace that with a call to the correct URL for a different REST call.

    """
Demonstration of using Python for a RESTful call to JIRA

Matt Doar
CustomWare
"""

import urllib
import urllib2
import cookielib

jira_serverurl = "http://jira.example.com:8080"
creds = { "username" : "admin", "password" : "admin" }
authurl = jira_serverurl + "/rest/auth/latest/session"

# Get the authentication cookie using the REST API
cj = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
req = urllib2.Request(authurl)
req.add_data('{ "username" : "admin", "password" : "admin" }')
req.add_header("Content-type", "application/json")
req.add_header("Accept", "application/json")
fp = opener.open(req)
fp.close()

add_component_url = jira_serverurl + "/secure/project/AddComponent.jspa?pid=10020&name=ABC4"
print "Using %s" % (add_component_url)

# Have to add data to make urllib2 send a POST
values = {}
data = urllib.urlencode(values)

# Have to tell JIRA to not use a form token
headers = {'X-Atlassian-Token': 'no-check'}

request = urllib2.Request(add_component_url, data, headers=headers)
fp = opener.open(request)

print fp.read()

Answer 4

I'd recommend considering using the very excellent requests library which provides a nice abstraction to make urllib2 a bit easier to use.

With requests you can simply do:

r = requests.get('https://api.github.com', auth=('user', 'pass'))

It supports all of the request methods needed to make REST calls as well (POST, PUT, DELETE, etc...).

You can find more here:

http://pypi.python.org/pypi/requests

---

If you absolutely MUST use plain old urllib2, here is an example of how it can be done:

import urllib2

theurl = 'http://www.someserver.com/toplevelurl/somepage.htm'
username = 'johnny'
password = 'XXXXXX'

passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
passman.add_password(None, theurl, username, password)
# because we have put None at the start it will always
# use this username/password combination for  urls
# for which `theurl` is a super-url

authhandler = urllib2.HTTPBasicAuthHandler(passman)

opener = urllib2.build_opener(authhandler)

urllib2.install_opener(opener)
# All calls to urllib2.urlopen will now use our handler
# Make sure not to include the protocol in with the URL, or
# HTTPPasswordMgrWithDefaultRealm will be very confused.
# You must (of course) use it when fetching the page though.

pagehandle = urllib2.urlopen(theurl)
# authentication is now handled automatically for us

More can be found here: http://www.voidspace.org.uk/python/articles/authentication.shtml