CODEWITHSUNDEEP
pythondjangopython-3.xurllib
TheInkedEngineer
TheInkedEngineer

Reputation: 241

urllib HTTPError not being caught in Python/Django

I am trying to handle an HTTPError with urllib. My settings are python3 in an anaconda virtualenv using Django 1.10. When the code gets the try, it doesn't go into except and crashes my page with Django telling me there's an HTTP error.

Here is the code: 

from urllib.request import Request, urlopen
from urllib.error import URLError, HTTPError

try:
  req = Request(api.lists.members.get(LIST_ID, client_email))
  response = urlopen(req)
except HTTPError as e:
  print('Error code: ', e.code)
else:
  print('everything is fine')

TRACEBACK:

Environment:

Request Method: POST
Request URL: http://127.0.0.1:8000/homepage/

Django Version: 1.10
Python Version: 3.6.1
Installed Applications:
['django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'website']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware']



Traceback:

File "/Users/plfiras/anaconda/lib/python3.6/site-packages/django/core/handlers/exception.py" in inner
  39.             response = get_response(request)

File "/Users/plfiras/anaconda/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
  187.                 response = self.process_exception_by_middleware(e, request)

File "/Users/plfiras/anaconda/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
  185.                 response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/Users/plfiras/vinhood/vinhood-website/website/views.py" in homepage
  52.             conn = http.client.HTTPConnection(api.lists.members.get(LIST_ID, client_email))

File "/Users/plfiras/anaconda/lib/python3.6/site-packages/mailchimp3/entities/listmembers.py" in get
  116.         return self._mc_client._get(url=self._build_path(list_id, 'members', subscriber_hash), **queryparams)

File "/Users/plfiras/anaconda/lib/python3.6/site-packages/mailchimp3/mailchimpclient.py" in wrapper
  25.             return fn(self, *args, **kwargs)

File "/Users/plfiras/anaconda/lib/python3.6/site-packages/mailchimp3/mailchimpclient.py" in _get
  100.             r.raise_for_status()

File "/Users/plfiras/anaconda/lib/python3.6/site-packages/requests/models.py" in raise_for_status
  928.             raise HTTPError(http_error_msg, response=self)

Exception Type: HTTPError at /homepage/
Exception Value: 404 Client Error: Not Found for url: https://us13.api.mailchimp.com/3.0/lists/7bdb42e5c9/members/d071e758df3554f0fe89679212ef95e8

Upvotes: 2

Views: 2991

Answers (2)

Wipqozn
Wipqozn

Reputation: 1314

You're catching the wrong exception. Take a look at the last line of your traceback:

 File "/Users/plfiras/anaconda/lib/python3.6/site-packages/requests/models.py" in raise_for_status
  928.  raise HTTPError(http_error_msg, response=self)

Take a look at line 31 of requests/models.py and you'll see the following:

from .exceptions import (
HTTPError, MissingSchema, InvalidURL, ChunkedEncodingError,
ContentDecodingError, ConnectionError, StreamConsumedError)

As you can see the HTTPError being raised actually comes from requests/exceptions.py. Take a look at the top of the file, and you'll see:

from urllib3.exceptions import HTTPError as BaseHTTPError


class RequestException(IOError):
    """There was an ambiguous exception that occurred while handling your
    request.
    """

    def __init__(self, *args, **kwargs):
        """Initialize RequestException with `request` and `response` objects."""
        response = kwargs.pop('response', None)
        self.response = response
        self.request = kwargs.pop('request', None)
        if (response is not None and not self.request and
                hasattr(response, 'request')):
            self.request = self.response.request
        super(RequestException, self).__init__(*args, **kwargs)


class HTTPError(RequestException):
    """An HTTP error occurred."""

Which reveals that HTTPError is being imported as BaseHTTPError, and the requests library has implemented it's own HTTPError which doesn't extend urlib3.HTTPError.

So to catch the error you need to import the HTTPError from the requests module, not urlib, like so:

from requests.exceptions import HTTPError

try:
  req = Request(api.lists.members.get(LIST_ID, client_email))
  response = urlopen(req)
except HTTPError as e:
  print('Error code: ', e.code)
else:
  print('everything is fine')

Upvotes: 5

TheInkedEngineer
TheInkedEngineer

Reputation: 241

For some reason it was not catching the HTTPError. By replacing HTTPError with Exception, it works.

from urllib.request import Request, urlopen
from urllib.error import URLError, HTTPError

try:
  req = Request(api.lists.members.get(LIST_ID, client_email))
  response = urlopen(req)
except Exception as e:
  print('Error code: ', e.code)
else:
  print('everything is fine')

Upvotes: -1

Related Questions