CODEWITHSUNDEEP
pythonpython-3.xloggingpython-requests
oleskii
oleskii

Reputation: 366

How to suppress http.client exceptions logging during requests.get() request with Python 3

I am using Python3 and making requests with the following code:

      try:
        resp = self.s.get(url, proxies=self.proxies)
        return resp.text
    except ConnectTimeout:
        self.logger.exception('{}: connection timeout!'.format(self.name))
    except ConnectionError:
        self.logger.exception('{}: ConnectionError!'.format(self.name))
    except RemoteDisconnected:
        self.logger.exception('{}: RemoteDisconnected'.format(self.name))
    except MaxRetryError:
        self.logger.exception('{}: MaxRetryError'.format(self.name))
    except Exception as e:
        self.logger.exception(e)

And if I get exception during session().get(url, proxies=proxies), I get the following in the log file:

2018-04-20 17:14:27 | RF5MR: ConnectionError!

Traceback (most recent call last):
  File "/opt/scripts/this-is-my-app/.venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/opt/scripts/this-is-my-app/.venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 387, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/opt/scripts/this-is-my-app/.venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 383, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.6/http/client.py", line 1331, in getresponse
    response.begin()
  File "/usr/lib/python3.6/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.6/http/client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/scripts/this-is-my-app/.venv/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/opt/scripts/this-is-my-app/.venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/opt/scripts/this-is-my-app/.venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 388, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.thesite.com', port=443): Max retries exceeded with url: /asdzxcqwe/2058706 (Caused by ProxyError('Cannot connect to proxy.', RemoteDisconnected('Remote end closed connection without response',)))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/scripts/this-is-my-app/fetcher.py", line 417, in make_request
    resp = self.s.get(url, proxies=self.proxies)
  File "/opt/scripts/this-is-my-app/.venv/lib/python3.6/site-packages/requests/sessions.py", line 521, in get
    return self.request('GET', url, **kwargs)
  File "/opt/scripts/this-is-my-app/.venv/lib/python3.6/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/opt/scripts/this-is-my-app/.venv/lib/python3.6/site-packages/requests/sessions.py", line 640, in send
    history = [resp for resp in gen] if allow_redirects else []
  File "/opt/scripts/this-is-my-app/.venv/lib/python3.6/site-packages/requests/sessions.py", line 640, in <listcomp>
    history = [resp for resp in gen] if allow_redirects else []
  File "/opt/scripts/this-is-my-app/.venv/lib/python3.6/site-packages/requests/sessions.py", line 218, in resolve_redirects
    **adapter_kwargs
  File "/opt/scripts/this-is-my-app/.venv/lib/python3.6/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/opt/scripts/this-is-my-app/.venv/lib/python3.6/site-packages/requests/adapters.py", line 502, in send
    raise ProxyError(e, request=request)
requests.exceptions.ProxyError: HTTPSConnectionPool(host='www.thesite.com', port=443): Max retries exceeded with url: /asdzxcqwe/2058706 (Caused by ProxyError('Cannot connect to proxy.', RemoteDisconnected('Remote end closed connection without response',)))

But what I want to get is only that string logged:

2018-04-20 17:14:27 | RF5MR: ConnectionError!

Could you please point me where my mistake is and how could I suppress those exception texts and get only the one I want to log?

Many thanks!

Upvotes: 2

Views: 719

Answers (1)

Dan D.
Dan D.

Reputation: 74655

Instead of calling self.logger.exception, call self.logger.error.

exception logs the message and the trace back. All of the other log methods only log the message at the priority given specific to the method.

See the documentation on the logging module.

Upvotes: 2

Related Questions