CODEWITHSUNDEEP
pythonpython-requests
DungeonTiger
DungeonTiger

Reputation: 717

Does requests properly support multipart responses?

I'm getting an error when receiving a multipart response.

WARNING connectionpool  Failed to parse headers (url=************): [StartBoundaryNotFoundDefect(), MultipartInvariantViolationDefect()], unparsed data: ''
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 399, in _make_request
    assert_header_parsing(httplib_response.msg)
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/response.py", line 66, in assert_header_parsing
    raise HeaderParsingError(defects=defects, unparsed_data=unparsed_data)
urllib3.exceptions.HeaderParsingError: [StartBoundaryNotFoundDefect(), MultipartInvariantViolationDefect()], unparsed data: ''

Does this mean that the library does not support multipart responses? The response from my server works in all other cases including to the browser so I'm a little confused.

Any ideas?

This is what is coming back from the server (of course body truncated for brevity):

HTTP/1.1 200 OK
X-Powered-By: Servlet/3.1
X-CA-Affinity: 2411441258
Cache-Control: no-cache
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Content-Encoding: gzip
X-Compressed-By: BICompressionFilter
Content-Type: multipart/related; type="text/xml"; boundary="1521336443366.-7832488688540884419.-1425166373"
Content-Language: en-US
Transfer-Encoding: chunked
Date: Sun, 18 Mar 2018 01:27:23 GMT

a

154e
<i  ʲ O  x\龅L   dre     Qyi  
/su  k

Of course this is encoded. If I decode it in Fiddler this is what it looks like:

HTTP/1.1 200 OK
X-Powered-By: Servlet/3.1
X-CA-Affinity: 2411441258
Cache-Control: no-cache
Expires: Thu, 01 Jan 1970 00:00:00 GMT
X-Compressed-By: BICompressionFilter
Content-Type: multipart/related; type="text/xml"; boundary="1521336443366.-7832488688540884419.-1425166373"
Content-Language: en-US
Date: Sun, 18 Mar 2018 01:27:23 GMT
Content-Length: 17419


--1521336443366.-7832488688540884419.-1425166373
Content-Type: text/xml; charset=utf-8
Content-Length: 15261

<?xml version="1.0" encoding="UTF-8"?>

Upvotes: 5

Views: 2349

Answers (1)

Beethoven&#39;s 7th
Beethoven&#39;s 7th

Reputation: 116

To answer your question: Yes, Requests handles multipart requests just fine. Having said that, I have seen the same error you're getting.

This appears to be a bug within urllib3 but possibly goes as deep as the httplib package that comes with python. In your case I would guess it comes back to the UTF-8 encoding of the response which obviously you can't do much about (unless you also maintain server-side). I believe it is perfectly safe to ignore but simply including urllib3.disable_warnings() doesn't seem to do the trick for me. If you want to silence this specific warning, you can include a logging filter in your code. (credit to the home-assistant maintainers for this approach)

def filter_urllib3_logging():
    """Filter header errors from urllib3 due to a urllib3 bug."""
    urllib3_logger = logging.getLogger("urllib3.connectionpool")
    if not any(isinstance(x, NoHeaderErrorFilter)
               for x in urllib3_logger.filters):
        urllib3_logger.addFilter(
            NoHeaderErrorFilter()
        )


class NoHeaderErrorFilter(logging.Filter):
    """Filter out urllib3 Header Parsing Errors due to a urllib3 bug."""

    def filter(self, record):
        """Filter out Header Parsing Errors."""
        return "Failed to parse headers" not in record.getMessage()

Then, just call filter_urllib3_logging() in your setup. It doesn't stop the warnings but it DOES hide them :D

!!PLEASE NOTE!! This will also hide, and thus, make it difficult to diagnose any error that is caused by parsing headers which occasionally could be a legitimate error!

Upvotes: 2

Related Questions