CODEWITHSUNDEEP
pythonapachemod-wsgiwerkzeugtransfer-encoding
GetFree
GetFree

Reputation: 42434

How to control the use of http chunking in werkzeug under mod_wsgi

Under the following environment: Apache -> mod_wsgi -> python -> werkzeug

How can I control whether http ckunking is used and how big the chunks are?

Follow-up:

What I worry about is the difference between this:

response = werkzeug.wrappers.Response()
response.response = very_long_string
return response

and this:

response = werkzeug.wrappers.Response()
response.response = [ very_long_string ]
return response

In the first case, werkzeug iterates through the string and sends one character at a time. I don't know if each character is sent in its own chunk or not, but I know for a fact that it's way slower than the second case in which the string is sent as a whole.

Is the difference in speed between these two cases due to chunking? or it's something else?

Upvotes: 0

Views: 934

Answers (1)

SingleNegationElimination
SingleNegationElimination

Reputation: 156158

In general, wsgi specifies that data should be sent to the client, without buffering, as soon as the application produces any portion of the response content; ie, if you do:

CONTENT = "a bit more content\n"

def my_slow_application(environ, start_response):
    start_response("200 OK", [("Content-Type", "text/plain")])
    yield CONTENT
    sleep(1)

Notably, without a Content-Length header, the gateway (apache/mod_wsgi in your case) is not able to guess that it has found all of the response content, and since it's not permitted to buffer, it must use the chunked transfer encoding.

On the other hand, consider:

def my_slowish_application(environ, start_response):
    start_response("200 OK", [("Content-Type", "text/plain"),
                              ("Content-Length", str(len(CONTENT))])
    yield CONTENT
    sleep(1)

since the application has specified a content length, and since the first yielded chunk is exactly that length, the gateway knows that no more data will come; it may or may not use chunked encoding, at its own discretion. Likewise 

def my_fast_application(environ, start_response):
    start_response("200 OK", [("Content-Type", "text/plain")])
    return [CONTENT] * 100

generally won't result in chunking; the response is a list, which has bounded size, so the gateway knows it has the whole response. It can find the content length even without an explicit header from the application using sum(map(len, app_iter)), and then send the response as fast as the network permits.

As for controlling how "big" the chunks are, the gateway might buffer chunks if the client connection is blocked; your app might 

yield "foo"
sleep(1)
yield "bar"
sleep(1)
yield "baz"

but if the client connection blocks while processing the "foo" chunk, the gateway might buffer bar and baz together, sending them as a single chunk when the connection becomes ready for reads again. In short, you can't control the chunking, you might force it to occur, and you can frequently prevent it from occurring (most gateways won't chunk if they don't have to); but it's not suitable for framing.

Upvotes: 1

Related Questions