Always send Content-Length in Apache?

Question

I'm loading a particularly large JSON string that is dynamically generated by PHP. To provide some feedback to the user, I want to show download progress.

I have the code figured out, and it works fine for static content such as images, JS files, etc. However, it doesn't seem to work for dynamic files.

This makes sense, since the dynamic files don't have predictable content length, but even if I add this in PHP:

ob_start(function($c) {
    header("Content-Length: ".strlen($c));
    return $c;
});

It still does not send the header (but if I add any other header it works fine).

Is there any way to force Apache to send the Content-Length header? Currently my only alternative is to save the output to a temporary file and redirect to it instead. This would work, but it's kind of messy so I'd rather avoid it if possible.

Answer 1

I had a few JS files that were around 1MB that didn't get Content-Length header (unlike other, smaller JS files). The fact that Content-Length header is not mandatory with HTTP/2 and having a somewhat flaky Internet connection resulted in a very hard to debug error where the files were sometimes delivered only partially. The solution was to increase DeflateBufferSize https://httpd.apache.org/docs/2.4/mod/mod_deflate.html#DeflateBufferSize

Answer 2

I had similar problem but in my case Content-Length header wasn't sent by Apache because the response was gzip compressed. When I disable compression the Content-Length was calculated and sent properly.

Below is htaccess setting to disable gzip compression for swf file only

<FilesMatch "\.swf$">
  SetEnv no-gzip 1
</FilesMatch>

Answer 3

From the ob_start documentation:

This function will turn output buffering on. While output buffering is active no 
output is sent from the script (other than headers), instead the output is stored 
in an internal buffer.

Note the "other than headers" bit -- the ob_start() callback is not called until the buffer is being flushed (or thrown away), at this point it's too late to use header(). I'm guessing you don't have error logging turned on, I see this in my error log:

PHP Warning:  Cannot modify header information - headers already sent 
  in /usr/local/apache2/apps/testing/test2.php on line 6

The callback lets you modify the buffer before sending, but as this contains only body data, you also cannot prepend a new header (it will appear in the content if you try, once for each chunk for chunked transfers).

Your code should call instead header("Content-Length: ...") once you know the size, before any other output. When a Content-Length: header is found, chunked transfer will not take place.

If you really need to force HTTP/1.0 (you don't), you can do it in httpd.conf with the special variables:

SetEnv downgrade-1.0 1
SetEnv force-response-1.0 1

You can put those in <Location> or <LocationMatch> for example, or use mod_rewrite's "env" flag for more control.

Answer 4

The HTTP protocol needs a way to determine when a response has ended. According to

RFC2616 Section 4

4.4 Message Length

The transfer-length of a message is the length of the message-body as it appears in the message; that is, after any transfer-codings have been applied. When a message-body is included with a message, the transfer-length of that body is determined by one of the following (in order of precedence):

1.Any response message which "MUST NOT" include a message-body (such as the 1xx, 204, and 304 responses and any response to a HEAD request) is always terminated by the first empty line after the header fields, regardless of the entity-header fields present in the message.

2.If a Transfer-Encoding header field (section 14.41) is present and has any value other than "identity", then the transfer-length is defined by use of the "chunked" transfer-coding (section 3.6), unless the message is terminated by closing the connection.

3.If a Content-Length header field (section 14.13) is present, its decimal value in OCTETs represents both the entity-length and the transfer-length. The Content-Length header field MUST NOT be sent if these two lengths are different (i.e., if a Transfer-Encoding

 header field is present). If a message is received with both a
 Transfer-Encoding header field and a Content-Length header field,
 the latter MUST be ignored.

4.If the message uses the media type "multipart/byteranges", and the transfer-length is not otherwise specified, then this self- delimiting media type defines the transfer-length. This media type MUST NOT be used unless the sender knows that the recipient can parse it; the presence in a request of a Range header with multiple byte- range specifiers from a 1.1 client implies that the client can parse multipart/byteranges responses.

   A range header might be forwarded by a 1.0 proxy that does not
   understand multipart/byteranges; in this case the server MUST
   delimit the message using methods defined in items 1,3 or 5 of
   this section.

5.By the server closing the connection. (Closing the connection cannot be used to indicate the end of a request body, since that would leave no possibility for the server to send back a response.)

For compatibility with HTTP/1.0 applications, HTTP/1.1 requests containing a message-body MUST include a valid Content-Length header field unless the server is known to be HTTP/1.1 compliant. If a request contains a message-body and a Content-Length is not given, the server SHOULD respond with 400 (bad request) if it cannot determine the length of the message, or with 411 (length required) if it wishes to insist on receiving a valid Content-Length.

All HTTP/1.1 applications that receive entities MUST accept the "chunked" transfer-coding (section 3.6), thus allowing this mechanism to be used for messages when the message length cannot be determined in advance.

Messages MUST NOT include both a Content-Length header field and a non-identity transfer-coding. If the message does include a non- identity transfer-coding, the Content-Length MUST be ignored.

When a Content-Length is given in a message where a message-body is allowed, its field value MUST exactly match the number of OCTETs in the message-body. HTTP/1.1 user agents MUST notify the user when an invalid length is received and detected.

You can't just randomly add headers and expect them to be obeyed (other headers can override). You need to control all possibly overriding headers generated in the first place.

According to this question (How to make PHP generate Chunked response) a good way to force "Chunked" is to set Transfer-Encoding and to flush. Maybe those two aren't strictly needed. Would there be any extraneous flushes anywhere before you start buffering?

Answer 5

My best guess is that you have some changes in the modules/http/http_filters because Apache by default sends Content-Length.