CODEWITHSUNDEEP
parsinghttptcp
Mark
Mark

Reputation: 6474

TCP segments with HTTP data

I'm implementing a http parser; As indicator of http data I'm searching for 'HTTP/1.? CRLF' in the stream. TCP layer may cut the application-provided buffer into chunks suitable for transferring over network. Is it possible to have http data (e.g. GET http://www.google.com/index.html HTTP/1.1 CRLF) NOT following immediately after TCP header? Also, is it possible to have, for example 'GET ..' query split across TCP segments?

Thanks.

Mark

Upvotes: 3

Views: 3008

Answers (2)

MattH
MattH

Reputation: 38247

Is it possible to have http data (e.g. GET http://www.google.com/index.html HTTP/1.1 CRLF) NOT following immediately after TCP header?

Possible yes. HTTP Pipelining makes it possible to have multiple requests in a single segment.

Also, is it possible to have, for example 'GET ..' query split across TCP segments?

Yes. The request size can be bigger than the segment size. Additionally fragmentation of the TCP segment can occur at the IP layer.

This would be an uncommon natural occurrence, but could be part of intentional evasion.

The only way to be sure is to re-assemble the stream, however this is expensive in terms of processing and memory. If you care that much you may be better off using a transparent HTTP proxy.

Upvotes: 3

rioki
rioki

Reputation: 6118

With TCP you need to basically assume that the segmentation can occur anyplace. You have to consider this while designing you parsing stack. You can't read one segment and assume that you have sufficient to read something. So the sensible way, is to view TCP as a stream and put it below your lexer. You just need accommodate for the fact that a call into the parser/lexer may take a bit longer or fault.

Upvotes: 3

Related Questions