CODEWITHSUNDEEP
pythonsocketsnetwork-programming
Vlad Havriuk
Vlad Havriuk

Reputation: 1451

Python3 socket - What is purpose of chunking network packets

I am new to socket module at Python 3. While researching this module I've seen many examples of code where socket.recv(n) function was called multiple times to "chunk" data. And almost all of these code examples used patterns like this:

with socket.socket() as sock:
    sock.connect((host, port))
    to_read = int(sock.recv(8))
    sock.sendall(b'GOT SIZE')

    chunks = []
    while to_read:
        chunk = sock.recv(chunk_length)
        chunks.append(chunk)
        to_read -= chunk_length
    data = b''.join(chunks)

So I have a question: What is purpose of chunking data if you can just use sock.recv(to_read)?

Upvotes: 0

Views: 195

Answers (1)

Steffen Ullrich
Steffen Ullrich

Reputation: 123461

What is purpose of chunking data if you can just use sock.recv(to_read)

recv does not return to_read bytes. It return up to to_read bytes. Especially with large values of to_read it is very likely that you need multiple recv to get all the requested bytes. Thus, "chunking" is needed to actually receive all the data.

Note that if the goal is to put all received data into a single buffer it would actually be more efficient to not read small chunks but to try to read all data at once and retry for the remaining data again and again until all data are received. This way the least number of recv calls and the least number of memory (re)allocations are used.

Upvotes: 1

Related Questions