CODEWITHSUNDEEP
python
yasser karimi
yasser karimi

Reputation: 329

How can I write Python code that uploads a file larger than RAM?

I had a vps with 1 GB ram and i wanted send a big file (2GB ) by curl and python.

python gave MemoryLimit error but curl send file without problem and use very low memory.

How curl did it?

How i can send a very big file in a Http post request with use low memory?

Upvotes: 0

Views: 613

Answers (1)

Tail of Godzilla
Tail of Godzilla

Reputation: 551

You might have loaded all file data into memory and then tried to send it. This obviously results in MemoryLimit error.

You could try to use mmap module. A sample code would be as follows:

import requests
import mmap

with open("hello.txt", "r+b") as f:
    # memory-map the file, size 0 means whole file
    mm = mmap.mmap(f.fileno(), 0)
    r = requests.post(URL, data=mm)
    print(r.status_code, r.reason)
    mm.close()

Upvotes: 1

Related Questions