CODEWITHSUNDEEP
phplinuxcurlcentos
Jake Cattrall
Jake Cattrall

Reputation: 461

Bypass PHP CURL 2GB download limit

I am trying to download a 7GB file using php-curl however it seems that it will only download the first 2GB.

There doesn't seem to be any documentation or talk about this.

Anyone have any ideas?

Upvotes: 0

Views: 2241

Answers (2)

Sammaye
Sammaye

Reputation: 43884

Here are two useful links on the topic:

Downloading a large file using curl

How to partially download a remote file with cURL?

Basically you may have two problems here:

  • You are reading into memory first as as such exhausting PHPs memory allocation
  • You need to chunk download the file to overcome certain restrictions in the HTTP protocol.

There are also file sytem limitations and what not so check your file system type as mentioned by @ajreal (i.e. FAT32 has a limit of 4GB, 99% chance your not using FAT but still it is an example).

As the OP found out it was do with the DB:

Turns out it was a database issue. File sizes were stored in a mysql database, the sizes were in bytes and max size for "int" column is 2147483648. Changing the column type to "bigint" fixed the issue.

Upvotes: 2

Vatev
Vatev

Reputation: 7590

Assuming your file system can handle files larger than 2GB you can try using copy

copy("http:://example.org/your_file","/tmp/your_file");

Also make sure you set an appropriate time limit (with set_time_limit(...)).

Upvotes: 0

Related Questions