CODEWITHSUNDEEP
phppostcurl
shulard
shulard

Reputation: 603

How to POST a large amount of data within PHP curl without memory overhead?

I'm currently using PHP curl extension to communicate with some HTTP APIs.

I use a bulk loader to perform a lot of operations at once. The bulk endpoint must be called with POST method so I use a code like :

<?php
$h = curl_init($url);
curl_setopt(CURLOPT_POST, true);
curl_setopt(CURLOPT_POSTFIELDS, $data);
curl_setopt(CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($h);
curl_close($h);

The bulk endpoint allows me to send a huge amount of data (more than 200Mo at once). For the moment I need to load the data inside a variable which require PHP to be able to use enough memory...

I need to set memory_limit to a high value just for the bulk load...

Is there a way to use a file stream to send data with the curl PHP extension ? I saw the CURLOPT_INFILE and CURLOPT_READFUNCTION but it seems to don't work with POST method...

I also saw that the curl command line tool is able to perform a --data "@/path/to/file/content" which seems to be what I need...

Any ideas ?

Upvotes: 6

Views: 5721

Answers (1)

Ninju
Ninju

Reputation: 2530

Use CURLOPT_INFILE

$curl = curl_init();
curl_setopt( $curl, CURLOPT_PUT, 1 );
curl_setopt( $curl, CURLOPT_INFILESIZE, filesize($tmpFile) );
curl_setopt( $curl, CURLOPT_INFILE, ($in=fopen($tmpFile, 'r')) );
curl_setopt( $curl, CURLOPT_CUSTOMREQUEST, 'POST' );
curl_setopt( $curl, CURLOPT_HTTPHEADER, [ 'Content-Type: application/json' ] );
curl_setopt( $curl, CURLOPT_URL, $url );
curl_setopt( $curl, CURLOPT_RETURNTRANSFER, 1 );
$result = curl_exec($curl);
curl_close($curl);
fclose($in);

Upvotes: 4

Related Questions