CODEWITHSUNDEEP
phpguzzle
ROunofF
ROunofF

Reputation: 730

File not closed on Request "Complete" using guzzle 5.3

We are uploading 1500+ files using a guzzle pool. Since we don't want to run into "too many open files" we figured we could use the event "before" to do the fopen and the "complete" to fclose the stream. PHP is not effectively closing the resource (and we hit the too many open files). Any idea what is happening / What we can do to fix this issue?

Here's the code:

    $client = new GuzzleHttp\Client();
    $requests = [];
    foreach($files as $fileName) {
        $options = [
            'debug' => false,
            'events' => [
                'before'=>
                    function (BeforeEvent $e) use ($fileName) {
                        echo 'Opening body|'.count(glob('/proc/'.posix_getpid().'/fd/*')).PHP_EOL;
                        $stream = \GuzzleHttp\Stream\Stream::factory(fopen($fileName,'r'));
                        $e->getRequest()->setBody($stream);
                    },
                'complete' =>
                    function (CompleteEvent $e){
                        echo 'Closing body|'.count(glob('/proc/'.posix_getpid().'/fd/*')).PHP_EOL;
                        $stream = $e->getResponse()->getBody();
                        $stream->close();
                    },
            ]
        ];

        $request = $client->createRequest('POST', $this->baseUri . $this->uploadPath, $options);
        $requests[] = $request;
    }
    Pool::batch($client, $requests, ['pool_size'=> $this->poolSize]);

Output : Opening body|31 Closing body|57 Opening body|57 Opening body|58 Closing body|59 Opening body|59 Closing body|61 Opening body|61 Closing body|62 Opening body|62 Closing body|63 ...

The number never goes down.

Upvotes: 1

Views: 1459

Answers (1)

Sam
Sam

Reputation: 36

You need to close the request's stream instead of the response one. The response stream seems to be closed automatically. So just replace

$stream = $e->getResponse()->getBody();

by 

$stream = $e->getRequest()->getBody();

in your complete event

Upvotes: 2

Related Questions