CODEWITHSUNDEEP
php
Trevor
Trevor

Reputation: 2922

How to non block process in php on windows server?

I am writing a php script to interact with a CouchDb server. The script reads an SQL database and creates documents and PUTs them on the server. Each script runs every 5 minutes and puts about 2000 documents (creates and updates).

Running sync, this takes about 3 minutes to PUT all the docs. In a test I did using node and promises, I found CouchDb can handle 100 asnyc puts at the same time and respond back in only slightly more time than it took to do a single document. I want to utilize this feature in PHP instead.
I have available, php 5.3 and php 7.0.10 on a Windows server.

How do I do this asnyc?

My first thought was using the pclose(popen()) trick, but that spawns a new process each time, and even if I restrict this to 100 docs at a time (my tests show up to 700 at a time is doable), that would still results in 6 scripts creating and recreating total of 600 new processes every 100/2000 docs every 5 minutes, or a total of 12,000 processes created and run every 5 minutes. I don't think Windows can handle that.

My second idea was to set up a basic node script to handle it, with PHP creating and formatting the data, writing to a file, and passing the file to a node script to process async and report back to PHP using exec. But I am hoping to find a pure PHP solution.

I currently send requests to couch like this

private function sendJSONRequest($method, $url, $post_data = NULL)
{
    // Open socket
    $s = fsockopen($this->db_host, $this->db_port, $errno, $errstr);
    if (!$s) {
        throw new Exception("fsockopen: $errno: $errstr");
    }
    // Prepare request
    $request = "$method $url HTTP/1.0\r\n" .
        ($this->db_auth === false ? "" : "Authorization: $this->db_auth\r\n") .
        "User-Agent: couchdb-php/1.0\r\n" .
        "Host: $this->db_host:$this->db_port\r\n" .
        "Accept: application/json\r\n" .
        "Connection: close\r\n";

    if ($method == "POST" || $method == "PUT") {
        $json_data = json_encode($post_data);
        $request .= "Content-Type: application/json\r\n" .
            "Content-Length: " . strlen($json_data) . "\r\n\r\n" .
            $json_data;
    } else {
        $request .= "\r\n";
    }

    // Send request
    fwrite($s, $request);
    $response = "";
    // Receive response
    while (!feof($s)) {
        $response .= fgets($s);
    }

    $headers = array();
    $body = '';
    $reason = '';

    if (!empty($response)) {
        // Split header & body
        list($header, $body) = explode("\r\n\r\n", $response);
        // Parse header
        $first = true;
        foreach (explode("\r\n", $header) as $line) {
            if ($first) {
                $status = intval(substr($line, 9, 3));
                $reason = substr($line, 13);
                $first = false;
            } else {
                $p = strpos($line, ":");
                $headers[strtolower(substr($line, 0, $p))] = substr($line, $p + 2);
            }
        }
    } else {
        $status = 200;
    }

    // Return results
    return array($status, $reason, $headers, json_decode($body));
}

My PHP knowledge is only basic, so examples to learn from would be greatly appreciated.

Thank you

Upvotes: 0

Views: 137

Answers (2)

Ice76
Ice76

Reputation: 1130

Guzzle is a PHP library that helps send HTTP requests and can do so asynchronously. The documentation for the async function can be found here.

Upvotes: 1

JohnnyDevNull
JohnnyDevNull

Reputation: 982

It's a litte bit ago since i have researched in this topic, but simply what you are looking for is a queue runner system. At my old employee i have worked with a custom built queue runner in php.

That mean, you have e.g. 4 queue runners. Thats are php processes which are watching a control table maybe "queue". Each time a queue process is inserted in maybe status "new" a runner lock this entry and start the process with a fork job.

PHP forking: http://php.net/manual/de/function.pcntl-fork.php

so... this 4 queue runners can maybe let's say fork 10 processes, than you have 40 parallel working processes.

To seperate them what each do is in best way another control tables from which each job selects a amount of data with LIMIT and OFFSET Queries. Lets say job 1 selects the first 0-20 rows, job 2 the 21-40 rows.

Edit:

After a little research this looks nearly similar to on what i've worked: https://github.com/CoderKungfu/php-queue

Upvotes: 1

Related Questions