CODEWITHSUNDEEP
phpmultithreadingshared-memory
stevendesu
stevendesu

Reputation: 16791

Sharing variables between child processes in PHP?

I'm sure what I'm trying is very simple, but I've never quite worked with multithreading before so I'm not sure where to start.

I'm using PCNTL to create a multithreaded PHP application. What I wish to do is have 3 functions running concurrently and I want their returned values merged into a single array. So logically I need either some variable shared among all children to which they append their results, or three variables shared only between a single child and the parent - then the parent can merge the results later.

Problem is - I have no idea how to do this. The first thing that comes to mind is using shared memory, but I feel like there should be an easier method.

Also, if it has any effect, the function which forks the process is a public class method. So my code looks something like the following:

<?php
    class multithreaded_search {
        /* ... */
        /* Constructors and such */
        /* ... */
        public function search( $string = '' ) {
            $search_types = array( 'tag', 'substring', 'levenshtein' );
            $pids = array();
            foreach( $search_types as $type ) {
                $pid = pcntl_fork();
                $pids[$pid] = $type;
                if( $pid == 0 ) { // child process
                    /* confusion */
                    $results = call_user_func( 'multithreaded_search::'.$type.'_search', $string );
                    /* What do we do with $results ? */
                }
            }
            for( $i = 0; $i < count( $pids ); $i++ ) {
                $pid = pcntl_wait();
                /* $pids[$pid] tells me the type of search that just finished */
                /* If we need to merge results in the parent, we can do it here */
            }
            /* Now all children have exited, so the search is complete */
            return $results;
        }
        private function tag_search( $string ) {
            /* perform one type of search */
            return $results;
        }
        private function substring_search( $string ) {
            /* perform one type of search */
            return $results;
        }
        private function levenshtein_search( $string ) {
            /* perform one type of search */
            return $results;
        }
    }
?>

So will I need to use shmop_open before I call pcntl_fork to create shared memory and save the results there, or do the children share class variables? Or do they only share global variables? I'm sure the answer is easy... I just don't know it.

Answers (for anyone who finds this)

I've got a few more years of experience, so I'll try to impart some knowledge.

First, there are two important distinctions to understand when it comes to implementing multiprocessing in your applications:

Threads, processes, forked processes

Choosing the proper tool for the job often is a matter of asking the question: "How often will you be spinning up additional threads/processes"? If it's not that often (maybe you run a batch job every hour and the job can be parallelized) then processes might be the easier solution. If every request that comes into your server requires some form of parallel computation and you receive 100 requests per second, then threads are likely the way to go.

Shared memory, message passing

A basic socket solution

First, I'll attempt to recreate my solution from 2012. @MarcB pointed me towards UNIX sockets. This page explicitly mentions fsockopen, which opens a socket as a file pointer. It also includes in the "See Also" section a link to socket_connect, which gives you a bit lower-level control over sockets.

At the time I likely spent a long time researching these socket_* functions until I got something working. Now I did a quick google search for socket_create_pair and found this helpful link to get you started

I've rewritten the code above writing the results to UNIX sockets, and reading the results into the parent thread:

<?php
/*
 * I retained the same public API as my original StackOverflow question,
 * but instead of performing actual searches I simply return static data
 */

class multithreaded_search {
    private $a, $b, $c;
    public function __construct($a, $b, $c) {
        $this->a = $a;
        $this->b = $b;
        $this->c = $c;
    }

    public function search( $string = '' ) {
        $search_types = array( 'tag', 'substring', 'levenshtein' );
        $pids = array();
        $threads = array();
        $sockets = array();
        foreach( $search_types as $type ) {
            /* Create a socket to write to later */
            $sockets[$type] = array();
            socket_create_pair(AF_UNIX, SOCK_STREAM, 0, $sockets[$type]);
            $pid = pcntl_fork();
            $pids[] = $pid;
            $threads[$pid] = $type;
            if( $pid == 0 ) { // child process
                /* no more confusion */
                $results = call_user_func( 'multithreaded_search::'.$type.'_search', $string );
                /* What do we do with $results ? Write them to a socket! */
                $data = serialize($results);
                socket_write($sockets[$type][0], str_pad($data, 1024), 1024);
                socket_close($sockets[$type][0]);
                exit();
            }
        }
        $results = [];
        for( $i = 0; $i < count( $pids ); $i++ ) {
            $pid = $pids[$i];
            $type = $threads[$pid];
            pcntl_waitpid($pid, $status);
            /* $threads[$pid] tells me the type of search that just finished */
            /* If we need to merge results in the parent, we can do it here */
            $one_result = unserialize(trim(socket_read($sockets[$type][1], 1024)));
            $results[] = $one_result;
            socket_close($sockets[$type][1]);
        }
        /* Now all children have exited, so the search is complete */
        return $results;
    }

    private function tag_search() {
        return $this->a;
    }

    private function substring_search() {
        return $this->b;
    }

    private function levenshtein_search() {
        return $this->c;
    }
}

$instance = new multithreaded_search(3, 5, 7);
var_dump($instance->search());

Notes

This solution uses forked processes and message passing over a local (in-memory) socket. Depending on your use case and setup, this may not be the best solution. For instance:

Upvotes: 11

Views: 12226

Answers (3)

Feras
Feras

Reputation: 2164

Use this class: http://pastebin.com/0wnxh4gY

http://framework.zend.com/manual/1.7/en/zendx.console.process.unix.overview.html

It utilizes shm functions to share variables across many processes with the setVariable method...obviously you shoud use it running PHP in some kind of cgi mode most likely php-fpm

Upvotes: 0

Abraham Covelo
Abraham Covelo

Reputation: 961

As long as father and children know the key/keys of the shared memory segment is ok to do a shmop_open before pcnlt_fork. But remember that pcnlt_fork returns 0 in the child's process and -1 on failure to create the child (check your code near the comment /confusion/). The father will have in $pid the PID of the child process just created.

Check it here:

http://php.net/manual/es/function.pcntl-fork.php

Upvotes: 1

Marc B
Marc B

Reputation: 360672

forked children will gain their own dedicated copy of their memory space as soon as they write anywhere to it - this is "copy-on-write". While shmop does provide access to a common memory location, the actual PHP variables and whatnot defined in the script are NOT shared between the children.

Doing $x = 7; in one child will not make the $x in the other children also become 7. Each child will have its own dedicated $x that is completely independent of everyone else's copy.

Upvotes: 5

Related Questions