Spreading server load with a mini CDN

Question

Using a caching plugin for my wordpress site reduced the server load by quite alot, but I'm trying to reduce it even more. The caching plugin creates a static html file that can be re-used rather than re-creating it with php every single request.

I want to copy this static file to multiple servers so it can be served from multiple locations, thus spreading server load.

The concept

I'm basically doing it like this:

Main server:

//Echo's what the pointer server will echo, which echo's what the sub-server echo's (which is the stored html file's content)
echo file_get_contents("http://$pointerserver?f=$fileNameWeAreRequesting";

Pointer server:

if(isset($_GET['f'])){ $requestedfile = $_GET['f']; }
/* Redirect user to file */
if(isset($_GET['redirect']))
{
    //Filename example: file.html (shares same filename as it's original, 
    //but the content holds a string of sub-servers that host the file, seperated by a comma)
    $servers = file_get_contents("filelocations/$requestedfile");
    $pieces = explode(",", $servers); //Sub-Servers to array
    //Check for online server
    foreach ($pieces as $server)
    {
        if(file_get_contents("servercheck/$server") == "1") //Check if sub-server online
        {
            /* Echo file from sub-server NOW */
            echo file_get_contents("http://$server?f=$requestedfile"); //Echo's whatever the sub-server echo's
            break;
        }
    }
}

Sub-server

/* Provide requested file */
echo file_get_contents("files/$requestedfile"); //Echo's the static html page stored on the sub-server

1. Does this even reduce the load on the main server at all? Or does file_get_contents work in such a way that the main server ends up "parsing"(putting together the file for html output) the file anyway.

2. How can I add a check so that if the sub-server is offline without the pointer server knowing about it, we serve the static html file from the main server instead.

Making the main server ready for this

The caching plugin handles the making of the static file, so for sending it I was thinking:

Main server:

//This doesn't send anything physical, just instructions to what file the pointer server needs to retrieve and then copy over to the sub-severs.
echo file_get_contents("http://$pointerserver?propagate&f=$fileNameWeAreRequesting";

Pointer server: (I know this doesn't work, it's to illustrate the idea)

    if(isset($_GET['f'])){ $requestedfile = $_GET['f']; }
/* Propagate file to sub-servers */
if(isset($_GET['propagate']))
{
    if ($handle = opendir('subservers/')) //Retrieve all available sub-servers
    {
        while (false !== ($server = readdir($handle))) //For-each subserver, send the file
        {
            if ($server != "." && $server != "..") 
            {
                /* CREATE AUTO-SUBMIT FORM HERE TO POST FILE CONTENTS (CONCEPT) */
                action="http://$server/receivefile.php?f=$requestedfile&send"
                $postThisStuff = file_get_contents("http://www.mainserver.xx/cache/$requestedfile"); //Reads the file contents from the main server so it can send it over to the sub-server
            }
        }
    }
}

The sub-server

/* Receive file */
if(isset($_GET['send']))
{
    //Write received post date to file
    $fh = fopen("files/$requestedfile", 'w') or die("can't open file");
    $stringData = $_POST['thefile'];
    fwrite($fh, $stringData);
    fclose($fh);
    die();
}

So my third and final question is; is this copying method a good approach, resource-wise? The static files on the main server rarely change, only when someone posts a comment or an article is edited (or a new one added) but it's still important to be resource-conserving.

Answer 1

first off, I had to up-vote the answer for the great presentation. Your idea is not bad either, but still: to answer your question: "Don't do it!".

1. If the WP plugin is configured to write actual HTMLs of the page, use them first? E.g. bypass the entire PHP stack by checking if a html file exists which could be served for the current request. Apache, Nginx or Lighttpd support this by means of URL rewriting (or even in core in terms of nginx). [Side-note: I have no idea which webserver or Wordpress plugin you use, so I omitted config snippets.]

2. If this doesn't help already, or if this is not enough, distribute the content using a reverse proxy (like Varnish). It's easy to define your main server as the origin server and let various Varnish servers in front of it handle the rest.

3. If Varnish is not good enough and your site can be served from a CDN entirely, have a look at someone like edgecast (speedyrails is a reseller of them). They allow you to do transparent caching:

   • http://my-blog.com is pointed to edgecast's server directly
   • you configure edgecast to check http://mainserver.my-blog.com if no cache is filled
   • configure cache headers to avoid frequent lookups
   • they also allow you to purge the cache (for when you update your blog)

4. If all of the above is not for you, I suggest you check out Cloudflare. They do what you want to do (and more). I'm test-driving them myself, works flawlessly so far.