How to automate Javascript combining and minification, in groups of files for different webpages?

Question

Let's say my page structure is :

1. one.html :
   includes ->   a.js , b.js , c.js ,d.js

2. two.html :
   includes ->   a.js , b.js,  x.js, y.js, z.js

3. three.html :
   includes ->   a.js , b.js, s.js,  x.js, y.js

and so on. Some pages are more visited than others, say 3 pages contribute 99% of all page views of the website.

I am looking for a solution to:

i) Combine and minimize files in groups which can be included in the pages.

ii) Has some logic to map files names of the group, to final combined file name.

iii) Includes a minifier like Google Closure compiler / YUI compressor.

One solution I have looked at is: PHP minify

which does most of it. However it has following drawbacks for me:

i) I would be hosting my static combined files on a CDN server,not on same web server hosting PHP minify, hence PHP minify's logic to server files by group name does not work for me.

ii) PHP Minify uses PHP CGI to process and serve the scripts, whereas I would want my minified content to be served directly from the CDN server.

Does PHP Minify have some functions to map group name to combined file name, which I can use in my webpage to directly set CDN path of the combined JS file. eg

<?php
  $groupName = array("onePage" => array('a.js','b.js','c.js','d.js');
?>

<script type="text/javascript" src="http://www.MYSTATICSERVER.com/js/<?php getMergedFileName($groupName)"></script>

Rather than calling PHP Minify's PHP script to get files of a group, which is actually a PHP page call,which then serves the javascript content from previously generated files:

<script type="text/javascript" src="http://www.MYWEBSERVER.com/min/?g=onePage" ></script>

( I agree most of this is doable by combining different solutions with custom deployment scripts and minifying tools eg ANT,FABRIC + YUICompressor/ClosureCompiler, but I am looking for a well developed configurable solution that I might have missed )

Answer 1

Updated to show example for minify

It does appear doable using minify. I'm not sure if you have tried this out, but putting it out there for anyone else who might need it

1) Minify can generate a combined and gzipped copy of your scripts and use it as a cache so that it need not process all your files for ever request. To enable this, just edit config.php file with the location of the temp directory

$min_cachePath = 'c:\\xampp\\htdocs\\min\\cache';

2) After you add all your groups in groupsConfig.php, just make a request to each group so that minify generates the outputfiles in the cache folder

3) For each group, you will find 2 files in the temp folder named so:

minify_g=group1_08da191ba9c60a0ed611d0de455bb7a4
minify_g=group1_08da191ba9c60a0ed611d0de455bb7a4.gz

4) You can rename the files and deploy them directly to your CDN as required

5) If your CDN allows url rewriting, you can rewrite your script url to serve up the JS, without the need to change the location in the pages that you serve.

Unless you have a huge number of different configurations, I'd recommend you do it using YUICompressor and deploy to your CDN network. It is actually quite trivial to automate something like that using a simple shell script. If however you have a really complicated setup, you can consider using a build tool like grunt that runs on top of node.js. I have been using Grunt to build JS files for different projects using the same codebase, and it works quite well. In addition, lint and compression are supported OOTB.

Answer 2

My own projects are much like yours; HTML with both external and interned JavaScript, CSS and images. I too have looked around for a ready-made solution and wasn't able to find it, so I've developed my own. Here's a run-down of the system I've created. If you're going that route I hope this gives you an idea of what to look for.

The system starts by checking out a particular revision of the project you wish to deploy. You need to establish a translation between paths on the local file system and their respective remote paths (e.g. CDN or web server); in my case, all assets are loaded from a fixed directory inside the project.

I start by parsing all HTML templates (I have an MVC to separate HTML from PHP code); this collects all JavaScript, CSS and image references:

• Image references inside HTML, JavaScript or CSS get replaced by their location on the CDN.
• Interned JavaScript and CSS get compressed on-the-fly using Closure Compiler and YUI respectively.
• Consecutive external JavaScript and CSS blocks get replaced by single bundles of compressed code / style; the bundle name is a hash of file names and their respective version (CDN will serve those files with maximum expiry date, so any change must cause a unique file name).

Then, all bundles get compressed, zipped, saved and uploaded to the CDN together with any images in preparation of deployment. PNG files are also "crushed" to reduce their size.

Finally, the whole project is uploaded and deployed (in case of multiple servers).

Conclusion it's a lot of work, but when it runs it's great to see it work :)

Answer 3

grunt is great. If that doesn't work, try cassette: http://getcassette.com/

Answer 4

Create a grouping configuration file(xml/php), which contains the grouping rules for use in static html you have.

<groupings>
    <group path='grouped_minified_js.js'><!-- multiple groups can be used for grouping js/css files. reuse this group by including the id in multiple views -->
            <resource>PATCH_OF_FILE_TO_BE_GROUPED</resource><!-- multiple file to be grouped -->
            .
            .
            .
        </group> 
        <group path='xyz.css' /> ...<!-- additional grouping rule -->
</groupings>

create a php script which reads this config and uses PHP minify to create grouped minified js/css files at the path specified in group definition.

Use these paths to include grouped files into the static html files. Run the script to test your pages and deploy the grouped files to the CDN.