CODEWITHSUNDEEP
npmgruntjs
RedRooster
RedRooster

Reputation: 33

grunt unused - loop subfolders in directory

I am attempting to remove all unused image links within multiple subdirectories using grunt-unused. Here is my folder structure for clarity:

|-- dist
|  |-- site-1
|  |  |—-index.html
|  |  |—-style.css
|  |  |—-app.js
|  |  |—-image1.jpg
|  |  |—-image2.jpg
|  |  |—-image3.jpg
|  |  |—-image4.jpg
|  |-- site-2
|  |  |—-index.html
|  |  |—-style.css
|  |  |—-app.js
|  |  |—-image1.jpg
|  |  |—-image2.jpg
|  |  |—-image3.jpg
|  |  |—-image4.jpg
|  |-- site-3
|  |  |—-index.html
|  |  |—-style.css
|  |  |—-app.js
|  |  |—-image1.jpg
|  |  |—-image2.jpg
|  |  |—-image3.jpg
|  |  |—-image4.jpg

I have wrote a 'forEach' function to target these child folders and then reference each folder, similar to this post but it is not working for the unused task. It does not loop through each directory and console detects no unused files in a very quick time, like it is not actually doing anything.

Where am I going wrong?

grunt.registerTask('prodOutput', function () {

    // read all subdirectories of dist/ folder, excluding the assets folder
    grunt.file.expand('dist/*', '!dist/assets/**').forEach(function (dir) {

        //get the current unused config
        var unusedConfig = grunt.config.get('unused') || {};

        //set options for grunt-unused, including the folder variable 'dir' as the reference of where to look
        unusedConfig[dir] = {
            option: {
                reference: dir + '/',
                directory: ['**/*.css', '**/*.html', '**/*.js'],
                remove: false, // set to true to delete unused files from project
                reportOutput: 'report.txt', // set to false to disable file output
                fail: false // set to true to make the task fail when unused files are found
            }
        };

        grunt.config.set('unused', unusedConfig);

    });

    grunt.task.run('unused');

});

Upvotes: 3

Views: 917

Answers (1)

RobC
RobC

Reputation: 24992

Issue

As mentioned earlier in my comment... The main issue is that you are dynamically configuring multiple Targets in your forEach Function prior to running the task, however grunt-unused does not support multiple target configurations.

Also grunt-unused expects the files that include references/links to other files (i.e. index.html) to be in a different directory/folder than the files it references (e.g. images, css, etc), however the folder structure provided in your post is flattened.

Solution

After a quick look around at other grunt plugins there doesn't seem to be one that will meet your requirement. The only way I think you can achieve this is to write your own custom Task/plugin to handle this.

To achieve this you could do the following:

Create a separate Javascript module that exports a Registered MutliTask. Name the file delete-unused.js and save it to a directory named tasks which resides in the same top level directory as your Gruntfile.js.

Directory structure

Your directory structure should be something like this:

.
├── dist
│   ├── site-1
│   ├── site-2
│   ├── ...
│   └── assets
│
├── Gruntfile.js
│
├── node_modules
│   └── ...
│
├── package.json
│
└── tasks
    └── delete-unused.js

delete-unused.js

module.exports = function(grunt) {

  'use strict';

  // Requirements
  var fs = require('fs');
  var path = require('path');

  grunt.registerMultiTask('unused', 'Delete unused assets', function() {

    // Default options. These are used when no options are
    // provided via the  initConfig({...}) papaparse task.
    var options = this.options({
      reportOutput: false,
      remove: false
    });

    var reportTxt = '';

    // Loop over each src directory path provided via the configs src array.
    this.data.src.forEach(function(dir) {

      // Log if the directory src path provided cannot be found.
      if (!grunt.file.isDir(dir)) {
        grunt.fail.warn('Directory not found: '.yellow + dir.yellow);
      }

      // Append a forward slash If the directory path provided
      // in the src Array does not end with one.
      if (dir.slice(-1) !== '/') {
        dir += '/';
      }

      // Generate the globbin pattern (only one level deep !).
      var glob = [dir, '*'].join('');

      // Create an Array of all top level folders (e.g. site-*)
      // in the dist directory and exclude the assets directory.
      var dirs = grunt.file.expand(glob).map(function(dir) {
        return dir;
      });

      // Loop over each directory.
      dirs.forEach(function(dir) {

        // Add the folders to exclude here.
        if (dir === './dist/assets') {
          return;
        }

        // Log status and update report
        grunt.log.write('\nProcessing folder ' + dir);
        reportTxt += '\nProcessing folder ' + dir;

        // Empty Array to be populated with unused asset references.
        var unused = [];

        // Define the path to index.html
        var pathToHtml = [dir, '/', 'index.html'].join('');

        // Create Array of file names and filepaths (exclude index.html)
        var assets = grunt.file.expand([dir + '/**/*.*', '!index.html'])
          .map(function(file) {
            return {
              fpath: file,
              fname: path.basename(file)
            };
          });

        // Log/Report missing 'index.html' and return early.
        if (!grunt.file.exists(pathToHtml)) {
          grunt.log.write('\n  >> Cannot find index.html in ' + dir + '/\n');
          reportTxt += '\n  >> Cannot find index.html in ' + dir + '/\n';
          return;
        }

        // Read the contents of index.html.
        var html = fs.readFileSync(pathToHtml, {
          encoding: 'utf8'
        });

        // Loop over each file asset to find if linked in index.html
        assets.forEach(function(asset) {

          // Backslash-escape the dot [.] in filename for RegExp object.
          var escapedFilename = asset.fname.replace('.', '\\.');

          // Dynamically create a RegExp object to search for instances
          // of the asset filename in the contents of index.html.
          // This ensures the reference is an actual linked asset and
          // not plain text written elsewhere in the document.
          //
          // For an explanation of this Regular Expression visit:
          // https://regex101.com/r/XZpldm/4/
          var regex = new RegExp("(?:href=|src=|url\\()(?:[\",']?)(.*"
              + escapedFilename + ")[\",',\\)]+?", "g");

          // Search index.html using the regex
          if (html.search(regex) === -1 && asset.fname !== 'index.html') {
            unused.push(asset); // <-- Not found so add to list.
          }
        });

        // Log status and update report
        grunt.log.write('\n  ' + unused.length + ' unused assets found:\n');
        reportTxt += '\n  ' + unused.length + ' unused assets found:\n';

        //Delete the unused asset files.
        unused.forEach(function(asset) {
          if (options.remove) {
            grunt.file.delete(asset.fpath);

            // Log status and update report
            grunt.log.write('  deleted: ' + asset.fpath + '\n');
            reportTxt += '  deleted: ' + asset.fpath + '\n';
          } else {
            // Log status and update report
            grunt.log.write('    ' + asset.fpath + '\n');
            reportTxt += '    ' + asset.fpath + '\n';
          }
        });

      });

      if (options.reportOutput) {
        grunt.file.write(options.reportOutput, reportTxt);
      }

    });
  });
};

Gruntfile.js

Configure your Gruntfile.js as follows.

module.exports = function(grunt) {

  'use strict';

  grunt.initConfig({
    // ...
    unused: {
      options: {
        reportOutput: 'report.txt',
        remove: true
      },
      dist: {
        src: ['./dist/']
      }
    }

  });

  // Load the custom multiTask named `unused` - which is defined
  // in `delete-unused.js` stored in the directory named `tasks`.
  grunt.loadTasks('./tasks');

  // Register and add unused to the default Task.
  grunt.registerTask('default', [
    // ...
    'unused',
    // ...
  ]);

  // Or add it to another named Task.
  grunt.registerTask('foobar', [
    // ...
    'unused',
    // ...
  ]);

};

Notes

  1. The grunt-unused plugin is no longer used, so you can uninstall it by running the following via your CLi tool: $ npm un -D grunt-unused

  2. The delete-unused.js custom module provides a similar configuration in Gruntfile.js with less options than those found in grunt-unused. The src Array in the config accepts a path(s) to the folder to be processed (i.e. ./dist/). and not a glob pattern - the glob pattern is generated inside grunt-unused.js.

  3. The default options for reportOutput and remove are set to false in grunt-unused.js. When you first run the Task I recommend that you initially set the remove option to false in your Gruntfile.js configuration. This will simply log to the console the unused assets and allow you to check whether it meets your requirement. Obviously setting the remove option to true will delete any unused assets when it's re-run.

  4. I noticed that in your post you provided the glob pattern '!dist/assets/**' to exclude the assets folder from being processed. Instead of passing the glob pattern to exclude via the src configuration this has been hard-coded in delete-unused.js. You'll see it in the lines that read:

// Add the folders to exclude here.
if (dir === './dist/assets') {
    return;
}

If there are additional directories inside the dist folder that you want to exclude you'll need to add them there: For example:

// Add the folders to exclude here.
if (dir === './dist/assets'
    || dir === './dist/foo'
    || dir === './dist/quux') {
    return;
}

  1. This solution checks whether the assets found inside the site-* folders are linked/referenced inside the corresponding index.html only and does not check whether they are referenced in any .css file.

  2. delete-unused.js utilizes a Regex to find whether the asset is actually linked to the index.html and is not as plain text written elsewhere in the document (inside a paragraph of text for example). An explanation of the custom Regular Expression used can be found here.

I hope this helps !

Upvotes: 1

Related Questions