Node.js & Gulp: Real piping sequence could differ from written '.pipe()'s chain?

Question

In the below code, fileShouldBePreprocessedBySass() will be called before console.log('intercepted!'); execution. Also, in fileShouldBePreprocessedBySass(targetFileAbsolutePath), parameter targetFileAbsolutePath will be undefined:

let currentSourceFileAbsolutePath;

return gulp.src(entryPointsSourceFilesPathsOrGlobs)

    // "gulpPlugins.intercept" is "gulp-intercept"
    .pipe(gulpPlugins.intercept( sourceVynilFile => {
      console.log('intercepted!');
      currentSourceFileAbsolutePath = sourceVynilFile.path;
      console.log(currentSourceFileAbsolutePath); // OK here
      return sourceVynilFile;
    }))

    // "gulpPlugins.if" is "gulp-if"
    .pipe(gulpPlugins.if(
        // currentSourceFileAbsolutePath is undefined !!!
        fileShouldBePreprocessedBySass(currentSourceFileAbsolutePath),
        gulpPlugins.sass()
    ));

// ...

fileShouldBePreprocessedBySass(targetFileAbsolutePath) {

    console.warn('---');
    console.warn(targetFileAbsolutePath); // undefined!

    const targetFilenameExtension = Path.extname(targetFileAbsolutePath);
    let targetFilenameExtensionIsSupportedBySassPreprocessor = false;

    for (const filenameExtension of SUPPORTED_FILENAME_EXTENSIONS__SASS_PREPROCESSOR) {
      if (filenameExtension === targetFilenameExtension) {
        targetFilenameExtensionIsSupportedBySassPreprocessor = true;
        break;
      }
    }

    return targetFilenameExtensionIsSupportedBySassPreprocessor;
}

Really, original code written in TypeScript, but I rewrite it to JavaScript to allow more people to understand the code. I said about it because TypeScript compiler somehow understood, what in pipe(gulpPlugins.if(/*...*/)), parameter currentSourceFileAbsolutePath is not initialized, so TS2454: Variable 'currentSourceFileAbsolutePath' is used before being assigned. error occurred.

I am confused because I have similar task which works without error (in right sequence):

let currentSourceFileAbsolutePath: string;

return gulp.src(entryPointsSourceFilesPathsOrGlobs)

    .pipe(gulpPlugins.intercept(sourceVynilFile => {
      currentSourceFileAbsolutePath = sourceVynilFile.path;
      return sourceFile;
    }))

    .pipe(gulpPlugins.pug())
    .pipe(gulpPlugins.intercept(compiledHtmlFile => {
      // currentSourceFileAbsolutePath is NOT undefined!
      if (shouldValidateCompiledHtmlRespectiveToSourceFile(currentSourceFileAbsolutePath)) {
        HtmlValidator.validateHtml(compiledHtmlFile);
      }
      return compiledHtmlFile;
    }))

    .pipe(gulp.dest(() => (
      // currentSourceFileAbsolutePath is NOT undefined!
      getOutputDirectoryForPreprocessedMarkupEntryPointFileByRespectiveSourceFile(currentSourceFileAbsolutePath)
    )));

What wrong in the first task? I missed some async call?

Answer 1

Gulp, by itself, does not reorder what goes through subsequent calls to .pipe. I'm not sure what is going on exactly in your case, but your code is unnecessarily brittle.

Consider this pattern:

gulp.src("...")
  .pipe(pluginA)
  .pipe(pluginB)

And suppose that pluginA just prints to the console pluginA and pluginB just prints to the console pluginB. You might think that the output would be:

pluginA
pluginB
pluginA
pluginB
pluginA
...

In other words, each file will go through pluginA and then this file will go through pluginB, then the next file will go through pluginA, etc. That is, pluginA won't see a new file until pluginB has processed the previous file that pluginA has finished processing. Though there are cases in which this order is exactly what will happen, for the general case this order is not guaranteed. You can just as well have:

pluginA 
pluginA 
pluginA 
pluginB 
pluginA 
pluginB 
pluginB 
pluginB
pluginA 
...

The files must be seen by pluginA first, but there's no guaranteed that pluginB will process a file before pluginA gets to its next file.

Your code works fine if the processing order happens to be perfectly interleaved like the first example I gave above. But in general, this order is not guaranteed, and your code will fail.

It so happens that the brittle part of your code is unnecessary. It is not necessary to record the file's path in a variable outside the file stream and then read it in a later part of the stream. gulp-if will pass the file it tests to the condition you give to it. Here is an example:

const gulp = require("gulp");
const gulpIf = require("gulp-if");
gulp.task("default", () =>
          gulp.src("src/**")
          .pipe(gulpIf((file) => {
            console.log(file.path);
            return file.basename === "a";
          }, gulp.dest("out"))));

This task will put in the out subdirectory all files whose basename is "a". You'll also see on the console the absolute path of each file that passes through the condition given to gulpIf.

You could modify fileShouldBePreprocessedBySass to accept a Vinyl file and just check file.path there instead of saving it and then retrieving it.