CODEWITHSUNDEEP
node.js
josh
josh

Reputation: 9538

Changing how nodejs require() fetches files

I'm looking to monkey-patch require() to replace its file loading with my own function. I imagine that internally require(module_id) does something like:

  1. Convert module_id into a file path
  2. Load the file path as a string
  3. Compile the string into a module object and set up the various globals correctly

I'm looking to replace step 2 without reimplementing steps 1 + 3. Looking at the public API, there's require() which does 1 - 3, and require.resolve() which does 1. Is there a way to isolate step 2 from step 3?

I've looked at the source of require mocking tools such as mockery -- all they seem to be doing is replacing require() with a function that intercepts certain calls and returns a user-supplied object, and passes on other calls to the native require() function.

For context, I'm trying to write a function require_at_commit(module_id, git_commit_id), which loads a module and any of that module's requires as they were at the given commit.

I want this function because I want to be able to write certain functions that a) rely on various parts of my codebase, and b) are guaranteed to not change as I evolve my codebase. I want to "freeze" my code at various points in time, so thought this might be an easy way of avoiding having to package 20 copies of my codebase (an alternative would be to have "my_code_v1": "git://..." in my package.json, but I feel like that would be bloated and slow with 20 versions).

Update:

So the source code for module loading is here: https://github.com/joyent/node/blob/master/lib/module.js. Specifically, to do something like this you would need to reimplement Module._load, which is pretty straightforward. However, there's a bigger obstacle, which is that step 1, converting module_id into a file path, is actually harder than I thought, because resolveFilename needs to be able to call fs.exists() to know where to terminate its search... so I can't just substitute out individual files, I have to substitute entire directories, which means that it's probably easier just to export the entire git revision to a directory and point require() at that directory, as opposed to overriding require().

Update 2:

Ended up using a different approach altogether... see answer I added below

Upvotes: 2

Views: 944

Answers (2)

josh
josh

Reputation: 9538

So rather than mess with the node require() module, what I ended up doing is archiving the given commit I need to a folder. My code looks something like this:

# commit_id is the commit we want
# (note that if we don't need the whole repository, 
# we can pass "commit_id path_to_folder_we_need")
#
# path is the path to the file you want to require starting from the repository root 
# (ie 'lib/module.coffee')
#
# cb is called with (err, loaded_module)
#
require_at_commit = (commit_id, path, cb) ->
    dir = 'old_versions' #make sure this is in .gitignore!
    dir += '/' + commit_id

    do_require = -> cb null, require dir + '/' + path

    if not fs.existsSync(dir)   
        fs.mkdirSync(dir)            
        cmd = 'git archive ' + commit_id  + ' | tar -x -C ' + dir
        child_process.exec cmd, (error) ->
            if error
                cb error
            else
                do_require()
    else
        do_require()

Upvotes: 0

Peter Lyons
Peter Lyons

Reputation: 146004

You can use the require.extensions mechanism. This is how the coffee-script coffee command can load .coffee files without ever writing .js files to disk.

Here's how it works:

https://github.com/jashkenas/coffee-script/blob/1.6.2/lib/coffee-script/coffee-script.js#L20

  loadFile = function(module, filename) {
    var raw, stripped;
    raw = fs.readFileSync(filename, 'utf8');
    stripped = raw.charCodeAt(0) === 0xFEFF ? raw.substring(1) : raw;
    return module._compile(compile(stripped, {
      filename: filename,
      literate: helpers.isLiterate(filename)
    }), filename);
  };

  if (require.extensions) {
    _ref = ['.coffee', '.litcoffee', '.md', '.coffee.md'];
    for (_i = 0, _len = _ref.length; _i < _len; _i++) {
      ext = _ref[_i];
      require.extensions[ext] = loadFile;
    }
  }

Basically, assuming your modules have a set of well-known extensions, you should be able to use this pattern of a function that takes the module and filename, does whatever loading/transforming you need, and then returns an object that is the module.

This may or may not be sufficient to do what you are asking, but honestly from your question it sounds like you are off in the weeds somewhere far from the rest of the programming world (don't take that harshly, it's just my initial reaction).

Upvotes: 1

Related Questions