Best way to create a "fresh and empty" Node.js JavaScript context within the same process?

Question

Question for Node.js and v8 experts.

I'm developing a new version of the Siesta testing tool.

By default, Siesta runs every test in the newly created Node.js process. However, I'd like to avoid the overhead of spawning a new process and instead provide the ability to run the test in the empty JavaScript context.

Such context can be created with the built-in vm module. However, the context created in this way is an empty JavaScript context, not an empty Node.js context. For example, it does not have global variable process:

> require('vm').runInNewContext('process')
evalmachine.<anonymous>:1
process
^

Uncaught ReferenceError: process is not defined
    at evalmachine.<anonymous>:1:1
    at Script.runInContext (vm.js:143:18)
    at Script.runInNewContext (vm.js:148:17)
    at Object.runInNewContext (vm.js:303:38)
    at REPL30:1:15
    at Script.runInThisContext (vm.js:133:18)
    at REPLServer.defaultEval (repl.js:484:29)
    at bound (domain.js:413:15)
    at REPLServer.runBound [as eval] (domain.js:424:12)
    at REPLServer.onLine (repl.js:817:10)
> 

So question is - what is the best way to create a fresh and empty Node.js context within the same process? I'd expect such context to have all regular globals, like process, require etc. Plus, I'd expect such context to have a separate and initially empty modules cache, so that even if some module is loaded in the main context, it will be loaded again in the new context.

Of course I could map the globals from the main context to the new context, but that would mean those globals are shared between contexts, and I'm aiming for context isolation. Plus the modules cache will be shared as well.

I believe the difference between JavaScript and Node.js context is that the latter is initialized with a certain script. Is it possible to obtain the sources of that script somehow and execute it in the new context?

Thank you!

Answer 1

This is what happens when NodeJS loads a new module:

(function(exports, require, module, __filename, __dirname) {
// Module code actually lives in here
});

They use what they call a Module Wrapper

So you would have to do something similar using VM

require('vm').runInNewContext('the code you are running', {
  module:  // your empty module or a wrapper
  exports: // a reference to the module.exports
  require: // your empty require or a wapper
  __filename: // your __filename
  __dirname: // your __dirname
  process
})

I was successful in creating valid new empty require functions after debugging the internals of the Module Class/Function

If you create these 2 following files and run the isolated.js you'll see that it is indeed reloading every time and at the same time the original cache stays intact (although I believe async executions could have unexpected results)

// isolated.js
const Module = require('module')
const vm = require('vm')
const path = require('path')

// this is just to show it wont load it again
const print = require('./print')
print('main file')

const getNewContext = () => {
  const mod = new Module()
  const filename = path.join(__dirname, `test-filename-${Math.random().toString().substr(-6)}.js`)
  const req = Module.createRequire(filename)
  req.cache = Object.create(null)

  return {
    module: mod, // your empty module or a wrapper
    exports: mod.exports, // a reference to the module.exports
    require: req, // your empty require or a wapper
    // require: mod.require, // your empty require or a wapper
    __filename: filename,
    __dirname: path.dirname(filename),
    console,
    process
  }
}

// print('runningInNewContext')
const jsCode = `
// What?
const log = console.log.bind(null, Date.now())
log({from:'print', cache: require.cache})
const print = require('./print.js')
log(print.toString())
print('evaluated code')
log({from:'print', cache: require.cache})
`

function customRunInNewContext (jsCode, context) {
  const { _cache } = Module

  Module._cache = Object.create(null)
  vm.runInNewContext(jsCode, context)
  Module._cache = _cache
}

customRunInNewContext(jsCode, getNewContext())
customRunInNewContext(jsCode, getNewContext())

// print.js
const log = console.log.bind(null, 'PRINT loaded at', new Date().toISOString(), module.parent.filename)
process.stdout.write('\n ====> loading print module\n\n')

module.exports = function print (...args) {
  log(...args)
}

When you run

node ./isolated.js

You should see the message ====> loading print module multiple times

Answer 2

One option you have is using 'child_process' fork() API, which spins a new Node process with it's own memory and V8 instance. However this might be just the same, or very similar (at least theoretically) as what you are doing right now.

Keep in mind that spawned Node.js child processes are independent of the parent with exception of the IPC communication channel that is established between the two. Each process has its own memory, with their own V8 instances. Because of the additional resource allocations required, spawning a large number of child Node.js processes is not recommended.

Note that this fork is not a cloning of the entire running process:

Unlike the fork(2) POSIX system call, child_process.fork() does not clone the current process.

Another option which looks promising, is using the Node async_hooks API to emulate executions on separated contexts. Check out node-execution-context, it may work out for your requirements.