Context-preserving eval

Question

We're building a small REPL that evaluates (with eval) javascript expressions as they are being entered by the user. Since the whole thing is event-driven, evaluation must take place in a separate function, but the context (that is, all declared variables and functions) must be preserved between the calls. I came up with the following solution:

function* _EVAL(s) {
    while (1) {
        try {
            s = yield eval(s)
        } catch(err) {
            s = yield err
        }
    }
}

let _eval = _EVAL()
_eval.next()

function evaluate(expr) {
    let result = _eval.next(expr).value
    if (result instanceof Error)
        console.log(expr, 'ERROR:', result.message)
    else
        console.log(expr, '===>', result)
}

evaluate('var ten = 10')
evaluate('function cube(x) { return x ** 3 }')
evaluate('ten + cube(3)')
evaluate('console.log("SIDE EFFECT")')
evaluate('let twenty = 20')
evaluate('twenty + 40') // PROBLEM

As you can see it works fine with function-scoped variables (var and function), but fails on block scoped ones (let).

How can I write a context-preserving eval wrapper that would also preserve block-scoped variables?

The code runs in a browser, DOM and Workers are fully available.

It should be mentioned that the desired function must handle side effects properly, that is, each line of code, or, at least, each side effect, should be performed exactly once.

Links:

JavaScript: do all evaluations in one vm | https://vane.life/2016/04/03/eval-locally-with-persistent-context/

Answer 1

TL;DR

Here is the recommended, best solution I came up with below, supporting all expressions including promise-based expressions like fetch(), making use of async/await and nesting evaluate() in the final then() of my fetch().

Note (also mentioned in full post below)

The result of the nested evaluate() expression is logged first. This is correct and to be expected as that nested expression runs within the fetch() that runs it. Once the entire fetch runs, it will return undefined just as a variable assignment would. For every other [non-recommended] solution in my answer below, the title variable will be evaluated if and after the fetch() statement has been fully evaluated successfully. This is because we are either forcefully deferring the evaluation of the title variable by means of setTimeout() or a pre-processed then(), or by means of forced sequential loading in the "BONUS" solution at the bottom of this solution.

var __EVAL = s => eval(`void (__EVAL = ${__EVAL.toString()}); ${s}`);

async function evaluate(expr) {
  try {
    const result = await __EVAL(expr);
    console.log(expr, '===>', result)
  } catch (err) {
    console.log(expr, 'ERROR:', err.message)
  }
}

evaluate('var ten = 10')
evaluate('function cube(x) { return x ** 3 }')
evaluate('ten + cube(3)')
evaluate('let twenty = 20')
evaluate('twenty + 40')
evaluate('let title = ""')
evaluate('fetch("https://jsonplaceholder.typicode.com/todos/1").then(res => res.json()).then(obj => title = obj.title).then(() => evaluate("title"))')

The madness explained

A few other solutions came very close here, so I must give credit to both Bergi and Brandon McConnell— Bergi for his/her clever use of closures with eval() and Brandon for his ingenuity in using a "stepped" result.

The correct solution does exist, and it does work with promises. For ease of use, I did use Bergi's solution as a foundation for my own, and if you do select my answer, I will gladly split the reputation bonus evenly with them.

Simply by making the evaluate() function async/await, you allow it to work with promises. From here, you have to decide how you would like for it to run— either organically, where fetch() statements run asynchronously as they normally would, or synchronously and wait for any Promise to be settled before proceeding to the next evaluate() call.

In my solution here, I chose to go the organic route as this is how JavaScript does actually work natively. To force all promises to run before proceeding would be to circumvent the nature of JavaScript. For example, if you were using this code to build a JavaScript engine or compiler, you would want the code to run the same with your engine as it would on the web for other users, so organic would be the wait to go.

BONUS ✨👀✨
If you would like to explore the non-organic, forced-sequential ordering idea I mentioned above, please scroll down to the bottom of this solution where I explain that concept in detail, link to an external resource that explains how to forcefully execute promises sequentially, and show a live working prototype of that idea in action.

If someone using your engine wants to wait for a fetch() to finish loading before proceeding, then they should adhere to the proper usage of then() as they would in other projects.

We can accomplish this by one of a few methods for fetch():

1. Include evaluate() in an actual then() statement ORGANIC & RECOMMENDED
2. Add a chaining command to our evaluate expression which will allow us to run one once another has completed. This can work out nicely for us in pure execution but it is not NOT RECOMMENDED as this adds special logic to the actual evaluation logic rather than the code being evaluated, so it is more of a server-side or back-end evaluation in a sense than the actual JS code running.
3. Add a delay on the next line using setTimeout() to provide time for the fetch() to complete. NOT RECOMMENDED as this does not guarantee the promise has been settled, whether resolved or rejected. Fetch and async/await both deal with promises, so we should use promises to wait for them as well.

Here are examples for all three methods:

1. Including a nested evaluate() expression in our fetch().then() ✅

Note: One important note here is that you will see the result of the nested evaluate() expression first. This is correct and to be expected as that nested expression runs within the fetch() that runs it. Once the entire fetch runs, it will return undefined just as a variable assignment would.

For every other [non-recommended] solution in my answer below, the title variable will be evaluated if and after the fetch() statement has been fully evaluated successfully. This is because we are either forcefully deferring the evaluation of the title variable by means of setTimeout() or a pre-processed then(), or by means of forced sequential loading in the "BONUS" solution at the bottom of this solution.

var __EVAL = s => eval(`void (__EVAL = ${__EVAL.toString()}); ${s}`);

async function evaluate(expr) {
  try {
    const result = await __EVAL(expr);
    console.log(expr, '===>', result)
  } catch (err) {
    console.log(expr, 'ERROR:', err.message)
  }
}

evaluate('var ten = 10')
evaluate('function cube(x) { return x ** 3 }')
evaluate('ten + cube(3)')
evaluate('let twenty = 20')
evaluate('twenty + 40')
evaluate('let title = ""')
evaluate('fetch("https://jsonplaceholder.typicode.com/todos/1").then(res => res.json()).then(obj => title = obj.title).then(() => evaluate("title"))')

2. Chaining then() onto evaluate() (NOT RECOMMENDED) ⚠️

Note: In order to add this chaining method to our evaluate() expressions, we must return a new promise each time we run then. These promises, however, can/will be self-settling, so they simply allow us to chain the then() statement to the end of any evaluate() calls. Here is how that would work:

var __EVAL = s => eval(`void (__EVAL = ${__EVAL.toString()}); ${s}`);

async function evaluate(expr) {
  try {
    const result = await __EVAL(expr);
    return new Promise(_ => _(console.log(expr, '===>', result)))
  } catch (err) {
    console.log(expr, 'ERROR:', err.message)
  }
}

evaluate('var ten = 10')
evaluate('function cube(x) { return x ** 3 }')
evaluate('ten + cube(3)')
evaluate('let twenty = 20')
evaluate('twenty + 40')
evaluate('let title = ""')
evaluate('fetch("https://jsonplaceholder.typicode.com/todos/1").then(res => res.json()).then(obj => title = obj.title)')
  .then(() => evaluate('title'))

3. Using setTimeout() (NOT RECOMMENDED) ⚠️

Note: The issue here which I mentioned previously is that there is no guarantee of when the promise will be settled without waiting for the promise itself. Using setTimeout() grants the advantage over option #2 in that this runs as pure JS and does not work around the JS by running extra processes in the background, but this solution would require you to guess at how long your fetch might take to complete. This is not recommended, whether for this evaluate() function or in practice on actual projects. Option #1 using fetch().then() is the only solution that offers the flexibility of waiting for the promise to settle within the actual code that is entered and also waits until the promise has successfully settled.

❗ Even sometimes when running the snippet below, after waiting a full second, the fetch() still has not completed, and the setTimeout() executes first producing a blank string rather than the actual title string as desired. After repeated testing, this appears to work most of the time but not all of the time.

var __EVAL = s => eval(`void (__EVAL = ${__EVAL.toString()}); ${s}`);

async function evaluate(expr) {
  try {
    const result = await __EVAL(expr);
    console.log(expr, '===>', result)
  } catch (err) {
    console.log(expr, 'ERROR:', err.message)
  }
}

evaluate('var ten = 10')
evaluate('function cube(x) { return x ** 3 }')
evaluate('ten + cube(3)')
evaluate('let twenty = 20')
evaluate('twenty + 40')
evaluate('let title = ""')
evaluate('fetch("https://jsonplaceholder.typicode.com/todos/1").then(res => res.json()).then(obj => title = obj.title)')
evaluate('setTimeout(() => evaluate("title"), 1000)')

---

BONUS: Forced-Sequential Execution (NOT RECOMMENDED) ⚠️

If you do want to force all evaluate functions to wait for the previous one to fully complete before proceeding, which I would highly discourage as this is not how JS will work in an actual browser, there is a great article on this exact concept of continuous/recursive promise chaining by James Sinclair which you can read here under the section labeled "A Sequential Solution". It is important if you choose to go this route that you do not simply use Promise.all() as that will not guarantee the order of execution of each promise. Instead, use a recursive chain of promise().then().

Here is how this might look in practice:

const exprs = [];
const evaladd = expr => exprs.push(expr);

var __EVAL = s => eval(`void (__EVAL = ${__EVAL.toString()}); ${s}`);

async function evaluate(expr) {
  try {
    const result = await __EVAL(expr);
    return new Promise(_ => _(console.log(expr, '===>', result)))
  } catch(err) {
    console.log(expr, 'ERROR:', err.message)
  }
}

function evaluateAll(exprs = []) {
  evaluate(exprs[0]).then(() => exprs.length > 1 && evaluateAll(exprs.slice(1)));
}

evaladd('var ten = 10')
evaladd('function cube(x) { return x ** 3 }')
evaladd('ten + cube(3)')
evaladd('let twenty = 20')
evaladd('twenty + 40')
evaladd('let title = ""')
evaladd('fetch("https://jsonplaceholder.typicode.com/todos/1").then(res => res.json()).then(obj => title = obj.title)')
evaladd('title')
evaluateAll(exprs)

Answer 2

The article you linked contains a crazy approach that actally works: during each eval() call, we create a new closure inside that eval scope and export it so that to we can use it evaluate the next statement.

var __EVAL = s => eval(`void (__EVAL = ${__EVAL.toString()}); ${s}`);

function evaluate(expr) {
    try {
        const result = __EVAL(expr);
        console.log(expr, '===>', result)
    } catch(err) {
        console.log(expr, 'ERROR:', err.message)
    }
}

evaluate('var ten = 10')
evaluate('function cube(x) { return x ** 3 }')
evaluate('ten + cube(3)')
evaluate('console.log("SIDE EFFECT")')
evaluate('let twenty = 20')
evaluate('twenty + 40') // NO PROBLEM :D

Answer 3

It is native JavaScript behavior to return 5 if you run 5; let a = 2 which is essentially what is happening here in the final statement of your program when you check for the value of twenty + 40. However, a quick workaround for this would be to gather both results, the full result fullResult and the result of just that step (stepResult). With both of these, once a success is met in your evaluate() function, we can check to see if stepResult is equal to undefined which occurs when assigning a new variable value.

If this is the case, we use that value undefined. Otherwise, we use the value of fullResult, which works in every case of the provided code in your question:

const pastEvals = [];

function* _EVAL(s) {
    while (1) {
        try {
            s = yield eval(s)
        } catch(err) {
            s = yield err
        }
    }
}

let _eval = _EVAL()
_eval.next()

function evaluate(expr) {
    pastEvals.push(expr)
    const fullResult = _eval.next(pastEvals.join(';')).value
    const stepResult = _eval.next(expr).value
    if (fullResult instanceof Error)
        console.log(expr, 'ERROR:', result.message)
    else
        console.log(expr, '===>', stepResult === undefined ? stepResult : fullResult);
}

evaluate('var ten = 10')
evaluate('function cube(x) { return x ** 3 }')
evaluate('ten + cube(3)')
evaluate('let twenty = 20')
evaluate('twenty + 40')

You will run into problems with this when trying to use more advanced JS functions such as async/await and fetch, but it should work just fine for these simpler use-cases. If you need to build something that works for more advanced uses, you may need to create a virtual DOM behind the scenes, create and destroy a new virtual DOM with each run, and also wait until any created promises are fulfilled and completed between iterations, as that is what would be required for any fetch-related operations.

Answer 4

If the user-entered code isn't meant to have any side-effects outside of their uses of evaluate, one approach is to concatenate the new input strings onto the old input strings. So, for example:

evaluate('ten + cube(3)')
evaluate('let twenty = 20')

results in the following being run. First time:

ten + cube(3)

Second time:

ten + cube(3)
let twenty = 20

This isn't very elegant since the code will have to run all code previously entered every time, but it'll at least make the repl functional.

function* _EVAL(codeToTry) {
    let userCode = '';
    while (1) {
        while (!codeToTry) {
            codeToTry = yield null;
        }
        try {
            const newCode = userCode + ';' + codeToTry;
            const result = eval(newCode)
            // No error, so tack onto userCode:
            userCode = newCode;
            codeToTry = yield result;
        } catch(err) {
            // Error, don't tack onto userCode:
            codeToTry = yield err
        }
    }
}

let _eval = _EVAL()
_eval.next()

function evaluate(expr) {
    let result = _eval.next(expr).value
    if (result instanceof Error)
        console.log(expr, 'ERROR:', result.message)
    else
        console.log(expr, '===>', result)
}

evaluate('var ten = 10')
evaluate('function cube(x) { return x ** 3 }')
evaluate('ten + cube(3)')
evaluate('let twenty = 20')
evaluate('twenty + 40') // PROBLEM