How to leverage event loop in JavaScript and Node.js?

Question

Is the concept of event loop general or specific to languages? I am looking for detailed explanations with examples to clearly understand the following:

1. how it works?

1. How/when should/can I leverage event loop in JavaScript?
2. Any changes introduced in ECMAScript-6/2015 regarding event loop?

Update: There's no dearth of answers to this but I'm looking for an easy to understand definition with examples.

Addendum:

Consider the following code:

var list = readHugeList();

var nextListItem = function() {
    var item = list.pop();

    if (item) {
        // process the list item...
        setTimeout( nextListItem, 0);
    }
};  

Is it correct to assume setTimeout() here uses the event loop to prevent stackoverflow? I read an explanation for this code that:

The stack overflow is eliminated because the event loop handles the recursion, not the call stack.

Update 2

The full explanation given for the setTimeout() in the code above is as follows:

The stack overflow is eliminated because the event loop handles the recursion, not the call stack. When nextListItem runs, if item is not null, the timeout function (nextListItem) is pushed to the event queue and the function exits, thereby leaving the call stack clear. When the event queue runs its timed-out event, the next item is processed and a timer is set to again invoke nextListItem. Accordingly, the method is processed from start to finish without a direct recursive call, so the call stack remains clear, regardless of the number of iterations.

The contradiction in explanations, aforementioned and in the answers here, is making it all the more difficult to understand.

Answer 1

Is the concept of event loop general or specific to languages?

At its highest level, general. But different environments will implement the details so differently that really you need environment-specific information to know what's going on.

how it works?

All the gory details are in the specification, chiefly in the section called Jobs and Job Queues.

There are a couple of keys aspects:

1. There is a queue of jobs (the JavaScript spec's term) or "tasks" (as the HTML spec calls them) waiting to be run by the JavaScript engine. When the engine finishes a job, it's put to work on the next job/task in the queue, if any.

2. Once a job is started, it keeps running until it's done; no other jobs can interrupt it. This is called run-to-completion and is very important to the definition of how JavaScript works.

3. The job queue is processed in order.

So consider this code:

console.log("one");
setTimeout(function() {
    console.log("two");
}, 1000);

If you run that code (via NodeJS or a browser), here's what happens (omitting some irrelevant details):

1. At the beginning, the queue is empty and the JavaScript engine is idle
2. The environment (NodeJS, the browser) queues a job to run the script
3. The JavaScript engine picks up the job and runs the script:
   • It outputs "one"
   • It sets a timer for the anonymous function we gave to setTimeout
   • The job ends
4. At some point, the timer mechanism in the environment determines that it's time to call the callback, so it queues a job to call it
5. The JavaScript engine picks up that job from the queue and runs the function
   • It outputs "two"
   • The job ends

Now consider this code:

console.log(Date.now(), "one");
setTimeout(function first() {
    console.log(Date.now(), "two");
}, 500);
setTimeout(function second() {
    var end = Date.now() + 1000;
    while (end > Date.now()) {
        // Busy-wait (normally this is a Bad Thing™, I'm using it here
        // to simulate actual work that takes significant time
    }
}, 100);

As you can see, it schedules a timed callback at 500ms and then another at 100ms. But the code in the callback at 100ms will take at least 1000ms to run. What happens?

1. The environment queues a job to run the script
2. The JS engine picks that job up
   • Outputs the time value and "one", say 1470727293584 one
   • Sets a timed callback to the function first for 500ms in the future
   • Sets a timed callback to the function second for 100ms in the future
   • The job ends
3. About 100ms later, the environment queues a job to run second
4. The JavaScript engine picks up the job and runs the function
   • It starts doing work (okay, so really it's just busy waiting)
5. About 400ms later (500ms from the time the timer was set), the environment queues a job to call first; since the JavaScript engine is busy with the previous job, the job sits in the queue
6. Meanwhile, the JavaScript engine is still working on the job calling first:
   • Eventually the work being done by the JavaScript engine from #3 is done
   • The job finishes
7. The JavaScript engine picks up the next job from the queue and runs the call to second
   • It outputs the time value and "two", say 1470727294687 two
   • The job ends

Note that the environment did something while JavaScript was busy; it queued a job to be done.

The fact that the environment can do things while the JavaScript engine is busy is very important.

(It's worth noting that when queuing jobs, some environments may not necessarily add the job at the end of the queue; in some browsers, for instance, "the queue" is actually more than one queue with slightly different priorities...)

how/when should/can I leverage it?

It's not so much leveraging it as knowing it's there. It's also important to note that while JavaScript has run-to-completion semantics, the environment in which it's running may be doing other things at the same time, even while the JavaScript code is running.

any changes introduced in ECMAScript-6/2015 regarding event loop?

Not really, although it's defined much more completely in the spec than previously. Probably the closest thing to a change is in relation to promises: The callback you schedule with then or catch will always be queued as a job, it will never be run synchronously. That's the first time the JavaScript specification has defined something that happens asynchronously (ajax and timers are not part of the JavaScript spec).

---

Below, you've asked:

Consider this code:

var list = readHugeList();
var nextListItem = function() {
    var item = list.pop();
    if (item) {
        // process the list item...
        setTimeout(nextListItem, 0);
    }
};

Is it correct to assume setTimeout() here uses the event loop to prevent stackoverflow?

(I assume there's a nextListItem(); call immediately after that.)

No, but it's doing something else important. The non-setTimeout version of that would look something like this:¹

var list = readHugeList();
while (list.length) {
    var item = list.pop();
    // process the list item...
}

That's a simple loop, there's no potential for stack overflow.

What it's doing is working cooperatively with the event loop, avoiding having a really long-running job that would tie up the JavaScript engine, preventing it processing any other jobs (such as I/O completions, clicks, or other events). So it's breaking the work up into small jobs by processing the items one at a time, helping ensure that the overall job queue keeps getting processed. That means it takes a lot longer to process the list, but doesn't block other things while doing so.

---

¹ It's true that the non-setTimeout version of that code could look like this:

var list = readHugeList();
var nextListItem = function() {
    var item = list.pop();
    if (item) {
        // process the list item...
        // then recurse
        nextListItem();
    }
};
nextListItem();

...and in that case, there's potential for stack overflow, but it would be a very odd way to write that code.