Async Frustrations: Should I be using callbacks and how do I pass them through multiple modules?

Question

I am attempting to make a simple text game that operates in a socket.io chat room on a node server. The program works as follows:

Currently I have three main modules

Rogue : basic home of rogue game functions

rogueParser : module responsible for extracting workable commands from command strings

Verb_library: module containing a list of commands that can be invoked from the client terminal.

The Client types a command like 'say hello world'. This triggers the following socket.io listener

socket.on('rg_command', function(command){
  // execute the verb
  let verb = rogueParser(command);
  rogue.executeVerb(verb, command, function(result){
    console.log(result);
  });
});

Which then in turn invokes the executeVerb function from rogue..

  executeVerb: function(verb, data, callback){
    verb_library[verb](data, callback);
  },

Each verb in verb_library should be responsible for manipulating the database -if required- and then returning an echo string sent to the appropriate targets representing the completion of the action.

EDIT: I chose 'say' when I posted this but it was pointed out afterward that it was a poor example. 'say' is not currently async but eventually will be as will be the vast majority of 'verbs' as they will need to make calls to the database.

...
  say: function(data, callback){
    var response = {};
    console.log('USR:'+data.user);
    var message = data.message.replace('say','');
    message = ('you say '+'"'+message.trim()+'"');
    response.target = data.user;
    response.type = 'echo';
    response.message = message;
    callback(response);
  },
...

My problem is that

1 ) I am having issues passing callbacks through so many modules. Should I be able to pass a callback through multiple layers of modules? Im worried that I'm blind so some scope magic that is making me lose track of what should happen when I pass a callback function into a module which then passes the same callback to another module which then calls the callback. Currently it seems I either end up without access to the callback on the end, or the first function tries to execute without waiting on the final callback returning a null value.

2 ) Im not sure if Im making this harder than it needs to be by not using promises or if this is totally achievable with callbacks, in which case I want to learn how to do it that way before I summon extra code.

Sorry if this is a vague question, I'm in a position of design pattern doubt and looking for advice on this general setup as well as specific information regarding how these callbacks should be passed around. Thanks!

Answer 1

Callbacks are fine, but I would only use them if a function is dependent on some asynchronous result. If however the result is immediately available, then the function should be designed to return that value.

In the example you have given, say does not have to wait for any asynchronous API call to come back with a result, so I would change its signature to the following:

say: function(data){ // <--- no callback argument 
    var response = {};
    console.log('USR:'+data.user);
    var message = data.message.replace('say','');
    message = ('you say '+'"'+message.trim()+'"');
    response.target = data.user;
    response.type = 'echo';
    response.message = message;
    return response; // <--- return it
}

Then going backwards, you would also change the signature of the functions that use say:

executeVerb: function(verb, data){ // <--- no callback argument
    return verb_library[verb](data); // <--- no callback argument, and return the returned value
}

And further up the call stack:

socket.on('rg_command', function(command){
    // execute the verb
    let verb = rogueParser(command);
    let result = rogue.executeVerb(verb, command); // <--- no callback, just get the returned value
    console.log(result);
});

Of course, this can only work if all verb methods can return the expected result synchronously.

Promises

If say would depend on some asynchronous API, then you could use promises. Let's assume this API provides a callback system, then your say function could return a promise like this:

say: async function(data){ // <--- still no callback argument, but async! 
    var response = {};
    console.log('USR:'+data.user);
    var message = data.message.replace('say','');
    response.target = data.user;
    response.type = 'echo';
    // Convert the API callback system to a promise, and use AWAIT
    await respone.message = new Promise(resolve => someAsyncAPIWithCallBackAsLastArg(message, resolve));
    return response; // <--- return it
}

Again going backwards, you would also change the signature of the functions that use say:

executeVerb: function(verb, data){ // <--- still no callback argument
    return verb_library[verb](data); // <--- no callback argument, and return the returned promise(!)
}

And finally:

socket.on('rg_command', async function(command){ // Add async
    // execute the verb
    let verb = rogueParser(command);
    let result = await rogue.executeVerb(verb, command); // <--- await the fulfillment of the returned promise
    console.log(result);
});

Answer 2

Asynchronousness and JavaScript go back a long way. How we deal with it has evolved over time and there are numerous applied patterns that attempt to make async easier. I would say that there are 3 concrete and popular patterns. However, each one is very related to the other:

1. Callbacks
2. Promises
3. async/await

Callbacks are probably the most backwards compatible and just involve providing a function to some asynchronous task in order to have your provided function be called whenever the task is complete.

For example:

/**
 * Some dummy asynchronous task that waits 2 seconds to complete
 */
function asynchronousTask(cb) {
  setTimeout(() => {
    console.log("Async task is done");
    cb();
  }, 2000);
}

asynchronousTask(() => {
  console.log("My function to be called after async task");
});

Promises are a primitive that encapsulates the callback pattern so that instead of providing a function to the task function, you call the then method on the Promise that the task returns:

/**
 * Some dummy asynchronous task that waits 2 seconds to complete
 * BUT the difference is that it returns a Promise
 */
function asynchronousTask() {
  return new Promise(resolve => {
    setTimeout(() => {
      console.log("Async task is done");
      resolve();
    }, 2000);
  });
}

asynchronousTask()
  .then(() => {
    console.log("My function to be called after async task");
  });

The last pattern is the async/await pattern which also deals in Promises which are an encapsulation of callbacks. They are unique because they provide lexical support for using Promises so that you don't have to use .then() directly and also don't have to explicitly return a Promise from your task:

/*
 * We still need some Promise oriented bootstrap 
 * function to demonstrate the async/await
 * this will just wait a duration and resolve
 */
function $timeout(duration) {
  return new Promise(resolve => setTimeout(resolve, duration));
}

/**
 * Task runner that waits 2 seconds and then prints a message
 */
(async function() {
  await $timeout(2000);
  console.log("My function to be called after async task");
}());

---

Now that our vocabulary is cleared up, we need to consider one other thing: These patterns are all API dependent. The library that you are using uses callbacks. It is alright to mix these patterns, but I would say that the code that you write should be consistent. Pick one of the patterns and wrap or interface with the library that you need to.

If the library deals in callbacks, see if there is a wrapping library or a mechanism to have it deal in Promises instead. async/await consumes Promises, but not callbacks.

Answer 3

1) Passing callback trough multiple layers doesn't sound like a good idea. Usually I'm thinking what will happen, If I will continiue doing this for a year? Will it be flexible enough so that when I need to change to architecture (let's say customer have new idea), my code will allow me to without rewriting whole app? What you're experiencing is called callback hell. http://callbackhell.com/ What we are trying to do, is to keep our code as shallow as possible.

2) Promise is just syntax sugar for callback. But it's much easier to think in Promise then in callback. So personally, I would advice you to take your time and grasp as much as you can of programming language features during your project. Latest way we're doing asynchronus code is by using async/await syntax which allows us to totally get rid of callback and Promise calls. But during your path, you will have to work with both for sure.

You can try to finish your code this way and when you're done, find what was the biggest pain and how could you write it again to avoid it in future. I promise you that it will be much more educative then getting explicit answear here :)