Invocation of javascript class methods is 20-40 times slower than same function

Question

While benchmarking code in my library I came across very interesting and disappointing finding - it looks like invocation of class method is 20-40 times slower than of the same standalone function.

Shortly, for me it means - "for better performance - do not implement very-frequently called functionality via JS classes". Accessing library functions via "namespace object" also has performance penalty. That problem may affect approach to design a library.

I feel I'm missing something. Please let me know if there is something wrong in perf-tests below.

Here is summary of results, windows + intel 9600k (nodejs 14, Chrome 86, FF82)

          ops/s in:   node14 slow   Ch86 slow   FF82 slow
function(a, b)          4195        3934        1625
namespace.method(a, b)   210 x20    2702 x1.5   1305 x1.24
instance.method(a, b)    126 x33     162 x24      89 x18
prototype.method(a, b)   105 x40     154 x25      57 x28
class.method(a, b)       110 x38     153 x25      57 x28

Test script for https://benchmarkjs.com/, portions where also copy/pasted to https://jsbench.me/ for in-browser testing.

const Benchmark = require('benchmark');
const suite = new Benchmark.Suite("function-vs-method");
const nCycles = 1000000;

suite
    .add('function(a, b)', function () {
        function method(a, b) {
            return a + b;
        }

        for( let i = 0; i < nCycles; i++ ) {
            method(i, i)
        }
    })
    .add('namespace.method(a, b)', function () {
        function method(a, b) {
            return a + b;
        }

        let namespace = {}
        namespace.method = method

        for( let i = 0; i < nCycles; i++ ) {
            namespace.method(i, i)
        }
    })
    .add('instance.method(a, b)', function () {
        function method(a, b) {
            return a + b;
        }

        function Test() {}
        let instance = new Test()
        instance.method = method

        for( let i = 0; i < nCycles; i++ ) {
            instance.method(i, i)
        }
    })
    .add('prototype.method(a, b)', function () {
        Test.prototype.method = (a, b) => {
            return a + b;
        }

        function Test() {
        }

        let proto = new Test()
        for( let i = 0; i < nCycles; i++ ) {
            proto.method(i, i)
        }
    })
    .add('class.method(a, b)', function () {
        class Test {
            method(a, b) {
                return a + b;
            }
        }

        let c = new Test()
        for( let i = 0; i < nCycles; i++ ) {
            c.method(i, i)
        }
    })
    // add listeners
    .on('cycle', function(event) {
        console.log(String(event.target));
    })
    .on('complete', function() {
        console.log('Fastest is ' + this.filter('fastest').map('name'));
    })
    .run();

Answer 1

It's not.

I setup a jsbench to isolate method invocation. https://jsbench.me/76ll8mwwr1/1

Short story: for less than the cost of a single statement, you can use classes. namespaces, instances, prototypes.

You never isolate method invocation. Some tests create objects each iteration, some do the test setup each time, etc.

Resolving the scope does have a slight overhead which is visible only when you have a something as short as "addition + return + incremental assign". For practical use, it's as much overhead as running with a gum wrapper in your pocket.

Answer 2

After reasonable comment from Barthy regarding "mixing initialization and test code within test method" I've restructured test suite and here are more findings

• Test suite from Barthy is also incorrect - it looks like javascript engine is clever enough and it just does not perform invocation of dummy method calls when their result is not used and nothing is modified. I'm doing such conclusion because performance of 'empty-body-test' is the same the as with invocation of dummy method

• However, invocation of any 'standalone/prototype/class-member' methods indeed have same cost. Correct implementation of test case is where invocation cannot be omitted: r += method(1, 2)

• My initial assumption was incorrect - assuming that mixing 'initialization' and 'test' code with ratio 1/1-mln using inner loop (1..1-mln) will count effect of 'initialization' part as negligible. Reason: cost of 'class construction'/'method invocation' is 6000/1 in worst cases.

Here are test results regarding cost of 'initialization part' in the test cases I used initially. (Note: there is huge difference for _method(a, b) and function(a, b) between nodejs14 and Chrome/Firefox which I could not find 'why')

Initialization cost:

      mln ops/s in:   node14 slow   Ch86 slow   FF86 slow
empty test              1021 ----   1039 ----   1516 ----
_method(a, b)            180 x1     1013 x1     1382 x1
function(a, b)           118 x1.5   1008 x1     1360 x1
namespace.method(a, b)    44 x4.1    260 x3.8   0.49 x2820
instance.method(a, b)   0.48 x375   0.57 x1777  0.41 x3365
prototype.method(a, b)  0.36 x500   0.45 x2251  0.43 x3213
class.method(a, b)      0.42 x428   0.46 x2202  0.23 x6000

and test suite for it

const Benchmark = require('benchmark');
const suite = new Benchmark.Suite("function-vs-method");
let r = 0

function _method(a, b) {
    return a + b;
}

suite
    .add('empty(1, 2)', function () {
        {}
    })
    .add('_method(1, 2)', function () {
        r += _method(1, 2)
    })
    .add('function(a, b)', function () {
        function method(a, b) {
            return a + b;
        }

        r += method(1, 2)
    })
    .add('namespace.method(a, b)', function () {
        function method(a, b) {
            return a + b;
        }

        let namespace = {}
        namespace.method = method

        r += namespace.method(1, 2)
    })
    .add('instance.method(a, b)', function () {
        function method(a, b) {
            return a + b;
        }

        function Test() {}
        let instance = new Test()
        instance.method = method

        r += instance.method(1, 2)
    })
    .add('prototype.method(a, b)', function () {
        Test.prototype.method = (a, b) => {
            return a + b;
        }

        function Test() {
        }

        let proto = new Test()
        r += proto.method(1, 2)
    })
    .add('class.method(a, b)', function () {
        class Test {
            method(a, b) {
                return a + b;
            }
        }

        let c = new Test()
        r += c.method(1, 2)
    })
    // add listeners
    .on('cycle', function(event) {
        console.log(String(event.target));
    })
    .on('complete', function() {
        console.log('Fastest is ' + this.filter('fastest').map('name'));
    })
    .run();

Answer 3

You are including the initialization in your tests instead of just comparing the method/function call.

Example: You are comparing "declaring and calling a function" vs "declaring a class, instantiating a class object and calling a class mehtod".

If you extract the setups outside of the actual tests, the results look quite different.

Additionally, each run of your code with nCycles and the for loop yielded quite different results. I changed it to one invocation and now see constant results across test runs.

Disclaimer: I haven't done much benchmarking so far but wanted to try my best at an answer. @everyone please feel free to explain why I might be on the wrong track. @op please don't blindly trust this answer.

const Benchmark = require('benchmark');
const suite = new Benchmark.Suite('function-vs-method');

function method (a, b) {
  return a + b;
}

let namespace = {};
namespace.method = function (a, b) {
  return a + b;
};


function Test () {
}

let instance = new Test();
instance.method = function (a, b) {
  return a + b;
};


TestProto.prototype.method = function (a, b) {
  return a + b;
};

function TestProto () {
}

let proto = new TestProto();

class TestClass {
  method (a, b) {
    return a + b;
  }
}

let c = new TestClass();

suite
  .add('function(a, b)', function () {
    method(1, 2);
  })
  .add('namespace.method(a, b)', function () {
    namespace.method(1, 2);
  })
  .add('instance.method(a, b)', function () {
    instance.method(1, 2);
  })
  .add('prototype.method(a, b)', function () {
    proto.method(1, 2);
  })
  .add('class.method(a, b)', function () {
    c.method(1, 2);
  })
  // add listeners
  .on('cycle', function (event) {
    console.log(String(event.target));
  })
  .on('complete', function () {
    console.log('Fastest is ' + this.filter('fastest').map('name'));
  })
  .run();

Log:

function(a, b) x 1,395,900,922 ops/sec ±0.11% (93 runs sampled)
namespace.method(a, b) x 1,397,715,256 ops/sec ±0.06% (98 runs sampled)
instance.method(a, b) x 1,397,494,031 ops/sec ±0.08% (96 runs sampled)
prototype.method(a, b) x 1,395,546,437 ops/sec ±0.09% (98 runs sampled)
class.method(a, b) x 1,398,311,922 ops/sec ±0.07% (96 runs sampled)
Fastest is class.method(a, b),namespace.method(a, b),instance.method(a, b),function(a, b)