Assert a function call in another module's dependency with Jest

Question

I have the following code I try to cover:

// @flow strict

import { bind, randomNumber } from 'Utils'
import { AbstractOperator } from './AbstractOperator'

export class Randomize extends AbstractOperator {
  // ...

  randomPick (dataset: Array<string>, weights: ?Array<number>): number {
    if (!weights) { return randomNumber(0, (dataset.length - 1)) }

    const sumOfWeights: number = weights.reduce((a, b) => a + b)
    let randomWeight = randomNumber(1, sumOfWeights)
    let position: number = -1

    for (let i = 0; i < dataset.length; i++) {
      randomWeight = randomWeight - weights[i]
      if (randomWeight <= 0) {
        position = i
        break
      }
    }

    return position
  }
}

And here is the test coverage:

import { Randomize } from './Randomize'

const dataset = [
  'nok',
  'nok',
  'nok',
  'ok',
  'nok'
]

const weights = [
  0,
  0,
  0,
  1,
  0
]

const randomNumber = jest.fn()

describe('operator Randomize#randomPick', () => {
  test('without weights, it calls `randomNumber`', () => {
    const randomizeOperator = new Randomize({}, [dataset], {})
    randomizeOperator.randomPick(dataset)

    expect(randomNumber).toBeCalledWith(0, dataset.length - 1)
  })
})

I'm trying to make sure that randomNumber is called but all I get is:

  ● operator Randomize#randomPick › without weights, it calls `randomNumber`

    expect(jest.fn()).toBeCalledWith(...expected)

    Expected: 0, 4

    Number of calls: 0

      33 |     randomizeOperator.randomPick(dataset)
      34 |
    > 35 |     expect(randomNumber).toBeCalledWith(0, dataset.length - 1)
         |                          ^
      36 |   })
      37 | })
      38 |

      at Object.toBeCalledWith (node_modules/jest-chain/dist/chain.js:15:11)
      at Object.toBeCalledWith (src/app/Services/Providers/Result/Resolvers/Operators/Randomize.test.js:35:26)

Answer 1

My two cents is that mocking the randomNumber dependency is not the right approach to test this functionality.

However, I'm going to answer the primary question here and see how we can make that test pass. Then will get to my additional thoughts about a better way to test this in a future update.

Asserting against randomNumber call

Intercept the import & mock

The actual issue with the code is that the randomNumber mock function is hanging in the air. As the error suggests, it's not being called.

Missing part is to intercept the module import and make it so that an outside call to Utils.randomNumber triggers the mock function; so that we can then assert against it. Here's how to intercept the Utils import & mock it:

// Signature is: 
// jest.mock(pathToModule: string, mockModuleFactory: Function)
jest.mock('Utils', () => ({
  randomNumber: jest.fn()
}))

Now every call to Utils.randomNumber during tests will trigger the mock function and it's no more hanging in the air.

If you're curious to see how it's working behind the scene, look into how babel-plugin-jest-hoist hoists jest.mock calls on top of imports which are being compiled to CommonJS Jest-hijacked require calls.

Depending on the situation, it might be problematic to mock a whole module. What if the test relies on the other exports of the Utils module? e.g. bind?

There are ways to partially mock a module, just a function, a class or two. However, to make your test pass, there's even a simpler approach.

Spy on it

The solution is to simply spy on the randomNumber call. Here's a full example:

import { Randomize } from './Randomize'
import * as Utils from 'Utils'

// Sidenote: This values should probably be moved to a beforeEach()
// hook. The module-level assignment does not happen before each test.
const weights = [0, 0, 0, 1, 0]
const dataset = ['nok', 'nok', 'nok', 'ok', 'nok']

describe('operator Randomize#randomPick', () => {
  test('without weights, it calls `randomNumber`', () => {
    const randomizeOperator = new Randomize({}, [dataset], {})
    const randomNumberSpy = jest.spyOn(Utils, 'randomNumber')

    randomizeOperator.randomPick(dataset)

    expect(randomNumberSpy).toBeCalledWith(0, dataset.length - 1)
  })
})

This is hopefully a passing test, but a very fragile one.

To wrap up this, these are very good reads on the topic in the context of jest:

• Jest Full and Partial Mock/Spy of CommonJS and ES6 Module Imports
• Understanding Jest mocks

---

Why it's not a good test?

Mainly because the test is tightly coupled to the code. To the extent that duplicate code is visible if you compare the test and the SUT.

A better approach is to not mock/spy on anything at all (look into Classist vs. Mockist TDD schools) and exercise the SUT with a dynamically-generated set of data and weights which in turn asserts that it's "good enough".

I'll elaborate more on this in an update.

---

A better test

Testing the implementation details of the randomPick is not a good idea for another reason as well. Such a test can't verify the correctness of the algorithm as it's only verifying the calls it's making. If there's an edge-case bug, it's not covering enough to be able to hit it.

Mocking/Spying is usually beneficial when we want to assert against the communication of objects; in cases that the communication is actually assertive of correctness, e.g. "assert that it hits the database"; but here it's not.

An idea for a better testcase might be to exercise the SUT "vigorously" and assert that it's "Good Enough" for what it's doing; picking a random element. The idea is backed by the Law of Large Numbers:

"In probability theory, the law of large numbers (LLN) is a theorem that describes the result of performing the same experiment a large number of times. According to the law, the average of the results obtained from a large number of trials should be close to the expected value, and will tend to become closer to the expected value as more trials are performed." — Wikipedia

Provide the SUT with a relatively large, dynamically generated set of random input and assert that it's passing every single time:

import { Randomize } from './Randomize'

const exercise = (() => {
  // Dynamically generate a relatively large random set of input & expectations:
  // [ datasetArray, probabilityWeightsArray, expectedPositionsArray ]
  //
  // A sample manual set:  
  return [
    [['nok', 'nok', 'nok', 'ok', 'nok'], [0, 0, 0, 1, 0], [3]],
    [['ok', 'ok', 'nok', 'ok', 'nok'], [50, 50, 0, 0, 0], [0, 1]],
    [['nok', 'nok', 'nok', 'ok', 'ok'], [0, 0, 10, 60, 30], [2, 3, 4]]
  ]
})()

describe('whatever', () => {
  test.each(exercise)('look into positional each() params for unique names', (dataset, weights, expected) => {
    const randomizeOperator = new Randomize({}, [dataset, weights], {})

    const position = randomizeOperator.randomPick(dataset, weights)

    expect(position).toBeOneOf(expected)
  })
})

---

Here's another perspective based on the same idea without necessarily needing to generate dynamic data:

import { Randomize } from './Randomize'

const exercise = (() => {
  return [
    [
      ['moreok'], // expect "moreok" to have been picked more during the exercise.
      ['lessok', 'moreok'], // the dataset.
      [0.1, 99.90] // weights, preferring the second element over the first.
    ],
    [['moreok'], ['moreok', 'lessok'], [99, 1]],
    [['moreok'], ['lessok', 'moreok'], [1, 99]],
    [['e'], ['a', 'b', 'c', 'd', 'e'], [0, 10, 10, 0, 80]],
    [['d'], ['a', 'b', 'c', 'd'], [5, 20, 0, 75]],
    [['d'], ['a', 'd', 'c', 'b'], [5, 75, 0, 20]],
    [['b'], ['a', 'b', 'c', 'd'], [0, 80, 0, 20]],
    [['a', 'b'], ['a', 'b', 'c', 'd'], [50, 50]],
    [['b'], ['a', 'b', 'c'], [10, 60, 30]],
    [['b'], ['a', 'b', 'c'], [0.1, 0.6, 0.3]] // This one pinpoints a bug.
  ]
})()

const mostPicked = results => {
  return Object.keys(results).reduce((a, b) => results[a] > results[b] ? a : b )
}

describe('randompick', () => {
  test.each(exercise)('picks the most probable: %p from %p with weights: %p', (mostProbables, dataset, weights) => {
    const operator = new Randomize({}, [dataset, weights], {})
    const results = dataset.reduce((carry, el) => Object.assign(carry, { [el]: 0 }), {})
    // e.g. { lessok: 0, moreok: 0 }

    for (let i = 0; i <= 2000; i++) {
      // count how many times a dataset element has win the lottery!
      results[dataset[operator.randomPick(dataset, weights)]]++
    }

    // console.debug(results, mostPicked(results))

    expect(mostPicked(results)).toBeOneOf(mostProbables)
  })
})

---

A more readable test

When tests are being polluted with "functionality noise" just like above, they become hard to read; they don't serve as documentation anymore.

In such situations, developing a custom matcher or test-double helps with readability.

test.each([
  // ...
])('picks the most probable: %p from %p with weights: %p', mostProbables, dataset, weights) => {
    const results = []
    const operator = new Randomize(...whatever)

    ;[...Array(420).keys()].forEach(() => results.push(
      operator.randomPick(...whatever)
    )

    expect(results).toHaveMostFrequentElements(mostProbables)
}

This custom toHaveMostFrequentElements assertion matcher helps eliminating "noise" from the test.