CODEWITHSUNDEEP
javascriptnode.jscode-coveragemocha.jsistanbul
Thiago Mata
Thiago Mata

Reputation: 2959

Nightmare.js and Code Coverage

Short version:

I am unable to see the code coverage from my tests that I have written using nightmare.js and mocha. I have already tried to use istanbul and _mocha with no luck so far.

Big version:

I have a little project:

/public/index.html

<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>My Website</title>
  <script src="./js/hello.js"></script>
</head>
<body>
  <h1>My Website</h1>
</body>
</html>

/public/js/hello.js

window.hello = function hello(){
  return 'world';
};

The site is running using express and forever.

When I am trying to test it using nightmare.js.

/test/test.js

var path = require('path');
var Nightmare = require('nightmare');
var should = require('should');

/*global describe */
/*global it */

describe('Simple demo', function () {
  this.timeout(15000);
  var url = 'http://localhost:9000';

  it('check hello world result', function (done) {
    new Nightmare()
      .goto(url)
      .evaluate(function () {
        /*global hello */
        return hello();
      }, function (value) {
        var expected = "world";
        if (value === null) {
          false.should.equal(true);
          return done();
        }
        value.should.equal(expected);
        return done();
      })
      .run();
  });

  it('should get the index title', function (done) {
    var expected = 'My Website';
    new Nightmare()
      .goto(url)
      .title(function (title) {
        title.should.equal(expected);
        done();
      })
      .run();
  });
});

The tests are passing

$ mocha


  Simple demo
    ✓ check hello world result (2089ms)
title =  Alexandria
    ✓ should get the index title (1947ms)


  2 passing (4s)

But, I am unable to get code coverage reports from my tests.

I have already tried some commands like:

$ istanbul cover _mocha -- test/test.js -u exports -R spec
No coverage information was collected, exit without writing coverage information

$ istanbul cover --hook-run-in-context _mocha -- -R spec
No coverage information was collected, exit without writing coverage information

So, someone was able to create code coverage reports of nightmare.js tests? If no, there is something close to that using another tools?

Upvotes: 2

Views: 1131

Answers (2)

Alexander Todorov
Alexander Todorov

Reputation: 2245

I want to propose another approach. It basically boils down to

  1. Instrument the application code with Istanbul
  2. Fetch the coverage data from the browser and pass it to the test process (so we can extract it)
  3. Run the reports

Here's a code snippet that does this

nightmare
  .evaluate(() => window.__coverage__) // this executes in browser scope
  .end() // terminate the Electron (browser) process
  .then((cov) => {
    // this executes in Node scope
    // handle the data passed back to us from browser scope
    const strCoverage = JSON.stringify(cov);
    const hash = require('crypto').createHmac('sha256', '')
      .update(strCoverage)
      .digest('hex');
    const fileName = `/tmp/coverage-${hash}.json`;
    require('fs').writeFileSync(fileName, strCoverage);

    done(); // the callback from the test
  })
.catch(err => console.log(err));

for detailed information and links to actual commits please check my blog: http://atodorov.org/blog/2017/08/12/code-coverage-from-nightmarejs-tests/

Upvotes: 0

Seweryn Bidolach
Seweryn Bidolach

Reputation: 26

I had exactly this same problem in my project. I couldn't find any libraries or configurations which solve this problem in simply way but with some implementation and Grunt configuration you can get code coverage from Grunt process.

Dependencies which I used in my project:

"chai": "^3.5.0",
"grunt": "^0.4.5",
"grunt-contrib-clean": "^0.7.0",
"grunt-contrib-copy": "^0.8.2",
"grunt-express-server": "^0.5.3",
"grunt-istanbul": "^0.7.1",
"grunt-mocha-test": "^0.12.7",
"istanbul": "^0.4.4",
"nightmare": "^2.2.0"

My project structure:

public/     -- public folder for index.html
src/        -- source folder for hello.js
test/       -- mocha tests implementation
server/     -- express implementation for server.js
coverage/   -- HTML report from code coverage
report/     -- HTML report from mocha tests    
dist/       -- folder which is used by express server to get content, generated by Grunt

Steps which you have to run from Grunt:

grunt.registerTask('test_hello', [
    'clean',            // clean dist/ folder
    'copy:public',      // copy public files from public/ (ex. index.html) to dist/ 
    'instrument',       // generate instruments (ex. hello.js) for code coverage from src/ by istanbul
    'express',          // run express server from dist/ folder
    'mochaTest',        // run mocha tests with nightmare and generate HTML report to report/ folder
    'report_coverage'   // generate HTML report from code coverage to coverage/ folder
]);

Grunt configurations:

module.exports = function (grunt) {

    grunt.loadNpmTasks('grunt-contrib-clean');
    grunt.loadNpmTasks('grunt-contrib-copy');
    grunt.loadNpmTasks('grunt-express-server');
    grunt.loadNpmTasks('grunt-istanbul');
    grunt.loadNpmTasks('grunt-mocha-test');

    grunt.initConfig({
        clean: {
            dist: ['dist/', 'report/', 'coverage/']
        },
        copy: {
            public: {
                expand: true,
                cwd: 'public/',
                src: ['**'],
                dest: 'dist/'
            }            
        },
        instrument: {
            files: ['**/*.js'],
            options: {
                cwd: 'src/',
                lazy: true,
                basePath: 'dist/'
            }
        },
        express: {
            dev: {
                options: {
                    port: 9000,
                    script: 'server/server.js',
                    background: true
                }
            }
        },
        mochaTest: {
            hello: {
                options: {
                    timeout: 10000,
                    captureFile: 'report/results.txt', // Optionally capture the reporter output to a file
                    quiet: false, // Optionally suppress output to standard out (defaults to false)
                    clearRequireCache: false // Optionally clear the require cache before running tests (defaults to false)
                },
                src: ['test/*.js']
            },
        }
    });

    grunt.registerTask('report_coverage', function () {
        var coverage = require('./test/utils/coverage');
        coverage.generateReport();
    });

    grunt.registerTask('test_hello', [
        'clean',            // clean dist/ folder
        'copy:public',      // copy public files from public/ (ex. index.html) to dist/ 
        'instrument',       // generate instruments (ex. hello.js) for code coverage from src/ by istanbul
        'express',          // run express server from dist/ folder
        'mochaTest:hello',  // run mocha tests with nightmare and generate HTML report to report/ folder
        'report_coverage'   // generate HTML report from code coverage to coverage/ folder
    ]);

}

I created also class which allows me collect code coverage from each Nightmare instance:

var istanbul = require('istanbul');
var reporter = new istanbul.Reporter(),
    sync = true;

var Coverage = function () {

    this.loadCodeCoverage = function (dom, done) {
        dom
            .evaluate(function () {
                return window.__coverage__;  // this variable store all information about code coverage
            })
            .then(function (coverageDate) {
                if (coverageDate) {
                    this.getCollector().add(coverageDate);
                }
                done();
            }.bind(this))
            .catch(function (error) {
                done(error);
            });
    }

    // load page by nightmare
    this.getCollector = function () {
        if (!this.collector) {
            this.collector = new istanbul.Collector();
        }
        return this.collector;
    }

    this.generateReport = function () {
        reporter.add('text');
        reporter.addAll(['lcov', 'clover']);
        reporter.write(this.collector, sync, function () {
            console.log('All reports generated');
        });
    }

}

module.exports = new Coverage();

For the above configuration file test/test.js should has the below structure:

var should = require('should');
var coverage = require('./utils/coverage');

/*global describe */
/*global it */

describe('Simple demo', function () {
    this.timeout(15000);
    var url = 'http://localhost:9000';

    before(function (done) {
        global.dom = new Nightmare()
            .goto(url)
            .evaluate(function () {
                return 'test';
            })
            .then(function(result) {
                done();
            })
            .catch(function(error) {
                done(error);
            });

    });

    after(function (done) {
        coverage.loadCodeCoverage(dom, done);
    });

    it('check hello world result', function (done) {
        dom.evaluate(function () {
                /*global hello */
                return hello();
            })
            .then(function(value) {
                var expected = "world";
                if (value === null) {
                  false.should.equal(true);
                  return done();
                }
                value.should.equal(expected);
                return done();
            })
            .catch(function(error) {
                done(error);
            });         
    });

    it('should get the index title', function (done) {
        var expected = 'My Website';
        dom.title(function (title) {
            title.should.equal(expected);
            done();
        });
    });

});

If everything works fine you should get on the console information and report from code coverage in HTML form should be generated in folder coverage/

Running "report_coverage" task
------------------------------------|----------|----------|----------|----------|----------------|
File                                |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
------------------------------------|----------|----------|----------|----------|----------------|
 js/                                |      100 |      100 |      100 |      100 |                |
  hello.js                          |      100 |      100 |      100 |      100 |                |
------------------------------------|----------|----------|----------|----------|----------------|
All files                           |      100 |      100 |      100 |      100 |                |
------------------------------------|----------|----------|----------|----------|----------------|

Problem which I still have is that for each test description I have to add method before and after. Could be nice to have these implementations only in one place, example somewhere in Grunt configuration that i needn't remember about them when I'm doing new test description.

It will be very appreciated if someone find more general solution for before and after methods.

Upvotes: 1

Related Questions