TL;DR If you just want to review the files, you can go here

Introduction

Testing is an integral part of every stable code base. It allows us to track various dependencies, without a deep familiarization with the code, it enables new developers to take risks, it gives us the chance to understand the maturity of a product and also create better and more scalable applications, by forcing ourselves to go ‘design first’, in methodologies such as test driven development.

continuous integration/continuous delivery

Often, the various tests will be integrated into all sorts of continuous integration/continuous delivery processes, which enable an automatic deployment and configuration of the product, in addition to various safety mechanisms that allow the reversion of bad code, until it stabilizes.

In order to achieve the most out of the automation, we need to integrate the browser tests into it as well.

Motivation

I was looking for the best way to achieve the above, and went over some wonderful tutorials, most notably this one however, it didn’t suit my needs since I wanted to use gulp and wasn’t using browserify.

Theory

This is the theoreticall part. If you’re interested in the practical application, feel free to skip here

The stack

The stack we will be using, is going to be made of the following:

  • gulp - our main entry point / task runner
  • mocha - our sync/async test framework
  • chai - an assertion library for mocha
  • phantomJS - a headless (gui-less) browser that will run the client code
  • istanbul - a code coverage framework

The flow

gulp browser testing flow chart

The flow depicted in the diagram above, is made out of several phases:

  • Instrumentation
  • Injection
  • Evaluation
  • Reporting

Instrumentation

In order to achieve the required statistics for the code coverage phase, the code needs to be initially instrumented, i.e; wrapped with additional code in order to count the amount of times it was accessed by the test framework. The original source files are instrumented and copied into a temporary directory called coverage/ which is later accessed by the injector.

The coverage data is then stored within a global variable named window.__coverage__

Injection

In the injection phase, the instrumented code, generated by istanbul is being injected into a static HTML file called index.html by a gulp task called gulp-inject, it will also scan the tests/ directory for all the test files and inject it as well.

This happens dynamically for the entire testing folder for every gulp test or gulp test-coverage invocation.

Evaluation

Within the evaluation phase, an instance of phantomJS will run index.html, which contains a reference to the instrumented source files and the test scripts. The result will be collected by mocha-phantomjs-istanbul which is a phantomJS hook that will grab the global window.__coverage variable and write it into a JSON file called coverage/coverage.json.

Reporting

The coverage.json file will be then parsed by gulp-istanbul-report and both testing and coverage reports will be created in any desired format.

Practice

We will need to create 3 files:

  • index.html - where all the client testing happens
  • package.json - will be in charge of the required node modules
  • gulpfile.js - will execute the for aforementioned tasks

index.html

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Title</title>
    <link rel="stylesheet" href="node_modules/mocha/mocha.css">
    <!-- inject:js -->
    <!-- all covered JS files will be injected here -->
    <!-- endinject -->
</head>
<body>
    <div id="mocha"></div>
    
    <!-- our mocha and chai testing libraries-->
    <script src="node_modules/mocha/mocha.js"></script>
    <script src="node_modules/chai/chai.js"></script>
    <script>
        // binding mocha with phantomJS
        window.initMochaPhantomJS && window.initMochaPhantomJS();
        mocha.setup('bdd');
    </script>
    
    <!-- inject:tests:js -->
    <!-- all the test files go here -->
    <!-- endinject -->

    <script>
        //initializing testing
        mocha.run();
    </script>
</body>
</html>
</body>
</html>

package.json

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
{
  "name": "browser-testing",
  "version": "0.0.1",
  "description": "A setup for testing browsers and integrating it with various automation processes",
  "main": "index.js",
  "scripts": {
    "test": "gulp test-coverage
  },
  "author": "Michael Katz (silicakes)",
  "license": "MIT",
  "devDependencies": {
    "chai": "^3.5.0",
    "gulp": "^3.9.1",
    "gulp-inject": "^4.1.0",
    "gulp-istanbul": "^1.0.0",
    "gulp-istanbul-report": "0.0.1",
    "gulp-mocha-phantomjs": "^0.11.0",
    "mocha": "^2.5.3",
    "mocha-phantomjs": "^4.1.0",
    "mocha-phantomjs-istanbul": "0.0.2"
  }
}

gulpfile.js

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
var gulp = require('gulp');
var inject = require('gulp-inject');
var istanbul = require('gulp-istanbul');
var mochaPhantomJS = require('gulp-mocha-phantomjs');
var istanbulReport = require('gulp-istanbul-report');

gulp.task('instrument', function () {
	return gulp.src(['src/js/**/*.js'])
	// Covering files
		.pipe(istanbul({
			coverageVariable: '__coverage__'
		}))
		// instrumented files will go here
		.pipe(gulp.dest('coverage/'))
});


gulp.task('test', ['instrument', 'inject'], function () {
	return gulp
		.src('index.html', {read: false})
		.pipe(mochaPhantomJS(
			{
				reporter: ["spec"],
				phantomjs: {
					useColors: true,
					hooks: 'mocha-phantomjs-istanbul',
					coverageFile: './coverage/coverage.json'
				}
			}))
		.on('finish', function () {
			gulp.src("./coverage/coverage.json")
				.pipe(istanbulReport({
					reporters: ['text', 'html']
				}))
		});
});


var paths = {
	"javascript": ["coverage/**/*.js"],
	tests: ["tests/**/*.js"]
};

gulp.task('inject', ['instrument'], function (cb) {
	return gulp.src('index.html')
		.pipe(inject(
			gulp.src(paths.javascript,{read: false}), {
				relative: true
			}))
		.pipe(inject(
			gulp.src(paths.tests, {read: false}), {
				relative: true,
				starttag: "<!-- inject:tests:js -->"
			}))
		.pipe(gulp.dest('.'))
});

Bringing it all together

Now that we are all set up, all we have to do is:

1
2
npm i
gulp test

and..Voila!

result

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
[16:57:19] Using gulpfile ~/dev/gulp-mocha-phantomjs/gulpfile.js
[16:57:19] Starting 'instrument'...
[16:57:19] Finished 'instrument' after 171 ms
[16:57:19] Starting 'inject'...
[16:57:19] gulp-inject 3 files into index.html.
[16:57:19] gulp-inject 2 files into index.html.
[16:57:19] Finished 'inject' after 20 ms
[16:57:19] Starting 'test'...
width: 400 height: 300

  sil.handler.ajax
    ✓ sil.handlers [click] should be defined 
    ✓ sil.handlers [click] should be the same length as the mock
    ✓ sil.handlers [click] should be an array
      
  sil.config.identifier
    ✓ should be equal to mock

  sil.utils.getTopMostWindow
    ✓ should return window top for this case


  5 passing (47ms)

[16:57:21] Finished 'test' after .738 s
------------------|---------|----------|---------|---------|----------------|
File              | % Stmts | % Branch | % Funcs | % Lines |Uncovered Lines |
------------------|---------|----------|---------|---------|----------------|
 sil.handlers/    |      50 |     12.5 |   30.77 |      50 |                |
  sil.handler.js  |      50 |     12.5 |   30.77 |      50 |... 5,96,97,120 |
 sil.config/      |   74.51 |    48.39 |   71.43 |   74.51 |                |
  identifier.js   |   74.51 |    48.39 |   71.43 |   74.51 |... 63,79,80,86 |
 sil.utils/       |   74.19 |       25 |   66.67 |   74.19 |                |
  sil.utils.js    |   74.19 |       25 |   66.67 |   74.19 |... 235,236,239 |
------------------|---------|----------|---------|---------|----------------|
All files         |   66.23 |    28.63 |   56.29 |   66.23 |                |
------------------|---------|----------|---------|---------|----------------|

A word about reporting

You can use all sorts of reporters, listed under require('istanbul').Report.getReportList() where you can select the one supported by your CI.

Conclusion

Now, you should be available to test and get coverage statistics for your code. Feel free to leave a comment if you feel something is missing.