Automated, test-driven, server-side Javascript with Maven, jsUnit & Sinon.JS

Here’s some background to a problem I faced recently: I have quite a few server-side Javascript scripts which I need to expand and refactor. Having moved from my usual comfort-zone of test-driven Java, I wanted to work in the same style to ensure the quality of the scripts I was writing.

The following describes how I reached a solution, with the many blind-alleys and wrong-turnings I took, ignored for simplicity.

Step 1: Find a Javascript unit-testing library

I needed a library that would work for server-side Javascript. The execution environment of the production code is Rhino, so I needed a compatible unit-testing framework. Unfortunately, what seem like the better Javascript testing frameworks, are focused on the problem of client-side multi-browser testing. These frameworks often adopt a server model, allowing tests to be submitted and run on different browsers. Even a headless browser would not match the specific environment (Rhino) so these frameworks were ruled out.

I found a couple of interesting projects that I began to look into:

https://github.com/stefanofornari/rhinounit-maven-plugin
https://github.com/stefanofornari/rhinounit

and, in the absence of documentation, a project that used these:

https://github.com/stefanofornari/subitosms-thunderbird-extension

Step 2: Get a simple test to run with Maven

By installing first RhinoUnit, and then the Maven plugin, in my local repository, I was able to incorporate Javascript tests into my build. I did this using the following Maven plugin in my pom.xml:

<plugin>
	<groupId>funambol</groupId>
	<artifactId>rhinounit-maven-plugin</artifactId>
	<version>1.0</version>
	<executions>
		<execution>
			<phase>test</phase>
			<goals>
				<goal>test</goal>
			</goals>
		</execution>
	</executions>
	<configuration>
		<testSourceDirectory>src/test/scripts</testSourceDirectory>
		<includes>
			<include>**/tools/datadictionary.lib.js</include>
		</includes>
	</configuration>
</plugin>

Note: implicit in the Maven plugin are two directories:

  • src/main/scripts, the directory relative to which the includes are applied
  • src/main/js, where the tests reside. Can be overidden, as I have above, using the testSourceDirectory configuration option.

Given the pom.xml configuration above, to run a basic test I need the following…

1: A valid project structure, e.g.

[project]
  - pom.xml
  - src/main/scripts/my/tools/datadictionary.lib.js
  - src/test/scripts/DataDictionaryTestSuite.js

2: A valid pom.xml

3: DataDictionaryTestSuite.js contains a test such as:

function DataDictionaryTestSuite() {
}

DataDictionaryTestSuite.prototype.test1 = function test1() {
  assertTrue(true);
}

Then everything should be ready to start automated testing.

Run ‘mvn test’, and you should see output like the following:

[INFO] [rhinounit:test {execution: default}]
> Initializing...
> Done initializing
> Running test "test1"
<testsuite time="0.003" failures="0" errors="0" tests="1" name="DataDictionaryTestSuite">
<testcase time="0" name="test1">
</testcase>
</suite>
> Done (0.005 seconds)
--------------------------------------------------------------------------------
Tests run: 1, Failures: 0, Errors: 0
--------------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------

Step 3: Write some real tests

The next step was to start writing some useful tests. I wanted to test the ‘delete’ action for a file management API. The API I was writing was dependent on an underlying API providing the raw functionality, this API was not Javascript, but Java objects injected into a Rhino as Javascript placeholders. As such, the underlying API needed to be stubbed. I decided to use a very comprehensive and test-framework-agnositic library for creating spies, mocks and stubs: Sinon.js. This library made the following tests possible, the first to test successful deletion, the second to test failed deletion (file not found):

DataDictionaryTestSuite.prototype.testDeletion = function testDeletion() {
	root.childByNamePath = sinon.stub().withArgs("/file1").returns(file);
	dataDictionaryDir.removeNode = sinon.spy().withArgs(file);

	var model = new dataDictionaryModel();
	var message = model.deleteFile("file1");

	assertEquals("File 'file1' deleted", message);
	assertTrue("removeNode(file) not called", 	
		dataDictionaryDir.removeNode.withArgs(file).calledOnce);
}

DataDictionaryTestSuite.prototype.testDeletionOfNonExistantFileReturns404Error = 
		function testDeletionOfNonExistantFileReturns404Error() {
	root.childByNamePath = sinon.stub().withArgs("/file1").returns(null);
	dataDictionaryDir.removeNode = sinon.spy();

	var model = new dataDictionaryModel();

	try {
		model.deleteFile("file1");
		fail("An exception should have been thrown");
	} catch(error) {
		assertEquals(404, error["code"]);
		assertEquals("Unable to delete file 'file1' does not exist", 
			error["message"]);
		assertTrue("removeNode(...) should not have been called", 
			dataDictionaryDir.removeNode.callCount==0);
	}
}

The assert… statements are provided by jsUnit and require no additional configuration. The stubs, spies and verifications are provided by Sinon.JS and are documented here.

I won’t go into the syntax of the tests above in too much detail, but essentially I am stubbing out the underlying API and ensuring the following:

  • A ‘removeNode’ action occurs for a successful deletions
  • A ‘removeNode’ action does not occur if the file could not be found
  • The response messages are relevant to the outcome

To use sinon.js, I found I needed to do the following adjustments:

1: Remove some code from sinon.js that conflicts with Rhino. The following extract is from line 1601 of sinon-1.1.1.js. Remove this code, and several of the associated functions. I am not sure exactly what should be removed, and even less sure of whether I am undermining the sinon library, but removing a few functions worked for me.

sinon.timers = {
    setTimeout: setTimeout,
    clearTimeout: clearTimeout,
    setInterval: setInterval,
    clearInterval: clearInterval,
    Date: Date
};

2: Locate sinon.js in /src/main/scripts and add the following include:

<include>sinon.js</include>

Step 4: Inject global mocks

You’ll notice that all the file API tests above refer to a variable ‘root’. This is a global variable which is injected into the Rhino context on the production system. For the Javascript in the includes directories to run, these global variables need to be present. I found it necessary to create a ‘global-mocks.js’ file in src/main/scripts containing the following:

var root = new Object();

Step 5: Run the tests

A failed test run, where the following assertion fails:

assertEquals("Unable to delete file 'file1' does not exist", error["message"]);

Will give you test output to indicate the nature of the failure:

[INFO] [rhinounit:test {execution: default}]
> Initializing...
> Done initializing
> Running test "testDeletionOfNonExistantFileReturns404Error"
testDeletionOfNonExistantFileReturns404Error failed
[object Object]
> Running test "testDeletion"
<testsuite time="0.05" failures="1" errors="0" tests="2" name="DataDictionaryTestSuite">
<testcase time="0.031" name="testDeletionOfNonExistantFileReturns404Error">
<failure type="jsUnitException">
Expected Unable to delete file 'file1' does not exist (string) but was Unable to delete file 'file1' not found (string)
</failure>
</testcase>
<testcase time="0.015" name="testDeletion">
</testcase>
</suite>
> Done (0.057 seconds)
--------------------------------------------------------------------------------
Tests run: 2, Failures: 1, Errors: 0
--------------------------------------------------------------------------------

WARNING: There are test failures.
--------------------------------------------------------------------------------

Finally, a successful build…

[INFO] [rhinounit:test {execution: default}]
> Initializing...
> Done initializing
> Running test "testDeletionOfNonExistantFileReturns404Error"
> Running test "testDeletion"
<testsuite time="0.045" failures="0" errors="0" tests="2" name="DataDictionaryTestSuite">
<testcase time="0.027" name="testDeletionOfNonExistantFileReturns404Error">
</testcase>
<testcase time="0.016" name="testDeletion">
</testcase>
</suite>
> Done (0.055 seconds)
--------------------------------------------------------------------------------
Tests run: 2, Failures: 0, Errors: 0
--------------------------------------------------------------------------------

Conclusion

This was my first attempt at test-driven Javascript, and I’d love to get some feedback if there are cleaner ways to achieve this, or even just some alternatives.

Advertisements

Testing Recipes, Part 2 – Utility methods

From part 1…

Should the testing frameworks or methodologies used, influence the design of the implementation code? I’m sure most would respond in the negative, but sometimes this is very hard to avoid. I have been attempting to overcome some of this influence by finding ways to test code that I had previously refactored to make it more ‘testable’. To achieve this, I have been building on use of the standard Mockito syntax, to apply the extensions provided by PowerMock (with Mockito).

Next up, private static utility methods.

When writing a service, as the complexity increases, it is useful to extract more services to provide sub-functionality. This approach is commonly the best way to extract inner functionality, but sometimes this functionality is very contained and local and does warrant a new service. In these scenarios, one will often start to refactor the code to produce utility methods.

Take the follow example: I wish to make a service to provide string padding (I have tried to simplify the context of this example, so excuse the unrealistic service). This service is not supposed to be a multi-purpose padding utility, but instead needs to provide a restricted functionality to its users. Here’s the interface:

public interface Padder {

	public String padWithSpaces(String text, int size);

	public String padWithHashes(String text, int size);

	public String padWithDashes(String text, int size);

}

A padder will pad the supplied text to the indicated size using spaces or hashes. So how do we go about building up the implementation using tests? Notice that the methods provide similar functionality but with varying initial configuration. We could start with the first method and explore all the edge cases with tests:

anEmptyStringInputWillReturnAStringOfSpacesWithALengthOfSize()
aSizeLongerThanTheLengthOfTheStringReturnsASpacePaddedStringUpToTheSize() 
aSizeShorterThanTheLengthOfTheStringReturnsAStringTruncatedToALengthOfSize() 
aSizeEqualToTheLengthOfTheStringReturnsTheSameString()  

An then we test the next method, where the first two tests can be ‘copied’.

anEmptyStringInputWillReturnAStringOfHashesWithALengthOfSize()
aSizeLongerThanTheLengthOfTheStringReturnsAHashesPaddedStringUpToTheSize() 

and again for the final method

anEmptyStringInputWillReturnAStringOfDashesWithALengthOfSize()
aSizeLongerThanTheLengthOfTheStringReturnsADashesPaddedStringUpToTheSize() 

Often this creeping addition of virtually identical tests happens as a service gets extended over time and often the duplication is missed.

My suggestion is that when the common functionality is identified through refactoring of the implementation, we address this common functionality at the test level (whilst considering if a separate service might be the cleaner route).

Testing private static utility methods using PowerMock

Let us assume that we intend to refactor the service to provide an internal private static utility method to manage the core functionality, as follows:

public class PadderImpl implements Padder {
	
	public String padWithSpaces(String text, int size){
		return pad(text, " ", size);
	}

	public String padWithHashes(String text, int size){
		return pad(text, "#", size);
	}

	private static final String pad(String text, String padding, int size) {
		//pad the string with a supplied padding
	}

}

Note that we want to keep the utility method private because we do not intend to expose this service to our users as it would complicate the interface.

We can now directly test the utility method, which is purporting to provide a generic padding service. We can use PowerMock’s WhiteboxImpl.invokeMethod() method to invoke the private utility method in the tests:

import static org.junit.Assert.assertEquals;
import static org.powermock.reflect.internal.WhiteboxImpl.invokeMethod;

import org.junit.Before;
import org.junit.Test;

public class PadderImplTest {
	
	private Padder padder;
	
	@Before
	public void setUp(){
		padder = new PadderImpl();	
	}
	
	@Test
	public void anEmptyStringInputWillReturnAStringOfPaddingWithALengthOfSize() throws Exception {
		String result = invokeMethod(padder, "pad", "", " ", 1);
		
		assertEquals(" ", result);
	}

	@Test
	public void aSizeLongerThanTheLengthOfTheStringReturnsAPaddedStringUpToTheSize() throws Exception {
		String result = invokeMethod(padder, "pad", "text", "*", 8);
		
		assertEquals("text****", result);
	}

	@Test
	public void aSizeShorterThanTheLengthOfTheStringReturnsAStringTruncatedToALengthOfSize() throws Exception {
		String result = invokeMethod(padder, "pad", "sandwich", "*", 4);
		
		assertEquals("sand", result);
	}
	
	@Test
	public void aSizeEqualToTheLengthOfTheStringReturnsTheSameString() throws Exception {
		String result = invokeMethod(padder, "pad", "sandwich", "*", 8);
		
		assertEquals("sandwich", result);
	}
	
	@Test
	public void paddingWithALongStringCausesThePaddingToBeTruncatedIfNecessary() throws Exception {
		String result = invokeMethod(padder, "pad", "$", "abc", 5);
		
		assertEquals("$abca", result);
	}

}

We now have a padding utility service that we can trust as tested within our service. Next we need to verify the behaviour of each of the service methods. This can now be done in terms of interaction with the utility method. Testing our static method requires the test to be altered to be run as a PowerMock test and to prepare the class with the static method using PrepareForTest:

import static org.junit.Assert.assertEquals;
import static org.powermock.api.mockito.PowerMockito.mockStatic;
import static org.powermock.api.mockito.PowerMockito.verifyPrivate;
import static org.powermock.reflect.internal.WhiteboxImpl.invokeMethod;

import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.powermock.core.classloader.annotations.PrepareForTest;
import org.powermock.modules.junit4.PowerMockRunner;

@RunWith(PowerMockRunner.class)
@PrepareForTest(PadderImpl.class)
public class PadderImplTest {

We can now add the final tests:

	@Test
	public void padWithSpacesCallsPadWithASpace() throws Exception {
		mockStatic(PadderImpl.class);
		
		padder.padWithSpaces("text", 10);
		
		verifyPrivate(PadderImpl.class).invoke("pad", "text", " ", 10);
	}
	
	@Test
	public void padWithHashesCallsPadWithAHash() throws Exception {
		mockStatic(PadderImpl.class);
		
		padder.padWithHashes("text", 10);
		
		verifyPrivate(PadderImpl.class).invoke("pad", "text", "#", 10);
	}

mockStatic() sets up our class for static verification, and using the verifyPrivate method, we can verify that the private static utility method is called correctly.

Adding an additional public method is as simple as describing only its interaction with the utility method, rather than copying the same set of more complex padding tests:

	@Test
	public void padWithDashesCallsPadWithADash() throws Exception {
		mockStatic(PadderImpl.class);
		
		padder.padWithDashes("text", 10);
		
		verifyPrivate(PadderImpl.class).invoke("pad", "text", "-", 10);
	}

One of the advantages with this approach is that refactoring to extract a service (where previously we had used a utility method) is much easier. This is because the tests reflect the service-like behaviour of the utility method rather than the black-box behaviour of the public methods.

There are problems with this approach however. If one is not careful to treat the utility method as ‘service-like’ then the coverage could be compromised and bugs introduced. It is not the way to test any internal method and it could be particularly dangerous if used to test entirely at a method-level. Generally speaking, the service should be treated as a black box, but occasionally it pays off to compromise and test internal method calls.

Testing Recipes, Part 1 – Self-service

Should the testing frameworks or methodologies used, influence the design of the implementation code? I’m sure most would respond in the negative, but sometimes this is very hard to avoid. I have been attempting to overcome some of this influence by finding ways to test code that I had previously refactored to make it more ‘testable’. To achieve this, I have been building on use of the standard Mockito syntax, to apply the extensions provided by PowerMock (with Mockito). This first example uses just Mockito.

The first problem I will address is how to test services that use their own services (‘self-service’). This scenario is best explained by the code below showing a ‘client’ service that allows the service-user to call methods on the client.

public class Client {

	public void create(){
		//perform action
	}

	public void delete(){
		//perform action
	}

	public void recreate(){
		delete();
		create();
	}

}

Three methods are available: create(), delete() and recreate(). Each method performs an action on the client, but recreate() is different because the action performed makes use of other service actions. More complex scenarios could be presented where a method would call other service methods multiple times, or vary calls according to business logic, but this simple example is suitable to explain a typical use-case.

The testing problem comes when wanting to test the recreate() method. Let’s assume that our client makes calls to a ‘databaseService’ to perform the underlying actions.

Approach 1 – Test the underlying interactions

A simple approach would be as follows:

	@Test
	public void recreateDeletesThenCreates() {
		client.recreate();

		//verify underlying actions
		verify(databaseService).runSQL('DELETE...');
		verify(databaseService).runSQL('INSERT...');
	}

The problem, I feel, with this test, is that we are not testing the functionality of the recreate() method, but instead we are treating the Client as a black box and testing how the client interacts with the underlying service. This would be the correct approach for the simpler create() and delete() methods but we are now open to duplication in our test suite. If the lower level methods become more complex, then tests for the higher level methods will increasingly duplicate already tested functionality. This problem begins immediately because the line

verify(databaseService).runSQL('DELETE...');

will presumably already be present in the test for delete(). If we want to refactor delete() then we have to change two or more tests.

Approach 2 – Extract an ‘extended’ service

Alternatively we can extract an ‘extended’ service which provides only the functionality that is defined in terms of the basic service methods:

public class BasicClient {

	public void create(){
		//perform action
	}

	public void delete(){
		//perform action
	}

}

public class ExtendedClient {

	private BasicClient client;

	public void recreate(){
		client.delete();
		client.create();
	}

}

We can then test the recreate() in terms of interactions with the BasicClient (the setInternalState() method is part of the Mockito Whitebox class):

public class ExtendedClientTest {

	@Mock BasicClient basicClient;

	ExtendedClient extendedClient;

	@Before
	public void setUp(){
		MockitoAnnotations.initMocks(this);

		extendedClient = new ExtendedClient();
		setInternalState(extendedClient, "client", basicClient);
	}

	@Test
	public void recreateCallsDeleteAndCreateOnTheBasicClient() {
		extendedClient.recreate();

		verify(basicClient).delete();
		verify(basicClient).create();
	}

}

The problem I see here is that we have let our testing needs influence our implementation. The approach of extracting services to reduce complexity is often the correct approach to writing more testable code. However, if no concept is reflected in the domain and the extended service is extracted purely for the purposes of testing the higher level methods, then extracting the service only adds complexity, and should therefore be avoided.

Approach 3 – Verify interaction between methods using spy

My suggested approach is to take advantage of Mockito’s spy() method. Creating a spy of the Client creates an object that can be treated like a mock (stubbing and method verification is possible), but maintains the original implemented code until stubbed.

public class ClientTest {

	Client client;
	
	@Before
	public void setUp(){
		client = spy(new Client());		
	}
	
	@Test
	public void recreateCallsDeleteThenCreate() {
		client.recreate();
		
		verify(client).delete();
		verify(client).create();
	}
	
}

We now have a Client that cleanly represents the service we wish to offer, but at the same time we can test method in terms of their interactions – testing methods in this way is generally best avoided but our Client presents a counter-example. Using this approach we must be careful to avoid the common pitfall of testing the implementation rather than the behaviour. I feel however that this approach allows us to build up interactions between methods inside a class without resulting in an explosion of test complexity.

Even the comments for the spy() method itself warn against it’s usage:

Real spies should be used carefully and occasionally, for example when dealing with legacy code.

and in general I concur, but the example presented above represents a scenario where attempting to avoid using this feature will comprise the design of the system.

These techniques are entirely compatible with a test-driven approach and provide a much cleaner way of adding higher level methods to services that already provide a set of lower level methods. I intend to explore testing internal private utility methods later in this series which is also closely related to this problem.

Thanks to Johan for recommending improvements to the code examples.