NUnitLite: Embedded, readable tests

I’m about to write some unit tests for my Lua-based game engine, and decided it was time to take a look at NUnitLite. Mainly because I want to play with its self-descriptive and, above all, forwards assertions:

Assert.That(actual, Is.EqualTo(expected));

// Output:
1) TestFoo (Tyler.Tests.TestTest.TestFoo)
Expected: 5
But was:  4

(I’ve ranted about xUnit’s backwards assertions before, so I won’t bore you with that again. Suffice to say, this is much, much cooler than NUnit.)

So I downloaded it and tried it out, and found that there are some gotchas. Nothing show-stopping yet, but some things that it would’ve been really nice to know about before I started. So I’ll share.

Getting NUnitLite

As of today, NUnitLite has not shipped any releases. So you’ll have to download the latest code from the NUnitLite source repository. You can download any changeset, but you probably want the latest (the topmost row in the grid — changeset 11989 when I downloaded). Click the disk icon in the left column to download a ZIP.

There’s a solution file in NUnitLite\src, but don’t bother opening it. It contains tests for NUnitLite itself, and those tests are NUnit tests — so they won’t even compile unless NUnit is wherever they expect it to be.

What you want is the NUnitLite\src\NUnitLite directory and project. Copy that into the solution directory where you want to start using NUnitLite, and add it to your solution. (NUnitLite is meant to be embedded, so you compile it in your solution just like all your other projects.)

Making NUnitLite return an exit code

Out of the box, NUnitLite doesn’t return a nonzero exit code when tests fail. This makes it completely worthless for any kind of automation. What good are unit tests if they don’t fail the automated build?

Fortunately, this is pretty easy to get around. You can do either of two things: modify the NUnitLite source code, or run the test-runner object yourself. I chose the latter, for now.

See below for some code you can put in your program’s Main() method to run NUnitLite properly, including return of a nonzero exit code when tests fail.

Quick overview of NUnitLite

As with most test frameworks, you define “test fixture” classes and put test methods on them. Setup and teardown are supported.

You can choose to make your test-fixture classes descend from the NUnitLite.Framework.TestCase class. This doesn’t seem to gain you much, though, and TestCase doesn’t have a default constructor, so you’d have to add an explicit constructor to your test fixture. Personally, I don’t bother descending from TestCase.

Test fixtures: Your test fixture must have the [TestFixture] attribute, whether or not it descends from TestCase. There’s no restriction on the class’s visibility, although I always make them public out of habit.

SetUp and TearDown: If you’re descending from TestCase, you can just override the SetUp and TearDown methods. Otherwise, make a method and tag it with the [SetUp] or [TearDown] attribute.

Test methods: Like DUnit, NUnitLite recognizes tests by name. Any public method whose name starts with Test (case-insensitive) is considered to be a test method. No more cluttering your code with all those stupid [Test] attributes! Yay! (Pity they didn’t do the same thing for SetUp and TearDown, but, oh well.)

Assertions: In addition to the uber-cool Assert.That() assertions, NUnitLite also supports the old NUnit-style Assert.AreEqual() assertions.

Setting up an NUnitLite test project

Create a new console application. Replace its Main() method with either of the following blocks of code.

Make sure that you’ve added the NUnitLite project to your solution. Then in your new console application, add a reference to NUnitLite (using the Projects tab of the Add Reference dialog). Also add a reference to the project containing the code you want to test.

Then you need to write the Main() method that starts up your app and runs the tests. I’ll present two different code blocks that both accomplish this; choose the one that suits your needs.

Non-automatable. If you will always rely on visual inspection to see if the tests passed or failed (i.e., you’re always running the tests manually, always looking at the output, and never automating anything), you can use this simple Main() routine:

using NUnitLite.Runner;

namespace <your namespace here>
{
public class Program
{
public static void Main(string[] args)
{
ConsoleUI.Main(args);
}
}
}

Automatable. If you actually care about making your tests automatable, you should use this Main() routine instead. It will return an exit code of 1 if any of the tests fail (or if any other type of exception occurs), so you can easily plug it into any style of automated or continuous build, and it will fail the build when the tests fail. (See below for a way to make it fail the compile right in Visual Studio.)

using System;
using System.Reflection;
using NUnitLite.Framework;
using NUnitLite.Runner;

namespace <your namespace here>
{
public class Program
{
public static int Main()
{
CommandLineOptions options = new CommandLineOptions();
ConsoleUI runner = new ConsoleUI(options, Console.Out);
try
{
TestResult result = runner.Run(Assembly.GetExecutingAssembly());
if (result.IsFailure || result.IsError)
return 1;
else
return 0;
}
catch (Exception ex)
{
Console.Error.WriteLine(ex.ToString());
return 1;
}
}
}
}

A sample NUnitLite test fixture

using NUnitLite.Framework;

namespace Tyler.Tests
{
[TestFixture]
public class AdderTests
{
private Adder _adder;

[SetUp]
public void SetUp()
{
_adder = new Adder();
}

public void TestZeroes()
{
Assert.That(_adder.Add(0, 0), Is.EqualTo(0));
}
public void TestTwoPlusTwo()
{
Assert.That(_adder.Add(2, 2), Is.EqualTo(4));
}
}
}

Failing the compile when the tests fail

If you want, you can set up your tests to run automatically every time you compile in Visual Studio, and to fail the compile if tests fail. Here’s how:

  • Make sure you used the “Automatable” Main() method (see above).

  • In Solution Explorer, right-click on your project, and go to Properties.

  • Go to the “Build Events” tab. In the “Post-build event command line” box, type the name of your test EXE. In my case, my project is called Tyler.Tests.csproj, so I just type Tyler.Tests for my post-build event.

Save everything and build. Now, every time you build your test project, your test project will automatically run, and if it returns a nonzero exit code, Visual Studio will show “The command ‘Tyler.Tests’ exited with code 1” in the Error List window. Then you can press Ctrl+Alt+O to view the Output window, where you can see the test output, and see which tests failed and with what messages.

Amusing side note: When I first tried hooking this in as a post-build event, I actually put it in as a pre-build event by mistake. And then I just couldn’t figure out why the test kept failing even after I fixed the code…

Leave a Reply

Your email address will not be published. Required fields are marked *