We did manage to trade in our Visual Studio Architect licenses, and get Developer instead. So I sat down to play with the unit-testing tools.
Verdict? The unit-testing support in Visual Studio 2005 is a toy. Nothing more.
Why do I say that? Because they let you write unit tests, but they don’t let you run them.
The unit-testing support is all part of Visual Studio, not of the .NET Framework itself. So that means you can only run your tests inside the Visual Studio IDE. Meaning, you can’t run your tests from the command line. Or as part of an automated build; there’s no way to tie the tests into MSBuild or Ant or NAnt or anything else. If you want to know that your latest official build actually passes all the tests, you have to open up Visual Studio so you can run the tests manually.
Correction: Apparently there’s a command-line tool called MSTask that lets you run the tests, so apparently you can run the tests from an automated build. Doesn’t help with running the tests as you develop in the IDE, though.
And you have to select the tests manually. There is no way to tell the IDE, “Run all the tests.” Oh, you can select a test from a list, and then click a toolbar button to run it. You can even Shift+click and Ctrl+click to select multiple tests, and then click the toolbar button to run them. But if you want to run all the tests, you have to manually select them all in the list. When you write a new test, you have to manually select it, too. And it doesn’t remember your selections the next time you start Visual Studio.
Don’t we use computers because they let us automate things? Just wondering.
You could, alternatively, hit Ctrl+Shift+X, which runs all the tests in the current assembly. But that only works if the code editor is currently on a test file. Meaning, you can write your test, and hit Ctrl+Shift+X to watch it fail; but then when you go edit your production code to make the test pass, and hit Ctrl+Shift+X again, it does nothing. Why, after all, would you want to run a test you’re not editing?
Why, indeed. Good God, what the hell is wrong with these people?
The most fundamental operation for unit tests is running all the tests. That’s the point. Yes, it can be useful to run a subset of the tests, but that’s a feature you add later. The first thing you write in any testing framework is “Run All Tests”. Because it’s something you do dozens of times a day. You do it all the time, because every time you change your code, you want to know you didn’t just break something. If you’re only running a single test after each change, and you get to the end of the day and find out you broke some other test, you have no idea when you broke it. Which is why you don’t run a single test. You run all the tests, unless you have a really good reason not to.
They just don’t get it. They give you tools for writing everything but unit tests. They give you tools for writing database tests, and ASP.NET server tests. And sure, you don’t want to run those dozens of times a day, because they’re slow, and you’d never get anything done. But unit tests are fast. Any unit test worthy of the name is only going to test your code — not the database, not the server, ideally not even the filesystem. A unit test takes maybe a few microseconds to run. You can run thousands of them at a time, dozens of times a day. It can be a lot of work finding a way to test your code without using the database, but it’s usually worth it, because it gives you the chance to run the test All. The. Time.
Unless, of course, you’re using Microsoft tools.
It’s ridiculous. Team System comes with this fancy new version-control system, when you can get a fabulous tool like Subversion for free. It comes with something Microsoft Marketing claims is a unit-testing tool, but it turns out we’re better off sticking with csUnit.
Team System is just looking more and more like a complete waste. Score another point for open source.