.NET code coverage: NCover

I spent some time today looking at .NET code-coverage tools. (We already know that Microsoft’s code-coverage tools are a waste of time.) Here’s what I dug up.

There are two open-source projects called NCover, and they have nothing to do with each other. One is hosted on SourceForge, the other at NCover.org.

ncover.sourceforge.net apparently works by instrumenting your source files. You have to run ncover-console on your source code; then you add NCover.dll to your project references, and recompile; then you run your unit tests; then you run nrecover to remove all the extra instrumentation code and put your source code back to its original state. Its output appears to be HTML-based, and looks decently readable, though fraught with typos.

ncover.org (which I think is the one that used to be on GotDotNet) works by using the .NET profiling API. You don’t have to modify your code; you just point NCover at your compiled assemblies (you also need PDBs for any assemblies you want it to instrument), and it does some profiling voodoo to instrument your code as it runs. Its output is XML, so it wouldn’t be too hard to write other tools that read it back in, and it comes with an XSL stylesheet that makes the output all pretty and collapsible (alas, the prettiness is only pretty in Internet Explorer, but it’s still kinda nifty).

I prefer having a tool that uses the profiling API, instead of modifying my source code, so I downloaded the NCover from ncover.org. It has an installer, but it doesn’t create any Start Menu icons; it just drops files into Program Files\NCover, and you have to specify the path when you run it. The only thing you need to watch out for is that you need two slashes on the command-line arguments to NCover itself (anything without two slashes is passed through as a command-line parameter to the app being profiled).

I didn’t have any unit tests set up yet (well, I did, but they weren’t usable), so I just ran NCover on my main executable:

"\program files\ncover\ncover.console" //a Budgeter.Classes Budgeter\bin\debug\budgeter.exe

It printed some debug output in the console window, and then my app started up. I clicked a few things, closed the app, and NCover generated its report (this is the NCover output from my little spare-time app).

It’s that simple: you run it, and it just works. And it generates useful output, and it isn’t nailed to the IDE. My kind of tool. Score yet another point for open-source.

Next up: figuring out which unit-testing framework to use…

Visual Studio 2005: Still wondering where the unit-testing tools are

We did manage to trade in our Visual Studio Architect licenses, and get Developer instead. So I sat down to play with the unit-testing tools.

Verdict? The unit-testing support in Visual Studio 2005 is a toy. Nothing more.

Why do I say that? Because they let you write unit tests, but they don’t let you run them.

The unit-testing support is all part of Visual Studio, not of the .NET Framework itself. So that means you can only run your tests inside the Visual Studio IDE. Meaning, you can’t run your tests from the command line. Or as part of an automated build; there’s no way to tie the tests into MSBuild or Ant or NAnt or anything else. If you want to know that your latest official build actually passes all the tests, you have to open up Visual Studio so you can run the tests manually.

Correction: Apparently there’s a command-line tool called MSTask that lets you run the tests, so apparently you can run the tests from an automated build. Doesn’t help with running the tests as you develop in the IDE, though.

And you have to select the tests manually. There is no way to tell the IDE, “Run all the tests.” Oh, you can select a test from a list, and then click a toolbar button to run it. You can even Shift+click and Ctrl+click to select multiple tests, and then click the toolbar button to run them. But if you want to run all the tests, you have to manually select them all in the list. When you write a new test, you have to manually select it, too. And it doesn’t remember your selections the next time you start Visual Studio.

Don’t we use computers because they let us automate things? Just wondering.

You could, alternatively, hit Ctrl+Shift+X, which runs all the tests in the current assembly. But that only works if the code editor is currently on a test file. Meaning, you can write your test, and hit Ctrl+Shift+X to watch it fail; but then when you go edit your production code to make the test pass, and hit Ctrl+Shift+X again, it does nothing. Why, after all, would you want to run a test you’re not editing?

Why, indeed. Good God, what the hell is wrong with these people?

The most fundamental operation for unit tests is running all the tests. That’s the point. Yes, it can be useful to run a subset of the tests, but that’s a feature you add later. The first thing you write in any testing framework is “Run All Tests”. Because it’s something you do dozens of times a day. You do it all the time, because every time you change your code, you want to know you didn’t just break something. If you’re only running a single test after each change, and you get to the end of the day and find out you broke some other test, you have no idea when you broke it. Which is why you don’t run a single test. You run all the tests, unless you have a really good reason not to.

They just don’t get it. They give you tools for writing everything but unit tests. They give you tools for writing database tests, and ASP.NET server tests. And sure, you don’t want to run those dozens of times a day, because they’re slow, and you’d never get anything done. But unit tests are fast. Any unit test worthy of the name is only going to test your code — not the database, not the server, ideally not even the filesystem. A unit test takes maybe a few microseconds to run. You can run thousands of them at a time, dozens of times a day. It can be a lot of work finding a way to test your code without using the database, but it’s usually worth it, because it gives you the chance to run the test All. The. Time.

Unless, of course, you’re using Microsoft tools.

It’s ridiculous. Team System comes with this fancy new version-control system, when you can get a fabulous tool like Subversion for free. It comes with something Microsoft Marketing claims is a unit-testing tool, but it turns out we’re better off sticking with csUnit.

Team System is just looking more and more like a complete waste. Score another point for open source.

Visual Studio 2005: Dude, where are the unit-testing tools?

So all the developers on our team recently got copies of Visual Studio 2005. (Year-end and leftover budget: two great things that go great together.)

The question came up of which edition to get: Architect, Developer, or Tester. The person on our end said, “Get Tester for all the QA people, and Architect for all the developers.” Sounded reasonable to me: in every development environment I’ve ever used, Architect is the edition with all the goodies.

Well, apparently not so with Microsoft these days, as I found when I tried to write some unit tests, using the new unit-testing framework that’s supposed to ship as part of Visual Studio 2005’s Team System thingy, and then did some research to find out why I couldn’t.

Apparently, Microsoft has decided that architects don’t write code. Oh heavens no. Architects don’t need tools for unit testing, or profiling, or code coverage, or static analysis. Apparently, architects just bring the specs down from the mountain, and then goof off until the next release.

In my world, the term “architect” does not mean “doesn’t sully their hands with code”. I can sorta see an architect not needing profiling and code coverage, if they’re on a team of, say, five hundred or more. But an architect would still need unit testing tools because the tests are part of the spec. The spec isn’t complete without acceptance tests that say, “The feature isn’t done until this test passes.”

Complete specs, it appears, are not a concern at Microsoft.

Money priorities

Sox is sick, Jennie lost her job, and we’re getting sued.

By the end of this week, I was really starting to stress about the money thing. We haven’t revised the budget since Jennie lost her job earlier in the week, but I had assured her it was okay, we had the emergency fund to cover things like this. But yesterday, after I found out we were getting sued for two thousand some-odd dollars (let this be a lesson: never, never, never, never co-sign someone’s apartment lease), I was starting to freak out.

And then last night it hit me: my priorities were backwards again. Here we are, planning to pay off one of our debts this month, and at the same time, we’re pulling money out of our emergency fund to pay for doctor bills and lost income and vet bills and court costs.

And once I realized that’s what we were doing, I couldn’t help but see how stupid it was. No matter how annoying these collections people may be, they are not a necessity of life. They can wait. We have to take care of us first. We’ll keep making our little payments to them, and we’ll pay them off when we can, but alas, they’ll have to keep paying someone to annoy us for a while yet.

So, we’re going to adjust our budget for the loss of Jennie’s income, we’re going to pay Sox’s vet bill, and we’re already dealing with the lawsuit thing as best we can for now. The annoying creditor won’t get paid off this month, but our emergency fund will continue to be funded, which sounds like a really good idea when there’s a lawsuit hanging over our heads.

I’m not totally over my panic yet, but it’s getting a lot better since I got my priorities straightened out. It’ll get better yet when we actually sit down and revise the budget, and I can see all the numbers, and prove to myself that it’ll be okay.

But first, I’ve got a sick cat to take to the vet.

How to write an elevator speech

wiki.ehow.com is a pretty cool site. I first found it when I was looking for instructions on how to mince garlic. They’re one of the RSS feeds on my Google homepage now.

One of the latest is “How to Develop Your Personal Elevator Pitch“. They don’t explain what they mean by “elevator pitch”, but I’ve heard the term before (though I heard it called “elevator speech”). One of my former bosses had started three different companies, and was pretty good at getting investor money, so he was well-versed in elevator speaking.

The idea is, you’ve just gotten onto an elevator with someone, and they say, “So, what do you do?” Your time limit: you have until the elevator stops. You’re not giving your life story, you’re just trying to get their interest so you can have them remember you, and give them a business card. You seldom give an elevator speech in an actual elevator, but the name gives the flavor of it: it’s a speech for any situation (and they are legion) where you have a very limited time to get to the point and get the other person’s interest.

The eHow elevator-pitch article isn’t about elevator speeches in general (though it has some good tips). It’s specifically targeted to job interviews, and having a prepared answer to the interview question, “Tell me about yourself.” If you’re on a job search, this article is definitely worth a read.

Loot 2005

We had a good Christmas.

It was good to see family (including cats), and of course there was lots of good food. There was also a load of loot. Serious loot. I mean, we’re talking 116-piece drill-bit set kind of loot. Massaging foot spa kind of loot. Forty-nine-piece food-container system with interchangeable lids on a spinning carousel as seen on TV kind of loot. And, of course, the big Christmas check from Mom and Dad (which, to our happy surprise, was much bigger than even in years past).

Not just that, but I had also been doing an evenings-and-weekends project at work with a couple of other developers. We finished that up just before Jennie and I left to go visit family. Between the bonus I got for that project, and the Christmas check from my parents, we had a little less than half a month’s extra salary lying around (!).

Of course that money went into the budget; we weren’t about to have it disappear and wonder where it went! That doesn’t mean we couldn’t have any fun with it, mind you; in fact, we added a “Toys” category to the budget this month. I bought a used laptop from work at a pretty good price (and it’s running pretty well for only having 256 MB of memory), and we picked up a digital camera on clearance.

But the bulk of that money is going to pay off another creditor, and that will be money well spent. Per the Financial Peace plan, we’re punting the lowest balance first. As an added bonus, they’re the ones that have been the biggest pains in the ass (sending our checks back because they “don’t take partial payments” and then calling to chew us out for not paying; it’s actually rather amusing). There are only two disappointments: one, that they might think their harassment tactics actually worked; and two, we won’t get to make fun of them anymore. Ah well. I think we’ll live, on both counts.

So the money thing is definitely going well. We’ll even be able to fully replenish our emergency fund (which we raided last month to fix the car’s radiator — I think this is the first time we’ve ever had a car crisis without a money crisis to go with it!) But the family thing was also great, the food was great, the gift-giving was great, the Christmas Eve church service where we sang “Silent Night” by candlelight was great, the time to read books and play Playstation was great, even the Trivial Pursuit was pretty good. Good downtime, good time.

I’m not sure I’m ready to go back to work yet, but I’d better be, since I’ve already been back for four work days. But it’s definitely been a great Christmas.

Now we just need to find time to take our tree down…

Editing and continuing

Sometimes, the most amazing things are the ones you barely notice.

I was just debugging in Visual Studio 2005. I wanted to know why, when I started in-place editing in a ListView, and then hit Enter without typing anything, it blanked out the item. So I put a breakpoint on the “node.Text = e.Label;” line in my AfterLabelEdit event. (I’m running the listview in VirtualMode, so I have to handle the edit events manually.)

Turns out, when you hit Enter without typing anything, the e.Label that gets passed to your event handler is null. I hunted through the online docs to see if they said this null was expected, but found not a shred of confirmation or explanation.

That’s some seriously bizarre behavior, but okay, I guess. If “hit Enter without typing anything” comes through as “null”, I can deal with that: I’ll look for null, and when I see it, I’ll just throw the change away. I don’t have to understand it, I just have to code for it. So I put an “if (e.Label != null) {” at the beginning of my method, and “} else e.CancelEdit = true;” at the end.

I didn’t remember that I was still stopped on the breakpoint. I only noticed that after I had finished making this change. “Well,” I thought, “here’s a good chance to try out that newfangled Edit-and-Continue feature.” So I hit F5 to continue running, expecting to see some sort of prompt asking me whether to apply my changes.

No prompt. My program just started running again.

I clicked on an item to go into edit mode, then hit Enter. It didn’t clear out the item this time. Visual Studio had applied my change, without needing to bother me with a prompt. What’s more, it had modified the method I was stopped at a breakpoint in the freaking middle of.

This is just so cool.

Thinking back to what I’ve read about EnC, I think I remember that it won’t modify any methods that you’re currently in the middle of, anywhere along the call stack (which eliminates the hairy questions about what should happen if you move or delete the line you’re in the middle of running). But for it to modify that method for next time you hit it, and to do it utterly transparently, without even a noticeable delay, is just mind-blowing.

They may still have some stupid bugs in their GUI controls, but in some things, .NET 2.0 and Visual Studio 2005 seriously rock.