Joe White’s Blog

I recently talked about something I called hybrid types: a strategic combination of existing Delphi compiler features that basically give you a polymorphic record. All kinds of languagey goodness: automatic memory management, virtual methods, operator overloading, and a touch of mix-in semantics all in one.

(Really it’s just a record with methods, wrapping an interface reference. But you can do interesting things with the idea, and I’m using them fairly extensively as I write DUnitAssertions.)

I’ve since found out, after several hours of troubleshooting, that the Delphi compiler doesn’t always work very well with these hybrid types. I’ll start with the bottom line:

// Don't do this.
MethodThatReturnsAHybridType.MethodOnTheHybridType;

In particular, I was putting a method on my test classes called Value(), that took a TValue and simply returned it; then my tests would chain calls like Value(5).Matches(...). TValue supports operator implicit; this was an easy way to cast integers, strings, etc. to TValue inside my tests. (I tried explicitly casting to TValue(...), which is supposed to work, but I got a compiler Internal Error.) The Value() function seemed like a good compromise, but apparently not. The compiler generates broken code if you do this; it tended to crash my test app on exit.

If a method returns a hybrid type, and you chain a call (as shown) to a method on that hybrid type, the compiler will do a double-free on the interfaced object that the hybrid type wraps. My QualityCentral report on the bug has more details and a full repro case. I tried grovelling around in the assembly code to figure out what exactly the compiler is doing differently with this particular code, but finally gave up.

It’s broken for sure in Delphi 2006. They still haven’t sent us our Software Assurance copies of Delphi 2007 (sound familiar?), so I can’t check whether it’s still broken or not in the latest and greatest compiler.

I shouldn’t be surprised; this is far from the first compiler bug we’ve found relating to records-with-methods (although the others were mostly obscure “internal errors” that failed the compile, not incorrect codegen that screws up the heap at runtime). It’s abundantly clear that the CodeGear developers never use this compiler feature themselves.

When you use a unit-testing framework to compare two records — two TPoints, say, or two TRectangles — would you expect it to stop at the first difference, or show all differences?

Ah, the joys of writing new test-assertion frameworks.

So if I wrote this code:

R1 := Rect(1, 2, 3, 4);
R2 := Rect(2, 3, 4, 5);
Expect.That(R1, Tis.EqualTo(R2));

…what should it output?

It could stop at the first difference, and tell you which one failed:

TRect.Left
Expected: 1
but was: 2

Or it could compare each value separately, and show all the failures:

TRect.Left
Expected: 1
but was: 2

TRect.Top
Expected: 2
but was: 3

TRect.Width
Expected: 3
but was: 4

TRect.Height
Expected: 4
but was: 5

Or maybe it should just show a compact form, and emit one failure:

Expected: Rect(1, 2, 3, 4)
but was: Rect(2, 3, 4, 5)

That last one has the advantage of compactness, but it also loses some context — you don’t exactly have Intellisense in the DUnit output window, to tell you which parameter is which.

Perhaps:

Expected: TRect(Left: 1, Top: 2, Width: 3, Height: 4)
but was: TRect(Left: 2, Top: 3, Width: 4, Height: 5)

But that could get really long, really quickly, if a record had lots of properties. And imagine extending the same thing to classes. Maybe just show the properties that are different?

I don’t know. Which makes the most sense to you guys? Are there other options you can see that might be better?

As I write DUnitAssertions, I’m baking in knowledge of how to compare floats intelligently. This means always providing an epsilon. If you choose to specify an epsilon of zero (or don’t provide an epsilon in the first place), I call Math.SameValue, which provides its own reasonable default epsilon.

While writing some tests for my code, I found an interesting bug in SameValue. It thinks that Infinity and NegInfinity are equal!

Just a bit off, that.

Now curious, I did some debugging.

SameValue starts out by seeing if you have zero as an epsilon (which you will, if you don’t specify the default parameter). If so, it calculates its own reasonable epsilon. Here’s the code it uses:

Epsilon := Max(Min(Abs(A), Abs(B)) * ExtendedResolution, ExtendedResolution);

So it takes ExtendedResolution (a very small number) and multiplies it by +INF. The result? +INF, of course. So anything within infinity is considered “close enough”.

Hmm. I think I may have found the problem.

And if Delphi’s standard library functions work this poorly with infinities, I may just not spend too much time worrying about how my framework handles them…

Marco Cantù today mentioned that Google Maps will give you driving directions from Italy to New York. Included in the directions is a step that says “Swim across the Atlantic Ocean — 5,572 km“.

To give you an idea of the kind of geek I am, I went on to look for other ocean routes that they support. Mainland to Hawaii? Nope, they can’t do it. U.S. to Australia? No can do.

Then I tried using Florida instead of New York. They routed me all the way up the East Coast. So apparently they only have a single route across the ocean, a single “road” if you will.

So then I tried to narrow down where this “road” starts and ends. (Am I scaring you yet?) I had two reasons for this: one, because I was curious; and two, because I wanted to know how long they thought the swim would take, so I could figure out how fast they think I can swim.

At this point, I found two bugs in Google Maps.

Bug #1: If you ask for a route directly from the starting point (Long Wharf, Boston) to the ending point (Terminal Grande-Bretagne, Le Havre, France), it doesn’t tell you to swim. It just tells you to “head south“. (Here’s a better version of the Google Maps swimming directions; it starts a block away, and does tell you to swim.)

Bug #2: They think you can make the 3,462-mile swim in 29 days 0 hours. Which means they think you can manage an average sustained pace of 5 miles an hour, or 2.2 meters per second, or 4.3 knots. Swimming. Average. If you swim 24 hours a day.

I mean, come on, guys. You could have at least done some research. The swim across the Atlantic actually takes 72 days.

(They also got the start and end cities wrong — it should have been Cape Cod, Massachusetts to Quiberon, France. But one mustn’t be too picky.)

For DUnitAssertions, I’ve been innovating on Delphi’s type system to create hybrid types. These are polymorphic, automatically memory managed, support implicit casts and operator overloading, and even have some aspects of mixins. And you don’t have to wait for a new compiler — they’re usable now.

It started with the universal value type. I needed implicit casts, so that you could have a variable (or, more often, parameter) of type TValue, and just assign any old type into it — Integer, Boolean, object, interface, enum, etc. Sounds like a record-with-methods (including operator Implicit) to me.

But I also wanted polymorphism. I needed an integer type and a floating-point type and an object type and an enum type and… You can’t do inheritance with Delphi’s records-with-methods, and I wasn’t about to write a variant record (shudder) or use old-style objects (double shudder).

What I really wanted was an interface that supported operator overloading. But interfaces don’t support operator overloading, because you need an implementation for the operator methods. Delphi interfaces, by definition, don’t have their own implementations; they’re pure interfaces, not mix-ins.

So I brought together the best of both worlds: I used a record type that contains an interface reference. The record type supplies the operator Implicit; the interface supplies the polymorphism (and the automatic refcounting).

This also turned out to be convenient when I was doing constraints (the Tis.EqualTo(...) family). You’ll be able to combine constraints using and and or; and those compound constraints will divide the work of building the assertion-failure message.

Here are some samples of what I’m planning for assertion-failure messages:

Expected: 4
but was: 2

Expected: 4
or 5
or 6
but was: 2

Expected: not 2
and not 3
and not 4
but was: 2

Expected: from 10 to 100
but was: 2

The interface supplies the expected value — everything after “Expected:” and before “but was:”. This makes it terribly easy for interfaces to decorate each other; TNotConstraint.ExpectedMessage just calls the ExpectedMessage of the constraint it’s wrapping, and then prepends “not “. TAndConstraint and TOrConstraint will be similarly easy to implement.

Then the record puts that together with the actual value, and builds the complete failure message. The record is the top-level object in the composite tree; it’s the only one that needs to build the complete message.

This is much nicer than the typical Composite pattern in Delphi, where you end up with every class in the hierarchy supporting both the “get the fine-grained value that my parent might need” and “build the whole whopping failure message”. Typically the “build the whole whopping failure message” logic doesn’t vary from implementation to implementation — it’s usually not even any different for the null objects!

Traditionally I would’ve used a base class that implements the non-polymorphic methods, and made all my interface implementations descend from it. But I think providing a top-level wrapper for the whole aggregate is a nicer way to go.

At work, we’ve added several items to Delphi’s Tools menu to make Subversion functions easy to get to from the IDE. I set some of them up on my home PC too, because they’re just so convenient. They’re easy to access from the keyboard, too (just hit Alt+T and then C to bring up a commit window), which is a minor miracle given that Delphi still doesn’t support customization of keyboard shortcuts.

We have a few more at work, but here are the “gotta-haves” that I’ve set up on my home machine:

• Svn Commit: Opens the TortoiseSVN commit window.

• Svn Diff: Shows diffs for the file currently being edited. (If you’ve configured an external diff viewer like Beyond Compare, this will use it.)

• Svn Modifications: Opens the TortoiseSVN modifications window, which shows a list of all modified files.

• Svn Update: Updates your working copy with the latest changes from the repository.

These should be trivial to set up with Delphi’s Tools > Configure Tools dialog, since they have macro expansions for things like the path to the current project. Unfortunately, Delphi’s Configure Tools macro expansions pretty much suck for this. TortoiseSVN requires that the parameter be “slash-parameter name-colon-value”, e.g. /path:C:\svn\DUnitAssertions. Delphi, on the other hand, is incapable of doing a macro expansion that isn’t preceded by a space, so that “colon before the value” thing is a killer. It doesn’t get any better when your paths can include spaces and you need to quote the path.

So I’ve got a script in Ruby, which reads the command-line parameters that Delphi is capable of producing, and translates them into something that TortoiseSVN is capable of understanding. The one I wrote at home looks like this (I saved this in C:\svn\RunTortoise.rb):

COMMAND = ARGV[2]
PATH = ARGV[3..ARGV.length].join(" ")
command = %Q|"c:/program files/tortoisesvn/bin/tortoiseproc.exe" | +
%Q|/command:#{COMMAND} /path:"#{PATH}" /notempfile|
puts command
system command

Then you can set up the Configure Tools items. Mine look like this (these assume that your project file is at the “root” of your Subversion checkout, i.e. that all the files you’re editing are in that directory or its subdirectories):

Title:	Svn &Commit
Program:	c:\ruby\bin\rubyw.exe
Working dir:	(blank)
Parameters:	C:\svn\RunTortoise.rb - -- commit $PATH($PROJECT) $SAVEALL

Title:	Svn &Diff
Program:	c:\ruby\bin\rubyw.exe
Working dir:	(blank)
Parameters:	C:\svn\RunTortoise.rb - -- diff $EDNAME $SAVEALL

Title:	Svn &Modifications
Program:	c:\ruby\bin\rubyw.exe
Working dir:	(blank)
Parameters:	C:\svn\RunTortoise.rb - -- repostatus $PATH($PROJECT) $SAVEALL

Title:	Svn &Update
Program:	c:\ruby\bin\rubyw.exe
Working dir:	(blank)
Parameters:	C:\svn\RunTortoise.rb - -- update $PATH($PROJECT) $SAVEALL

You never want to do exact comparisons on floating-point numbers. They’re finite precision — and 0.1 is a nonrepeating fraction in binary. So 1.1 plus 0.1 does not necessarily equal 1.2.

So float comparisons should always have an epsilon, or fudge factor — something that says “if the difference is only 0.000001, that’s close enough”. That’s why Delphi has the SameValue function (which provides its own reasonable epsilon if you don’t specify your own), and why DUnit has overloads for CheckEquals that take an epsilon.

NUnitLite apparently doesn’t do epsilons, though, so I’m on my own for coming up with a meaningful syntax as I write DUnitAssertions. Here are a few of my brainstorms:

Expect.That(1.1 + 0.1, Tis.EqualTo(1.2, 1E-6));
Expect.That(1.1 + 0.1, Tis.EqualTo(1.2), 1E-6);
Expect.That(1.1 + 0.1, Tis.EqualTo(1.2), Tis.Within(1E-6));
Expect.That(1.1 + 0.1, Tis.EqualTo(1.2), To.Within(1E-6));
Expect.That(1.1 + 0.1, Tis.EqualTo(1.2), Plus.Minus(1E-6));
Expect.That(1.1 + 0.1, Tis.EqualTo(1.2).ToWithin(1E-6));

I like the third one (To.Within) the best, except that it won’t compile (to is a reserved word in Delphi). Obviously I could make it &To but that’s kludgy; a Delphi test framework should compile in Delphi without escaping the method names. Jennie suggested “plus or minus”, which is what prompted Plus.Minus. The last one might work too, although I’d have to fiddle with it.

I’m also not sure how discoverable any of these are. Ideally I’d want something where anyone could sit down with nothing more than the test framework, Intellisense, and one example (Expect.That(2 + 2, Is.EqualTo(4), 'Something is wrong');), and be able to figure the rest out. I already know that’s an impossible goal in Delphi/Win32, because of the issues with enums, but it’s still a decent aim.

Any preferences among these examples? Suggestions?

All four cats are home. We got to bring Tycho home from the vet on Friday, and Goober yesterday. (Tycho was so ready to go home. I could almost hear him saying, “Give me the keys! I’ll drive!”)

All four cats’ values are back down to well within the normal range — even Goober’s, which started out so far off the charts, less than a week ago, that we had no idea whether he was going to survive. He is all better. (There may be long-term effects, five or ten or fifteen years down the road. But that’s no worse than some of the other health problems our cats have turned up with; in my mind, that’s as good as healthy.)

They all get re-checked on Wednesday, and then at progressively longer intervals thereafter.

Life is much better.

Well, mostly better. Now we’ve got to give them subcutaneous fluids every day. That means that we (not, repeat, not trained professionals) stick an IV needle into each cat’s back, and then try to get them to sit still and not flinch for five or ten minutes. (I used to donate plasma, and at the end of the process, they would run fluids back into the IV to replace what they took. That stuff was freaking cold. So I know how the cats feel. But it’s still gotta be done if we’re going to make sure they stay healthy.)

We try to remind them that it’s either five or ten minutes a day of something they don’t like, or twenty-four hours a day of being at the vet. They don’t seem convinced.

Ah well. Band-Aids are cheap.

Stefan and Noel got to come home today.

All four cats are doing really well. The intravenous fluids have been working very well on all of them. Hopefully the same will end up being true for everyone else’s cats who had been eating m/d and are now stuck in the hospital.

We’ll have to give Stefan and Noel subcutaneous fluids once a day for a while (the doctor wasn’t there when we visited, so we’ll have to wait till tomorrow to find out how long this arrangement will last). This basically amounts to attaching an IV needle to a bag of saline solution, sticking the needle in the scruff of their neck, and getting them to sit still for ten minutes or so. It’s not really as bad as it sounds. They showed us how to do it before we brought them home. Both cats put up with it a lot better than I thought they would; it didn’t seem to hurt them at all. They just didn’t want to sit still that long.

The other two cats will be at the vet a while longer. Tycho’s blood-test values were normal when they checked him today, but they’re keeping him another day or so just to make sure, since his values were high when we brought him in. Goober probably won’t be able to come home until Saturday at the absolute earliest, maybe Monday.

Jennie brought a milk ring along when we went to visit Goober this morning, and he was playing with it enthusiastically. He’s back to himself.

I’m not sure what to think about Tycho. He’s been friendly. To the lab techs. At the vet’s office. This is not like him. He even let one of the doctors scratch him on the nose, and he didn’t try to bite or anything. Jennie and I have started calling him “pod-cat”. It’s not that we mind him being nice at the vet’s office. Usually they have to sedate him to do anything to him at all, and they have to get out the metal-studded elbow-length heavy-duty leather gloves to be able to hold him long enough to sedate him. So it’s not that we mind not having to deal with that. It’s just that, well, to quote an old Garfield comic strip (which ended with Garfield in a straitjacket in a padded room), “People don’t want nice… people want consistency.”

Goober doesn’t seem to mind being at the vet’s office too much — he’s enough back to his usual autistic self that I’m not entirely convinced that he even notices where he is. But Tycho is desperate to come home. He’s being (unusually) nice and all, but sounded so pitiful when we left this evening. He just about smashed my fingers against the cage bars when he head-butted my hand.

I really hope everything looks good again tomorrow so he can come home. I guess I’m just a sucker for a sad yowl.

DUnit has several overloads of the CheckEquals method, but we have our own testcase that descends from theirs and adds lots more overloads. Most of them are for enums. We have things like:

type
TMyTestCase = class(TTestCase)
strict protected
procedure CheckEquals(Expected, Actual: TFormBorderStyle; Msg: string = ''); overload;
function EnumToStr(Value: TFormBorderStyle): string; overload;
...
procedure TMyTestCase.CheckEquals(Expected, Actual: TFormBorderStyle; Msg: string)
begin
CheckEquals(EnumToStr(Expected), EnumToStr(Actual), Msg);
end;
procedure TMyTestCase.EnumToStr(Value: TFormBorderStyle): string;
begin
Result := GetEnumName(Ord(Value), TypeInfo(TFormBorderStyle));
end;

Obviously, if you want other methods to support an enum too — CheckNotEquals, for example — you have to overload them too. This really makes me wish I were working in a language with a unified type system.

When I started thinking about DUnitAssertions, enums were one of the first things I thought about, because you’d have to add overloaded methods in more than one place. Consider this assertion:

Expect.That(Form.BorderStyle, Tis.EqualTo(bsNone));

You’d have to add TFormBorderStyle overloads to two methods to make this work — Expect.That and Tis.EqualTo — instead of just having to overload CheckEquals like in DUnit. Yuck.

After poking and prodding at this for a bit, I found what I think is a decent solution: I made a universal value type, which I’m calling TValue. It’s a record type that supports operator Implicit, so you can supply an integer, an Int64, a floating-point value, a Boolean, a string, an object, an interface, or any enum type it knows about. Then all the assertions just take TValue parameters. So you can pass any type of value to those methods, without the methods themselves needing tons of overloads. The overloading is effectively moved from the methods onto TValue, and centralized there. It’s actually kind of slick.

“But,” you may be thinking, “why a new type? What about array of const? What about Variant? Why make another universal value type, when Delphi already has at least two of them?”

Simple answer: because Delphi’s “universal” value types suck rocks when it comes to enums. With both array of const and Variant, if you pass in an enum value, they turn it into the ordinal value — so bsDialog becomes 3. All type information is utterly lost, and if the assertion ever failed, you would get something extremely unhelpful like “Expected 0 but was 3″. In my view, that’s unacceptable. If you’re comparing enums, the assertion needs to say “Expected bsNone but was bsDialog”.

Having a custom-coded universal value type does fix that. It does introduce a few wrinkles of its own, though. For one, you have to write some code every time you want it to support a new enum. But that’s not a huge deal; you already had to do that to compare enums in DUnit. Actually, with the universal value type, you only need to write one method instead of two or more:

class operator TValue.Implicit(Value: TFormBorderStyle): TValue;
begin
Result := TValue.Create(TEnumValue.Create(Ord(Value), TypeInfo(TFormBorderStyle)));
end;

Once you’ve added the operator Implicit for your enum, everything else will just work — Tis.Not.EqualTo will just work with no extra effort. Even Tis.GreaterThan would come along for the ride, if you were sadistic enough to want to do greater-than / less-than comparisons on enums.

A somewhat bigger issue is dependencies. With ordinary DUnit, you can write your own assertion methods that operate on enums, and put them in your own unit; there’s no need to edit TestFramework.pas from the DUnit distribution. Not so with a universal value type that the test framework depends on. If you want to add a new enum to be supported, you’ll have to edit Values.pas, which in turn is used by all the other DUnitAssertions units. So if you have a TMyProprietaryEnum type in your ProprietaryStuff unit, you’ll actually have to change the Values unit in the DUnitAssertions distribution to depend on ProprietaryStuff.pas.

This is distasteful, but remember that your test project already depends on ProprietaryStuff. The only case where I can see genuine problems with making Values.pas depend on ProprietaryStuff.pas is if you have multiple different DUnit test EXEs, and you don’t want them all to depend on the same code base. In this case, you would have to have multiple separate copies of Values.pas, one for each project, and set your search paths accordingly so that each project finds its own TValue. Also distasteful, but it’s the best option Delphi’s type system leaves available.