Emergency funds

You know what? Having an emergency fund doesn’t take away the stress.

Okay, let me amend that: Having an emergency fund does take away the stress, but only if you never have to touch it.

Back when we started budgeting, we made a choice. We decided that car breakdowns would fall under the heading of “Emergency”. So would sick cats. And so we’ve had to dip into the emergency fund probably half a dozen times in the past year. And every time, we stress about it.

Actually, I think those choices were probably the right ones, the first few months we were budgeting. But not anymore. And so we’re finally learning, and our October budget has two new line items in the “Saving” category: “Sick Cat Fund” and “Sick Car Fund”. Next time the inevitable happens, we won’t have to dig so deeply into the emergency fund. If we’re lucky, we won’t have to touch it at all.

In retrospect, we should have put those categories in our budget from the beginning. If there were months when we couldn’t put anything in them, fine, but they still would’ve been a reminder that we really ought to be doing something about this.

9/11: Five years

There’s a place in my brain labeled “Things about 9/11 I will Never Forget”. I’m embarrassed to say that I’m pretty sure there were once three things in there, and now I can remember only two. But those two are: Yasser Arafat donating blood; and the British palace guard, for the first time in history, standing at attention during the “Star Spangled Banner”.

9/11 brought the world together. I hope that that, and not fear, will be its lasting legacy.

Unit tests for parsers

I’m still tinkering with my ANTLR-based Delphi parser, but I’ve also started playing around with Racc. Today, I found out that there are going to be some fundamental differences in how I write unit tests for Racc.

ANTLR generates a recursive descent parser, which means that it can’t handle any sort of ambiguity in the grammar. (Is “at” a keyword or an identifier? “It depends” is not generally an answer ANTLR can accept.) But there’s an upside, too: the code it generates is very easy to understand (and debug, for that matter). If I have a parser rule “expression ::= atom operator atom“, then ANTLR generates a function called expression(), which consists of calls to atom(), operator(), and atom(). There’s some other decorations along the way, but it’s basically that simple.

This makes it really easy to write unit tests for the expression rule in isolation: I just stuff some tokens into the lexer, call parser.expression(), and then make sure all the lexer’s input has been consumed. It’s not really a unit test of expression(), since I’m also testing everything expression() calls, but it’s been working reasonably well so far.

Racc, on the other hand, makes an LALR parser. It should, like yacc, be able to handle all sorts of ambiguity, as long as there’s some way to eventually resolve it. It generates a table-driven parser, basically a big state machine (so I hope I never have to debug it).

And it doesn’t have entry points for individual parser rules. This presents a problem for unit testing.

I see several options here:

  • Don’t write unit tests for the parser. (Fat chance.)
  • Force the state machine into the right state, then throw some tokens at it, and check its final state. The problem with this is, I’m pretty sure there’s more than one starting state for “expression”, since it will show up multiple places in the grammar, each with its own exit conditions. (Which state do you move to if you’re in an expression and you hit a semicolon? Depends entirely on which state you were in when you started…)
  • Feed in a complete source file’s worth of syntax for every test. If I’m testing the expression rule, then each test would prepend “unit Foo; initialization implementation begin” and append “end.” to the expression I’m testing. This might be doable, though it would force me to implement the grammar starting from the unit rule and working down (I had been going the other direction with the ANTLR parser), which would necessitate some really good record-keeping to know how close I was to being done.
  • Whip up some magic tokens that will never be generated by the real tokenizer, but can be injected by tests. Then make the top-level rule able to parse either projects (“program Foo; …” or “library Foo; …”), units (“unit Foo; …”), packages (“package Foo; …”), expressions (“<secret token for expression> 1 + 2”), uses clauses (“<secret token for uses clause> uses Foo, Bar;”), etc. This also would be doable, if I could automate the repetitive parts of the syntax; and it would let me build the grammar from the bottom up, which has its advantages.

Has anyone out there written an LALR grammar test-first? Does anyone have any suggestions, or see anything I hadn’t thought about?

On using Rake.run_tests

Okay, an update to my previous Rake post. Rake does have some nice facilities for running tests, but the code I posted isn’t quite the right way to do it.

When you use rake/runtest, you should put the require inside your test task (inside each test task, if you have more than one). Like so:

task :test do
  require 'rake/runtest'
  Rake.run_tests "**/*tests.rb"

require 'rake/runtest'
task :test do
  Rake.run_tests "**/*tests.rb"

The reason: the WRONG version will always (indirectly) require ‘test/unit’, which will always run your tests, even if you go through a code path that doesn’t register any.

Here’s what it looks like if you have the WRONG version and run rake -T (list all the interesting targets to the console, but don’t run any of them). Note that it’s running the tests, even though there aren’t any to run.

C:\svn\dgrok-r>rake -T
(in C:/svn/dgrok-r)
rake all  # Runs everything
Loaded suite c:/Program Files/ruby/bin/rake

Finished in 0.0 seconds.

0 tests, 0 assertions, 0 failures, 0 errors


Here’s what it looks like if you use the RIGHT code above. It still runs the tests when you execute the “test” target, but when you’re not doing anything that involves test running, it doesn’t give you all that nonsense about how it ran 0 tests in 0.0 seconds:

C:\svn\dgrok-r>rake -T
(in C:/svn/dgrok-r)
rake all  # Runs everything


The coolness that is Rake

Rake is cool enough as a build tool, but wait until you check out its library.

First off, if you use make or ant, you owe it to yourself to look into Rake. The syntax is so much more readable than either, and you have a full-fledged programming language right there in your build script. The docs have a concise introduction to Rakefiles; Martin Fowler has a longer article called “Using the Rake Build Language“.

But wait! There’s more!

So Rake’s documentation is pretty sketchy. Mention is made of some cool libraries that ship with it, but no details. So I decided to RTFS, and I found much of coolness.

Coolness #1: All the coolness comes right out of the box. You don’t even need to require anything from your rakefile (unless you’re doing really fancy stuff). Just write a rakefile containing some tasks, run rake, and you’re off and running.

Coolness #2: They add an ext() method to the String class. Treats the string as a filename; returns a new filename with the extension changed to whatever you specify. Equivalent to Delphi’s ChangeFileExt() or .NET’s Path.ChangeExtension. Something that’s sorely lacking from Ruby out of the box.

Coolness #3: A built-in command to shell out to Ruby. They have a method to shell out, and another one specifically to shell out to another Ruby process.

When you want to shell out (e.g. to run your compiler), you should use the sh() and ruby() methods, which will automatically fail the task if the process returns a nonzero exit code. (By default, the value returned by a task block is ignored; if you want to fail the task, you need to do it explicitly with fail() or raise or some such, or call something like sh() that does it for you.)

rule '.rb' => ['.y'] do |t|
  sh "racc", t.source, "-o", t.name
task :test do
  ruby "test_all.rb"

Coolness #4: Built-in support for running unit tests. I already wrote a test_all.rb, but if I’d looked at Rake, I wouldn’t have had to. This works just as well:

[Note: It turns out that this sample code isn’t the right way to do it; see this post for corrected code. Still the same number of lines, just better-behaved.]

require 'rake/runtest'
task :test do
  Rake.run_tests "**/*tests.rb"

If you follow their convention of having your tests in test/test*.rb, you don’t need to pass an argument to Rake.run_tests at all. I like my tests to be next to the files they’re testing, so the above example is what I prefer.

There’s also a TestTask if you want fancier options, like modifying your $LOAD_PATH before running the test. (TestTask actually launches a new Ruby process via the ruby() method; Rake.run_tests runs them in-process.)

Coolness #5: FileList.

FileList is a list of files, much like an array. But instead of giving it individual filenames, you can give it patterns, like ‘lib/**/*’, and it comes back as an array of matching filenames. (It can take both include and exclude patterns.) It lazy-initializes its contents, so if you create a FileList but then don’t run the task that uses it, it’s basically free. By default, it ignores CVS and .svn directories.

But coolest of all, FileList has shorthand methods: sub(), gsub(), ext(), egrep(). Want to return a new copy of a FileList with all the extensions changed? No sweat. Want to grep through every file in a FileList, and do something with each file whose contents match a given regex? egrep() to the rescue. I may never again have to write six whole lines of code to recursively grep through a directory tree of source files.

Neighborhood (Game) Watch

It’s been nice and cool outside, so we had all our windows open last night and today.

Earlier this afternoon (and still ongoing at sporadic intervals), I’ve been hearing some very loud, angry-sounding shouting from one of the houses up the street. I was debating with myself over whether to call the police to look into a domestic disturbance.

Then I remembered that there’s a Husker game this afternoon.


Pity the grass is still wet. Otherwise I’d go out and finish mowing the lawn.

Assertions the right way around (in Ruby, anyway)

I’ve always hated the common xUnit convention of writing things like AssertEquals(expectedValue, actualValue). Who came up with the idea of doing things backwards?

Your tests have to say things like “assert that 5 is what you get when you add 2 and 3”. It’s stilted and awkward (unless, perhaps, you’re trying to write test poetry). And it puts the effects before the causes. “I should get this. Oh, by the way, here’s what I’m doing.” Answer first, then question. I mean, what is this, Jeopardy?

This isn’t how things are done in real life. If I put the pizza into the oven for fifteen minutes, take it out, and then preheat the oven, my dinner is not going to taste very good (unless I have something else for dinner). And if I tried to take that pizza home before I paid for it, Wal-Mart would be well within its rights to take a dim view of the situation, even if I did eventually come back and pay for it.

Why not just do it the other way around? “Assert that 2 plus 3 equals 5.” Cause first, then effect: “I should do this calculation, and here’s the result I should get.” Isn’t that nicer?

But most xUnit frameworks put the effect first, so that’s what people are used to, for good or ill. I want to write my own custom assertions that put the cause before the effect, but that would just confuse people (myself among them) who have been putting it the other way around for years.

Well… I was just tinkering in Ruby, and I found what I think is a nice way to flip the parameters, and leave a nice readable clue that that’s how they’re supposed to be:

assert_lexer "*" => [:times, "*"]

Ruby’s => syntax lets me show how things are flowing: the “*” should turn into the array [:times, “*”]. Input on the left, expected output on the right, and visibly so. Neat, huh?

For anyone not familiar with =>, it lets you lump several name => value pairs together at the end of a parameter list. They all get scooped up into a single Hash, which is passed as the method’s final parameter. So the above is equivalent to (but much prettier than):

hash = Hash.new
hash["*"] = [:times, "*"]