Joe White’s Blog

Life, .NET, and Cats


Using ImageMagick for palette-swapping

February 24th, 2014

A common thing in console RPGs is palette-swapped monsters, where the same image is given a different set of colors, so an Imp becomes an Imp Captain, Dark Imp, etc. without the need for brand-new artwork. I used this technique to generate the various hair colors in my Ponytail and Plain hairstyles that I posted on OpenGameArt and contributed to the Universal LPC Spritesheet.

You can do palette swaps fairly straightforwardly with the ImageMagick command line. You pass something like the following to the convert command line:

-fill #FFFF00 -opaque #999999

This replaces all light gray pixels (#999999) with eye-searing yellow (#FFFF00). Note that it’s target color first, then source. You can repeat this pair of commands as often as you like.

It gets a little odd when you get to semitransparent colors. Say your source image has some pixels that are #9C59919C (that’s ImageMagick’s oddball RGBA syntax, with the alpha at the end). If you try to replace #9C5991 with some other color, it won’t affect the semitransparent pixels — -opaque does an exact match, and #9C59919C is not an exact match for #9C5991[FF]. So you need to explicitly specify the source and target alpha with each semitransparent color:

-fill #FF00FF9C -opaque #9C59919C

Note for GIMP users: If you use ImageMagick to replace semitransparent colors, and then open the output file in GIMP, GIMP may not show the semitransparency. This seems to be a GIMP-specific bug with indexed-color images (i.e., those stored with a palette). Just switch GIMP to Image > Mode > RGB and it’ll display correctly.

Combine this with the trick we saw last time to remove the embedded create/modify dates and you get something that looks like this (though all on one line):

convert
    infile.png
    -fill #FFFF00 -opaque #999999
    -fill #FF00FF9C -opaque #9C59919C
    +set date:create +set date:modify
    outfile.png

Using ImageMagick to write PNGs without embedded create/modify dates

January 12th, 2014

I’m writing some Rake tasks to generate a bunch of PNGs to check into version control, and ran into a problem: every time ImageMagick generated the files, they showed up as diffs in Git, even if the pixel content was identical. The files’ hashes were different each time I generated them.

I did some hunting, and found a discussion thread “create/modify-date written into file”, but its fix (adding +set modify-date +set create-date to the command line before the output filename) didn’t work for me — if I generated the same content twice, the two files were still different.

So I looked at the output of identify:

identify -verbose generated.png

Which spews out quite a lot of output, but in it was this:

  Properties:
    date:create: 2014-01-12T08:01:31-06:00
    date:modify: 2014-01-12T08:01:31-06:00

Maybe they just renamed the properties in a later version of ImageMagick? (The thread was from IM 6.4; I’m on 6.8.7.) So I tried adding +set date:create +set date:modify.

And that did it: I could generate the same file twice, and the two files were binary equal.

file 'generated.png' => ['infile.png', 'Rakefile'] do
  sh "convert infile.png[64x64+0+0] infile.png[64x64+128+0] +append " +
    "+set date:create +set date:modify generated.png"
end

Growl for angular-seed test-status notifications

October 20th, 2013

When you run angular-seed’s test runner (scripts\test.bat), it runs all the tests once, and then enters watch mode. If you make a change to any of your source files (either production code or test code), it will automatically re-run the tests. This is pretty cool, but you need some way to see the test results.

If I were doing Angular development at work, I’d have dual monitors, and I could put my editor on one monitor and the test runner’s console window on the other monitor, so I’d get instant feedback. But then I’d go to look something up in the documentation, and now the console is covered up by a Web browser, so I’d have to juggle that. And in my case, Angular is just a hobby, and I don’t have dual monitors at home.

So I went looking for a way to show the build status via toast alerts. And it turned out to be pretty easy.

Growl

The Jasmine test runner has lots of extensions available, and one of them shows build status via Growl notifications.

Growl for Windows in action

Growl is a toast-alert platform for the Mac, and there’s also a Growl for Windows. I’ve included a screenshot of a couple of alerts so you can see what it looks like. Each alert sticks around for a few seconds, so I made my test fail, saved, quickly made it pass, saved, and took a screenshot.

So this is pretty cool. You’ll still have to look at the console window to see which tests failed and why, but if you expected it to pass and it did, you can just keep storming along.

Configuring angular-seed and Growl

  1. Download and install Growl for Windows. The installation is pretty simple.

  2. Go to a command prompt, cd to your project directory (the directory with the package.json in it), and type:

    npm install --save-dev karma-growl-reporter
    

    This will create a node_modules directory inside your project directory, and install the karma-growl-reporter Node.js module (and all its dependencies) into it, where they need to be for Jasmine to find them.

    It will also (because of the --save-dev option) automatically modify your package.json file to say that you depend on the karma-growl-reporter package. This isn’t important now, but will be when you want to check out your code on another computer; see the next step.

  3. Tell your version-control system to ignore the node_modules directory. You don’t check this in.

    If you’re using Git, you do this by editing the .gitignore file in your project directory, and adding a line saying

    node_modules/*
    

    You do check in package.json. If you check out your code on another computer, you can rebuild your node_modules directory by running npm install to install all the dependencies listed in your package.json.

  4. Edit your config/karma.conf.js and make the changes labeled “STEP 1″ and “STEP 2″. (The below config file also includes the changes to make angular-seed run CoffeeScript tests.)

    module.exports = function(config){
        config.set({
        basePath : '../',
    
        files : [
          'app/lib/angular/angular.js',
          'app/lib/angular/angular-*.js',
          'test/lib/angular/angular-mocks.js',
          'app/js/**/*.js',
          'test/unit/**/*.coffee',
          'test/unit/**/*.js'
        ],
    
        preprocessors: {
          '**/*.coffee': ['coffee']
        },
        coffeePreprocessor: {
          // options passed to the coffee compiler
          options: {
            bare: true,
            sourceMap: false
          },
          // transforming the filenames
          transformPath: function(path) {
            return path.replace(/\.js$/, '.coffee');
          }
        },
    
        autoWatch : true,
    
        frameworks: ['jasmine'],
    
        browsers : ['Chrome'],
    
        plugins : [
                // STEP 1: Add 'karma-growl-reporter' to this list
                'karma-growl-reporter',
                'karma-coffee-preprocessor',
                'karma-junit-reporter',
                'karma-chrome-launcher',
                'karma-firefox-launcher',
                'karma-jasmine'
                ],
    
        // STEP 2: Add the following section ("reporters") -- or if you already have
        // a "reporters" section, add 'growl' to the list:
        reporters: ['progress', 'growl'],
    
        junitReporter : {
          outputFile: 'test_out/unit.xml',
          suite: 'unit'
        }
    
    })}
    

Then run scripts\test.bat, and you should see a toast alert pop up, telling you your tests are passing.

Writing CoffeeScript tests with angular-seed

October 18th, 2013

The first thing I did, after snagging a copy of angular-seed, was try to write a test in CoffeeScript. It didn’t work out of the box, so I went hunting to figure out how to make it work.

(I also found a few GitHub projects that are prepackaged angular-seed with CoffeeScript, but I didn’t see any that have been kept up-to-date with new Angular versions like angular-seed has. Maybe that’ll change when Angular 1.2 ships.)

Why CoffeeScript rocks for Jasmine/Mocha tests

As awesome as nested describes are, they’re even more awesome in CoffeeScript.

Jasmine tests look like this:

describe('calculator', function() {
    it('adds', function() {
        expect(calculator.add(2, 2)).toBe(4);
    });
});

CoffeeScript has some nice advantages for code like this.

  1. CoffeeScript has a built-in lambda operator, ->, that replaces function() {. So that’s a little less typing, and a lot less of the line taken up by furniture.
  2. CoffeeScript doesn’t need curly braces. Instead, you indicate nested blocks by indenting them (and heck, you were doing that anyway).
  3. CoffeeScript lets you leave off the parentheses around a method’s arguments. (Unlike Ruby, you do still need the () if you’re not passing any arguments.)

Combine these, and you get code that looks like this:

describe 'calculator', ->
    it 'adds', ->
        expect(calculator.add 2, 2).toBe 4

Each line is shorter than before, so there’s less typing and a little better signal-to-noise. But you also don’t need those }); lines at the end, because you don’t need the close-curlies to end the lambdas, and you don’t need the close-parens to end the argument lists for describe and it — in both cases, the indentation suffices. Less ceremony, more essence.

Until you see it on your screen, it’s hard to appreciate just how much it improves your tests, not having all those }); lines. Many tests will consist of a single assertion, so by cutting that worthless extra line, you go from almost 33% noise-by-line-count to 0%. Plus, the entire test goes from three lines to two — so now you can fit 50% more tests on your screen. That’s a win.

The syntax is uncluttered enough that it can even be reasonable to put the whole test on one line:

describe 'calculator', ->
    it 'adds',      -> expect(calculator.add 2, 2).toBe 4
    it 'subtracts', -> expect(calculator.subtract 6, 2).toBe 4

When you’ve got a bunch of little tests that are all on the same theme, this can work really well. That would look a lot uglier if it was function() { instead of ->, and if you had to find a place for the }); as well.

And last but not least, it’s really easy to find a text editor that can do code folding based on indentation (and not as easy to find an editor that can collapse based on curly braces). I use Sublime Text, which is a bit pricey, but you can also do indentation-based code folding with the free SciTE if you put it in Python mode. So if you’ve got a whole nested describe for your calculator’s trig functions, and you want to collapse that because now you’re focusing on hex arithmetic, you just fold that whole section of your code up into a single line.

CoffeeScript tests with angular-seed

It takes some ritual to get angular-seed working with CoffeeScript, but the good news is, you only have to do it once (well, once per project).

I tested these instructions with Angular 1.2.0-rc2 and with Karma 0.10.2.

All the changes are in config/karma.conf.js. Look for the parts I tagged “STEP 1″, “STEP 2″, and “STEP 3″ — or just paste the whole thing over your karma.conf.js and review the diffs.

module.exports = function(config){
    config.set({
    basePath : '../',

    files : [
      'app/lib/angular/angular.js',
      'app/lib/angular/angular-*.js',
      'test/lib/angular/angular-mocks.js',
      'app/js/**/*.js',
      // STEP 1: Add 'test/unit/**/*.coffee' to this list:
      'test/unit/**/*.coffee',
      'test/unit/**/*.js'
    ],

    // STEP 2: Add the following two sections ("preprocessors" and "coffeePreprocessor"):
    preprocessors: {
      '**/*.coffee': ['coffee']
    },
    coffeePreprocessor: {
      // options passed to the coffee compiler
      options: {
        bare: true,
        sourceMap: false
      },
      // transforming the filenames
      transformPath: function(path) {
        return path.replace(/\.js$/, '.coffee');
      }
    },

    autoWatch : true,

    frameworks: ['jasmine'],

    browsers : ['Chrome'],

    plugins : [
            // STEP 3: Add 'karma-coffee-preprocessor' to this list:
            'karma-coffee-preprocessor',
            'karma-junit-reporter',
            'karma-chrome-launcher',
            'karma-firefox-launcher',
            'karma-jasmine'
            ],

    junitReporter : {
      outputFile: 'test_out/unit.xml',
      suite: 'unit'
    }

})}

Once that’s done, you’re off and running: you can drop a .coffee file in your project’s test/unit directory, and the test runner will pick up all the tests in it and run them.

Note that, at least with the current versions of angular-seed and jasmine, you’ll have to stop and restart scripts\test.bat before it’ll pick up the new test file. (This is true any time you add a new test file; it’s not CoffeeScript-specific.) Watch mode apparently doesn’t extend to watching for new test files. Just Ctrl+C to stop it, say Yes (or hit Ctrl+C again) to the “Terminate batch job (Y/N)?” prompt, and then run scripts\test.bat again, and you’re on your way.

Home page redesign

October 15th, 2013

The excastle.com home page has been looking pretty drab for years now. Recently I decided to flex my creative muscles and make it look prettier, with the power of Bootstrap and OpenGameArt (props to extradave and pixel32).

Before:

ExCastle.com, pre-2013

After:

ExCastle.com, circa 2013

This was a fun way to spend a Saturday, and I like how it turned out. I’m still not going to call myself a graphic designer, but at least the page won’t put you to sleep right away.

Latest project: a video game in AngularJS

October 14th, 2013

I’ve been playing with AngularJS (“Superheroic JavaScript MVW Framework”) for a while. And, as is my wont, I’m trying to learn it by writing a video game with it.

Here’s an introduction to what I’m doing. I’ll go into more details in the near future.

The video game

I’ve been writing and re-writing the same video game for probably at least ten years now, sometimes in Delphi (with or without DirectX), sometimes in WinForms or XNA or XAML, now in HTML5/JavaScript. (It’s a spare-time project, which means I don’t do it to finish something; I do it to learn something.)

The game I’m trying to write is a 2D console RPG, in the vein of the early Final Fantasy games. I haven’t decided whether I’ll target a Web browser, or a Windows Store app; possibly both. Gameplay will be much like Final Fantasy if it were written with a touchscreen in mind. Artwork will come largely from OpenGameArt, especially the Liberated Pixel Cup-style artwork (including some of my own) and the Universal LPC Spritesheet.

And yes, I’m going to try to do this using Angular. Hey, you don’t really know a tool until you’ve tested its limits, right?

A brief introduction to Angular

Some JavaScript frameworks, like jQuery and Handlebars and Knockout, try to solve one small problem and solve it well. Angular, on the other hand, tries to do it all — everything from templating to two-way databinding to extending HTML to dependency injection to testing. And it does it all with relatively few mechanisms — you’ve got services for shared code, directives for manipulating the DOM, and controllers/scopes, and that’s about it.

One nice perk is that Angular uses Jasmine for testing, which means you get those beautiful nested describes. God, I wish I could do that in .NET for my day job.

Angular has a fairly steep learning curve, so I’m not going to cover the mechanics here. The Angular web site has good documentation, including a tutorial. There’s also some good stuff on YouTube, and I picked up a copy of “Mastering Web Application Development with AngularJS”, which is pretty decent.

angular-seed

The angular-seed project looks like it’s probably the best starting point for writing a generic Angular app. Sure, you could start from zero and reference Angular via CDN, but angular-seed gives you a few niceties like a local Web server (node scripts/web-server.js), and it comes pre-configured for you to be able to run your tests (scripts\test.bat) — in watch mode, yet, so they automatically re-run whenever you save one of your source files. Pretty nice.

(There are other starting-point projects out there too, so if you want something with Bootstrap already baked in, or a Node.js/Express backend, there may already be something better than angular-seed for you. But I don’t want any of that stuff, so angular-seed is great for me.)

My only real complaint about angular-seed is that it doesn’t have any Grunt support built-in. It’d be nice to at least have a Grunt task to run the tests, especially since Grunt doesn’t look nearly as easy to get started with as something like Rake. But otherwise, angular-seed seems to be a great jumping-off point.

So… Angular for a video game? Really?

I don’t know whether it would be practical for the overland map, but I think Angular has some potential for the battle screen. There’s a lot of state management that would work well with an MV* framework, like showing a hero’s menu when it’s his turn. And the upcoming Angular 1.2 has some nice support for CSS3 animations — and there are a lot of little animations during a battle: hero takes a step forward, swings his sword, bad guy flinches from the hit, damage numbers appear, etc. I think I can make all those animations happen with databinding — which would mean it’d all be just as testable as I wanted it to be.

Of course, if you’re just using the built-in databinding, then Angular doesn’t tell you when the animation is done so you can start the next one. But Angular is extensible, and if you write your own directive, you can get those notifications easily. I’ve been working on an animation queue to coordinate all these animations across different scopes and elements, and it’ll be interesting to see if I can pull it off.

What’s next?

Angular-seed gives you a workable start. But to be honest, it’s not really a comfortable environment, at least not right out of the box. In particular, Jasmine tests are much nicer in CoffeeScript than they are in JavaScript. And you need a good way to see whether your tests are passing or failing. More on those next time.

Stefan White, 1998-2013

April 22nd, 2013

Our oldest cat, Stefan, died this morning. It was sudden. Yesterday he was outside, lying in the grass, happy as can be. This morning, he woke us up at 4:30, yowling over and over. We got him to the emergency vet, and probably within five or ten minutes of us getting there, he was gone. The vet said it was probably a stroke.

We got him the second year we were married. The small company I was working for hit some rough times, and we had to move into a smaller apartment — but one that allowed cats. When we went to the shelter, they showed us the cats that were up for adoption, and told us they had one more in the playroom, drying off; he’d just had a bath. We asked to see him, and this little gray tabby jumped, soaking wet and purring up a storm, into Jennie’s arms. She called me over, and he walked out onto my outstretched arm, and perched there and purred. So it wasn’t like we had much choice; he picked us. He was about a year old then, lean and rangy enough to balance on my arm.

We couldn’t take him home that day (I don’t remember why), so they put him back in the playroom. The door was one of those with a tall, narrow window up high; and Stefan jumped up, hung by his claws from the bottom of the window, and meowed piteously. It was hard to leave him.

He was always a purr-motor. Scratch him behind the ears for a few seconds and he’d purr for five or ten minutes. Walk past him in the hallway and he’d start purring. And he would come and curl up with us and purr and purr and purr.

He used to play fetch. Jennie would throw her necklace, and he would tear off after it and bring it back. He’d have the chain in his mouth, dragging the necklace behind him. He would lay his ears down sideways as he dragged it, kind of like we might make a funny face if we were carrying something heavy. One time he jumped down off the second-floor landing to chase the necklace. He was fine, but we were a little more careful where we threw it after that.

That apartment had a shared laundry area, and we would let Stefan out to prowl the hallways when we did laundry. He loved to explore. More than once, he wandered into a neighbor’s apartment to make friends.

Later, we moved to a rental house with an unfinished basement — more of a cave, really; the ceiling was only about five feet high, so it was no good for anything except maybe storage. We used to let him prowl down there, and he’d come back with cobwebs in his whiskers and ears. One night he woke us up with a gift of half a dead mouse. We kept the basement shut after that.

We had a cable guy over one time, and he tried to put a tool back in his toolbelt, only to find that Stefan had curled up in that pouch. Stefan just looked up at him like, “What?”

One time a neighborhood cat got into our screened-in porch, and Stefan was very interested, curious, sniffing at this other cat. The other cat just hissed at him, but Stefan kept on sniffing and trying to be friends. That’s when I realized Stefan was lonely; Jennie and I were both working full-time, and Stefan missed our company. So we went to get him a little brother, and brought home a little orange puff-ball we named Tycho. But unlike the neighborhood cat, this time Stefan was having none of it. We had to keep one of them upstairs and one downstairs, and there was no door in the doorway to the second floor; we had to lean a mattress against the doorway to block it off. (Later they did warm to each other — they would curl up together to nap, or get into fights over who got to give who a bath.)

After we moved to Omaha, and Noel adopted us, Stefan and Noel did not get along; they would fight all the time. That is, until they were on top of the fridge (fighting) and both fell behind it, and were stuck there. This was our spare fridge in the spare bedroom (so at least they fell onto carpet) and we had to move some furniture around to get them out, which included moving books off bookshelves, so it took us a while. But evidently the time in close quarters did them some good; they got along much better after that.

The first time we took him to the vet in Omaha, when the vet came into the exam room, Stefan purred and hissed at the same time, while he twirled lovingly around her legs. She laughed and called him a twit.

After we bought a house, we started letting the cats out to enjoy the weather in the back yard. Stefan liked to eat grass, and then would usually come back inside and throw it back up (generally on the carpet, naturally). But he loved being outside; he would lie there, basking in the sun, or else sitting under the tree, listening to the birds, his nose twitching at everything there was to smell.

Jennie’s computer used to be on the floor of her office, and he would come and curl up on the floor between her and the keyboard, purring mightily. She would play music on the computer, and whenever “Who Will Shoe My Pretty Foot?” came on, she would sing along and tickle his feet, and he would grumble and pull his feet away and keep right on purring.

He went to the vet for a checkup last Thursday. He’s been overweight for years now, and he had bladder stones a few months ago, so they did an ultrasound on him to check on his heart and his bladder. They gave him a clean bill of health. They also removed some teeth that had been giving him trouble. Whatever anesthetic they gave him, it must have been good stuff, because after we got him home, he purred nonstop for the rest of the day.

Just yesterday, he was lying out in the grass, happy as could be.

The vet this morning said there was nothing we could have done. It sucks that he had to be in pain and scared, but at least it wasn’t for long.

Goodbye, Stefan. We’ll miss you terribly.

Making DataContractSerializer play nice with UpdateControls

January 18th, 2013

I’m really psyched about UpdateControls. I haven’t used it in too many projects yet, but it’s already changing the way I think about INotifyPropertyChanged, and opening new horizons yet unexplored.

(If you’re not familiar with UpdateControls, here’s my intro. tl;dr: If you do MVVM, you want to learn about UpdateControls. And it’s free.)

But UpdateControls just takes care of what happens between your model and your viewmodel and your view. There are other concerns your app has to figure out too, like moving data between your model and some kind of persistent storage. I’m writing a game, so I don’t have a database back-end or a service tier; I just want to save state to a local file. Sounds like a job for DataContractSerializer.

Simple enough, right? I just slap a [DataContract] attribute on my model, [DataMember] on all the public properties, and write it out, no problem. Then I try to load it back in, and… my app promptly crashes with a NullReferenceException. What went wrong?

War of the Constructors

The crux of the problem is that an UpdateControls-based model has to do some initialization in its constructor before you can start using its properties, whereas DataContractSerializer doesn’t acknowledge that the constructor even exists.

Recall that, when you define a model object using UpdateControls, you wrap each of your data fields in an Independent<T> that handles the bottom half of the notification magic. Our model object looks something like this:

[DataContract]
public class FooModel {
    private readonly Independent<bool> _active = new Independent<bool>();
    [DataMember]
    public bool Active {
        get { return _active; }
        set { _active.Value = value; }
    }

Notice that the _active field is initialized to a new instance of Independent<bool>. When this code is compiled, that assignment actually gets inserted at the beginning of the IL for FooModel’s constructor(s).

DataContractSerializer, when it deserializes a stream back into an object, doesn’t call any constructors; it just grabs an empty hunk of memory and calls it an object. I don’t know why it skips the constructors; it makes things awfully weird — as well as making it hard to delegate the property’s storage to another object, like you do with UpdateControls. Since FooModel’s constructor never ran, _active is null; so when the deserializer tries to set the Active property, the _active.Value assignment causes a NullReferenceException.

You can get different behavior by removing the [DataContract] attribute and serializing the object as a Plain Old CLR Object (POCO). If you’re deserializing a POCO, DataContractSerializer actually will call the constructor. But, it won’t serialize sub-objects — for POCOs, it looks like it only operates on properties with primitive types. No good for me — I don’t want to cram my entire application’s state into a single object.

Classic solution: Persistence objects

If I was using Entity Framework, I wouldn’t even be thinking about saving my model objects directly; I’d already have a separate set of entity classes that represent tables in the database. These classes wouldn’t depend on UpdateControls, and I’d have to manually copy the data between my model and my entity objects. Similarly, if I was saving to a service tier in a multi-tier application, I’d have the same situation with data transfer objects: I’d have to copy my model object’s contents to a DTO and back.

I could do the same thing with DataContractSerializer. I could define some persistence classes — dumb data contracts that are only used to save and load — and then copy data between those and my models.

The thing is, that “copy data back and forth” step is (a) hard, (b) boring, and (c) easy to screw up. I strive to be constructively lazy, and hard/boring/easy-to-screw-up is high on my list of things to avoid.

At work, we automate the hard/boring parts (and unit-test the easy-to-screw-up parts) with AutoMapper, which works really well. I think it would play pretty nicely with UpdateControls. But there isn’t (yet?) a WinRT version of AutoMapper, so that won’t help me with my WinRT app.

And even if I could use AutoMapper, it feels redundant to make a whole separate set of classes, with all the same properties as my model objects, unless it’s absolutely required by some persistence framework. Even if AutoMapper was there to remove the grunt work (and warn me when I forget to add a property to my persistence object), creating all those extra classes still feels unnecessary. DataContractSerializer should be able to serialize objects; that’s its job. The only problem is that it doesn’t run any of our initialization code. If that was solvable, we’d be golden.

OnDeserializing

If you poke around the System.Runtime.Serialization namespace, you’ll find the OnDeserializingAttribute.

The documentation for this attribute is pretty vague: it just says that it lets you designate a method to be “called during deserialization of an object”. When during deserialization? Before any fields are deserialized? After all the fields?

But since there’s also an OnDeserializedAttribute, I think I’m fairly safe in guessing that these follow the usual pattern: first the -ing method is called, then some other work (deserializing the object’s properties) is done, then finally the -ed method is called. Assuming that’s true (and I think it is, since my tests are passing), then you can use it to make a constructor stand-in for deserialization.

So you can make these changes:

  1. Add a new method called InitFields.
  2. Remove all the initialization expressions from the field declarations, and move those assignments into InitFields.
  3. Remove the readonly from the fields, since they’re now being assigned in a method, not in the constructor.
  4. Add an OnDeserializing method and tag it with [OnDeserializing], and give it a parameter of type StreamingContext. (I don’t know what the parameter is for, but you get a runtime error if it’s not there.)
  5. Call InitFields from both your constructor and your OnDeserializing method.

If you’re using ReSharper, then you’ll also want to add the JetBrains.Annotations NuGet package, and mark the OnDeserializing method as [UsedImplicitly] so ReSharper doesn’t warn you about the unused parameter.

Et voilà — you now have an UpdateControls-based model that you can serialize and deserialize successfully with DataContractSerializer! Here’s what it looks like:

[DataContract]
public class FooModel {
    private Independent<bool> _active;
    private void InitFields() {
        _active = new Independent<bool>();
    }
    public FooModel() {
        InitFields();
    }
    [OnDeserializing, UsedImplicitly]
    private void OnDeserializing(StreamingContext context) {
        InitFields();
    }
    [DataMember]
    public bool Active { ... }

This works, but it feels a bit clumsy, what with the extra two methods, and the assignments being separated from the field declarations. Just for fun, I decided to see if I could go one better.

Automating the process

Why not make a base class that uses Reflection to find all of our Independent<T> fields, and instantiate them automatically as needed?

My first thought was to plug this logic into both the constructor and OnDeserializing. Here’s what a model would look like in that case:

// Note: this example isn't compatible with the final version of SerializableModel
[DataContract]
public class FooModel : SerializableModel {
    [UsedImplicitly] private readonly Independent<bool> _active;
    [DataMember]
    public bool Active { ... }

If the base constructor instantiates the Independent<T>s for us, then _active doesn’t need an initializer anymore. But then ReSharper’s static analysis warns us that the field is never assigned, and we need to suppress the warning by adding an attribute to tell it that the field is used implicitly (i.e., via Reflection).

I thought I was being clever by removing all the “duplicate code” of Independent<T> instantiation, but ReSharper’s warning was my first hint that no, this really wasn’t technical elegance — it was a code smell. And after playing around with it for a couple of days, I had to agree.

It comes down to cognitive load. Our brains are only so big. When the code does what it says, you don’t have to waste as much of your brain capacity on technical minutiae; you have more brainpower available to actually solve the problems at hand. It’s not worth “removing duplication” if it means the code no longer does what it says. Any time you have to stop and think about where that field is being instantiated, it derails your train of thought.

As an added bonus, doing things this way also means that you can make an existing UpdateControls-based model into a serializable model just by changing its base class, and nothing else:

[DataContract]
public class FooModel : SerializableModel {
    private readonly Independent<bool> _active = new Independent<bool>();
    [DataMember]
    public bool Active { ... }

I like the way this code reads. All the UpdateControls-based stuff looks exactly like you’d expect. The only unusual thing is that you’re descending from SerializableModel, and that’s almost as declarative as the [DataContract] attribute.

One big caution: if you pass a default value to the Independent<T> constructor, that default won’t be used for deserialization. This isn’t a problem in the usual case, where you’re about to load that property’s value from storage. But if you’re reading an older stream that doesn’t contain that property, you might have to figure something out.

Without further ado, here’s the SerializableModel class. This was written for WinRT, but should also work in .NET 4.5. (If you’re using an older version of .NET, the Reflection APIs are totally different.)

[DataContract]
public class SerializableModel
{
    private IEnumerable<TypeInfo> Ancestry
    {
        get
        {
            for (var type = GetType(); type != null; type = type.GetTypeInfo().BaseType)
                yield return type.GetTypeInfo();
        }
    }

    private void CreateIndependentFields()
    {
        var independentFields =
            from type in Ancestry
            from field in type.DeclaredFields
            let fieldType = field.FieldType.GetTypeInfo()
            where fieldType.IsGenericType &&
                  fieldType.GetGenericTypeDefinition() == typeof(Independent<>)
            select field;

        foreach (var field in independentFields)
        {
            var instance = Activator.CreateInstance(field.FieldType);
            field.SetValue(this, instance);
        }
    }
    [OnDeserializing, UsedImplicitly]
    private void OnDeserializing(StreamingContext context)
    {
        CreateIndependentFields();
    }
}

I consider this code to be more of a useful technique than actual copyrighted intellectual property, so feel free to use the above code in any context you wish (no credit necessary). But I’d appreciate hearing about it if you find this useful.

WinRT’s default “save on suspend” isn’t good enough

January 5th, 2013

If you create a new WinRT app, using the templates that ship with Visual Studio (e.g. the Grid App template), the default code will save your app’s state when the app gets suspended.

“Save on suspend” seems like a reasonable model. The trouble is, your app can get killed without ever getting its suspend notification.

If you close a WinRT app (either by pressing Alt+F4, or dragging from the top of the screen to the bottom), Windows switches away from the app immediately, but it actually waits about ten seconds before firing the app’s OnSuspending method. (That “about ten seconds” comes from my own observations of how long it takes until the timestamp changes on the autosave file. That’s on a physical machine, not a VM, so it should be about best-case.)

If, during that ~10-second window after the app gets closed, the user decides to start the app again, then Windows kills the previous instance, flat-out, and starts a new instance. In this case, the previous instance never gets a chance to save.

I suspect (though it’d be tricky to try to confirm) that your app could even get terminated while it’s in the middle of an async save operation — at which point your save file is only partially written, and therefore corrupt, and the user loses all their data.

So if the only time you save your state is in your app’s OnSuspending method, you’re asking for trouble. If a user closes your app, and then re-launches it again less than ten seconds later, they will lose all the data they entered during their previous session, and maybe everything.

Now, why would somebody do that — close the app and then re-launch it right away? Beats me. Maybe they close it and then think of something else they wanted to do. Maybe your app doesn’t have a Home button and they’re frustrated with trying to find their way back. Maybe the app doesn’t correctly refresh the date when you resume it the next day (Microsoft’s XBox games, with their date-sensitive daily challenges, are really bad at this).

But you know what? It doesn’t matter why they do it. Closing and reopening is a totally innocuous action. It should not screw the user out of their data.

Right now, I don’t have a great alternative to recommend; I just ran into the problem, I haven’t solved it yet. I’ll probably keep the “save on suspend”, so I have a chance to save the latest and greatest state; but I’ll probably also add some background saves as well. But of course all those saves will be async operations, so I have to come up with a way to make sure they don’t step on each other. Good times.

Stop writing INotifyPropertyChanged, start using UpdateControls

December 16th, 2012

If you use MVVM, you know all about INotifyPropertyChanged, and mostly you know that it’s a pain. Simple properties are a lot of code, but otherwise not too bad; but once you start having derived properties, it’s hard to make sure you fire all the right change events at the right time (and even harder to keep it right as you refactor).

Can’t we do things the way Knockout.JS does, with a framework that automatically discovers the dependencies for you?

Why yes, yes we can. Last weekend I discovered UpdateControls.

Introduction to UpdateControls

UpdateControls is free and open-source (MIT license), is available via NuGet, and works pretty much everywhere: WinRT, WPF, Silverlight, Windows Phone 7 — even WinForms! Update: WP8 is supported too.

The idea is:

  • Don’t implement INotifyPropertyChanged on your viewmodel.
  • Wrap each of your independent variables inside an Independent<T> instance. Expose the underlying value through properties.
  • For computed properties, just write the code to compute the values.
  • Before you assign your top-level viewmodel to a DataContext, wrap it with a call to ForView.Wrap.

And that’s it. You don’t need to fire notifications manually; UpdateControls does it for you.

An example:

public class MyViewModel {
    private Independent<string> _firstName = new Independent<string>("");
    private Independent<string> _lastName = new Independent<string>("");
    private Independent<bool> _lastNameFirst = new Independent<bool>();

    // Expose independent values through simple wrapper properties:
    public string FirstName {
        get { return _firstName; }
        set { _firstName.Value = value; }
    }
    public string LastName {
        get { return _lastName; }
        set { _lastName.Value = value; }
    }
    public bool LastNameFirst {
        get { return _lastNameFirst; }
        set { _lastNameFirst.Value = value; }
    }

    // Dependent (computed) properties are just code:
    public string Name {
        get {
            if (LastNameFirst)
                return string.Format("{0}, {1}", LastName, FirstName);
            else
                return string.Format("{0} {1}", FirstName, LastName);
        }
    }
}

// And in the view:
DataContext = ForView.Wrap(new MyViewModel());

And like magic, all your properties, all your logic, will start notifying the UI of changes. Between the ForView.Wrap wrapper at the top and the Independent<T> at the bottom, UpdateControls is able to automatically track all the dependencies, and triggers change events for computed properties whenever their ingredients change. This even works if a property changes which ingredients it uses.

ForView.Wrap wraps your top-level viewmodel in an internal class called DependentObject<T>. The first time a databinding asks for a property, the DependentObject<T> switches into recording mode, and then invokes your viewmodel property. Until that property returns, the DependentObject<T> takes note of every time you read the value from any Independent<T> anywhere; and whenever you do, it hooks that Independent<T>‘s change event.

Then the DependentObject<T> caches the property value and returns. Next time XAML asks for the value, it just returns it. (Its ingredients haven’t changed, so why not?) But as soon as any of the ingredients change, it clears its cache and fires PropertyChanged. There’s a lot of magic under the covers, but using it is dead simple. In principle.

As with any magic, there are some subtleties and gotchas, which I go into below. But first, some important notes about the documentation.

API drift, or, an UpdateControls documentation survival guide

UpdateControls has a fair bit of documentation, including several good videos. Unfortunately, the API has changed in a couple of significant ways over time, but a lot of documentation still refers to the old APIs. So some places you see it one way, and some places you see it the other way, which is hella confusing.

The old APIs do still work (though not on every platform and not with every edition of Visual Studio), but there are easier — and more universal — ways to do things now.

Here are the two things you really need to know before you start reading the UpdateControls documentation:

Codegen vs. Independent<T>

Independent properties used to take more code than what I showed above. It was so tedious that the author even wrote a Visual Studio add-in to do the code generation for you. But now we have Independent<T>, which is enough simpler that you don’t need codegen. The old way still works (as long as you don’t have Visual Studio Express, which doesn’t support add-ins), but Independent<T> does a better job of encapsulating the boilerplate code, so IMO it’s the way to go.

Unfortunately, most of the code samples and screencasts still use the old way. You’ll see (and hear, in the videos) lots of references to Ctrl+D G, which tells the VS add-in to do its code generation. And you’ll see a lot of code like this:

private string _firstName;

#region Independent properties
// Generated by Update Controls --------------------------------
private Independent _indFirstName = new Independent();

public string FirstName
{
    get { _indFirstName.OnGet(); return _firstName; }
    set { _indFirstName.OnSet(); _firstName = value; }
}
// End generated code --------------------------------
#endregion

My advice: Don’t install the MSI (in these days of NuGet, the MSI is really just there to install the VS add-in). Ignore the VS add-in entirely. Tune out whenever the videos talk about Ctrl+D G.

Just add the NuGet package to your project, and then use Independent<T>. The above code becomes this, which is simple enough to write and maintain by hand:

private Independent<string> _firstName;

public string FirstName {
    get { return _firstName; }
    set { _firstName.Value = value; }
}

By the way, that’s not a typo in the getter. You can return _firstName.Value if you like, but Independent<T> also has an implicit conversion to T, so as long as it’s not Independent<object> you can indeed just return _firstName;.

{u:Update} vs. plain old bindings

Originally, UpdateControls had a XAML markup extension ({u:Update}) that you were supposed to use in place of {Binding}. It took care of turning on dependency-recording mode whenever values were bound to the UI, so that it knew how to watch for changes and update the UI when ingredients change later.

Unfortunately, not all the XAML platforms support markup extensions. Silverlight didn’t get them until v5. Windows Phone 7 doesn’t have them; not sure about 8. WinRT doesn’t support them. {u:Update} isn’t going to cut it for those platforms. I suspect that {u:Update} also wouldn’t play well with visual designers like Blend; they’re all about the {Binding}.

So UpdateControls added ForView.Wrap(), which you can use in combination with plain old {Binding}. If the objects you expose to your XAML and your DataContexts and your Bindings are wrapped for view, you get the same magic, but without needing markup extensions.

Any time you see {u:Update} in the documentation, just mentally substitute {Binding}, and make sure the viewmodel gets wrapped with ForView.Wrap before the view sees it. For example:

// Using the PersonViewModel class from the first code sample, above:
var viewModel = new PersonViewModel();
view.DataContext = ForView.Wrap(viewModel);

<!-- This will automatically update whenever ingredients (FirstName, LastName) change. -->
<TextBlock Text="{Binding Name}"/>

Gotchas

If you’re starting with UpdateControls on a new project, I would imagine it’s not too hard to follow the rules and have things work well. If you’re integrating it into an existing code base, you’re more likely to run into gotchas. Here are some of the problems I ran into when trying to refactor my project to use UpdateControls.

Don’t use INotifyPropertyChanged

If your viewmodel implements INotifyPropertyChanged, UpdateControls assumes you know what you’re doing, and ForView.Wrap won’t wrap that class; instead it presents your class as-is to the view. The consequence is that the computed-property notification magic won’t work for your INPC class or anything inside it. If you implement INotifyPropertyChanged, it’s all on you.

Which means that, if you’re trying to refactor from INPC to UpdateControls, you can’t do it one property at a time. To use Independent<T>, you can’t have INotifyPropertyChanged. You’ll have to change your entire viewmodel class at once: remove INotifyPropertyChanged, add Independent<T>s for everything, all in one fell swoop.

If you do viewmodel composition — where one viewmodel exposes other viewmodels as properties (think nested panels inside a larger screen) — then you probably want to start at the top level (the whole screen), and work toward the leaves (nested panels).

Corollary: Don’t use ObservableCollection<T>

I had viewmodels that exposed collections, and nothing was happening when the collection items’ properties changed. It turns out that ObservableCollection<T> implements INotifyPropertyChanged (makes sense when you think about it; you might want to bind to its Count) — so UpdateControls just exposed the collection instance directly to the UI, without attaching any of the dependency-recording logic: not for the collection itself, and not for the items inside it, either.

So don’t use ObservableCollection<T> with UpdateControls. Use IndependentList<T> instead (it’s the UpdateControls equivalent of an observable collection). Or just a plain old List<T> if the contents aren’t going to change. Or even a LINQ query (great for filtering — as long as the filter conditions boil down to Independent<T>, changing them will automatically re-run the LINQ query). But if you start using UpdateControls, you should get rid of all your references to ObservableCollection<T>.

ForView.Wrap and dependency properties

This one may be peculiar to my code, but I had some custom dependency properties whose value was a custom class. Once I started using UpdateControls, their value wasn’t that custom class anymore; the view layer (where dependency properties live) operates on the wrapper objects, not the actual viewmodels. Since my DependencyProperty.Register call specified that the property value should be of type MyCustomClass, but the actual instance was a DependentObject<MyCustomClass>, the bindings saw a type mismatch and never set my property. To get around this, I had to change my DependencyProperty.Register call to specify object as the property’s value type. (And of course my property’s change event had to use ForView.Unwrap<T> instead of casting directly to the class type.) This probably won’t affect many people, but it’s worth noting since I burned some time trying to figure it out.

Debugging wrapper objects

If you’re in the debugger, and you’re looking at a wrapper object (from ForView.Wrap), the tooltip will show the ToString() from the wrapped class — which is often the name of the wrapped class. It can be hard to even realize that the object you’re looking at is a wrapper, and not the viewmodel itself.

It’s not hard if you know how, though. If you expand the tooltip, wrappers will have a “HasErrors” property, and not much else. That should be easy to tell apart from your actual viewmodel (which will typically have lots of properties).

If you then want to drill down to see the properties of the wrapped instance, you can expand “Non-Public Members”, then “_wrappedObject”.

Debugger tooltip with UpdateControls

Integrating with Caliburn.Micro

I tried combining both UpdateControls and Caliburn.Micro in the same project, mostly because I was already using Caliburn.Micro’s IViewAware so my viewmodels could fire animations on the view and wait for them to complete. Here’s what I learned:

  • Don’t use Caliburn.Micro’s base viewmodel classes, because they implement INotifyPropertyChanged (see above).
  • Caliburn.Micro doesn’t know about UpdateControls’ wrapper classes. If you’re doing viewmodel-first (where Caliburn.Micro automatically discovers and instantiates the correct view type for you), and your DataContext is a DependentObject<PersonViewModel>, Caliburn.Micro’s conventions will probably look for a DependentObjectView or something — it doesn’t know that it’s supposed to look for a PersonView. This isn’t hard to deal with, though; Caliburn.Micro has good extensibility points for this sort of thing.
  • The IViewAware stuff won’t work out of the box either, because Caliburn.Micro takes the DataContext and says “does it implement IViewAware?” and of course it doesn’t, because it’s got the wrapper object, not your viewmodel. None of the lifecycle stuff (IGuardClose and the like) would work either, because it’s also based on interfaces. Again, you could hook CM’s extensibility points to make this work.
  • No idea how Caliburn.Micro’s {x:Name}-based binding conventions would play with UpdateControls. I didn’t get that far.

In the end, I decided not to use Caliburn.Micro for this app. I had a lot of custom UI, so the conventions weren’t much use to me; I was really only using CM for IViewAware, which would require some coding to integrate with UpdateControls. Easier to roll my own at that point.

But I’m confident you could use the two together. If anyone digs into this further and wants to share their code, drop me a line and I’ll either post it or link to you.

Conclusion

I’m pretty excited about UpdateControls. Yeah, I spent a lot of time here talking about the downsides and gotchas. But I wouldn’t have bothered if it didn’t show so much promise.

If you’ve used Knockout, or if you’re tired of keeping track of which calculated properties to fire PropertyChanged events for whenever something changes, you owe it to yourself to take a look at UpdateControls.

Here’s where to start:

  • First, look through the notes above about API drift. You’ll want to be able to mentally translate the out-of-date code samples as you see them.
  • Then head over to the UpdateControls documentation and read through all half-dozen or so pages; they’re short. Also take a quick look through the other sections in the sidebar (“Examples”, “Tips”, “Advanced Topics”).
  • Then check out the videos, which cover different material than the docs, and which dig a lot deeper. They’ll really give you an idea of what UpdateControls can do.

Joe White's Blog copyright © 2004-2011. Portions of the site layout use Yahoo! YUI Reset, Fonts, and Grids.
Proudly powered by WordPress. Entries (RSS) and Comments (RSS). Privacy policy