TechEd 2008 notes: Evolving Frameworks

This session was aimed at people who write frameworks: low-level code used by thousands of people. When you’re writing a low-level framework, you have to be very cautious about how you change APIs, lest you break code in the field. If nobody outside your department consumes your code, and you compile all your code every time you build — probably the most common case — then most of this stuff is only of academic interest to you. But it’s interesting nonetheless.

This is my last session-notes post about TechEd 2008. I’ll probably post more about the con later — I think it’d be interesting, for example, to contrast the philosophies different presenters had about unit-testing best practices — but it’ll probably be a few days or weeks before I get back to that; writing 22 blog posts that are up to my own editorial standards is a lot of work, and I need a break!

Once again, those of you reading via DelphiFeeds are only getting the posts about general programming topics. If you want to also see the ones about features not likely to be directly relevant to Delphi anytime soon (e.g., lambda expressions, expression trees, LINQ, .NET-style databinding, add-ins, F#, the Provider pattern), you can look at my entire list of TechEd 2008 posts.

Evolving Frameworks
Krzysztof Cwalina
Program Manager, .NET Framework team
Microsoft

Team has dual charter:

  • Basic APIs (used to be on BCL team, now on higher-level application model, cross-cutting features)
  • Architectural and design quality of the whole framework
  • Framework produced by many (over 1,000) people. Goal to make it look like it was designed by one person. Consistency guidelines.
  • More recently looking into evolving APIs and improving the evolution process.

Frameworks deteriorate over time

  • OO design community has already done much research into how to change requirements
  • It’s even worse with APIs
  • Still many forces require changes over time
    • Requirements change
    • Ecosystem changes: new tools, language changes
    • People change

No silver bullet. But there are some techniques to design APIs that will be easier to evolve, and some tricks that allow modifications that used to be breaking.

Slow down framework deterioration

  • With thoughtful architecture
  • With proper API design (micro-design guidelines)
  • With framework evolution idioms

Libraries, Abstractions, Primitives

  • Three different kinds of types in frameworks

Library types

  • Definition: types that are not passed between components. Instantiate, use, then maybe keep a reference or maybe let the GC collect it.
  • Examples: EventLog, Debug.
  • Easy to evolve: leave old in, add new one.
  • Cost to consumers, of introducing duplication, is nonzero. Shouldn’t be done lightly, but is doable.

Primitive types

  • Definition: types that are passed between components and have very restricted extensibility (i.e., no subtype can override any members).
  • Examples: Int32, String, Uri
  • Hard to evolve
  • Little need to evolve. Usually very simple. Not much policy went into designing them.

Abstractions

  • Definition: types that are passed between components and support extensibility (i.e., interfaces or classes with members that can be overridden)
  • Examples: Stream, IComponent
  • Lots of policy; contracts usually quite strict
  • Hard to evolve
  • Unfortunately, there’s quite a bit pressure to evolve abstractions
  • Extremely difficult to design abstractions out of the blue
    • The most successful abstractions in the .NET Framework are those that have been around for many years
    • “What should a stream do?” is pretty well established.
    • Interface with too few members won’t be useful. Interface with too many members will be hard to implement.

Evolving libaries

  • Can write a new class and tell people to start using it. Problematic if there isn’t a good migration path.
  • Architecture
    • Dependency management
  • Design
  • Toolbox
    • Type forwarders — lets you move a type from one assembly to another without breaking binary compatibility
    • EditorBrowsableAttribute
    • ObsoleteAttribute
  • Some people say a library should be at least 10 times better before you should consider replacing the old one.

Dependency management

  • Mostly applicable to APIs with more than one feature area, esp. if they evolve at a different pace or are used for different scenarios.

Framework Layering

  • Within each layer, have “components” (groups of classes) that each evolve together
  • Manage dependencies between the components
  • Lower layers shouldn’t depend on higher layers

Basics of dependency management

  • API dependency: A depends on B if a type in B shows in the publicly accessible (public or protected) API surface of a type in A. Might be parameter type, base type, even an attribute.
  • Implementation dependency: type in A uses a type in B in its implementation.
  • Circular dependency (including indirectly)
  • Dependency going to a lower layer: OK
  • Dependency going to a higher layer: Not allowed
  • Dependency within a layer: discussed by architects to see if it makes sense

Design principles

  • Focus on concrete customer scenarios
    • Much easier to add to something simple
    • Does this minimal component meet your needs?
  • Keep technology areas in separate namespaces
    • Mainly applies to libraries
    • Single namespace should be self-contained set of APIs that evolve on the same time schedule and in the same way
  • Be careful with adopting higher level APIs (usually libraries) for lower layers
    • E.g., Design a high-level API, then realize you can make it general, so you try to move it to a lower layer.
    • This rarely works when it’s not thought through from the beginning.
    • Don’t do it just because you can.
  • Don’t assume that your library is timeless
    • XML DOM should not be in System.Xml namespace

Toolbox: Type forwarders

[assembly:TypeForwardedTo(typeof(SomeType))]
  • Lets you move a type to a different assembly without breaking already-compiled code
  • Put in assembly where the type used to be
  • Forces a compile-time dependency on the assembly the type has been moved to
    • Can only be used to move a type down?

Toolbox: ObsoleteAttribute

[Obsolete(...)]
public void SomeMethod() {...}
  • Take the API out of developers’ minds. Present simplified view over time of “This is the framework”.
  • Caution: many people think Obsolete is non-breaking, but that’s not entirely true because of “Treat warnings as errors”.
    • “Yes,” you may say, “but that’s only when you recompile.” True, but some application models, like ASP.NET, recompile on the fly.

Toolbox: EditorBrowsableAttribute

[EditorBrowsable(EditorBrowsableState.Never)]
  • Hides from Intellisense, but you can still use it without warnings.
  • Often this is good enough.

Evolving primitives

  • Minimize policy (keep them simple)
    • Int32 should be no more than 32 bits on the stack
  • Provide libraries to operate on primitives
    • Consider using extension methods to get usability

Extension methods and policy

// higher level assembly (not mscorlib)
namespace System.Net {
    public static class StringExtensions{
        public static Uri ToUri(this string s) {...}
  • Policy-heavy implementation in a library that’s isolated from the primitive
  • High usability because it’s an extension method

Evolving abstractions

  • HARD!
  • Plan to spend ~10x as long designing abstractions as you do designing policies or libraries
  • Right level of policy
  • Right set of APIs

Interfaces vs. abstract classes

  • Classes are better than interfaces from an evolution point of view
  • Can’t add members to interfaces, but can add them to classes
  • That’s why it’s Stream instead of IStream
  • Were later able to add timeouts to streams, and it was much easier to add than it would have been with an IStream.
  • Imagine that it had been IStream from the beginning, and later they’d decided to add timeouts.
    • Adding members to an existing framework interface is never allowed.
    • When adding timeout, would have had to make a new descendant interface ITimeoutEnabledStream.
    • Wouldn’t need CanTimeout.
    • Problem is, base types proliferate (e.g. Stream property on a StreamReader). So casts would proliferate as well. And your “is it the right type” is effectively your CanTimeout query.
    • Less usability, since new member doesn’t show up in Intellisense.

Summary

  • Primitives, abstractions, libraries
  • Dependency management
  • Controlling policy
  • API malleability
    • Classes over interfaces, type forwarders, etc.

Q&A:

Q: Have there been times you did an abstract class and later wished it had been an interface?
A: Really not yet; he’s still waiting to hear from a team who’s done a class and later wishes they hadn’t. There are some situations where you do need interfaces (e.g. multiple inheritance). Sometimes it’s still a judgement call.

Q: Guidance on when to use extension methods?
A: Working on some guidelines for the next version of the Framework Design Guidelines book. There are some proposed guidelines at LINQ Framework design guidelines (scroll down to section 2, then look for the list of “Avoid” and “Consider” bullet points); if those stand the test of time, they’ll eventually become official guidelines.

Q: When would you split a namespace into a separate assembly?
A: When you design assemblies and namespaces, they should be two separate design decisions. Feature areas have a high correlation with namespaces. Assemblies are for packaging, servicing, deployment, performance. Make the decisions separately.

Q: Why not fix design flaws when moving from 1.1 to 2.0?
A: As current policy, they don’t remove APIs. (Not promising that it will never happen.) They think they can evolve the framework in a relatively healthy way. They’re even brainstorming ways to add more things like type mappers, e.g. moving static methods from one type to another (but no, it’s not in a schedule). Didn’t have some of these mechanisms when they were writing 2.0.

Q: How does the CLR team resolve conflicts when reviewing a design? Consensus? Vote?
A: Many processes at MS revolve around “orb”. One for compatibility, one for side-by-side, etc. Groups of four roles: owner, participants, reviewers, approver (escalation point). Try to concentrate on owner and participants, to reach a conclusion by consensus. When that fails, go to the reviewers, then the approver. Approver rarely has to make the decision; more likely to educate than override.

Q: Long overloaded parameter lists vs. parameter objects?
A: They’ve done overloads in that case. Ideally, each shorter one just loses one parameter from a longer one (be consistent about ordering, etc.) Best if the leading parameters are similar, for Intellisense usability reasons. They do use parameter objects in a few cases, but mostly in cases where you don’t want to, or cannot, have overloads; e.g., an event. Also don’t want an interface with lots of overloads.

TechEd 2008 notes: How not to write a unit test

How not to write a unit test
Roy Osherove
Typemock
Blog: ISerializable.com

All the things no one ever told you about unit testing.

Will have two parts: presentation-like (what really works), and interactive (questions, prioritized).

Early questions:

  • Data access
  • Legacy code (what not to do)
  • Duplication between unit tests and functional tests
  • Testing non-.NET code, e.g. ASP.NET
  • Testing other languages, e.g. F#, IronRuby)
  • Unit tests and refactoring
  • Testing UI
  • How do you mock the world?
  • How important are tools? Mocking tools, refactoring, etc. Can you write unit tests with just what VS provides?
  • Did you bring your guitar? — No. Wanted as much time for information as possible.
  • Where did you get your T-shirt? — Was being given away at another conference.

A unit test is a test of a small functional piece of code

  • If a method returns a boolean, you probably want at least two tests

Unit testing makes your developer lives easier

  • Easier to find bugs.
    • That’s the common line. But not necessarily — e.g. if your test has bugs, or if you’re testing the wrong things
    • If you can’t trust your tests to find bugs (and especially if you don’t know you can’t trust them), then the opposite may be true — you may be confident you don’t have bugs when you do.
    • If you don’t trust them, then you won’t run them, they’ll get stale, and your investment in writing them was wasted.
  • Easier to maintain.
    • But 1,000 tests = 1,000 tests to maintain
    • Change a constructor on a class with 50 tests — if you didn’t remove enough duplication in the tests, it will take longer than you think to maintain the tests
    • We will look at ways to make tests more maintainable
  • Easier to understand
    • Unit tests are (micro-level) use cases for a class. If they’re understandable and readable, you can use them as behavior documentation.
    • Most devs give really bad names to tests. That’s not on purpose.
    • Tests need to be understandable for this to be true.
  • Easier to develop
    • When even one of the above is not true, this one isn’t true.

Make tests trustworthy

  • Or people won’t run them
  • Or people will still debug for confidence

Test the right thing

  • Some people who are starting with test-driven development will write something like:
[Test]
public void Sum()
{
    int result = calculator.Sum(1, 2);
    Assert.AreEqual(4, result, "bad sum");
}
  • Maybe not the best way to start with a failing test
  • People don’t understand why you want to make the test fail
  • Test needs to test that something in the real world is true: should reflect the required reality
  • Good test fails when it should, passes when it should. Should pass without changing it later. The only way to make the test pass should be changing production code.
  • If you do TDD, do test reviews.
    • Test review won’t show you the fail-first. But you can ask, “So can you show me the test failing?”

Removing/Changing Tests

  • Don’t remove the test as soon as it starts passing
    • If it’s a requirement today, chances are it’ll still be a requirement tomorrow
    • Duplicate tests are OK to remove
    • Can refactor a test: better name, more maintainability
  • When can a test fail?
    • Production bug — right reason (don’t touch test)
    • Test bug (fix test, do something to production code to make the corrected test fail, watch it fail, fix production code and watch it pass)
      • Happens a lot with tests other people wrote (or with tests you don’t remember writing)
    • Semantics of using the class have changed (fix/refactor)
      • E.g., adding an Initialize method that you have to call before you use the class
      • Why did they make that change without refactoring the tests?
        • Make a shared method on the test class that instantiates and Initializes
    • Feature conflict
      • You wrote a new test that’s now passing, but the change made an old test fail
      • Go to the customer and say, “Which of these requirements do you want to keep?”
      • Remove whichever one is now obsolete

Assuring code coverage

  • Maybe unorthodox, but Roy doesn’t like to use code-coverage tools
    • 100% code coverage doesn’t mean anything. Finds the exceptions, but doesn’t prove the logic.
  • Better: change production code and see what happens
  • Make params into consts
  • Remove “if” checks — or make into consts (if (true)). Will a test fail? If not, you don’t have good coverage.
  • Do just enough of these kinds of tweaks to make sure the test is okay.
  • Test reviews are still valuable if you do pair programming, just maybe less often. Good to bring in someone else who didn’t write the code, with an objective eye.
  • Quick test review of yesterday’s code at end of stand-up meeting?

Avoid test logic

  • No ifs, switches or cases
    • Yes, there are always exceptions, but it should be very rare
    • Probably only in testing infrastructure
    • Most of the time, there are better ways to test it
    • Sometimes people write conditionals when they should be writing two tests
    • Don’t repeat the algorithm you’re testing in the test. That’s overspecifying the test.
  • Only create, configure, act and assert
  • No random numbers, no threads
  • Test logic == test bugs
  • Fail first also assures your test is correct

Make it easy to run

  • Integration vs. Unit tests
  • Configuration vs. ClickOnce
  • Laziness is key
  • Should be able to check out, run all the unit tests with one click, and have them pass.
  • Might need to do configuration for the integration tests, so separate them out.
  • Never check in with failing tests. If you do, you’re telling people it’s okay to have a failing test.
  • Don’t write a lot of tests to begin with, and have them all failing until you finish everything. If you do that, you can’t check in (see previous point). Write one test at a time, make it pass, check in, repeat.

Creating maintainable tests

  • Avoid testing private/protected members.
    • This makes your test less brittle. You’re more committed to public APIs than private APIs.
    • Testing only publics makes you think about the design and usability of a feature.
    • Publics are probably feature interactions, rather than helpers.
    • Testing privates is overspecification. You’re tying yourself to a specific implementation, so it’s brittle, and makes it hard to change the algorithm later.
    • Sometimes there’s no choice; be pragmatic.
  • Re-use test code (Create, Manipulate, Assert) — most powerful thing you can do to make tests more maintainable
  • Enforce test isolation
  • Avoid multiple asserts

Re-use test code

  • Most common types:
    • make_XX
      • MakeDefaultAnalyzer()
      • May have others: one already initialized, with specific parameters, etc.
    • init_XX
      • Once you’ve already created it, initialize it into a specific state
    • verify_XX
      • May invoke a method, then do an assert on the result. Pulling out common code.
  • Suggestion: by default, the word new should not appear in your test methods.
    • As soon as you have two or more tests that create the same object, you should refactor the new out into a make method.

Suggestion: don’t call the method directly from Assert.AreEqual(...). Introduce a temp variable instead. (This relates back to the 3A test pattern.)

Aside: Test structure

  • One possibility: Each project, e.g. Demo.Logan, has tests Demo.Logan.Tests. Benefit: they’re next to each other in Solution Manager.
  • Test files: one file per tested class?
    • That’s a good way to do it. Aim for a convention like MyClassTests so it’s easy to find.
    • If you have multiple test classes for one test, make multiple classes.
    • Consider nested classes: making a MyClassTests, and putting nested classes, one per feature. Make the nested classes be the TestFixtures.
      • Be careful of readability, though.
      • Roy said his preference would be to keep one test class per source file, rather than using nested classes to put them all in one file.
      • Decide for yourself whether you’d prefer one class per file, or all tests for one class in one place.

Enforce test isolation

  • No dependency between tests!
  • If you run into an unintended dependency between tests, prepare for a long day or two to track it down
  • Don’t run a test from another test!
  • You should be able to run one test alone…
  • …or all of your tests together…
  • …in any order.
  • Otherwise, leads to nasty “what was that?” bugs
  • Almost like finding a multithreading problem
  • The technical solution (once you find the problem) is easy. That’s why God created SetUp and TearDown. Roll back any state that your test changed.

Avoid multiple asserts

  • Like having multiple tests
  • But the first assert that fails — kills the others. If one test fails, the others still run. Asserts aren’t built that way.
  • Exception: testing one big logical thing. Might have three or four asserts on the same object. That’s possible, and doesn’t necessarily hurt.
  • Consider replacing multiple asserts with comparing two objects (and also overriding ToString so you can see how they’re different when the test fails).
    • My experience: this doesn’t work well when the objects get really big and complicated. Could work well for small objects, though.
  • Hard to name
  • You won’t get the big picture (just some symptoms)

Don’t over-specify

  • Interaction testing is risky
  • Stubs -> rule. Mocks -> exception.
  • Mocks can make you over-specify.
  • “Should I be able to use a stub instead of a mock?”
    • A stub is something you never assert against.
  • There’s only one time when you have to use mocks: when A calls a void method on B, and you have no way to later observe what B was asked to do. Then you have to mock B and verify that it was called with the right parameter.
    • If the other class does return a value, then you can test what your class did with that result. You’re testing your class, after all, not that other object — that’s why you’re faking it.

Readability

  • If you do another test that tests basically the same thing, but with different parameters, he suggests appending “2” to the end of the test name. But that’s assuming you already have a really good naming convention for the base test name! (Remember the serial killer.)
  • Bad: Assert.AreEqual(1003, calc.Parse("-1"));
  • Better:
int parseResult = Calc.Parse(NEGATIVE_ILLEGAL_NUMBER);
Assert.AreEqual(NEGATIVE_PARSE_RETURN_CODE, parseResult);
  • If you can send any kind of number, and the specific value you pass doesn’t matter, either use a constant, or use the simplest input that could possibly work (e.g. 1).

Separate Assert from Action

  • Previous example
  • Assert call is less cluttered

TechEd 2008 notes: Advanced Unit Testing Topics

He repeated a lot of his favorite patterns that I already took notes about in his earlier session, “Design and Testability“. If you haven’t read that one yet, go check it out.

Advanced Unit Testing Topics
Ray Osherove
Typemock
Blog: http://iserializable.com/

The Art of Unit Testing (book he’s working on)

Assumption for this session: You have some experience with writing tests

Mocking 101

  • Replace dependencies with fakes so you can test something
  • If the Storage class calls a non-virtual void method on EmailSender, you can’t test that interaction.
  • Pass an interface instead of a class. “Dependency injection”. Then we can write a FakeEmailSender and pass that in, and it can do whatever we want.
    • Do nothing
    • Set a flag saying “I got asked to send the e-mail”
  • Can create the fake manually or with a mock-object framework
  • Fake vs. mock
    • Fake: fake to make the system more testable. They won’t fail your test. (In another session, I heard this called a “stub”, with “fake” as a top-level term for both stubs and mocks.)
    • Mock: something we will assert against. These can be used to detect conditions where your test should fail.
  • More complex scenario: force EmailSender to throw an exception, so you can test how StorageManager interacts with ILogWriter.
    • FakeEmailSender, FakeLogWriter — where the LogWriter is a mock (you’ll assert that it actually got asked to log the error), and the e-mail sender is a fake (you don’t assert against it, it just replaces the real one)
    • Usually, in a test, you’ll only have one mock, and maybe many fakes

Side note: Test names should be extremely readable. Imagine that the person reading your code is a serial killer who knows where you live. Don’t make him mad.

Mock-object libraries

TypeMock:

[Test]
[VerifyMocks]
public void Typemock_Store_StringContainsStar_WritesToLog()
{
    ILogger log = RecorderManager.CreateMockedObject(typeof(ILogger));
    // Tell the mock object what you expect to happen in the future
    using (var r = new RecordExpectations())
    {
        log.Write("*");
        // Tell it to check parameter values, rather than just expecting
        // the method to be called
        r.CheckArguments();
    }
    // Now run the code under test
    var sm = new StorageManager(log);
    sm.Store("*");
}
  • The assert isn’t in the code — it’s implicit in the [VerifyMocks] attribute. Can also do it explicity through a method call.

RhinoMocks:

[Test]
public void Typemock_Store_StringContainsStar_WritesToLog()
{
    MockRepository mocks = new MockRepository();
    ILogger log = mocks.CreateMock();
    using (mocks.Record())
    {
        log.Write("*");
        // RhinoMocks checks arguments by default.
    }
    // Now run the code under test
    var sm = new StorageManager(log);
    sm.Store("*");
}
  • Looking back at that, I wonder if I missed something, because I don’t see any sort of VerifyMocks() call in that code snippet. I probably missed copying it down from the slide.

What happens when you can’t extract an interface?

  • Derive from the class under test

What happens when you can’t change the design?

  • Can cry to our managers
  • Typemock lets you mock things that are static or private, without modifying the design.
    • So suppose the Storage class instantiates the Logger itself, and you can’t change it. How do you break the dependency so you can test the code?
[Test]
[VerifyMocks]
public void Typemock2_Store_StringContainsStar_WritesToLog()
{
    using (var r = new RecordExpectations())
    {
        new Logger().Write("*");
        r.CheckArguments();
    }
    var sm = new StorageManager2();
    sm.Store("*");
}
  • Yep, it can record, and later fake out, a new call.
  • Uses the .NET Profiling API to do this. Deep black magic.

Mock object frameworks save time and coding, but sometimes fake objects are the simpler solution

Testing ASP.NET WebForms

  • Integration testing: use tools that invoke the browser and check stuff out
  • To test it in-process, and fake out the HttpContext and all that: Ivonna (which uses Typemock underneath)
    • Lets you set values on controls, and then process postbacks in-process

Extending the test framework with extension methods

  • Create a domain-specific language on your objects
"abc".ShouldMatch("\\w");
List<int> ints = new List<int> { 1, 2, 3, 4, 5, 6 };
ints.ShouldContain(4);
3.ShouldBeIn(ints);
5.ShouldBeIn(ints);
  • Make a static class with static methods
  • First parameter has this
  • Call asserts from the extension method
  • Import the namespace in order to be able to use the extension methods
public void ShouldBeIn(this object o, IEnumerable items)

Testing (and Mocking) LINQ Queries

  • LINQ is not a test-enabling framework by default
  • Use Typemock, then duplicate the LINQ query in your test (to record the calls)
  • The problem is, you’re over-specifying your tests, because you have to know what query is being run underneath — you’re specifying the internal behavior
  • So, avoid doing this unless you have to
  • May be better to abstract away the queries
  • What about projections? E.g. selecting an anonymous type?
    • Typemock can record anonymous types. He already had a demo for this.
    • Can be really, really weird.
using (RecordExpectations rec = new RecordExpectations())
{
    var p1 = new { Name = "A", Price = 3 };
    rec.CheckArguments();
    string a = p1.Name;
    rec.Return("John");
}
var target = new { Name = "A", Price = 3 };
Assert.AreEqual("John", target.Name);
  • The above actually passes. Yes. Look closer. They’re mocking the return value of the anonymous type, so even though you construct it with “A”, it returns “John”. Like I said above, deep black magic.

Testing WCF

  • Again, maybe the best thing is not to unit-test it. Consider an integration test instead.
  • Integration test: create a ServiceHost, ChannelFactory
    • Still might want to use Typemock for things like the security context
    • Be pragmatic about things. If it’s one line of code to use Typemock, vs. introducing lots of interfaces, maybe it’s worthwhile (as long as it’s maintainable).
  • Worth making a WCFTestBase<T, TContract> to pull out common code.
    • If you do integration testing, get used to base test-fixture classes to remove duplication.

Database-related testing

  • To mock or not to mock the database?
    • Mock:
      • Tests will be fast, no configuration, don’t need a real database.
      • But the database itself also has logic. Keys, indexes, integrity rules, security, triggers, etc.
  • Team System has a tool for testing against databases, but it’s not really “there” yet.
  • Can do an integration test against the database.
  • If you change external data, roll back between tests. Good solution: lightweight transactions with TransactionScope. Create in SetUp (with TransactionScopeOption.RequiresNew), and call Dispose in TearDown.
    • Will work even if the test creates its own transaction (as long as it doesn’t also do RequiresNew).

Put integration tests in a separate project, so it’s easy to run the unit tests (which are fast) all the time, without being slowed down by the integration tests (which are slow, require configuration, etc.)

Testing events

  • Hook the event as usual
  • Hook it with an anonymous delegate
  • Do your asserts inside that anonymous delegate
  • Make an eventFired variable and set it true from inside the delegate
  • Assert eventFired in the main test body
  • Yay closures!

Musical postlude

TechEd 2008 notes: Lessons Learned in Programmer (Unit) Testing

This one was a mixed bag. Some of his lessons learned are worth looking at, like the 3A pattern, and he covered classics like inversion of control and avoiding ExpectedException. But he was pretty dogmatic about a couple of things that just didn’t make much sense.

Lessons Learned in Programmer (Unit) Testing: Patterns and Idioms
James Newkirk, CodePlex Product Unit Manager

Been writing software for close to 25 years.
When he was first hired after college:

  • Company made EEs work in manufacturing for 2 years
    • Goal: teach them how to build things that could be manufactured
  • Software people just started writing stuff
    • Never taught how to build things that were testable, maintainable. Up to them to discover that.

JUnit Test Infected: Programmers Love Writing Tests
(Has anyone actually met programmers who love writing tests?)

  • Programmers weren’t writing tests
  • Couldn’t prove that what they had worked
  • James ported JUnit to .NET as NUnit, released as open-source
  • How can we use open-source to enable different kinds of innovation?

“Unit testing” means something to testing people. Some think, if cyclomatic complexity is 5, then I need 5 unit tests. Not the same thing as tests for test-driven development. So we’ll probably not use the term “unit test”.

What is programmer testing? Per Brian Marick

  • Technology vs. Customer
  • Support vs. Critique
  • Support + technology: Programmer tests
  • Support + customer: Customer tests
  • Critique + customer: Exploratory tests (should still happen during the development process)
  • Critique + technology: “ilities” — non-functional. Scalability, usability, performance, etc.

Why do programmer testing?

  • “There is no such thing as done. Much more investment will be spent modifying programs than developing them initially.” [Beck]
    • Whenever we think we’re finished, we’re probably wrong. There can always be more features.
    • Can define “done” locally: “We’re done with this release.”
    • Programmer testing lets me say as a programmer, “I believe that I am done.”
  • “Programs are read more often than they are written.” [Beck, “Implementation Patterns” book]
    • You’re writing a book on your code.
    • The only artifact of any value is the code. It has to be the communication mechanism.
  • “Readers need to understand programs in detail and concept.” [Beck]

Total development cost:

  • Develop (illustrated in green): small. Fun!
  • Extend/Maintain (orange): big. Sucks, especially if you didn’t write the green part!

“I might break something!”

  • Fear lowers our productivity
    • Programmer tests help, because if you broke something, you know. You know the consequences of your actions.
  • Where do I start?
    • Programmer tests are very useful in this understanding phase

Lesson #1: Write tests using the 3A pattern

  • Attributed to Bill Wake (xp123.com)
    • Arrange — Set up the test harness
    • Act — Run the thing you actually want to test
    • Assert — Verify the results
[Fact]
public void TopDoesNotChangeTheStateOfTheStack()
{
    // Arrange
    Stack<string> stack = new Stack<string>();
    stack.Push("42");
    
    // Act
    string element = stack.Top;
    
    // Assert
    Assert.False(stack.IsEmpty);
}
  • Benefits
    • Readability
    • Consistency
  • Liabilities
    • More verbose
    • Might need to introduce local variables
  • Related issues
    • One assert per test?
      • Much agile dogma says there should only be one assert
      • Pragmatic view: test should only test one thing, but if that one thing takes multiple asserts, that’s fine

Lesson #2: Keep your tests close

  • Tests should be as close as possible to the production code
  • Same assembly?
  • Treat them like production code
  • They have to be kept up-to-date
  • Visibility (let you see internal)
    • (I would argue that, if you think you need to test your internal state, what’s really going on is that you’ve got another object that wants to get out.)
  • Liabilities
    • Should you ship your tests?
      • If so, you need to test your tests. Whole new level.
      • Need to make sure your tests don’t modify the state of the system
      • If No, how do you separate the tests from the code when you release?
    • Various tool issues

Lesson #3: ExpectedException leads to uncertainty

[Test]
[ExpectedException(typeof(InvalidOperationException))]
public void PopEmptyStack()
{
    Stack<string> stack = new Stack<string>();
    stack.Pop();
}
  • Problem #1: where’s the assert? A: it’s in the framework. Violates 3A.
  • Obfuscates the test to some extent.
  • Better:
[Fact]
public void PopEmptyStack()
{
    Stack<string> stack = new Stack<string>();
    Exception ex = Record.Exception(() => stack.Pop());
    Assert.IsType(ex);
}
  • Makes the exception more explicit.
  • Lets you inspect the exception object.
  • Another possibility is NUnit’s Assert.Throws():
[Fact]
public void PopEmptyStack()
{
    Stack<string> stack = new Stack<string>();
    Assert.Throws<InvalidOperationException>(delegate {
        stack.Pop(); });
}
  • Downside: Act and Assert are in the same spot.
  • Other problems with ExpectedException:
    • Don’t know which line of code threw the exception. Test can pass for the wrong reason. (This, IMO, is the most compelling reason to avoid ExpectedException.)
    • ExpectedException forces people to use TearDown. With Record.Exception or Assert.Throws, you can keep the teardown in the test if you like.

Alternatives to ExpectedException

  • Benefits
    • Readability
    • Identify and isolate the code you expect to throw
  • Liabilities
    • Act and Assert are together in Assert.Throws
    • Anonymous delegate syntax leaves something to be desired

Lesson #4: Small fixtures

  • Your code should be a book for people to understand
  • If the fixture has 1,000 tests, it’s hard for someone to know where to start
  • Better: fixtures focused around a single activity
    • Maybe even a fixture for each method
    • Can use nested classes for this. Outer class is named for the class you’re testing, with nested classes for each behavior.
  • Benefits
    • Smaller, more focused test classes
  • Liabilities
    • Potential code duplication
      • May consider duplicating code if it’s for the purpose of communication (but use this carefully)
    • Issues with test runners — not all can deal with nested classes
  • Related issues
    • Do you need SetUp and TearDown?

Lesson #5: Don’t use SetUp or TearDown

(I smell a holy war)

  • If the tests are not all orthogonal, you can end up with a test that doesn’t need anything from the SetUp method.
  • If SetUp takes a long time, you’d be paying the price for every test, even those that don’t need all of it.
  • If there are tests that don’t need all of SetUp, readability suffers.
  • It’s a hacking point. It’s a place where people can add code that you might or might not like.
  • When asked about the duplication, he suggested pulling the duplicated code into a method, and calling it at the beginning of each test. That wouldn’t necessarily be a totally bad idea, if SetUp was the only thing he was trying to get rid of.
  • Benefits
    • Readability (if it removes duplication)
    • Test isolation (but only if it means you get rid of all fixture state, and put absolutely all your state into local variables)
  • Liabilities
    • Duplicated initialization code
    • Things he didn’t mention, that I will (having run into all of them):
      • Duplicated cleanup code
      • Try..finally in every test to make sure your cleanup code gets run
      • Signal/noise ratio: if a test gets too long, it’s much harder to tell what it’s actually trying to accomplish
      • Poor test isolation: if you have any fields on your fixture class, you will forget to initialize some of them in some of the tests, which will introduce test-order dependencies (since NUnit shares the same fixture instance among all the tests in the suite)
  • Related issues
    • Small fixtures

Lesson #6: Don’t use abstract base test classes

He’s apparently specifically referring to the practice of putting tests on a base class, and inherit them in descendants. We use this technique, though sparingly; you do take a big hit in readability, so it really has to be worth having the same tests (including newly-written tests) apply in more than one place.

What’s wrong with base classes?

  • If the tests don’t all apply to all the derived classes, you’d have a problem. (Um, duh. You’d notice when they failed, wouldn’t you?)
  • Removes duplication at the expense of readability. (Fair point.)
  • He wants us to put the test logic into a utility function instead, and call that from the different descendants. (Trouble is, that doesn’t help make sure that all new tests get added to all the fixtures. You don’t use this pattern unless you want all the tests in all the descendants.)
  • Benefits
    • Readability
    • Test isolation
  • Related issues

Lesson #7: Improve testability with Inversion of Control

  • Martin Fowler article: Inversion of Control Containers and the Dependency Injection pattern
  • Dependency Injection
    • Constructor injection
    • Setter injection
  • Don’t want errors to cascade across all parts of the program.
  • Benefits
    • Better test isolation
    • Decoupled class implementation
  • Liabilities
    • Decreases encapsulation
    • Interface explosion
  • Related issues
    • Dependency injection frameworks are overkill for most applications

Side note: Mocks are good for interaction testing, bad for state testing.

TechEd 2008 notes: How to Make Scrum Really Work

This was a small group, in a small room with a whiteboard, so it was fairly interactive. That means lots of Q&A, which means we jumped all over the place and it looks pretty haphazard in written form. Oh well.

How to Make Scrum Really Work
Joel Semeniuk
Imaginet Resources Corp

Scrum teams are 6.5 times more effective than waterfall teams. (Pity they didn’t cite a source. Anyone got a reference?)

How Scrum works

  • Lots of feedback mechanisms: between team members, re quality of software, with user community
  • Processes that support feedback mechanisms: daily scrums, sprint
    • Sprint = iteration. Generally 2-4 weeks. Design, code, test, deploy.
  • Sprint review: demo to customers, get feedback.
  • Sprint retrospective: Not necessarily every sprint (though that’s debatable, see Juan‘s comments). How did it go? How was the process? Did we feel like this was a successful sprint? What made it successful? If weak, what was the problem? Be constructive. Make sure everybody knows what the team did well.
  • Scrum is a process framework: there are no absolutes beyond the key principles. Continuous process improvement (with the retrospect).
  • House: “You don’t tell your dog to stop peeing on the carpet once a year.”

Roles:

  • Scrum Master. Coach. They take stuff out of the way of the team, to make you more productive. Take away impediments.
  • Team members (everyone involved in building the software, so includes QA, people who set requirements, etc.)
  • Pigs and Chickens
    • Chicken doesn’t have deliverables.
    • Pig has skin in the game.
    • When you make ham and eggs, the chicken is involved, but the pig is committed.
    • In some organizations, at the daily scrum, only the pigs talk. Chickens can observe.
    • Pass a token; only the person with the token can talk.

Is the scrum first thing in the morning? — Do all your devs get in at the same time? (Ha.)

How do you keep a daily scrum short? (Especially when remote.) — Scrum Master is the moderator, and will say, “Rat hole.”

  • What did we do yesterday?
  • What are we going to do today?
  • What are our impediments?

Backlog

  • Bucket of stuff that needs to get done
  • Can assign backlog items to sprints
  • Sprint planning: reconfirm what you have, do decomposition to make it more real
  • If you do internal development, you can plan as you go. If you do contracting or fixed-bid work, you need to spend more time on planning.

Scrum is about what’s next. What about management wanting deliverable dates, when Agile tends to be about discovering stuff as we go?

  • Ken Schwaber: Have a bigger preparation phase, lay out a vision
  • Convince the customer that you can allow change
  • Change is gonna happen
  • We’re allowing the customer to change their mind, by re-prioritizing, changing the schedule, replacing features with others of the same size
  • Prioritizing of backlog is absolutely necessary
  • Track everything: changes in priority, changing out features. Just because we’re doing agile doesn’t mean we throw out best practices about change management.
  • Might be able to win projects even if you refuse to do a fixed bid. Can’t fix all three aspects of the Iron Triangle, which makes for an adversarial customer relationship. Can say, “I understand your budget. We think it’ll be this much. Let’s keep features flexible, and stay aware of the business value of each one.”
  • Sometimes Scrum isn’t the right model, especially when there’s a lack of trust.
  • Aside: Every team should have a nap room.

Suggested story pattern: “As a <role> I want <ability> so that <benefit>.”

Team System plug-ins to help manage sprints electronically:

  • Conchango
  • MSF for Agile
  • Lightweight Scrum Process Template
  • eScrum template (Microsoft)

User stories

  • Three things:
    • What I’m trying to do
    • Conversation about that user story: what fields? what reports? Record that in the work item.
    • Acceptance test, in the terminology of the user.
  • Suggestion: “Sprint 0” = planning sprint.
  • Product backlog: User Stories.
  • Sprint backlog: tasks that need to be done to complete those user stories.

Estimation. One possible technique:

  • Rate each story for:
    • Complexity (1 to 5)
    • Business value (1 to 5)
  • If it’s got a complexity of 5, you must decompose.
  • Rock/paper/scissors-style estimation. If you’re off by more than one, we don’t have the same understanding of the problem.
  • Why use a made-up scale instead of hours? — Don’t want it to turn into a budget for the developers. “Student syndrome”. If it was estimated at a day, the programmer thinks they have a day to do it. Instead, the dev should do the necessary work, no less and no more.
  • Try to discourange single-point estimation. (I think they meant single-axis, which is why they suggest rating both complexity and value. It’s been a few days, though, so I might be remembering wrong.)
  • Another suggestion: minimum / most probable / maximum time.
  • Another suggestion: estimate + certainty.
  • The guy who did order-of-magnitude estimation does hours during execution: how many hours spent, how many estimated hours remaining.

Impediments (obstacles)

  • They have a flag on their work-item database called “Issue”. That seems way too slow-response to me — shouldn’t you just walk over and talk to the customer?
  • Risk management.
  • Mitigate risk. How to lower impact or lower probability?
  • Trigger. When is it not a risk anymore, and now a Problem? What’s the contingency?
  • Mitigations are tasks in your backlog.
  • Can become a rat-hole.
  • Bottom line: anticipate problems.

Scrum of Scrums

  • Each team has their own scrum meeting
  • A few people from each team do a combined scrum, so the teams have some idea of where other teams are

What do you do when your backlog is hundreds of stories long?

  • Consider feature-driven development. Major features, feature sets, features. Structure your backlog that way.

They don’t like teams bigger than 7.

Scrum scaling: “team of teams”. But even so, after about 50 people, scaling on team of teams degrades quickly. FDD scales better.

Amazon: Two-large-pizza rule. A team can’t be bigger than can be fed by two large pizzas. (So, two people, right?)

How to deal with scope creep? — Need a big, burly scrum master. Bring the customer in, lay the cards on the table, and ask what they want to give up. As long as you haven’t started the task, you can change the sprint by trading something out.

Time, Resources, Features: Pick 2.

Side quote: “Drive-by chickens”

So you do the estimates, make the commitment, more requirements emerge, and they don’t want to take more time for them? — Maybe you can’t do agile. Do requirements up front and then change-report the snot out of it.

If your velocity suddenly changes and you figure it out mid-sprint, figure out why. Don’t re-estimate; adjust your velocity.

Some metrics:

  • Stories per sprint
  • Complexity points per sprint
  • Burndown chart

Audience suggestion: Anyone can stop the sprint and call a meeting to resolve something.

End of iteration and not done? — Cover it in the review, move it to the next sprint. Drop from sprint release; remove from the build. That work didn’t get done. Milestones: nothing is 82% done.

If you really don’t want wiggle room at the end of the sprint, put the end of the sprint on a Wednesday. Student syndrome — not thinking about it until the last minute.

TechEd 2008 notes: Design and Testability

Design and Testability
Design Dilemmas in the Real World
Roy Osherove
Typemock
Blog: ISerializable.com (will have slides there)

  • Dilemmas that he’s faced, we’re probably starting to face, Microsoft is facing
  • Many have to do with change
  • Design has changed very little in the last few years, but maybe it’s time for a new thought, or some new compromises, or lack of compromises

Has been on two sides of the same coin

  • Pure design, dependency injection, hardcore freaks
  • Being pragmatic
  • What you can do about testability when tooling is not available to help you (or you choose not to use it)
  • Benefits of testable designs

Agenda

  • Why should I care about testability?
  • Defining a testable system
  • Examples of implementing and refactoring testable code
    • Extract interface, virtual methods…
    • Dependency injection and IoC containers
  • Legacy code, and what happens when you can’t change your design
  • Silver bullets
  • Why current systems make this hard
  • Patterns

Me Me Me

  • The Art of Unit Testing
    • ArtOfUnitTesting.com
  • Typemock
    • Next generation isolation platform (Isolator)
    • Next generation testing tools (shhhh…)
    • Some things are changed, some are a little more powerful, some things feel a little weird

Before we start

  • This is just one way to do design
  • Assuming you have experience with NUnit or MS Test

Why should I care?

  • Framework authors
  • Maintainable system. Testing is about quality, and a security net that lets you know when you screw up.
  • Extensible system, like LEGOs.
  • Code quality

Automated unit/acceptance testing is important

  • Helps with regression testing
    • Maintainable system
  • Finds bugs early
    • Code quality
  • Better understanding of requirements
    • Most people write code from the inside out: write the method, without caring about who’s going to be calling it
    • The test is using the API, so you have to think about requirements, design, usability
    • Also means it makes your coding take longer… but code quality and maintenance are better, and release cycle is actually shorter
  • API documentation
  • “Executable” requirements. It’s good for the customers, because they can run actual tests.
  • Big database change — not a big problem.
  • Even bad tests can help more than you think.

Unit level testing requires testability

  • How easily can you…
    • Create a new instance? Do you need to create a configuration file first? Configure a third-party library?
    • Get a consistent result? Do methods have side effects? Talk to a database or Web service?
    • Test just X without Y?
    • Control inputs and outputs from X?
    • Isolate X from its dependencies?

Two kinds of testability

  • Testable application
  • Test enabling framework
    • ASP.NET MVC, WebForms. WebForms code depends on the WebForms framework, which is not test enabled. Much easier to test code built on ASP.NET MVC.
    • SharePoint. Untestable by default: lots of statics, global objects. Testing your logic is almost impossible.
    • MS CRM. Even more horrible than SharePoint.
    • WCF, WF. Not test enablers by default. WF has lots of sealed classes, multiple processes, multiple threads, no places to intervene. Lots has to do with security, but lots has to do with the way things are done.
    • If you’re a framework author, making your framework test-enabled should be one of your main concerns.

What is a unit-testable system?

  • Lets us write good tests easily
  • For each piece of coded logic, a unit test can be written easily enough to verify it works as expected, while keeping the PC-COF rules

PC-COF

  • Partial runs are possible
  • Configuration is not needed
  • Consistent pass/fail result
  • Order does not matter
  • Fast (what is “fast”? We’ll talk about that)

Consistent pass/fail result

  • Can you trust your test?
  • Control of dependencies (you want to test a method, but it starts with a call to a static method e.g. to validate)

Interface-based design

  • Design: Extract interface for Validator
  • Test: send in a “fake” validator
    • He implements with a public field saying what IsValid should return
  • Inject the IPersonValidator via e.g. a constructor parameter
    • OO purists are panicking that you’re letting someone else decide how to do a core part of your functionality

Side note: if you write a test, give it a good name. Don’t call it Test1, Test2, Test3. He will personally choke you.

  • His name suggested naming convention: MethodBeingTested_UnderTheseCircumstances_DoesThis
  • Don’t name it starting with “Test”, because the attribute already tells you it’s a test

This works, but it kind of sucks. You end up with lots of interfaces, and your classes have lots of dependencies.

“Injecting” dependencies

  • Constructor injection
    • Use for dependencies that are not optional.
  • Property injection
    • That means it’s an optional dependency.
  • Factory injection (especially if you already have a factory)
  • Virtual method injection

The dependency problem

  • People just want to use the code that you wrote. They don’t care about passing in a logger and a validator.
  • To create a logger, you need an exception handler and a data writer.
  • Etc.

Inversion of Control containers

  • Kind of like really smart factories. You ask them for a new instance of some type, they’ll look at the constructor’s dependencies, and if you’ve configured it correctly, it’ll know which objects to pass for those dependencies. If those also have dependencies, it handles it recursively.
  • Spring.NET
  • Castle.Windsor
  • StructureMap (not officially released lately, must build from source)
  • Microsoft Unity Application Block — Unlike many other Application Blocks from Microsoft, it doesn’t suck.
  • Very configurable, from XML or from code

Demos: Windsor and Unity

Windsor

  • Create a WindsorContainer
  • Add components — generic syntax. Either give it a class, or an interface and a class.
  • Have to also add the type you’re going to resolve (instantiate).
  • container.Resolve() — constructs something.
  • Some people believe your tests should call the constructor directly with your fake dependencies, instead of using containers from tests.

Unity

  • Create a UnityContainer
  • Call RegisterType
  • Fluent interface — you can chain calls to RegisterType.
  • Don’t have to register the class you’re going to resolve (instantiate).
  • Tries to use the constructor with the most parameters. Gives an exception if the container doesn’t know how to create one of thoes parameters.

Important: Need to make sure everyone uses container.Resolve instead of new.

Fast run times

  • A test that takes one second is not fast.
  • Want someone to be able to get latest version, right-click, and run all the tests. It should be that simple. And all the tests should pass.
  • If they have to wait a long time, they won’t run all the tests.

No configuration needed

  • How easy is it to get latest and run the tests?
  • If a config file is needed, how do you test a bad connection string? It quickly becomes not easy and not fast to write that sort of test.

Solving configuration with override

  • Works if you don’t want to add interfaces
  • Extract dependency into a virtual method
    • This virtual method needs to be the stupidest code you’ve ever written. Absolutely trivial.
  • Class can’t be sealed, so you can’t do this with WCF, WF, SharePoint, etc.
  • Easier; pollutes the code less; but it forces you to make a lot of derived classes in your tests
  • Good for simple scenarios, maybe not when you get more complexity

Test order should not matter for consistent results

  • Some people don’t mind that tests share data
  • If tests share data, you have dependencies between tests
  • That’s BAD, because you’ll kill yourself when you need to maintain tests. You remove a test and something fails. You modify a test and something else fails.
  • Debugging test dependencies is awful. You can spend days on this.
  • If tests use a shared resource (e.g. a database), you need to fix the tests, not the application.
  • Need to restore shared state in TearDown. For databases, create a TransactionScope in SetUp, and Dispose it in TearDown.

Partial runs of tests should produce consistent results

  • Sometimes you want to run only some of the tests. Sometimes tests do take a long time; that’s the real world.
  • Some test frameworks let you re-run just the failed tests.

Are singletons evil?

It depends. .Instance probably is. Getting it from an IoC container, maybe not so much.

  • How do you instantiate it for testing? (And re-instantiate it, e.g. if you modify the configuration that it loads on startup)
  • Shared resource between tests
  • Single Responsibility.
    • There’s a class being held, and the one doing the holding and making sure there’s one instance.
    • Can refactor into testable ones: separate holder to a separate class, or refactor using a container.
      • Unity: RegisterType(instance)
  • Global variables
  • Tight coupling
  • State that lasts forever
  • Threading?

The GOD Method

  • You know it’s important; you don’t know why it does what it does; sometimes it’s cruel to you
  • One huge do-it-all method
  • Prevents maintenance, impossible to test
  • Can’t refactor for testability, because you might introduce bugs. Solution: integration-level tests.
  • Avoid this by design
  • Keep single responsibility principle
  • Calls to little methods

Legacy code

Test Driven Development/Design

  • Tests have to ues a testable class
  • Test-first means testable-design-first
    • Test can later show you how you’d like the API to be used
  • Decoupled design and architecture
  • More control on state simulations

Design Guidelines

  • Avoid BDUF (Big Design Up Front). Tends to result in highly untestable code. Do EDUF (Enough Design Up Front) instead, but your design should be able to change as you go.
  • Interface-based designs. Tend to be easier to understand in terms of roles. A Logger is a role.
  • Try to avoid Singletons
  • IoC containers
  • Avoid GOD methods — use polymorphism instead
  • Virtual by default
  • new X() -> Factory.MakeX(). When you abstract away the creation, you can replace the creation.
  • Single Responsibility for class and method. If our class or method is elaborate and does more than one thing, it’s usually a design smell.

Key points

  • Design for testability leads to agility
    • You need tests when someone changes the requirements
  • Common patterns
    • Recognizing test challenges in the design
    • Refactoring the design for testability
  • Test-driven development leads to a testable design
  • Testable code == well designed, decoupled design

Musical postlude

TechEd 2008 notes: The Gentle Art of Pair Programming

I already know a fair bit about pair programming, but the session description said, “For those already working in a pairing environment, Wendy and Oksana include some novel viewpoints and interesting discussions on familiar topics.” I think that worked out okay — it’s always good to hear how other people are doing agile, and they do a few things differently than we do.

The description also said there would be “interactive and fun games”, but the room was way too big and spread-out for that to work. There also wasn’t much time for things like that once they’d answered all the questions; people were really curious about a lot of things. Questions and answers interspersed below.

The Gentle Art of Pair Programming
Oksana Udovitska
Wendy Friedlander

Usually an interactive session with games and some hands-on practice, but they got put in a lecture room instead, with not enough time for games

Agenda

  • Who are those chicks? (their words, not mine — I just copied the slide)
  • Let’s get to know each other
  • What this pair programming is all about
  • Personalities and Pairing
  • Daily routines
  • It’s all about the team
  • Respect and communication
  • Wrapping it up

Who Are Those Chicks?
Oksana Udovitska and Wendy Friedlander

  • Software developers
  • On cutting edge of agile practices
  • Wendy likes to code nonstop, do things her way, so was hesitant about pair programming initially; but changed her mind after trying it
  • Oksana enjoys pairing more than regular programming, because it’s very social

A Day In Life Of Pairing

  • Morning stand-up
    • This is when you break into pairs and pick up tasks
  • “War room hours”
    • Uninterrupted coding
    • Try not to schedule meetings or get distracted.
    • ~2 hours in morning and ~2 hours in afternoon.
  • Break for lunch, yummy!
    • Pair programming is very intense — very emotionally and intellectually draining. Need breaks. Sustainable pace.
  • More “War room hours”
  • Go home, yippee!

Morning Stand Up

  • Standing by the story board
  • Talk to the board
    • Worked on this story
    • Finished these tasks
    • This is what we have left
  • They switch pairs at stand-up, and again after lunch.
  • Promiscuous pairing: the person who is newest in the task stays on
  • Rotate pairs — don’t always pair with someone who thinks like you. Can use a chart to keep track.

Losing code ownership. Can stand up from a task in the middle, and have someone else take over. Trust that it will be done OK.

May not have personal desks, but there is team space. So you need to like the people you work with — but you can post pictures of your wife all over the place. Little knickknacks. Space Invaders screenshots. It seems scary, but you get used to it.

Also, there are 2-3 hours a day when you’re doing other things: making calls, researching on the Internet, going to meetings.

More than two people in the room; you’re in the same room with the other pairs on the team. They called this “osmotic information”: even if you’re not focused on it, you have some awareness of what others are doing. If another pair seems stuck, you can offer to help. If you’re stuck, you can ask for help. If you need to make a decision that could affect the whole team for the next couple of weeks, you can talk to the group about it. (But try to only interrupt if you need to, because of context switching.)

Do you have to match experience levels? — Surprisingly enough, yes and no. Make sure you both understand what you’re trying to do, but even if you’re not the best person to complete the task, you still can have good ideas on how to approach it, and you learn really fast. Oksana’s experience, as a newbie in a pairing environment, was that she didn’t feel she was dragging anyone down. You immediately learn the things that you need to know.

Depending on who you’re pairing with, it’s your responsibility to customize your behavior or put on a different pairing hat. If one person feels like they’re dragging the pair down, that’s the fault of the more experienced person. (Trading the keyboard often may help with this.)

There is no driver: there’s a typer. Both people should write every line of code. If you do it right, you’re both engaged all the time.

Coding-style conflicts: very different when you’re pairing. Wendy said she writes code in a very different style when she’s pairing than when she’s alone. Oksana pointed out that you can change the code’s style; nothing will stop you except your pair (who will probably point out that you could be writing new code instead). It becomes a lot more fluid, and people tend to start writing code, not necessarily in the same style as everyone else, but in a style that everyone else can read. Anyone can try something new, and if others don’t like it, it’ll get deleted (so don’t take it personally — must be a friendly environment). It becomes really easy to agree; the things that make sense stick around, and the things that don’t make sense don’t stick around, and you don’t even remember who did it.

Don’t you get half as much done when you have two people on the same task? — No, because:

  • More focus
  • Blocked less often
  • More knowledge — people can take vacations without everything else grinding to a halt
  • Compare cost of fixing early vs. fixing after a bug gets into production.
  • Start with a pilot project on something big and risky, and see if the benefit shows.
  • If the team is self-managing (as it should be in an agile shop), just do the pairing. As long as management is happy with the work, try stuff to see if it makes sense. Then reflect on it to decide whether it worked.

How many developers do you need to get this started? — Probably 4-6 developers; 6 is a little large for getting everyone to agree all the time.

What if there’s an odd number of people? — Can work alone, just needs a review afterward. Also lean on continuous integration. Or if you don’t want to work alone, you can work on spikes (research into new technology, doing the quickest, dirtiest proof of concept), or maintain the build server, or review old code.

Velocity: Different pairs of people work at different speeds. So you just have to look at how many story points you average as a team, and know that it won’t be exact. Many agile teams don’t put too much time into estimation, especially long-term estimation (and “three months out” changes every two weeks as the customer changes priorities).

How do you do employee reviews? — Everyone knows everyone’s strengths. Some people are good at coding, some people make the pair happy, some people make the pair unhappy. It becomes a team thing — the team did well, so the team gets a raise. Then maybe, if one person is especially good, they get a little extra.

Becomes a very healthy environment. Don’t have people being pathologically competitive and protective of their code and knowledge.

Documentation: customer-facing documentation only, if needed. No design docs or comments (your tests are your comments). You probably will tend to have a wiki to store knowledge like port numbers, how to set up the build environment, etc.

Do it together? Do it yourself?

  • People are social animals
  • We can do things alone or together
  • Which do you prefer?
  • Learning to communicate with a pair can help you learn how to communicate with customers

What is pair programming?

  • Team effort
    • Everyone participates
    • No heroes
    • Good to put people who are inexperienced in a task on that task
      • Have to be open to failing your commitment because you’re not putting the hero on the task all the time.
  • Collaboration and sharing
    • Knowledge transfers — not knowledge transfer!

Why pair?

  • More interactive
  • Team spirit
  • Confidence
    • Okay to ask questions and show vulnerability
    • Know others agree with your code and like it
  • No more code reviews
  • Readability and consistency
  • Discipline
  • You can take vacation!

Why is it hard?

  • Personal interaction
  • Everyone is unique
  • Less personal space and time
  • Very intense
  • Have to do the wrong thing sometimes to keep the flow going (don’t spend two hours discussing why it’s going to fail — just write it and let it fail)

My way, your way, our way?

  • Talk
  • Compromising isn’t good. You can’t really compromise between two different patterns; they don’t mix. Choose your battles — better to do it in a poor way, but completely. Or try it, show that it can’t work, and then do the other way (and maybe you were wrong and it is good enough).
  • YAGNI
  • Know when to ask the team

Getting started

  • How?
  • Commit. Make sure everyone on the team agrees that it’s worthwhile to try, and try it for at least a week.
  • Switch pairs. Helpful if you’re sitting in the same room, or at least nearby.
  • Can be done remotely, as long as you know you’ve got the technology working (VPN, screen sharing, etc.) But you do lose osmotic information; you don’t know what other pairs are doing. Cameras are helpful. Skype + VNC. Pairing is probably the easiest agile practice to do from home.
  • Know your results.
    • Know how to reflect on what’s working and what isn’t.
    • Don’t be afraid to bring up things that should change.
    • Go around the room, have everyone say something we should keep doing and something we should change.
    • If you’re doing agile, retrospectives are core. Otherwise you’re being dogmatic.

The most important thing

  • Having fun!
  • If the walls are blank and everyone’s staring at the monitors, you’re doing something wrong. Joking, talking, arguing (in moderation).

Respect and communication

  • Communicate
  • Respect your pair
    • Be open to their suggestions
    • If something bad happened at home and you’re in a bad mood, you have to leave it behind
    • Hiring process is really tricky. You need people on your team who will interact well.
  • Express yourself and listen

Knowing when to mentor

  • Pairs not matched
    • It’s rare that one person won’t be mentoring. If you’ve got two experts pairing, they’re probably not experts in the same thing.
  • Frustration
  • Mentoring mindset

They think nine developers is too large for one team — they suggested forming two sub-teams, each working on its own domain, and letting people move between sub-teams.

TechEd 2008 notes: Microsoft Software Licensing and Protection Services

I didn’t know Microsoft even had an API for software licensing. They do, but there’s a lot of stuff it doesn’t do, most of which is supposed to be coming in the next version.

Microsoft Software Licensing and Protection Services
(SLP Services)
Terrence J. Nevins (tnevins@microsoft.com)
Rehan Hamid

Software Economic Models

  • Licensing market: $228B today, growing to $305B by 2010
  • Subscription: $113B for Internet access growing to $200B by 2010; IPTV growing to $17B by 2010
  • Advertising: $27B+ today, $80B+ in 2010
  • Transactional: $10.7B sold on Amazon in 2006; $52B sold on eBay in 2006; $2B in music downloads; $1B in ringtones

Key Challenges for Licensing

  • Fighting piracy and low compliance, reverse engineering
  • Meeting customer demands: try before they buy, subscription models
  • Reducing costs and enabling integration

Big tip

  • Getting started
    • Request an Evaluation (your MSDN subscription key is good for their license services, but don’t use it when you’re playing around — get an eval key instead)
    • Activate your account
    • Use the evaluation key as your “sandbox”
    • Preserve your potent MSDN subscriber key
      • Use MSDN key for commerical use, not for testing
      • One year service subscription
      • A starter set of free activations included

www.microsoft.com/slps/ to request an eval key

PROTECT

  • Protect against reverse engineering, binary patching
  • Balance protection and performance
  • Make it easy for your customers to stay in compliance

Code Protection. You want to use IL and reflection, but you want to protect your IP.

  • Select app to be protected
  • Identify methods to be transformed
  • Transform method IL to SVML
  • Distribute protected app with SVM

SVML — Secure Virtual Machine Language

  • MSIL is easy to reverse engineer. SVML is very difficult to reverse engineer, and unique to your organization.
  • SVM — Secure Virtual Machine. Because SVML is unique to you, your SVM is unique as well.
  • Not obfuscation — transformation.

Developers should drive the decision of which methods to protect. Marketing can drive a lot of the policies, but not method selection.

Browse to folder (“Medical” in the example), select methods to protect, click Protect. Creates a “Medical.Protected” directory with a protected EXE and several DLLs.

There’s also a command-line version for automated builds. Can be integrated with Visual Studio. Can use attributes to specify which methods you want to protect. Not really good MSBuild or VSTS CI integration yet, but they’re working on it.

Every time you call a protected method, it calls into the SVM.

Performance impact: Absolute worst case is thousands of times worse. If you have critical code in tight loops, you’ll want to strategically choose which methods to protect.

Supports .NET 1.1 upwards.

What does it cost? — It’s complicated, and it’s going to change in a month or two anyway.

Can you use the protection without the licensing? — Today yes, after July 28 maybe no.

Can you do this with native code? — You wouldn’t really need the protection, so you’d just be doing the licensing. You can do that; you’ll take a managed-code dependency.

Does the end-user need to have a connection to the activation server? — If you want silent activation, yes. Otherwise, you can save a machine “fingerprint” to a USB stick, walk that to an Internet-connected machine, download a license file onto the USB stick, and walk it back to the disconnected machine.

What goes into the machine fingerprint? — It’s complicated. Don’t use MAC addresses. Processor serial numbers aren’t reliable in multi-processor machines. You need some balance, especially when you start implementing it onto an existing customer base. Keep the honest people honest; think about the end-user experience.

Activate by phone / manual activation without USB key? — It was dropped from the current shipping version. It’s currently lower on the internal priority list, so it’s hard to tell when / whether it will come back. For today, you need a homegrown solution to this.

These solutions have to be reviewed by an MS crypto board any time they don’t use the crypto API. It’s reviewed by 20 pairs of eyes that know crypto inside and out. Obviously it’s not totally secure (once you put it on someone else’s machine, it’s not your code anymore), but it’s probably as reasonably secure as it can be. (Note: this was an acquired company.)

Generating keys

  • Go to Microsoft’s portal, log in
    • Create a product, and specify its name and version
      • Major version is part of license. If you want license keys to be good for a time period but any version (including major upgrades), they think that can be done, but would require followup with their devs.
    • Add features
  • Log into local app
    • In property grid, select the product name you created in the portal
    • Select methods and associate them with features
      • If the app isn’t licensed yet, you get a UI prompting you to activate.
        • What about services (where you can’t show GUI)? — Huh? Licensing a service? Apparently nobody ever thought of that. (WTF?!)
        • Can you make it throw an exception instead of popping up GUI? — There’s some kind of API for that.
        • Can the dialog be branded? — Yes.
        • Can you put the licensing into the MSI? — There’s a demo online for that.
  • Go back to portal and create a key (5×5)
  • User selects “Activate over the Internet”, enters that 5×5 into the dialog
    • Contacts the service, gets back a license file, does its magic
  • If you try to use an unlicensed feature (but you have already activated), you get a dialog saying you can’t do that

Concurrency model: License pooling. This is big and complicated. They don’t do that yet.

When you active or reactivate, usage data gets sent to the server automtically. So you get to know how many times they clicked Print.

Microsoft runs their own license server, but you can run your own in your company. There are really only a couple of reasons to do that: regulation; international laws; federal government.

Limited Windows users can activate, though obviously they can’t install an MSI.

Current version does not support ClickOnce deployment. It’s been discussed but is lower priority.

Their Web site has guidance on privacy issues.

If you want to capture feature usage, the feature has to be in a method that you protect (today, at least).

Interesting note: if you’re an early start-up, it can be really valuable to know how many people activated your product. You can prove, to your venture capitalists, how many people activated from different machines; and because of the fingerprint, you can’t fake it by installing it 42 times on the same box.

Is there a Web service for the portal? — Currently no, but the client application does have APIs that are documented and available. So you can do it, it’s just not Web-service buzzword-compliant.

Is there an API to find out whether a feature is enabled, so you can disable / rearrange GUI? — Not in the current version.

The Road Ahead…

  • CRM integration
  • More activation scenarios and licensing models
  • More software asset management and BI analytics
  • Partnering with complementary code protection technologies
  • IDE integration

TechEd notes: 10 Ways To Improve Your Code

10 Ways To Improve Your Code
Neal Ford
Sr. Software Architect / Meme Wrangler
ThoughtWorks

Where did this topic come from?
Book: “The Productive Programmer”, Neal Ford
Exists in two parts, “mechanics” and “practices”.
Mechanics: how to speed you up, learning keyboard shortcuts.
Today is more Practical (and philosophical).

“The unexamined life is not worth living.” — Socrates

“Unexamined code isn’t worth executing.” — Neal, stealing from Socrates

1. Test driven design.

When you’re really rigorous at test-driven development, it has effects on your design. You’re creating the first consumer of your code.

  • Consumption awareness. You’re aware of how the rest of the world will use the code, before they use it.
  • Mocking of dependent objects. If you’re really unit testing, you’re mocking out the dependencies, which forces you to think about the collaboration between these objects.
  • Introverted vs. extroverted objects:
    • If you do TDD, you’ll learn to avoid “extroverted objects” that reach out to create other objects, fire constructors, allocate resources, etc.
    • Instead, you’ll use “introverted objects” where all dependencies are injected via parameters, properties, etc. You tend to move object creation to a few simple places, and can use things like real dependency injection. Also called “shy objects”.

2. Metrics and static analysis.

Compilation is a really, really weak form of unit testing. You can compile any sort of trash. “It compiles” doesn’t tell you much.

Source Monitor (can do many languages including C# and Delphi). Lots of metrics, including cyclomatic complexity. Can set thresholds and have it show the outliers. Has both GUI and command line. Costs you virtually nothing, and will help you find problem spots.

Cyclomatic complexity (1978): measure the complexity of your code. Has to do with the number of paths through the code, though it’s not quite that simple; it’s based on number of branches and decisions. But higher cyclomatic complexity tends to correlate to more bugs.

FxCop.

3. Good Citizenship.

Cooperation with the rest of the world.

Properties != Encapsulation.

  • See too many devs who use a tool to spit out public read/write properties for every one of their fields, and then engage their brain.
  • “Knee-jerk propertiation.”
  • Much better off passing values to a setter method like SetAddress, which means you can’t be a bad citizen by e.g. setting City without setting State — your object should go atomically from one good state to another good state.
  • Objects are the keepers of state, and they should guard that jealously.
  • Only create properties when you need to call them from real code.

Constructors

  • Specific contract for how to create valid objects.
  • How often is a valid object blank? (from a business standpoint) … Never!
  • Don’t provide default constructors for domain objects.
  • Push back on frameworks that try to require this
  • If you need a null object (and have a language that requires you to have a stateful object to say you have no state)…

Static methods

  • Should be a black box
  • You never worry, when you call the sqrt function, that the next time you call it it’s going to give you the cube root.
  • Statics should be non-stateful.
  • Where you get problems is when you mix statics with state.

Mixing static + state

  • The Evil Singleton Pattern
  • Bad because
    • Mixes responsibilities (doing stuff and maintaining its instance)
    • Untestable
    • Object version of global variables
  • Better: object and factory, where the factory handles the instantiation policing
    • Factory can even call object’s private constructor through reflection. Then you’re enforcing that nobody can instantiate the singleton accidentally.

4. YAGNI

Picture of gold-plated toilet

Discourage gold-plating.

  • Build simplest thing we need right now!
  • No speculative development. Speculative development:
    • Increases software entropy. The more code, the more complexity, even if you aren’t using it.
    • Only saves time if you can guarantee you won’t have to change it later
    • Leads to frameworks.
      • Frameworks are not inherently bad, but we have framework-itis.
      • Let’s face it, building a framework is cooler than what you should be doing right now.
      • Frameworks are written by people engaged in ivory-tower building.
      • The best frameworks are not created by somebody who wants to build a framework. The best are extracted from working code.

Changeability. Code with anticipatory design is larger and harder to refactor and maintain.

5. Shaving with Occam

Sir William of Occam: “Given multiple explanations, the simplest is best.”

Simplest is really hard to come by.

Dave Thomas was brought in to solve problem with volume of internal mail. Inter-office memos were constantly getting misrouted. So they brought in an OCR machine, and brought Dave in to write software for this thing. At one point, Dave said, “Couldn’t you solve this problem with colored envelopes?”

It’s really hard to understand the essence of the problem and address that.

We are drowning in complexity.

  • Essential complexity: We have a hard problem that we’re trying to solve.
  • Accidental complexity: We’ve made the problem hard.

The Goal

  • Simplify essential complexity
  • Kill accidental complexity

This also affects the kind of tools you use. E.g., “static” vs. “dynamic” languages. (I already noted this “ essence vs. ceremony“.) Something nasty and multithreaded would be better written in something like F#.

Interlude: the Top Ten Corporate Code Smells

  1. We invented our own web / persistence / messaging / caching / logging framework because none of the existing ones was good enough.
  2. We bought the entire tool suite (even though we only needed about 10% of it) because it was cheaper than buying the individual tools.
  3. We use BizTalk because… (I always stop listening at this point) Okay, it has some legitimate uses, but it’s been much abused.
  4. We can’t use any open source code because our lawyers say we can’t.
  5. We have an Architect (note the capital A) who reviews all code pre-checkin and decides whether or not to allow it into version control. (New feature in VSTS.)
  6. The only XmlDoc is the default message explaining how to change your default XmlDoc template.
  7. We keep all of our business logic in stored procedures… for performance reasons. There is some tortured logic at work here, but that ignores that dealing with stored procs will slow down your development. Let’s have a talk about premature optimization.
  8. We don’t have time to write unit tests — we’re spending too much time debugging.
  9. The initial estimate must be within 15% of the final cost, the post-analysis estimate must be within 10%, and the post-design estimate must be within 5%. (Punished for coming in too low as well as for coming in too high. The team will overestimate on purpose, get done early, not tell you about it, twiddle their thumbs until they hit the golden window, and then say, “Okay, all done!”)
  10. Blue Screen of Death.

Side note: Unless you work for NASA, your problem isn’t harder than everyone else’s. You’re just doing it wrong.

6. Question authority.

Dave Thomas’ story of angry monkeys. (The story of the stepladder, the bananas, and the ice water. “That’s the way we’ve always done it.”)

[Test]
public void UpdateCacheAndVerifyThatItemExists()

Long camel-cased names are hard to read, and test names tend to be long. Neal’s suggestion: use underscores just for test names.

[Test]
public void Update_cache_and_verify_that_item_exists()

Fluent interface:

ICar car = Car.describedAs()
              .Box
              .Insulated
              .Includes(Equipment.Ladder)
              .Has(Lining.Cork);

instead of setting lots of properties, to make it more readable for non-programmers. Thing is, it violates the rules of properties, by making a property getter (Box) that mutates the object. But in the context, it makes for much clearer code.

Killing I

  • Interfaces define semantic intent
  • No implementation details…
  • …except in the name!
  • Don’t name interfaces with a leading “I”
  • Name concrete classes with a naming pattern
  • (I’m not sure that I would agree with Neal on this, but it’s worth thinking about)

What’s Bad About Standards?

  • Forces you to create default constructors
  • Property setters return void
  • Can’t use PONOs for fluent interfaces

Non-intuitive

  • Pair programming (seems like it would be 50% slower, but in reality, once you settle into it, you develop 15% slower, but have 50% fewer defects)

Read AntiPatterns.

  • GiveMeEstimatesNow. Presented with a problem, asked for an estimate out of the blue; you don’t know anything about it, but the boss asks for a guess. That becomes an ironclad contract.
  • StandingOnTheShouldersOfMidgets. Have a disastrous tool or framework that you’re required to use in every application, because it’s a corporate standard.

Anti-patterns are the lore of software. Take advantage of this prior art. Don’t think that the problems at your job are uniquely yours.

7. Composed Method.

“Smalltalk Best Practice Patterns” by Kent Beck. Smalltalk people were some of the first OO programmers, and came up with elegant solutions. (They’ve got a head start on the rest of us.)

  • Every public method consists of steps implemented as private methods.
  • You can tell something’s wrong with the existing code if there have to be comments.
  • Extract methods.
  • Find opportunities for code reuse. Some of those methods you extract might start to look like Template Methods, which opens up more opportunities.

8. Polyglot Programming

Leveraging existing platforms with languages targeted at specific problems and applications

We have lots of .NET languages. Why not take advantage of them, rather than thinking there’s a framework for everything that you can use from the One True Language?

Looming problems / opportunities

  • Massively parallel processing (multithreading)
    • Use a functional language: F#, Haskell.net
  • Schedule pressure
    • Use a dynamic language: IronPython, IronRuby
    • Use an alternate Web framework: Django, Ruby on Rails
  • New approaches
    • Domain-specific languages (DSLs)
    • Fluent Interfaces

Write your multithreaded logic in F#, your GUI in C# or VB.NET, your unit tests in IronPython or IronRuby, and use DSLs

  • Wouldn’t that add complexity?
  • In the past, language == platform
  • Now, language != platform

Ola’s Pyramid

  • Stable language at the bottom, e.g. C#. Static-typed, statically-verified. Maybe even stricter than C#, like a mathematical language.
  • Above that, a dynamic (non-ceremonial, e.g. F#) language that lets you write code quickly.
  • Above that, a DSL that gets you closer to the problem domain.

9. Learn Every Nuance

  • Reflection
  • Regexes
  • Dependency injection
  • Closures
  • Lambda expressions
  • Extension classes
  • And once you’ve learned it, teach it to your co-workers. E.g., regular expressions.
  • Can save you orders of magnitude of work.

10. Anti-objects

Comes from OOPSLA paper called “Collaborative Diffusion”.

“The metaphor of objects can go too far by making us try to create objects that are too much inspired by the real world.”

Figure vs. ground. Sometimes you need to see the problem a different way.

Pac-Man console had less memory than a 10-year-old cell phone. Problem to solve for the ghosts: What is the shortest distance between two moving objects in a maze? Solution: don’t model the ghost; model the maze. Intelligence was built into the maze itself. Each cell had state. Invented “Pac-Man smell”. A cell he just moved off of had “maximum smell – 1”, and it decayed quickly. Ghosts just had to smell Pac-Man. The ghosts wander randomly until they pick up the scent, and then they move into the cell with higher smell. So they will never cut him off at the pass (on purpose)… but it was a very simple and memory-efficient solution.

TechEd 2008 notes: Why Software Sucks

This was a crowded, entertaining, fast-paced session. It’ll repeat later in the week, so if you’re reading this and you’re at TechEd, you should go see it.

Why Software Sucks
David S. Platt
Supreme and Exalted Dictator-For-Life
Rolling Thunder Computing, Inc

Some of this talk has been adapted from the book “Why Software Sucks… and What You Can Do About It” by David S Platt, Addison-Wesley

So, to start:

Users Think That Your Software Sucks.

Usually audiences will argue with that. It’s like alcoholics in denial. Word #1 in rule #1 of any twelve-step program is “Admit”.

Industries with the top number of complaints to the Better Business Bureau: #2. Internet Shopping Services. #5. Computers — Software & Service. #7. Internet Services. All in the software industry. Worse than used car dealers (#8) and collection agencies (#9). Only beat out by cell phone companies (#1), and they’re probably software problems too.

How did it get this way? Our audience has changed, and we haven’t adapted.

  • 1994 — about 2 million Web users
  • 2006 — about 1000 million Web users
  • And almost all of them use PCs for almost all Web access
  • The Web is now ubiquitous. Florida doesn’t even put their state name on the license plate — they put their Web address (myflorida.com) instead!

Platt’s First, Last, and Only Law of User Experience Design:

  • Know Thy User For He Is Not Thee
  • Engrave that on your heart with “Buy Low, Sell High” and “Always Cut the Cards”

Audience survey: What percentage of today’s attendees are female? In this room: less than 5%. Overall population is 52% female. User population used to be predominantly male, but not anymore.

Audience survey: What percentage of today’s attendees do NOT have a college degree (and aren’t currently working toward one)? In this room: one or two. User population used to be mainly college-educated, but today, 82% of population, 72% of US adults, don’t have a college degree.

Audience survey: What percentage of today’s attendees drive a car with a manual transmission? How many more WOULD if your spouses would let you buy one? Around 75% of people raised their hands. But in the general population, 12-14% of cars are sold with a manual transmission. Normal people don’t drive stick-shifts. They’re not interested in the driving process in and of itself. They don’t want to drive somewhere, they want to BE somewhere. Normal people don’t prize fine-grained control over ease of use.

Users don’t want to use your programs. They want to HAVE USED your programs. They want the benefit, not the process.

You don’t sell software. Your user does not come to you to buy software.

What is your customer actually buying? Let’s look at some non-software examples.

  • Lottery ticket: they’re not buying a lottery ticket, they’re buying futility. Well, okay, that’s true, but they think they’re buying a dream.
  • Toilet paper: They don’t buy it so they can admire the roll. They buy it because they want to use it for the designated purpose. Let’s just say they’re buying hygiene and leave it at that.
  • “The Sex-Starved Marriage” book: They’re really buying futility. Well, they think they’re buying sex and/or hope.
  • Insulin: They’re not buying needles and liquids. They’re buying life.

Still think you’re selling software? You’re not. You’re selling what software does.

Every single user says the exact same thing:

I just want it to work!

So: Make It Just Work. Sounds simple, but the laughs from the audience tell us all we need to know about why software sucks.

GOOD AND BAD EXAMPLES

  • Microsoft Word’s menu bar, floating free over the document. Has anyone ever done this because you wanted to? Have you seen anybody doing this because they wanted to? Heard of anyone?
    • Why did this happen? There was a programmer too geeky to get a date on Saturday night. So he went into the lab and took his frustration out on us!
    • No thought as to whether it would make the user more happy or less happy.
    • If you go to click on the menu and overshoot by a couple of pixels, you end up dragging it instead. Floating toolbars force us to be more precise. It forces the user to be more like a computer, and punishes you if you don’t.
  • Good example from Office: type hte and have it automatically change to The.
    • Understands and recognizes the humanity of the user.
    • Yes, I understand that humans are not diligent, thorough, precise. The computer will be those things for you. “I got your back.”
    • Enhances the humanity of the user. It’s as good as the previous one is bad, for the same reason.
  • www.google.com, in Sweden, comes up in Swedish.
    • Hmm, the user is a human being. Human beings like to be spoken to in their native language.
    • Take their best guess: you’re in Sweden, so you likely speak Swedish.
    • But there’s a “Google.com in English” link right there, which fixes it permanently (persistent cookie).
  • UPS tries the same thing, but doesn’t consider the user.
    • www.ups.com asks you to select your location, and doesn’t let you do anything until you do. You can’t track a package, order supplies, or send them an e-mail saying “Your site sucks, fix it.”
    • If you speak Swedish, you have to drop down the list, hit S, and hit down arrow twenty-six times, select it, then hit OK.
    • Compare to the post office. The clerk doesn’t tell you to click this button thirty times to tell him what language to talk to you in. He knows what country you’re in.
    • UPS didn’t put themselves in their users’ heads. They didn’t say, “How can we make this easy for our users?” Instead they said, “Let’s tell our users what to do.”
    • Amusing: You can type a UPS tracking number into Google, they’ll give you a link to track it, and one click takes you there. That means Google is a better UPS than UPS is.
  • Who has backed up their hard drive in the last week? (Ha.)
    • Like any human, we have a hierarchy of needs. Putting it off one more day probably won’t kill you today.
    • PC Editor’s Choice Backup Manager. (Genie Backup Manager Home Edition 7.0)
    • First choice: Normal or Easy?
    • So… Easy is the opposite of Normal… and Not Easy is the default?
    • Treeview with checkboxes… immensely powerful and flexible, but when was the last time you wanted to back up anything other than everything?
    • Backup type: Normal, Increment (so Increment isn’t Normal?), or Mirror (which is neither Increment or Normal, and what the heck is it, anyway)?
    • You do not want to back up your hard drive. You want to have backed up your hard drive.
    • Carbonite (Internet backup) has a picture of a guy in a hammock. Very few choices. Choices are “My Documents”, “Everything”, and “Pick and Choose”. Nobody has ever selected “Pick and Choose”. It’s there because the programmers insisted.

CONVINCING THE BOSS

How do we convince the boss that our apps need to Just Work?

MAKING IT JUST WORK

Plattski’s Handful

  • Add a Virgin to the Design Team
    • Somebody who doesn’t know the internal workings of the problem
    • “The text in the Untitled file has changed. Do you want to save the changes?”
    • Exposing internal workings directly to the user; forcing user to learn and understand how the program is written internally. User shouldn’t have to know about dirty buffers.
    • “Blindness improves your vision” — Po Bronson
    • Put a virgin on your team, and don’t shout them down — listen to them.
  • Break Convention When Needed
    • MS Money doesn’t ask when to save to disk. It just works.
  • Don’t Let Edge Cases Complicate the Mainstream
    • A theorem that’s true 99 times out of 100 is a false theorem.
    • A program that makes 99 out of 100 users happy is doing pretty darn good.
    • The Starbucks store finder asks you for a search radius. Very few people ask, “How many Starbucks are there within five miles?” They ask, “Where’s the nearest Starbucks?” The one guy who does care dictated the user experience for everybody else.
  • Instrument — Carefully
    • Track what the users are doing, and analyze it
    • Ultimate way of Knowing Thy User
    • Microsoft has this, but they won’t license it out, so everyone else has to write their own
    • knowthyuser.com — David is working on a framework for this
  • Always ask: Is This Individual Design Decision Taking Us Closer to Just Working, or Farther Away?
    • Sum of many, many small decisions

Make software not suck. Make it Just Work.