WHOO! *Happy dance*

Had to blog this first thing.

I have, in my hot little backpack, a two-CD set of the Diamondback Preview. They handed them out to everyone after the What’s New in Diamondback / Meet the Delphi Team session tonight. (Now aren’t you sorry you missed the con?)

New toy. Hehehe…

Yes, it’s beta, yes it’s going to do some crashing. But… refactoring, man. Find References. Sync Edit. Debugging .NET and Win32. Delphi and C# in the same project group. I’m going to have some serious fun with this thing, especially after I get back home.

Oh, and a note for the guys back home: The Diamondback Win32 compiler still supports old-style objects. So we could use this thing’s refactoring tools to migrate them to classes. How long have we been waiting for that?


What *wasn’t* in Danny’s Diamondback session

Stuff that came up after Danny‘s talk (which I briefly summarized here). Even those of you who were at the talk didn’t hear this stuff yet. (Well, except for Allen, but I suspect he already knew all of this anyway.)

Delphi for Mac?

Now, don’t get your hopes up. There is no official word on this yet. (It may be worth noting that this is not an official “no”.)

Yes, you can get partway there with Mono, but there’s not much there for GUI. However, when someone was asking Danny about this after the session, he pointed out that Diamondback will be a multi-personality IDE (with the three personalities out of the box — Delphi/Win32, Delphi/.NET, and C#/.NET). And personalities will be part of the Open Tools API. (Not sure if that’ll be totally cleaned up for Diamondback, but it’s there.) Open Personality API?

So, whether Borland writes a Mac personality or not (and mum’s the word on that so far, so you can still hold out hope), somebody else might be able to. This is a pretty darned intriguing concept, I think.

The features are coming! The features are coming!

I spent a while grilling Danny about what will be in Diamondback for Win32, and what will sneak in later. Here are the results. (Tabular format makes the holes a little more glaring, doesn’t it? Still, this is good stuff to hear word on…)

Feature Diamondback Diamondback+1
‘strict private’ and ‘strict protected’ in Win32 Yes
‘static’ keyword in Win32 No Forgot to ask
Class constructors in Win32 No Forgot to ask
Nested classes in Win32 Yes [Note 1]
Records with methods in Win32 No Yes [Note 2]
Operator overloading in Win32 No Yes [Note 3]
Custom attributes in Win32 No Forgot to ask
Generics in Win32 No Researching
Multicast delegates in Win32 No Researching

Note 1: Yes, they’ll be there, but you won’t be able to use them in published sections because RTTI won’t be able to cope with them.

Note 2: Records with methods will be there primarily because they’re a prerequisite for operator overloading.

Note 3: Will only be supported for record types.

Usual disclaimers about forward-looking statements, but if Danny thinks it’s going to be in a release, I’d say the odds are pretty darned good…

Danny’s “What’s New in the Diamondback Compiler” session

Covering Danny Thorpe‘s session about what’s new in the Diamondback compiler.

I’ve already posted several times about what’s new in Diamondback. I won’t duplicate any of that information here. And Danny was only talking about the compiler. Not about refactoring, not about databases, not about ASP.NET, not about ECO, not about VCL. Just the compiler.

Therefore, this is going to be a long post.

(Also, be sure to see my next blog post, with stuff Danny didn’t say during the session. In particular, when are some of the other Delphi/.NET language features coming to Win32?)


  • Loops over the elements of a collection, without actually requiring you to care about indexing into the collection
  • When is it useful?
    • When you don’t care about the index, but just want the elements
    • When you don’t want to do an indexed loop because getting a count and/or doing random access is expensive (linked lists, database tables)
    • Danny gave the example of asynchronous retrieval (a background thread is going out and getting the next element before you ask for it), which doesn’t work so well with array indexes
    • Unordered / non-indexed data (sets, hashtables)
  • You should think of the returned data as read-only. Do not change the collection while iterating. (You can change properties on the object you got, but don’t try to replace the object in the collection)
    • Modifying the underlying collection should cause the enumerator to throw an exception the next time you call MoveNext. (So don’t modify the collection while you’re iterating!)
    • Win32 implementation may allow writeback someday, but it might be full of gotchas. Assume that you should never modify the collection while you’re iterating it.
  • Because you treat the data as read-only (you are treating the data as read-only, right?), an enumerator should give you a stable view of the data.
    • That means that, if you call Reset on the enumerator, and enumerate a second time, you should get the same data.
    • Not guaranteed to be in the same order the second time, though.
    • This is the theory. In reality, things like databases may not guarantee a consistent view, for performance reasons (because doing a read-only transaction might be cost-prohibitive). But if they did offer a consistent view, wouldn’t that be nice?
  • Syntax: for MyObject in MyCollection do { statement };
    • Why not foreach?
    • Because Danny was reluctant to add a new keyword
    • Besides, should it have been ‘foreach‘ or ‘for each‘?
    • As a mathematician, Danny asked, “Wouldn’t ‘for all‘ be better?”
    • So, no new keywords for this.
  • Supported in both Win32 and .NET
    • .NET uses IEnumerable if it’s there, otherwise looks for the code pattern
    • Win32 only supports the code pattern (probably because interfaces carry all that reference-counting baggage)
  • The collection is held in a temporary variable.
    • So, if you did for MyRecord in LoadCollectionFromDatabase do, you’d still be fast, because LoadCollectionFromDatabase would only be called once.
    • Much like the way the for loop already caches its maximum value.
  • Collection must fit one of these:
    • Implement IEnumerable (.NET only, for now)
    • Have a method called GetEnumerator
    • Be an array, set, or string (or enumeration? Danny didn’t remember for sure)
  • Mechanics: (very familiar if you already know how C# does foreach)
    • GetEnumerator returns an enumerator object, with MoveNext and Reset methods and a Current property
      • You start with no current element (on a “crack”, per Danny) and must call MoveNext to make the first element current
      • MoveNext returns False when there are no more elements (if the collection is empty, it returns False the first time you call it)
      • Reset rewinds the collection to the beginning. for..in does not call Reset.
    • The enumerator is freed automatically at the end of the loop (inside a finally block).
  • As I mentioned before, many classes in the VCL will support for..in.
    • Not all, because when generics come out, they’ll all be generics and it will only need to be implemented once
    • TList has TListEnumerator, even in .NET (and even though TList is just a thin wrapper around ArrayList), simply because Win32 does need TListEnumerator. (It’s there as a placeholder for code compatibility between Win32 and .NET, for code that uses the enumerator directly instead of using for..in)

Multi-unit namespaces

  • Goal: Put symbols into a specific namespace while preserving unit make logic (which works by having a relationship between the unit name and the file on disk) and the syntax and the package support
    • Relationship between assemblies and namespaces is many-to-many
    • There is no relationship between the namespace and the file on disk
  • uses Namespace; always refers to a namespace, never to a filename
  • In Delphi 8: (the old way)
    • Unit name = .NET namespace. This works as an interim solution, let’s get it shipped.
    • Problem: Exposes structure to the world. Component and library authors want to split code into multiple files, but that’s a pain for their customers.
    • More segregated than most .NET namespaces
    • Can’t inject code into a namespace you don’t control (not that this was necessarily recommended practice to begin with)
  • Diamondback: (the new way)
    • unit Borland.Vcl.Classes;
      • Rightmost segment is dropped. Types in this unit end up in namespace Borland.Vcl. (Borland.Vcl.TList)
      • ‘Classes’ is still there behind the scenes (buried in the metadata), but it’s not in the C# programmers’ face
    • uses Borland.Vcl.Classes;
      • Compiler can find file on disk by appending ‘.pas’.
      • Compiler can also find this unit within a package, and only pulls symbols for that single unit.
      • This is using a unit, not a namespace.
    • uses Borland.Vcl;
      • This is using a namespace.
      • Only works for code that’s already compiled into an assembly. Cannot work for static linking (because there’s no way to find the .pas file).
      • Wide open; pulls in everything in the namespace, not just one unit.
    • uses Borland.Vcl.*;
      • This is either using a namespace, or using multiple units specified by a filespec. They amount to the same thing.
      • Works against code in packages, and against source/dcus.
      • Be careful. If you’re going against source, this loads a lot of symbols from disk. (Compilation will be slow.)
      • The smart linker will be even smarter for this. If a unit is dragged in by the wildcard, but you don’t ever reference any of its types, then its initialization section will not be compiled into the EXE.
      • This is here for completeness. Danny doesn’t recommend using it normally.
      • Danny didn’t talk about this, but I imagine it would be a bad idea for Borland.Vcl.Classes.pas to say using Borland.Vcl.*;
      • Globals (global procedures/functions, global constants, global variables) are as discussed in my earlier post: Borland.Vcl.Classes.MyGlobal becomes Borland.Vcl.Units.Classes.MyGlobal.
        • The ‘Units’ is stuck in there because Delphi already allows a global named ‘Classes’ inside the unit named ‘Classes’. (News to me! I thought that was a compiler error for duplicate identifier.) To avoid breaking it, they stuck an extra level in there.
    • Simple unit names (no dots in the filename) are an exception to this rule. When the rightmost segment is in fact the only segment, it’s not dropped.
      • But if the project has a default namespace, it’s considered part of the filename.
      • So if the default namespace is Borland.Vcl, and the file is Classes.pas, then the “fully-qualified filename” (I’m not sure if this is Danny’s terminology) is considered to be Borland.Vcl.Classes.pas, so the Classes is dropped.

A couple of interesting digressions

  • Arrays in .NET can’t be preinitialized, so if you have a global const array, it’s actually initialized in code in the unit’s class constructor (I’ll have to reread the IL book — I know there is global data, since that’s where strings are stored, but evidently it can’t be used for arrays)
  • Note on unit initialization sections: If you have circular unit dependencies, the initialization order may not be the same between Win32 and .NET. This is because, when unit A touches something inside unit B, unit B’s class constructor will fire immediately in .NET, whereas in Win32 the initialization section might not fire until later.

Function inlining

  • Involves persisting the compiler’s node tree to the .dcu. Not been done before.
  • Can produce significant speed boost in certain cases
  • Caution: Code bloat risk!
  • Not quite like C macros, because these aren’t textual replacements — instead, it grafts nodes into the syntax tree (it’s after parsing, not before)
  • Works for most anything: procedures, functions, methods, and local procedures
  • ‘inline;’ directive, looks like a calling convention
    • Didn’t Turbo Pascal have this a long time ago? Maybe it was just for assembler…
  • Works for both Win32 and .NET (and may beat the JIT compiler in some cases, since the JITter goes for fast optimizations and low impact)
  • The ‘inline’ directive is just a suggestion to the compiler. The compiler can disregard it if it thinks you’re wrong or stupid.
    • Could decide it would make the call site too complex (by needing too many temps and blowing your registers)
    • The same procedure may be inlined at some call sites and not others
  • This is still a top-down compiler. It must see the body of the inline procedure before it’s used.
    • In some cases with circular unit references, the compiler disables inlining on purpose.
  • Doesn’t work for procedures with an asm body, because that doesn’t emit compiler tree nodes
  • Spliced into node tree before optimization
    • Generates temps for stuff that’s passed into the procedure
    • Then probably optimizes those temps right back out
  • Restrictions on inlining stuff that access class privates
    • Hard and fast rule in .NET: Can’t inline these into another class
    • May relax this rule in Win32 code, post-Diamondback
  • Best practices:
    • Use only for very small functions
    • Always test performance and measure the benefit — don’t assume it will speed up your code. It may slow it down!
  • Caveats:
    • Hidden temps; burns registers and stack
    • Can actually hurt performance
    • Causes unit dependency brittleness: users depend not just on your interface, but also your implementation. May not matter much for our code, but for this reason, expect Borland to be very conservative about using inline within the RTL and VCL!
  • Since the procedure isn’t always inlined, the compiler actually does emit a “real” procedure body as well
    • @ operator returns this
    • Presumably the smart linker is vigilant, and if every call is inlined, the real body will be eliminated
  • {$INLINE ON/AUTO/OFF} directive
    • Affects the call site
    • AUTO = compiler picks “small routines” to inline. Per Danny, this option is “scary” and will never be the default.

Miscellaneous new features

  • MOVZX (Optimization for P4/PIII processors. Will cause smaller/maybe-slightly-faster code on newer processors, at the expense of running slightly slower on older processors)
  • UTF8 and UCS2 source for both Win32 and .NET
    • Danny, weren’t you going to show some amusing examples of this?
    • No Unicode allowed in published properties or methods (I think this restriction is Win32-only)
  • Improved overload discrimination (can overload on ‘type of’ types)
  • Forward declarations for record types (.NET only)

Coming in Diamondback+1

  • Whidbey support (well, yeah, you can compile against Whidbey in D8, but you can’t consume generics so you’re really a second-class citizen)
  • Generics (for .NET; see below)
  • May do something along the lines of partial classes
  • Records with methods for Win32?
  • (Also see my next blog post)


I had a lot of questions, and they answered most of them right away:

  • Transactions? Yes, full transaction support including rollback and disaster recovery
  • Client/server? Yes (but also supports local in-process)
  • SQL? Yes (SQL-92)

Here’s the rest of my notes:

  • Short answer: Yes, this is a real database.
  • ADO.NET and BDP drivers
  • SQL-92 entry-level compliance
  • When running locally, can run in-process (remote is TCP/IP)
  • Transaction support
    • Log-based, crash recovery, rollback
    • Row-level locking (plus table locking if you want it)
    • Support for read-only transactions
    • High-availability support (automatic and manual failover, incremental backup)
    • Disaster recovery
  • Stored procs, user-defined functions, triggers are coded in .NET
    • Direct access to transactional context (you’re running in-process on the server)
  • Managed code protects the database kernel
  • Single DB kernel for multiple editions
    • A lot of vendors prefer to have a “server kernel” and a “micro-kernel” that aren’t quite compatible in one way or another
    • NDataStore doesn’t; local, server, and high-reliability server all have the same disk structure, identical protocol, same transactiton management, same SQL, etc.
  • All this in a single, 1.1 MB DLL
    • That’s for ADO.NET. If you want BDP, there are additional DLLs.
  • If you want to do client/server, there’s a client assembly available that’s even smaller.
  • Performance: Managed code doesn’t make it slower.
    • 64-bit support
  • Licensing: Same as JDataStore ($60/seat for desktop version; $500/server for normal server; $1000/server for high-reliability server)
  • Faster than mySQL in most benchmarks
    • Note that mySQL does not support disaster recovery (doesn’t force writes to flush to disk immediately, so a commit isn’t really a commit), and NDataStore is still faster in several benchmarks
    • Obviously, these are vendor benchmarks, so salt grains are required
  • Single-file data store
  • Zero-admin database: keeps transaction logs only as long as needed (no need to truncate or anything messy like that)
  • Next steps:
    • Apply for NDataStore field test (drop off a business card, or e-mail ggoldfield at borland dot com)
    • Will need to sign an NDA, so you may not see me blogging about it 🙁
  • Interesting note: They actually maintain a single Java codebase, and run a tool to translate into C# with every build!
    • 95% of code is shared
    • Other 5% is .NET platform support (like ADO.NET), and written in C#
  • No Delphi/Win32 client yet, but they’ve broken ground on this and are evaluating it

Sigh. If only they’d had this on the market a month ago. The folks back home (especially John and Jeff) will know what I mean.

Opening session and opening keynote

I’m posting this a little late, since Nick already posted about this stuff. I’ll just add on to what he had to say.

Opening session (Sunday night)

There was the obligatory light show (the lightning effects were pretty cool; I wonder what kind of equipment they needed to make that work), techno music (they should sell the soundtrack), and abstract computer-generated videos (I particularly liked the mood-lit clocks and the plasmafied ASCII). It was interesting to watch DavidI getting bouncy. And the dancing girls were an interesting bit of spice, with their 80’s hair, and what couldn’t exactly be called miniskirts — they were really more like mini denim loincloths.

This is Dale’s sixth BorCon, DavidI’s 19th, and DavidI’s mom’s first. (He apologized to her about the dancing girls. It’s not clear whether he actually even knew about them.)

The new buzzword is “Software Delivery Optimization”, which extends ALM. They basically make it a continuum:

  • Build software (Delphi, etc.)
  • Build software right (ALM: CaliberRM, StarTeam, etc.)
  • Build the right software (SDO)

The basic idea is to extend the process up to the decisionmakers, and make sure they’re involved in knowing what the costs and risks are of changes. I’ll be interested to see what all they come up with for this.

They pointed out that people have tried this before, with things like CASE tools, but those focused on the business needs and not the code. Borland, being Borland, focuses on the code and grows from there.

I’ll have to see if I can drag our director of development to BorCon next year. There’s a fair bit of stuff for him to chew on.

Opening keynote

I agree with Nick: San Jose Taiko’s performance was very cool. The song they started with was called “Matsuri”, which means “festival”.

More about SDO, including the skit. I liked the programmer’s comment, when the “project manager” came over and said “good morning”: “Why is it a good morning? It’s 8:28 am. Talk to me at 10:00.”

The interesting statistics they cited:

  • 30% of software projects are canceled before completion
  • 54% are over budget
  • 66% are not considered successful by their companies
  • 90% were delivered late

And the big reason is that management makes decrees that have a huge impact on the timeline (their example was “we need a new feature, and we need to move the deadline up by a month, and we need to take five people off the project” — the end result was “we can do it, but we need to keep those five people on the project”).

They also talked about new projects on the horizon: Project Themis (team infrastructure, coming first half of 2005), Project Hyperion (visibility and predictability, coming in 12 to 18 months), and Project Prometheus (enterprise resource planning, also coming in 12 to 18 months).

Lots of buzzwords. I hope they offer some good sessions next year that give us a solid sense of what this stuff means.

Allen Bauer’s “What’s New in Diamondback” session

Covering Allen Bauer‘s BorCon session on what’s new in the Diamondback IDE.

Looks like Nick wasn’t doing a live blog of this one, so I’ll post my comments.

Consider this to be a follow-up to my notes from John Kaster’s preconference tutorial on Diamondback. I won’t cover any of the overlap here.

Allen started by saying he would mostly cover IDE features. Fair enough. Danny (who knows me by name, BTW) will cover the compiler stuff this afternoon, and the “Meet the Team” tonight will cover even more.

He’s talking about features in a product that’s not finished yet, BTW. Ship dates haven’t been decided yet, features haven’t been decided yet. This is a very new thing for Borland. (He mentioned that he’s allowed to talk about anything except release dates, product names, and bundling/pricing. Very cool.)

  • Debugging features
    • Debug .NET code hosted within a Win32 process
      • May not be able to seamlessly “Step Into” from Win32 code into .NET code
    • AppDomain support in Module view
    • Better stack traces in Win32 for frames without debug info. Think EnumFonts; you didn’t used to be able to see a stack trace when Windows calls your callback function, but now you will.
    • Locals view and stack frames: Yes, it will work even after exceptions! Woot!
    • Look at different threads’ call stacks and Locals views
    • Same operations in both Win32 and .NET debuggers
  • Refactoring
    • No distributed refactoring in Diamondback, but they have underlying architecture to support it
    • Refactoring works across projects within a project group, even between C# and .NET
    • Based on actual uses list, not just what’s in the .dpr
    • Only works within current IFDEFs
    • Sync edit: Can Tab through distinct symbols
    • “Live Templates” (Ctrl+J templates with this Tabbing to fill gaps) won’t be in Diamondback but are on the plate
  • IDE is theme-enabled! Transparent panels & checkboxes, even at design time
  • Component palette is drag-and-drop even in VCL
    • Marketing has always claimed that Delphi supports “drag-and-drop” GUI development, but Allen commented that it was really “click-and-click”
    • Now you can actually drag a component off the palette and onto the form
    • Yes, click-and-click development still works. So do double-click and Shift+click.
  • Component Palette in Diamondback probably won’t support one component in multiple categories, yet.
  • Object Inspector shows properties for files and projects. Can rename a file using Object Inspector, even if it’s not saved yet!
    • Carries rename through to the disk file
    • Doesn’t do anything to the old .dcu file (alas)
  • Rename Class refactoring even reaches into the .dfm
    • Also, if you rename a form class, it will rename the global Form1 variable in that unit
  • History view: If you had 30 revs of a file, and you change the setting to only save 10 revs, it will not delete the old revs, and will continue to keep 30 revs for that file.
  • Borland devs use AutomatedQA apps (I should’ve asked which apps)
  • IDE uses VirtualTreeView