Non-trivial example of unmanaged exports

Chris Bensen just posted an example of passing non-trivial types — strings and interfaces — between Win32 and .NET using unmanaged exports. If you’ve ever wondered how to use unmanaged exports to pass anything more complicated than integers, this post is for you.

(And in case anyone was wondering, I’m the one who sent the question he’s responding to.)

Now I just need to find some spare time to tinker with this, because it’s prompting a number of questions in my mind. Can I put some attributes on the .NET interface to get rid of the IDispatch requirement? Do you have to use a var parameter to return an interface, or can you use an out parameter or a function? What happens when you throw an exception in .NET and catch it in Win32, and vice versa? How easy is it to pass callbacks (delegates) back and forth?

Of course, I don’t know if I’ll get that tinkering done very soon, for two reasons: (1) I’ll have to play with it on a work PC (my home PC only has Turbo Delphi for Win32), and (2) next month is NaNoWriMo.

DGrok 0.8: Faster and more configurable

DGrok 0.8 is now available.

You can now configure compiler options (IFDEFs, IFOPT, etc.) from the GUI. Your settings (compiler options, search paths) are saved from session to session. And the parser is probably around 20-25% faster (thanks to Brian for profiling and finding the hot spots).

Here’s the list of new GUI features:

  • FIX: .dproj-file exclusion got broken in 0.7. Made it work again.
  • CHG: Rules are now shown in a treeview instead of as hyperlinks. And the “Parse” button is back (along with its Alt+P accelerator).
  • ENH: Much-requested: Added “Compiler Options” page, where you can specify what to do with IFDEFs and IFOPTs.
  • ENH: Much-requested: Save settings (including search paths and compiler options) on exit, and reload on program startup.
  • ENH: Sped up parsing.
  • ENH: Show elapsed time after parsing.
  • ENH: When running an action, show a summary node with the action name and the number of hits.
  • ENH: Give a better error on duplicate filenames (it now shows these errors in the tree, instead of popping up an “unhandled exception” dialog).
  • ENH: Added Find (Ctrl+F) and Find Next (F3) to source-edit boxes.

New “find stuff” actions:

  • ENH: Added FindAsmBlocks action.
  • ENH: Added FindVariantRecords action.

Things you’ll see if you’re using the parser in your own code:

  • FIX: ToCode() was throwing exceptions on nodes with a trailing, empty ListNode (commonly seen with “procedure of object”). Fixed.
  • CHG: All node-type properties now end with the word Node.
  • ENH: Add a Projects collection to CodeBase.
  • ENH: Added ParentNode property to AstNode.
  • ENH: Added ChildNodes property to AstNode. When you need to iterate a node’s children, this is easier than using Properties. It also skips over nulls.
  • ENH: Added ParentNodeOfType<> method to AstNode.

DGrok 0.7 released: putting the parser to work

DGrok 0.7 is now available.

What’s new:

  • FIX: Initialized variables and typed constants now accept the empty list, ‘()’. The grammar doc already indicated this, but the code didn’t do it. It does now.
  • CHG: In package files, allow assembly attributes after the ‘requires’ and ‘contains’ clauses, rather than before. (I know they can come after, and that’s what I currently need to parse. If they can come other places, I’ll have to extend this later.)
  • ENH: Added support for undocumented {$SOPREFIX}, {$SOSUFFIX}, and {$SOVERSION} compiler directives.
  • ENH: The “search paths” box now allows a semicolon-delimited list of directories. As a side effect, there’s no longer a Browse button.
  • ENH: The “search paths” box now lets you put “\**” at the end of a search path. This means “include subdirectories”. Without that, it only does one specific directory. In a semicolon-delimited list, this gives you the ability to recurse into some directories and not others.
  • ENH: Added features to find all “with” statements, nested methods, and global variables in the code.

You’ll notice that there’s no longer a “Parse” button; instead it’s a hyperlink. This means that it no longer has a keyboard accelerator. Sorry — I miss it too. I’ll probably add a shortcut at some point.

There are five hyperlinks in this version:

  • Parse All: parses all the source files in the search path(s). You have to do this before any of the other links will enable.
  • Show Parse Results: resets the display to show the results of Parse All. This is only useful after you’ve clicked one of the later links, and want to get back to the thing that shows which files did and didn’t parse successfully.
  • FindGlobalVariables: Gives you a list of all global variables.
  • FindNestedMethods: Gives you a list of all nested methods (procedure inside a procedure, etc.).
  • FindWithStatements: Gives you a list of all the “with” statements.

Those last three are the bare beginnings of DxCop, a tool I’m eventually planning to write on top of the DGrok parser. DxCop (named after the .NET Framework’s FxCop) will analyze your code, looking for places where you aren’t following best practices. FxCop looks at your compiled code, while DxCop will look at the source, but it’s the same idea.

For those interested in looking at the code, all three “find” features will show you how to put the Visitor pattern to use. They’re only the beginning. Feel free to suggest other such features to add (or to send me code) — but keep in mind that DGrok doesn’t yet have symbol-table support, so anything involving “find references” isn’t possible just yet.

Mere-moments guide to creating custom ring tones for your Verizon RAZR V3m

I finally broke down and got a cell phone. Then I had a challenge: how to customize the ring tone.

See, I’ve always insisted that I won’t get a cell phone until I can set the default ring tone to Carmina Burana. I couldn’t lose face in front of my geek friends by admitting that I couldn’t figure out how to do it.

But apparently Verizon wants to make it hard for anyone to get ring tones without giving more money to Verizon, which I refuse to do on general principle. (And besides that, I kind of doubt that they sell “Carmina Burana” as a ringtone.)

It took me over nine hours, but I got it. (Stubborn little cuss, ain’t I?)

Now I’m sharing. Here’s how to make custom sound clips into ring tones for your Verizon RAZR, without voiding the warranty, in mere moments or less.

On not voiding the warranty

The RAZR supports custom ring tones. It’s Verizon that makes it hard. From what I can tell, they deliberately disable some of the phone’s functionality before they sell it, presumably so they can make more money selling their own ringtones.

There’s plenty of information out there about ways to re-enable the functionality, via SEEM edits and other forms of hacking the phone’s firmware. I didn’t do any of those, because they would have voided the warranty. Even though I’m not happy with Verizon for disabling features, I’m not going to re-enable them if I’m the one that’s going to get screwed. So I stuck to strictly non-hack methods.

What didn’t work

I bought the Media Essentials kit with my phone, so I tried hooking the phone up to my computer with the USB cable. Of course that didn’t work. You can use their software to transfer “songs” to the phone, but they differentiate between “songs” and “sounds”, and “songs” can’t be used as ringtones. And no, they don’t let you mount the phone as another drive on your computer so you can transfer “sounds” to the phone.

The phone can record “sounds” through its built-in microphone, and use them as ring tones. So I tried holding the phone up to the computer speakers. Result: volume nearly inaudible, quality far too crappy and under-watery. I didn’t expect this to work very well, but it was worse than I thought. So I’m not doing that.

I found some message board posts that suggested using the sound-recording capability to make a one- or two-second file, move it to the flash drive, take the flash drive out of the phone, plug it into a PC, and replace the files with identically-named files (including their weird extension) that are actually MP3s. This sounded promising, and it did get me sound files that play beautifully on the phone. But it still was a no-go. Why? Because apparently Verizon has caught on to this trick, and they’ve made it so that, as soon as you move a sound recording onto the flash drive, it’s forever unavailable to be a ring tone. Once you move it, it’s locked out. Even if you move it back to the phone’s internal memory, they won’t let you use it as a ring tone. Drat. (I spent $15 on an SD card reader, too. Isn’t it amazing how we Americans will spend money to avoid spending money?)

What did work

As of October 2007, here’s something that does work. (If you find in the future that it doesn’t work anymore, please leave a comment, so this page stays relevant.)

What you’ll need
  • A supported phone (see below)
  • Verizon service (these steps are very Verizon-specific)
  • Sound-editing software like GoldWave or Audacity
  • A sound file on your computer (if you need to rip a song from a CD, do that first)
  • One of the following:
    • A plan where you can get 250 (or whatever) incoming text/picture messages before paying extra, OR
    • A plan where you get unlimited incoming text/picture messages, OR
    • The willingness to pay Verizon 35 cents to get your custom ring tone onto your phone
  • Mere moments or less

I’m guessing that this should work on any phone that can receive sound attachments to picture messages. Here’s a list of phone models that have been confirmed as either supported or unsupported. If you have updates to this list, feel free to post a comment and let me know.

Supported Not Supported
Note: Aside from the RAZR V3m, these lists are based entirely on user feedback, and are not guaranteed to be correct. When in doubt, try it yourself.
The short short version

If you know what you’re doing, here’s the short short version: save the file as a 30-second-or-less, 22KHz WAV, and use Verizon to send it to your phone.

The rest of this document takes you through this process, step by detailed step. So let’s get started.

Step 1: Create a Verizon vzwpix account

First, go to (You’ll get redirected to another Verizon URL, but in some places they still refer to it as vzwpix.)

Find the Login box on the left side of the page, and click the “Register” link. Then go through the steps to register. (This step can’t exactly be done in mere moments, but that’s Verizon’s fault, not mine.)

You’ll probably need to have JavaScript enabled to go through the registration process. Yes, this is stupid.

Once you’ve created an account, you’re ready to start moving a sound file to your phone. But first, you need to convert your sound file to have the proper length and format.

Step 2: Create the sound file

Your sound file cannot be in MP3 format. (Even if you rename the extension, Verizon will see that it contains MP3 content, and block the file. It’s not clear why they won’t accept MP3 content, since the phone can clearly play it.)

WAV format is what worked for me. It also has to be 30 seconds or less, and have a sample rate of 22.050 KHz or less.

I used the GoldWave trial edition to trim my sound file to less than 30 seconds, add a fade at the end, and save it as WAV. In GoldWave’s Save dialog, I set “Save as type” to “Wave (*.wav)”, and “Attributes” to “Microsoft ADPCM 22.050 KHz, 4 Bit, Mono”.

I’ve also heard a lot of good things about the Audacity audio editor, which is free. I’ve used GoldWave before; it has a lot of features but isn’t always quite intuitive. I’ve never used Audacity, so I don’t know what it’s like.

Mac users: see NL’s comments on Mac software for editing the sound file.

Step 3: Send the clip to your phone

Now that you’ve got a 22KHz WAV of 30 seconds or less, it’s time to get it onto your phone.

You’ll almost certainly need JavaScript enabled for this part.

Start by logging into your account. Once you’re logged in, you’ll see the “Picture & Video Messaging” page.

Screenshot of the Picture & Video Messaging page you see after logging into

In the bottom center of the page, you’ll see a red “Upload Media” button. Click it once. You may have to wait a long time for anything to happen, but if you click it more than once, the site gets confused, so don’t do that.

Eventually you’ll see this:

Screenshot of the Upload Media box you see after clicking the Upload Media button

Click the top Browse link. Browse to your WAV file and click OK. Then click the red “Upload” button, and wait a bit more.

After the “Upload Media” box closes, you’ll see your sound file in the “Uploads” box on the left. In this screenshot, I’ve uploaded two sound files. Notice that Verizon assigned them nonsense names, which I haven’t found a way to change (though I haven’t looked too hard).

Screenshot of the Picture & Video Messaging page after uploading sound files

Now, on the right side, notice the tall red box with stuff in it. A little over halfway down, there’s some white-on-red text that says “Drag audio here”.

Find your sound file on the left side, and drag it onto that “Drag audio here” text.

If you did it right, you’ll now see two tall red boxes. Ignore the second one.

Screenshot of the Picture & Video Messaging page after dragging the sound file to the right side

The first tall red box will now have a Play button (which for some reason is red, rather than the traditional green) in place of “Drag audio here”. You can use this to make sure you got the right audio file (since you can’t tell which one you’re dragging because of the nonsense names they assigned). If you got the wrong sound file, click the “X” (delete) button just above the tall red boxes, and try again.

You can just ignore the “1/1, 5 sec” stuff above the red boxes. I left that stuff alone and everything worked.

Okay, so you’ve got two tall red boxes, and the first one has the right sound file in it. You’re almost done. Click the red “Preview & Send” button in the lower right.

Screenshot of the Preview and Send screen

When the “Preview and Send” box opens, a 5-second clip of your sound file will play in your browser. Don’t panic! They didn’t lose the rest of the file. They have the whole file, it’s just that this screen only plays a 5-second clip.

In the “Send To” box, type your 10-digit cell phone number, without any punctuation. For example: 4025551212

Tip: If you just got your cell phone and don’t remember your phone number yet, here’s how to check it on a RAZR V3m. You can go to Menu > SETTINGS & TOOLS > System > Device Info > My Number. Or you can click the Voice button to the right of the display, wait for the menu to come up, and say “Check my phone number”.

Now type a subject line and click Send. The sound file will be sent to your phone, attached to a “picture message”.

Step 4: Save the clip as a ringtone

It will take a minute or two for the picture message to be sent to your phone. When your phone bleeps and tells you you’ve received a new picture message, select “View Now”, and your sound file will play on your cell phone, in all its glory.

You’re not done yet. Select “Options” (right soft button) and “Save Sound”. Give it a name and OK. It will pause for a moment, and then display “Sound saved”.

And… you’re done. Now you can select the sound as your master ring tone in the usual way, or assign it to individual contacts. Custom Verizon ringtones in mere moments or less.

Happy ringing!

Also in the Mere Moments series: Mere-Moments Guide to installing a Subversion server on Windows

DGrok 0.6.5 released: fixed $IFNDEF

DGrok 0.6.5 is available. It’s a minor bugfix release, hence the half-version.

What’s new:

  • FIX: {$IFNDEF UnrecognizedSymbol} was ignoring its contents. That’s the right thing for {$IFDEF} but not for {$IFNDEF}. Fixed.
  • ENH: Allow attributes in package files.
  • ENH: Add support for {$INLINE} and {$STACKCHECKS} undocumented compiler directives.

Ordinarily I would have gotten more done in an evening. It’s all Sam’s fault for distracting me with his Erlang talk last night.

DGrok 0.6 released: enter Visitor

DGrok 0.6 is now available for download.

New in this version:

  • All documented compiler directives are supported (plus one undocumented one, {$VARSTRINGCHECKS})
  • Added support for hard casts to the file keyword, e.g., file(AUntypedVarParameter)
  • Changed PointerType to allow any type, not just QualifiedIdent. This allows things like ^string.
  • Fixed runtime cast error when a program or unit has a beginend block or asm block for its initialization section. For simplicity’s sake, all init sections are now represented by InitSectionNode.
  • Ignore .dproj files when you search for *.dpr.
  • Ignore {$DEFINE} and {$UNDEF} when they occur inside a false {$IF...} block. Oops.
  • Ignore unrecognized compiler directives if they’re inside a false {$IF...} block. This lets me ignore, say, FreePascal compiler directives (since they usually occur inside {$IFDEF FPC} blocks).
  • Allow dotted names for packages (in the “Package Name” line).
  • Contains clauses now support “in filename” specifiers. (Actually, contains clauses are now handled by the same rule as uses clauses.)
  • Added support for [assembly: Attribute(...)] syntax. Yes, mostly I’ve been focusing on parsing Win32, but I have some need to deal with .NET code as well.
  • Changed it so that not-yet-defined symbols are treated as false by {$IFDEF} rather than explicitly raising an error. The error served its purpose earlier, but I reached the pain point and shut it off.
  • Added a Visitor class.

Whew. Not bad for one evening’s work.

I think Visitor will really be the way you put the parser to work. I was hoping to get some examples into today’s build, but I ran out of time. Play around if you like.

8.3 backward compatibility, or, why *.dpr returns .dproj files

CodeGear, it turns out, chose poorly when they picked .dproj as the extension to replace .bdsproj.

If you have Delphi 2007 and have tried to run DGrok 0.5, you’ve already seen what might look like a bug in DGrok: when the file masks include *.dpr, DGrok looks at all of your .dproj files as well. Obviously this is wrong, right? I mean, you searched for *.dpr, not *.dpr*. And since .dproj files are XML (which doesn’t so much fit with the Delphi grammar), every .dproj file is listed as a parse failure.

The thing is, it’s not really a DGrok bug (although it is broken, whoever’s fault it may be, and I will have a fix in 0.6). It’s a combination of a Windows backwards-compatibility feature, and someone at CodeGear not knowing about that Windows backwards-compatibility feature (and making a bad choice as a result).

First, a bit of background. Apparently there exist applications — perhaps vital line-of-business apps that run Fortune 500 companies — that still, 12 years after Windows 95 was released, can’t deal with anything but 8.3 filenames. That isn’t too surprising, really; if the app still works, who’s going to mess with it? But (here’s the kicker) some of these 8.3-only apps use the full Win32 API to search for files. I haven’t quite puzzled out how that combination comes to be — it’s not like there was a time when there were Win32 APIs but not long filenames — but after a few years of reading Raymond Chen‘s blog, I’m forced to believe that such apps really do exist somewhere (and I suspect I really don’t want to know how many there are).

Every file has a long filename and and a short (8.3) filename. (I think you can turn off 8.3 filename generation, but nobody does, because 8.3 filenames are handy for apps that can’t deal with spaces in path names.) So if you create a file called, say, MyProj.dproj, it will also be given an 8.3 name like, say, MyProj.dpr.

And because of the possibility of 32-bit 8.3 apps, whenever you supply a search mask, the 32-bit file-search APIs (FindFirstFile, etc.) will match that mask against both the long filename and the short filename. Which means, if you search for *.dpr, Windows will happily give you both .dpr files (whose long and short names both match *.dpr) and .dproj files (whose short names match *.dpr, even though their long names do not). And it doesn’t even have the decency to return the short filename when that’s what it matched — no, it returns the full long filename, the one that specifically didn’t match the mask you gave it.

Yes, it’s all very weird, and the Windows behavior isn’t exactly friendly. Then again, it’s been that way for, let’s see, 12 years. This behavior is a known, a constant. And it’s up to us, as software developers, to deal with it. It’s just another developer tax we have to pay, like it or not.

Which means that, CodeGear, you blew it with that .dproj extension. By choosing something that’s 8.3-ambiguous with .dpr, you made life hard for every third-party vendor who wants to do anything with Delphi source files. Here’s the rule (for future reference): When you invent a new extension, its first 3 characters should never be the same as an exactly-3-character extension you already own.

I just coded a workaround for 8.3 compatibility, which will be in the next release of DGrok. Specifically, I added a loop that looks at each filename and grabs its extension. If the extension is “.dproj” (case insensitive), I skip over that file. (This means that if you’re silly enough to type in a mask of “*.dproj”, the new version won’t find any files at all.) Clumsy? Yeah. But .dproj files are the only ones (currently) likely to come up as false positives, and they’ll never contain valid Delphi code (unless, of course, you specifically rename them to taunt me). And an awkward fix is better than a broken app.