Security permissions in Windows Forms #.NET #sharplayout

Whenever I'm writing a custom WinForms control, unless I have some reason to do otherwise, I always try to write it so it'll work in a partial-trust environment, like embedding the control inside a Web page (Internet zone), or running a program from a network drive (Intranet zone).

Windows.Forms does not tend to make this easy.

So I'm working on my FlowLinkLabel class, right? There's some text (blue and underlined, of course (well, by default anyway)), there's possibly an image. And depending on the length of the text string, there may be a bunch of empty space to the right of the text, or below the image. I don't want the control to react to clicks in that empty area. Only clicks on blue underlined text (or on the image) should count. Okay.

That means I need to do my own click tracking; I need to override OnMouseDown and OnMouseUp. No problem. I also need to process WM_CANCELMODE. Problem!

If the user presses the left mouse button, and then (without letting up on the mouse button yet) Alt+Tabs away, or presses the Windows key to bring up the Start menu, or some ill-behaved application steals the focus because the user just got an instant message, then my OnMouseUp handler will never fire. But I still need to clear my "_isLeftButtonPressed" flag. Granted, if I can't clear that flag, it will only cause real problems in certain pathological edge cases. But I don't like leaving edge cases open, even pathological ones.

As I learned back in my Delphi days, Windows tells me about all these rude happenings by sending me a WM_CANCELMODE message. Lovely, says I. I'll just override WndProc and — boom! Oops, there's a security demand on WndProc. If I override WndProc in FlowLinkLabel, then nobody in the Internet zone can instantiate FlowLinkLabel. If they try, they get a SecurityException.

The interesting part is this: The SecurityException is not thrown by the call to new FlowLinkLabel(). Instead, it's called when the Just-In-Time compiler tries to JIT the method containing the call to new FlowLinkLabel(). That is, it blows up when it's partway through trying to JIT-compile InitializeComponent. Which means the exception's stack trace is marvellously unhelpful; all I see is that an exception occurred in the Form1 constructor, on the line where it tries to call InitializeComponent. I never see anything that tells me which line within InitializeComponent is actually causing the problem, heavens no; so if this is the first time I'm trying to run my app under restricted permissions, I don't have a clue which third-party component is being ill-behaved. Nice, eh?

Back to FlowLinkLabel. I can't override WndProc if I want partial trust to work. I was getting started on an elaborate workaround, and then happened to be looking through Reflector and saw that Control.WndProc calls something named OnNotifyMessage. I fire up the help, and read that OnNotifyMessage is specifically meant for partial-trust scenarios! It can't modify or suppress the message; all it can do is watch the message going by. Which is exactly what I need for WM_CANCELMODE.

Lovely, says I. I'll just override OnNotifyMessage, and do the appropriate SetStyle() to tell WinForms it's there. Run it under the Internet zone — it runs! Hooray! Write some more code. Run and — boom!

Undo checkout. Was it my override of OnMouseDown? Try that again, see what happens. No exception. Well, I know it wasn't my override of OnNotifyMessage, I already tested that. Try again anyway, just to make sure. Hmm. Scratch my head. Finally go through all the motions again, testing much more often this time.

It's when I add an if statement inside OnNotifyMessage. An if statement that does nothing but read the message struct's Msg int property.

Open Reflector again. There's a security demand on the Message structure. It requires unmanaged-code permissions. Which I very much do not have in the Internet zone.

So they give me a method I can override where I can see all these messages flying by. But I can't see a blooming thing of what's in those messages. That is frustration in action, folks. It also smells like poor design — OnNotifyMessage exists only for partial-trust scenarios; the docs even go so far as to say you don't need to bother calling base.OnNotifyMessage(), which heavily implies that Microsoft will never, ever bother to put any of their own code into OnNotifyMessage. And aside from permissions, anything you can do with OnNotifyMessage can be done just as easily by overriding WndProc. OnNotifyMessage exists for one purpose, and yet, it's useless for that purpose.

Sigh.

I worked around it. The first time FlowLinkLabel.OnNotifyMessage is called, I enter a try..except, where I call another method that tries to read message.Msg (has to be a separate method, since it's the JIT throwing the exception — if I did it inline, my try..except would never finish getting JITted). If a SecurityException is thrown, I just don't watch for WM_CANCELMODE anymore, and those pathological edge cases remain open. But any time it turns out that the code can read message.Msg, it will process WM_CANCELMODE quite happily. (Then, of course, I short-circuit my implementation on later calls, so I'm not throwing an exception every time a Windows message is received.) Decent compromise, I guess.