markbetz.net

I use Microsoft’s SyncToy for nightly backups. In the past I’ve used many different solutions, from custom xcopy scripts to robocopy to the built-in backup program that started shipping with Windows back in the NT days. I also keep complete disk images for those rare occasions when a full recovery is needed. But for making sure that my daily work is saved I haven’t found a better alternative than SyncToy and a second hard disk.

SyncToy uses the Microsoft Sync Framework 2.0 to manage pairs of folders and keep them in sync according to certain rules. To keep some set of files safe all you do is organize them under one or more folders, then use SyncToy to define named folder pairs that match the original content folders with sync folders on the second drive. There are several modes, but I use the “echo” action, which propagates changes from the source folder to the sync folder. I have a scheduled task that runs every night to tell SyncToy to process all the defined folder pairs.

This setup worked great for months, or at least I thought it did. It’s one of those deals where I set something up, make sure it works, and then forget about it. In this case it turns out that the SyncToy command didn’t quite work as well as I thought. The problem was my Outlook.pst mail file. I have Outlook open all the time, minimized to the system tray, and when Outlook is open SyncToy can’t read the .pst file. When I happened to check the log I saw the read errors and realized that my voluminous mail file had never been backed up.

So, what I needed was a command that would close Outlook, run SyncToy, then restart Outlook. After Googling around and experimenting this is what I came up with:

1. @echo off
2. echo Synchronizing folders...
3. echo Shutting down Outlook.exe...
4. "c:\Mark Personal\System Tools\NirCmd\nircmd.exe" closeprocess outlook.exe
5. :wait
6. timeout 2
7. tasklist /FI "IMAGENAME eq outlook.exe" 2>NUL | find /I /N "outlook.exe">NUL
8. if "%ERRORLEVEL%"=="0" goto :wait
9. echo Outlook closed; executing SyncToyCmd...
10. "C:\Program Files\SyncToy 2.1\SyncToyCmd.exe" -R
11. echo Sync completed; starting Outlook.exe...
12. start "" /B /MIN C:\Users\Mark\Desktop\Outlook
13. echo Outlook.exe started; exiting
14. @echo on

This script first sends Outlook.exe a shutdown command using a nifty command line tool called NirCmd.exe, from NirSoft. You could also use taskkill, but NirCmd is a gentler method that gives Outlook a chance to properly close its data files, and if you know anything about Outlook, you know you want it to properly close its data files.

After telling Outlook to close the script loops until it can no longer find outlook.exe in the running task list, and then proceeds to run SyncToy for all active folder pairs, and then restart Outlook after it completes. Pretty neat, and I can’t take credit for any of it. Just some stuff cobbled together from bits and pieces of others’ wisdom around the net.

I installed a new main system drive last week, a Western Digital Caviar Black 500GB to replace the aging SATA 1.0 74GB Raptor that has been in my development box since 2005. The new drive is three times faster, and lots bigger. So, to celebrate all that empty space I started dragging out old games to see if I could get them to install and run. Just to set the stage: my main development system is a Core 2 E8500 with 8GB of DDR3 RAM, on an Asus motherboard. The graphics card is an NVidia GTS-250. Audio-wise I use the onboard Realtek for Skype, and an Audigy 4 Pro for games/movies/etc. The operating system is Windows 7 Enterprise 64-bit.

I had some successes and some failures. Rise of Nations, circa 2003, ran fine in widescreen with a little hacking. Age of Empires II, circa 1999, worked but the graphics were messed up, and widescreen/hi-res modes were unavailable. Railroad Tycoon 3, circa 2003, worked fine, including widescreen with a hack to the engine.cfg file, but when patched with the Coast-to-coast expansion it broke. But the backward compatibility award of the day goes to Empire Earth, designed by Rick Goodman of Age of Empires, developed by Stainless Steel Studios and released in 2001 by publisher Sierra. Empire Earth installed and ran flawlessly in 1920 x 1200 widescreen mode. Here’s a shot: The game ran perfectly in version 1.0 right out of the box. Worked perfectly after patching to 1.04 and then 2.0 as well. Sound worked, all available resolutions were supported, the graphics looked great, the game play was as I remembered. This is quite a feat for a nine year-old title. When Empire Earth was released the minimum system requirements included a Pentium II running at 350 Mhz., and at least 64MB of RAM. If you wanted to enjoy the game’s visuals you were requested to have at least 1024 x 768 pixels available. I think I have more pixels in some of my desktop icons now. Windows XP had just been released, and the hot video cards were the GeForce3 and the original Radeon. The other supported chipsets read like the walls in a technology mausoleum: 3dfx, Savage, Matrox, PowerVR. Network play? You’ll need a 28.8 kbps modem, pal.

As a developer myself I am thoroughly impressed anytime something this old installs and runs the way it was originally designed to. Microsoft gets slapped around a lot these days for being stodgy and unhip, not to mention monopolistic. But they don’t often get the compliments they deserve for getting the big picture right, establishing sensible and well-designed APIs, and then managing them well over the decades. The developers who worked on Empire Earth should be proud of themselves as well. It’s got to feel good to know that someone can take your nine year-old efforts, pop them onto a modern 64-bit DirectX 11 platform, and just have it all work. Well done, guys.

About a year ago I started work on a set of Silverlight libraries providing controls and classes that implement the primary features of a drawing application. Some of the results of that effort made their way into demos on this site, i.e. the ColorTools and GradientMaker demos. Eventually I also posted a partially complete demo of a drawing program, however not long after that I joined the J. Becker team, and have been heads down working with a small, creative group on some cool Silverlight media management stuff. The upshot of it all is that the drawing libraries, and the application they were designed to support, have been gathering dust.

This morning I received a request from Sweden for the source to GradientMaker, which I’m happy to share. I had previously released the source for the color controls in ColorTools, but the GradientEditor control hadn’t been added at that point. When I got around to implementing it I created a dependency on another small library which I hadn’t released, and so I put off adding the GradientEditor to what was publicly available. Prompted by this morning’s request I have whipped it all into shape and uploaded it to the site. See the download links at the end of the post.

The code is released under a new BSD license. In order to package it I ripped the projects for the two libraries out of the solution I created for the app last year, and placed them into a solution called SilverDraw. It’s a little aspirational at the moment, but I hope to add in some of the other drawing code I have written and release some updates to it soon. There is no documentation, but the code is pretty well-commented. If you have any questions feel free to email me.

Downloads: SilverDraw Assemblies, SilverDraw Source

Over the last couple of days I was working on a system that generates HTML emails. The email content is created by loading and rendering ascx controls that perform the value substitutions into a template, and comments had been included in the templates describing what data items each email required. This morning I realized that the comments were being included in the email bodies, so that using ‘View Source’ would display them. Not a huge deal, but it would be better not to have them in there, so I went looking for a method to strip the HTML comments. Everything I found seemed more cumbersome than necessary, so I threw together this simple recursive method to remove HTML comments from a string. It’s naive with respect to the question “what is a comment in HTML?” For my purposes a comment is the text between the <!– and –> tags, and that’s it. Posting this in case someone finds it useful.

1. private string StripHtmlComments(string html)
2. {
3. int open = html.IndexOf("<!--");
4. if (open > -1)
5. {
6. int close = html.IndexOf("-->");
7. if (close > open)
8. {
9. string newHtml = html.Remove(open, (close - open) + 3);
10. return StripHtmlComments(newHtml);
11. }
12. else throw new FormatException("The input HTML contains mismatched comment tags");
13. }
14. else return html;
15. }

Sometimes programming in .NET just drives me up a wall. Not that I would prefer to go back to the Win32 API and writing my own message loops, but when a framework is as all-encompassing and opaque as .NET is, it’s doubly important to make the internals work right and sensibly, and doubly likely that somewhere they will do neither. Case in point: I probably spent four hours today trying to inject some custom behavior into a WCF channel only to find out that the problem was the lack of a space in front of a comma. Or, more accurately, four spaces in front of four commas.

My goal was to use message inspectors to sniff out an authentication cookie from an incoming WCF service reply, and inject it into subsequent requests, as part of a piggyback authentication scheme with an ASP.NET website. I won’t say much about message inspectors. They let you get your hands on various parts of an incoming or outgoing message before it is handed to the client or proxy, however the case may be. They work as advertised and there are a lot of good posts on how to use them. Microsoft has one too… it’s just not good. If you want to read specifically about managing cookies using inspectors this is the post that got me started.

If you read any of those posts you’ll notice that to implement a message inspector requires also creating a custom behavior that implements IEndpointBehavior. It’s a very simple class that just creates an instance of your inspector class on demand. If you’re willing to deal with creating the behavior for every service and every channel manually you can stop there, since there are imperative ways to do this. What I wanted was to be able to specify the behavior declaratively, in the .config file where the endpoints are defined, so that it is added transparently to all channels when they are used. In order to do that you need one more class: a custom extension of BehaviorExtensionElement that gets registered in the system.serviceModel/extensions/behaviorExtensions configuration scope. This class creates the behavior, the behavior creates the inspector, and so on and so forth.

And that’s where the fun began. The syntax for declaring the extension element and behavior looks like this:

1. <system.serviceModel>
2. <extensions>
3. <behaviorExtensions>
4. <add name="extensionName" type="extensionType" />
5. </behaviorExtensions>
6. </extensions>
7.  
8. <behaviors>
9. <endpointBehaviors>
10. <behavior name="behaviorName">
11. <extensionName />
12. </behavior>
13. </endpointBehaviors>
14. </behaviors>
15. ...
16. </system.serviceModel>

Looking at the behaviorExtensions/add element, the name attribute value is arbitrary, and is used later to declare the behavior itself. The type attribute value must be the fully qualified typename of the class that implements the behavior extension. When I first wrote the code I assumed that “fully qualified” meant: include the namespace. My declaration looked something like this:

1. <behaviorExtensions>
2. <add name="myExtension" type="MyNamespace.MyExtension" />
3. </behaviorExtensions>

When I got everything written and fired it up I got a configuration error from the parser, indicating that it could not find the named assembly. Hmm… /facepalm. I had implemented the extension class in a separate assembly, so of course it couldn’t find it. I tried adding the assembly name:

1. <behaviorExtensions>
2. <add name="myExtension" type="MyNamespace.MyExtension, MyAssemblyName" />
3. </behaviorExtensions>

That didn’t work either. A little more poking around on the web and I discovered that the parser absolutely required the complete strong name of the assembly. So I added the missing fields, and my declaration looked like this:

1. <behaviorExtensions>
2. <add name="myExtension" type="MyNamespace.MyExtension, MyAssemblyName, version=1.0.0.0, Culture=neutral, PublicKeyToken=null" />
3. </behaviorExtensions>

Still no go, but now the error changed. Instead of complaining that the assembly could not be found the parser now squeaked about the manifest data not matching the reference. Aha! Progress! I googled a bit more and discovered another post describing problems with breaking the string and having spaces after the commas. So I removed the spaces. I won’t bore you with another listing. Just look at the one above and imagine taking out the spaces after the commas in the type attribute string. It still did not farking work.

Damn, /facepalm, the sequel. My assembly is signed. Just because Microsoft’s example used null for the PublicKeyToken attribute doesn’t mean I can. I used sn.exe to generate a public key token and replaced the null with a string of letters and digits. Again, no listing, just use your imagination. It still didn’t work. But the error changed again! Instead of complaining about the mismatched manifest data the error now simply stated that the element couldn’t be loaded. I changed the key back to null – mismatched manifest data. Put back the token – can’t load element. What the hell? It was pretty obvious that I was getting past the assembly part… but what now?

Google provided no additional help, except for one or two posts that mentioned the unusual pickiness of the config parser with respect to this element. And it was while ruminating over one of these that I remembered taking those damn spaces out earlier in the process. I put them back in.

And it worked. Just why the presence of spaces in a comma-separated list of substrings should be important to the config parser I don’t know. I don’t even care. All I know is that it worked with spaces, and it didn’t work without them. Microsoft must have its reasons because the issue has been brought up a number of times and they have refused to acknowlege it as a bug. To me it looks like a bug, or at least a massively anal design decision, but what do I know? All I can do is kick the damn thing until it works.

If you’re using the System.IO.File.Exists method to determine whether a file is present in a particular location, then the answer to the question posed in the title of this post might be: when you don’t have permissions to see it. This might seem obvious, but actually it caught me a little bit off guard this morning. In general I’m conditioned to think that a permissions failure will throw an exception. After all, one way to interpret the rule might be: you don’t have permissions to access the file, therefore there is no way to answer your question. Another way is: you don’t have permissions to access the file, therefore as far as you’re concerned it doesn’t exist. Here’s the MSDN lowdown on the Exists method:

If the caller does not have sufficient permissions to read the specified file, no exception is thrown and the method returns false regardless of the existence of path.

Obviously Microsoft’s library designers agreed with the second interpretation. Not sure I like it, but at least now I know that the file might be there even when File.Exists says it isn’t.

I like it when, every now and then, some ridiculously low-tech requirement comes along and I can spend a few minutes whipping up a class to make it easier and more flexible. Such things often go into the utility library and see a lot of reuse. An opportunity arose recently when I was working on an email notification and reporting engine strapped to a large database. Every day the data gets updated six ways from Sunday and we generate a bunch of naturally tabular data that has to get embedded into emails. The usual approach to this, for myself and everyone else on the team, is to hard code the output. Last night I was looking for something relaxing to work on, and I decided to whip up some stuff to make generating fixed-width text tables a lot easier. The result is a class called TextTable.

I don’t think TextTable is going to win me any ACM awards any time soon, but it’s one of those things that is pretty handy when you need it. I hope to add some extensions to it in the future to support proportional fonts. In the meantime, if you’re interested, you can download the source or read the project post.

A couple of events over the last week have me thinking about the concept of aliasing. I first encountered this idea when I started messing with computer graphics back in the late 80’s, and indeed the first event to which I refer involved someone on a forum bringing up the use of “antialiasing” techniques in games and other graphical applications. The term aliasing in general means a loss of information. In graphics people use the word aliasing to mean the jagged lines that result from rendering continuous geometrical features to a pixelated frame buffer. The reason that you see jagged edges on lines and polygons is because information has been lost where there is no pixel to hold it.

The second case that arose also involved a loss of information, but it was less obvious to the people working on it. In this instance a system produces changes to a database, and each change triggers a message out to an enterprise service buss. The problem is that the messages are unordered, and often involve sequential changes to the same field in a record. The designers created a multi-threaded sender for the resulting messages, and cannot guarantee the order of receipt, nor did they provide for any closure message to signal the end of a transaction. Given such constraints, can the state of the entity from the view of message consumers ever be guaranteed to be consistent with the state as seen by the publisher?

The answer is no. In this case, as in the aliasing of edges in a rendered image, information has been lost that cannot be replaced by the target system. In the graphical image it was the color of the missing pixels. Here it is the order of changes to an entity. Once that data is gone, it’s gone. Graphical applications can compensate to some extent by making assumptions. For example, they might assume that the color of a pixel neighboring two other pixels in a jagged line should be set to an average of the surrounding pixels, thus filling in some of the missing information with a best-guess at what might have been there. This is in fact how graphical anti-aliasing works, and it can produce dramatic improvements.

There are similar techniques for sound, although less successful because sound is less forgiving of error. Business information is less forgiving still. If a bank account is supposed to go through a series of transformations leaving it with a balance of x, there is no way to guess what x should be if some of those changes are applied in the wrong order. How could we fix the underlying problem in this particular design? The only way is to provide enough information to consumers to allow them to reconstruct the order of changes. But really the issue is the basic design: the object should not have been published to observers of the transaction on the buss until it was in a consistent and stable state.  Lacking either of these remedies there is nothing clients can do to restore the missing information and guarantee integrity. Ultimately it’s no different than running a pristine digital image through a lossy compression algorithm: you’re throwing stuff away, and you can’t get it back.

The answer is : probably not very often. Setting the IsThreeState property of a CheckBox control to True enables the user to click through all three states (unchecked, indeterminate, checked), but that is rarely what you want. You might do this for a survey, with yes/no/unsure choices, but I doubt most users would even pick up on the mechanism. You’d be better off with radio buttons. More often than not what you want that third state for is to show that some but not all of a parent’s child items are checked. In other words, the parent’s check state is checked if all its children are checked, unchecked if all its children are unchecked, and indeterminate otherwise.

The logic for this is going to go into some code-behind, or in the accessor methods for a data item property. It is not something the user should control. Fortunately, the IsThreeState property of a CheckBox control has no effect on its ability to display three states. It only affects the response to user actions. So in this scenario what you want is a normal 2-state CheckBox, and then when a parent meets the criteria for being indeterminate you’ll set its IsChecked property to null. It’s as simple as that. To see what I mean copy-paste the following into XamlPad:

1. <Grid xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
2. xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
3. <CheckBox x:Name="CheckBox1" Content="I'm a 2-state CheckBox">
4. <CheckBox.IsChecked><x:Null /></CheckBox.IsChecked>
5. </CheckBox>
6. </Grid>

As you can see, setting the 2-state CheckBox’s IsChecked property to null causes it to display the third, indeterminate state. There’s no need for IsThreeState to be true for most scenarios where CheckBoxes are used.

Consider the following scenario: a rich client communicates over the Internet with an ASP.NET service that does not use forms authentication, and also sends requests to a content site that does use forms auth. Initial user authentication is performed by the client and service using a proprietary protocol. The service then creates an authentication ticket for the user that will permit access to the content site. The ticket is created using the FormsAuthentication.GetAuthCookie method, and passed back to the client to be used in later requests. On the content site the web.config file specifies that the authentication ticket timeout is 120 minutes on a sliding reset, however the ticket consistently expires in 30 minutes. What’s wrong?

My colleagues and I recently faced this exact problem on a current development project. In researching the issue we first made sure that our web.config settings were correct. Using the IIS service manager we verified that the values for the timeout, sliding expiration, cookie name, and machine keys were being picked up and used by the server. The timeout was clearly being set to 120 minutes, and everything else was correct, but still the ticket was expiring at 30 minutes. Obviously our config files weren’t at fault.

Configuration can be a tricky thing in ASP.NET, however, and sometimes the properties that end up driving your system are not the ones you thought were at the wheel. Further research led us to a Microsoft support article on forms auth tickets and cookies. The last paragraph in the article contained the following statement:

“If the forms authentication ticket is manually generated, the time-out property of the ticket will override the value that is set in the configuration file.”

Aha! Surely we had the culprit in hand. Someone had set an explicit timeout in the code that generated the cookie, and that timeout was overriding the values set in web.config. It made perfect sense, except for the fact that when we opened up the code the line in question did not set a timeout. In fact we could see no way of specifying a timeout in the call to GetAuthCookie. Lacking an explicit timeout the value in web.config should be used. We were prepared to head back to the drawing board, or at least Google, when one of my smarter coworkers wondered if, in fact, some default timeout was being set during the call to GetAuthCookie? With the help of a friend who had a copy of .NET Reflector handy we peeled System.Web.Security. Ultimately the public versions of FormsAuthentication.GetAuthCookie call a private overload. The disassembly of that method looks like this:

1. private static HttpCookie GetAuthCookie(string userName, bool createPersistentCookie,
2. string strCookiePath, bool hexEncodedTicket)
3. {
4. Initialize();
5. if (userName == null)
6. {
7. userName = string.Empty;
8. }
9. if ((strCookiePath == null) || (strCookiePath.Length < 1))
10. {
11. strCookiePath = FormsCookiePath;
12. }
13. FormsAuthenticationTicket ticket = new FormsAuthenticationTicket(2, userName,
14. DateTime.Now, DateTime.Now.AddMinutes((double) _Timeout),
15. createPersistentCookie, string.Empty, strCookiePath);
16. string str = Encrypt(ticket, hexEncodedTicket);
17. if ((str == null) || (str.Length < 1))
18. {
19. throw new HttpException(SR.GetString("Unable_to_encrypt_cookie_ticket"));
20. }
21. HttpCookie cookie = new HttpCookie(FormsCookieName, str);
22. cookie.HttpOnly = true;
23. cookie.Path = strCookiePath;
24. cookie.Secure = _RequireSSL;
25. if (_CookieDomain != null)
26. {
27. cookie.Domain = _CookieDomain;
28. }
29. if (ticket.IsPersistent)
30. {
31. cookie.Expires = ticket.Expiration;
32. }
33. return cookie;
34. }
35.  

The part that caught our eye right away was the constructor call to create the FormsAuthenticationTicket. Specifically the fourth argument to the call, which is calculated from DateTime.Now.AddMinutes((double) _Timeout). Hmm, _Timeout, a private field, and obviously relevant to our issue. But where was it getting a value? The first line of GetAuthCookie is a call to FormsAuthentication.Initialize, and so we disassembled that, too. In it we found the following line:

_Timeout = (int) authentication.Forms.Timeout.TotalMinutes;

So the timeout value being passed to the constructor was coming from the forms element of the authentication section in our web.config! But we had already verified the web.config was correct… errrm… actually, we had only verified that the web.config values on the content site were correct. The cookie was being created in the service implementation, a different site altogether, and one with its own configuration. When we looked in that web.config we saw that the forms element did not specify a timeout for the ticket. Since it didn’t the value that was ultimately assigned to _Timeout and passed to the constructor for the ticket was the default, and that is… you guessed it, 30 minutes.

Ultimately the solution that worked was to specify the same timeout value in web.config both on the content site where the ticket would be used, and in the service implementation where it was created. And while having a dependency between those two was less than satisfying, getting this vexing bug to go away more than made up for it.