I’ve written here in the past about the nature of software, and how to visualize it. An article I read this morning while browsing Slashdot (one of my favorite ways to get the day rolling) directed my thoughts toward the notion of what software is, and what quality it is that allows a particular person to be better or worse at conceptualizing and producing good software. The record of my past accomplishments does not make me a natural candidate to opine on the topic. I’ve written some moderately complex software, that has been used fairly widely. How’s that for adroit use of qualifiers? The best example is probably MVP Backgammon, a game I wrote back in 1991 that has been downloaded and used a hundred thousand times or so in the last fifteen years. Incredibly it is still selling.

Maybe the worst example is CardSite, a platform for web-based credit card account management that I helped develop in 1998, and which was ultimately used by a few million credit card account holders. Whereas with backgammon I did a deep dive on the subject and concepts for over a year, CardSite was developed while I was trying to get a technology business off the ground, and taking on simultaneous duties of design, development, sales, administration, finance, etc. Ultimately we did get the company off the ground, but CardSite was a pain in the ass from day one.

The article that got me thinking along these lines appears in Scientific American, and was written by Daniel Jackson, one of the creators of Alloy, an automated tool for checking the correctness of software designs.

That in itself is an interesting objective, and leads to a subject too deep to cover in what I intended to be a quick blurb here. It’s worth reading the article if you have an interest in this kind of thing. Suffice it to say that asserting the correctness of a software system requires iterating through all of its possible states and validating that no undersireable outcomes proceed from them. This is generally recognized as an NP-complete problem, i.e. one for which the only solution is to patiently apply every possible set of variables to see what works. In more concrete terms, to solve an NP-complete problem requires interating through the whole solution space looking for acceptable solutions – an analog is the classic “Travelling Salesman” problem that has been taught in CompSci classes for years.

But I digress. There were two statements in the article that got me going. The first occurs on page 4: “Bad abstraction choices lie at the root of many unnecessarily complicated or unreliable systems.” I think most experienced developers intuitively recognize the truth of that statement. You can teach somebody the syntax of a programming language, and perhaps even imbue or awaken in them some appreciation of the process of implementation. But how do you teach them to visualize a problem space and then decompose it into component parts in a way that produces a cohesive, complete, logically consistent model of the system? Is this art or skill or both? It seems to me that this is a weakness of tools like Alloy. The search technology is impressive, but to achieve correct results you need substantial skills in design, modelling, and expression. You could argue that every developer should have these skills, but I have met many who do not.

The second statement occurs in the last paragraph of the article: “At some point, there may come a time when software becomes so essential to our day-to-day infrastructure that society will no longer tolerate bad software.” We’re very close to that point now, I think, and the tools we have for producing software are evolving much less rapidly than is our dependence on what they produce. In areas of critical software development (military and engineering systems, for example) where lives are at risk, correctness is ensured by strict management of the implementation team and exhaustive testing, with the support of a few tools that largely automate tactical areas of the problem.

It will be interesting to see where all this goes, but in at least one sense it has already affected the way I manage the team of developers that work for me, in that it has changed the nature of the questions I ask candidates. Too many interview processes for technical positions rely on demonstrating memory of rote knowlege, like how an API works. There is too much of this stuff for any developer to remember these days. If you focus on this you either miss good thinkers who have general knowlege, or you get highly-specialized people who have immersed themselves in one specific technology. Things change much too rapidly for the latter to be a good investment, so I have refocused on questions about the thought processes that will hopefully produce more of the former sort of person.