Wednesday, May 07, 2003

Hearse and Buggy, Part I

One of the greatest lies that we tell ourselves about software is that we can find bugs. But bugs, or errors, or malfunctioning code, or whatever, are not only unavoidable, they are probably necessary. Today, buggy software inflicts $68 billion of damage on the U.S. economy per year, according to the National Institute of Standards and Technology; I think that is probably an underestimation by at least one order of magnitude, especially when we consider not just fatal bugs, but aggravating, annoying, inconvenient bugs that just cause the users extra time, attention, or hassle.

Before I go on, I must provide yet another diversion and talk about Grace Murray Hopper. As you ought to know, Admiral Hopper is one of the great founders of computer science. She is often cited as the originator of the term "bug," due to a story that in 1945, when looking for a problem in the Mark II, one of the first computers, she found a moth. This was taped into her lab notebook with the inscription "First actual case of bug being found." (read the Navy's official version, with pictures) Now the story is true as far as it goes, but the whole point is that it was a joke: if the term "bug" hadn't been in use already, it wouldn't have been funny. The term bug goes back at least to Edison. Hopper and her team may have been the first to use it in a computer context, but they hardly invented the term.

Anyway. Our mistake is in thinking that we can find these bugs, but the complexity of modern software means that we can't possibly find them all. We must accept as a basic fact of life, some kind of equivalent to a law of thermodynamics, that the software will contain bugs. (Jon Bentley once observed that all programs contain at least one bug, and all programs can be reduced by one line. The corrolary of these two laws is that all programs can be reduced to one, non-working line.)

So what do you do? Modern software companies seem to have approached this problem by laying off their QA departments, which while I applaud the willingness to embrace new realities, isn't really what I'm talking about. We need to start writing software with the assumption built in that it will fail.

This isn't a new idea; another founder of the field, John von Neumann, wrote a paper in 1952 called "Probabilistic Logic and the Synthesis of Reliable Organisms from Unreliable Components"; but it's fifty years later and we still aren't following his advice.

The final point is a suggestion that bugs may not be a bad thing. In evolution, it is the presence of tons of excess, non-coding, irrelevant, or even malfunctional genes that act as a repository for functional innovation. When the environment changes somehow, and the species must adapt, it is often the "bugs" or non-optimal designs that provide the basis for changes. As we try to make software more adaptive and autonomous, bugs may similarly become the wellspring of change and innovation.