Hidden Clause

I recently had a conversation about software development. Well, truth be told, I have conversations about software development all the time, but this one was a conversation I had been thinking about for a long time: why does so much software stink? And how do we fix that?

Typically, when anyone discusses the state of software development the use of tools, management practices, etc always get tossed about and everyone walks away feeling like the problem will be solved one day by the brute force of our backs (our brains, really) as we devour the elephant one bite at a time or climb the mountain one step at a time or however it is you feel like thinking about it.

A few years ago I realized that there will never be a tool or collection of tools that will solve this problem. The problem is not technology; it is people. Developers who inflate their abilities; managers who think managing a software team is the same as leading a squad of Green Berets into battle. This was going to take a more extreme solution that everyone is going to fight, well most would fight, but would take care of the quality problem once and for all. In some cases, the problem might get worse, but overall things will be better. Money will be spent to make sure that software works, developers have the background they need, certifications will mean something and a certain amount of prestige will come to the work of development and developers.

The solution to the software problem will be solved by… lawyers.

Yes, you read that right. Lawyers. The guys about whom Mario Puzo once said “A lawyer with a briefcase can steal more than a thousand men with guns.” In order to see how this is going to work I need to go into a little history and talk about that other profession folks love to malign (no, not accountants): doctors.

It is easy to forget that, in the United States, about a hundred years ago anyone could profess to be a doctor (at least that was true in the US. The Islamic world had a medical school before 872 AD). While a hundred years may sound like a long time in the grand scheme of things it might as well be yesterday. Even though the University of Pennsylvania School of Medicine, which began by giving lectures on anatomy in 1765, was one of the first schools to offer a medical degree the number of unlicensed/bogus physicians was quite high due to the general public having a deep-seated mistrust of experts. Kind of like today.

The road from charlatan to professional for the medical profession was a long, hard and ugly one; some would say that the fight is still going on, but very few would want to return to the days of unlicensed doctors no matter what you think of HMOs or malpractice insurance. If anyone could become a doctor they would (who wouldn’t want the money or the prestige?). So what keeps every well-meaning or discontented or idiotic person from practicing medicine? Perhaps the lure of homeopathy as a career? No, it is the same thing that keeps every well-meaning or discontented or idiotic person from practicing civil engineering: the law and, by extension, lawyers. Paying incredible fines and/or prison terms are a fantastic deterrent to fraud even if it doesn’t always work.

Do individual doctors, civil engineers, and accountants go to jail for fraud? Sometimes, but usually it is their employers who primarily pay the price: hospitals, construction firms and accounting firms pay the fines and they are never happy about it. Sometimes they even lose certifications so that the firm is no longer allowed to do business in their respective domains. Is this a happy ending? Of course not, but in our society it would appear that fines and jail time are a great way to get people to do the right thing. Not always, but usually.

So how does great medicine get done? Really, only two ways: you either pay a lot for it or you have a cadre of great practitioners who just get it right. Expensive or done by superstars.

Yeah, yeah, health care can be reasonably priced in countries outside the US, but we don’t have enough data points to determine if that sort of model holds up over time. I hope it does, but I don’t know and I am glad that there are countries willing to fund the experiment to find out.

So what we have are professions where the standards are so high that anyone who attempts to enter the field finds that the bar is set quite high. To some extent that is what defines a professional (well, other than doing a job for money): someone who has a skill or knowledge that places them in a position where their skill or knowledge is needed for very specialized work. Yes, plumbers and carpenters fit this definition, but I would like to concentrate on those professions that require higher education to achieve the skill and knowledge that is required.

What does this have to do with software development? Most of you have already figured it out: when bug-ridden software is released into the open market no one wins. Is it possible to release software with few or no bugs? Of course, just ask the guys who develop software for the space shuttle. “But isn’t that expensive?” I hear you say. Damn straight.

Great software only comes in two flavors: expensive or written by superstars. Sounds suspiciously like medicine. Or law. Or accounting. Or any of a number of other professions where the practitioners can be sued into the ground. It beats being tarred and feathered, but not by much.

But what about hackers (the good kind)? Can we expect folks who are writing software for the fun of it to be held responsible for writing software that someone else uses improperly? Perhaps, but if we think of developers today as the first doctors of a hundred or so years ago we know that as time progresses software will only become more and more ubiquitous and the effect of freely distributed software will be akin to the creation of medicine that has not had the benefit of FDA approval: it might work, or it might not, but it will be up to the user to decide if they want to take that chance. And if that medicine causes someone harm the maker of that medicine is held responsible (as opposed to someone deciding that taking some Liquid Plumber will help their digestion. You can’t hold the makers of LP responsible for someone else’s basic idiocy).

This is not a diatribe against open source software. Let me say that again.

This is not a diatribe against open source software. Far from it.

I am against companies that put out junk software for any reason. While the quality of open source software has been debated, and I don’t want to do that right now, the one thing we know is that when a company releases software that causes records to be lost, or leaves security holes where private information can be stolen, no one wins. Skyscrapers would not be built with the regularity that they are today if there was not the threat of prison if they came down with the same regularity. The same goes for bridges and anything else where there is the chance of losing gobs of money or lives. Yes, there is corruption and painful growing pains as a field that is not in a position to be sued suddenly finds itself in front of Judge Judy or Judge Scalia and has to explain why there are no industry standards for how their members are trained and held responsible when things go wrong that could have been avoided.

In defense of open source developers: I know they don’t have money. And they mean well. I am writing this using OpenOffice on a Kubuntu distribution on a WordPress blog. I love working on a Unix variant using tools that absolutely awesome developers have built giving up valuable time and effort to make it possible for me to accomplish the simple task of putting my thoughts on (virtual) paper so that I can say that as software developers, if we want to move to that next level of career maturity, not to mention money, we have to be in a position where we can be sued for doing a bad job.

But that is just me.

Thanks to Andrew Wardly for his comments.