The Trainwreck: HealthCare.gov Meets The Mythical Man-Month

A classic work on software engineering and project management can help us to understand what’s going on with the Obamacare website.

The analogy I’ve encountered most often when reading about the Oct. 1st launch of HealthCare.gov is train wreck. (The next most common is trainwreck.) And the joint criticism most frequently levelled — by both opponents and proponents of Obamacare — is they had three years! and spent hundreds of millions of dollars! producing a train wreck!

How could this have happened? And can The Trainwreck provide us with any clues about the future of the federalizing of healthcare?

We will look at, first, the main reason software projects go awry; then, at the lynchpin idea of conceptual integrity; finally, at how software systems tend to reflect the organization that produces them.

Software Projects Go Awry

Yuval Levin reported, Oct. 17th, that “the contractors believed the time they were given for development was totally inadequate”. An Associated Press article, Oct. 22nd, provided some colorful details:

Project developers for the health care website who spoke to the AP on condition of anonymity — because they feared they would otherwise be fired — said they raised doubts among themselves whether the website could be ready in time. They complained openly to each other about what they considered tight and unrealistic deadlines. One was nearly brought to tears over the stress of finishing on time, one developer said. Website builders saw red flags for months.

Not enough time: that’s an easy and common excuse, of course, in many kinds of situations. But it’s so common in software projects because it’s so commonly true. As Frederick Brooks observed, “More software projects have gone awry for lack of calendar time than for all other causes combined” (p. 25).

Brooks is the author of The Mythical Man-Month, a book first published in 1975, with an expanded edition in 1995. He analyzed the various factors that contribute to not-enough-time software disasters; I will paraphrase a few:

• the “unvoiced assumption” that all will go well, when all usually does not go well;
• confusing effort with progress, though they are hardly synonymous; and,
• “Adding manpower to a late software project makes it later” (dubbed “Brooks’s Law”, p. 25).

The first item ought to be familiar to everybody. But the second and third might need some explanation.

The second item, confusing effort with progress, may give pause to non-programmers. They may think along these lines: when I mow the lawn, effort is progress; when they build automobiles on the assembly line, effort is progress; when they resurface the highway, effort is progress. So, when they program computers, why isn’t effort progress?

Brooks explains it this way: “In many creative activities the medium of execution is intractable. Lumber splits; paints smear; electrical circuits ring…. Computer programming, however, creates with an exceedingly tractable medium. The programmer builds from pure thought-stuff: concepts and very flexible representations thereof” (p. 15).

Thus, the writing of a poem or an essay is a better comparison to programming than are more concrete operations, like lawn-mowing and highway-building. Indeed, computer programming can be thought of as a highly specialized kind of writing that, to an extraordinary degree, employs logic and requires attention to minutiae while circumscribed by rigid, and somewhat arbritrary, rules.

Several factors combine in the third item, Brooks’s Law:

• programmers vary widely in natural ability, education, and experience, and they are not interchangeable at will;
• each software project has its own protocols and algorithms — its own ways and means, its own ins and outs — which must be learned by training and practice: the knowledge cannot be infused at will;
• handing on this knowledge, and acquiring it productively, takes time and cannot be done flawlessly at will.

The upshot? Adding more people to a software project can, itself, cause delays (even if it goes well) and introduce more problems and even more delays (if it does not go well). The further along the project, the greater the chance that these complications will become significant. Moreover, being “the best and the brightest” isn’t all it takes: intellect and ability on their own do not automatically possess the required knowledge.

Thus, Brooks declared the “man-month” a myth; that is, the notion that progress can be calculated by multiplying the number of programmers by the amount of time they work is false. When falling behind schedule, he argued, it is plausible to get a realistic revised schedule by adding time, but not by adding staff.

Of those pundits writing about The Trainwreck whose work I’ve seen, only Megan McArdle understands this:

Even if you could somehow assemble a dream team of crack developers and tech managers, and parachute them in to take over this operation, that wouldn’t magically enable the administration to fix this malfunctioning site by Nov. 15. In fact, it would pretty much ensure that the site didn’t get fixed in time. The new team would have to spend weeks figuring out how the site was put together, and who did what, and who they needed to talk to in order to purchase office supplies.

Conceptual Integrity

Lack of calendar time is, surely, one of the causes of The Trainwreck. But Brooks identified another issue that is even more fundamental in the success or failure of software projects: conceptual integrity.

To explain, he first discusses the wide variety of architectural styles that can be seen in many a European cathedral — that is, different plans and styles in the same building — brought about by the succession of different ideas from one generation to another over a span of decades or even of centuries. He then leaps to computer systems:

Even though they have not taken centuries to build, most programming systems reflect conceptual disunity far worse than that of cathedrals. Usually this arises not from a serial succession of master designers, but from the separation of design into many tasks done by many men. I will contend that conceptual integrity is the most important consideration in system design. It is better to have a system omit certain anomalous features and improvements, but to reflect one set of design ideas, than to have one that contains many good but independent and uncoordinated ideas….

Simplicity and straightforwardness proceed from conceptual integrity. Every part must reflect the same philosophies and the same balancing of desiderata. Every part must even use the same techniques in syntax and analogous notions in semantics. Ease of use, then, dictates unity of design, conceptual integrity….

Conceptual integrity does require that a system reflect a single philosophy and that the specification as seen by the user flow from a few minds. (pp. 42, 44, 49)

The phrase conceptual integrity may be unfamiliar; the idea itself may seem arcane. We can think of it as consistency and coherence from beginning to end.

The idea has many applications. William Strunk was writing about conceptual integrity in his 1918 classic Elements of Style:

Vigorous writing is concise. A sentence should contain no unnecessary words, a paragraph no unnecessary sentences, for the same reason that a drawing should have no unnecessary lines and a machine no unnecessary parts. This requires not that the writer make all his sentences short, or that he avoid all detail and treat his subjects only in outline, but that every word tell. (# 13)

And Abraham Lincoln understood the necessity of conceptual integrity:

In law it is good policy to never plead what you need not, lest you oblige yourself to prove what you can not. Reflect on this well before you proceed. (February 20, 1848)

And we all live each day with the unfortunate results of lack of conceptual integrity. Examples can be found in many areas: highways that were poorly integrated with existing roads; groups that disintegrate and disband because they can’t agree on a specific purpose or goal; parking lots that were, obviously, an afterthought once the building plans had been drawn up; store layouts that just make it so hard to find anything you actually want; and, of course, websites that are frustratingly difficult to navigate. Others might be a novel or a movie or even a television series that is well-done in certain parts or in certain ways, but the whole “just doesn’t work” and is unsatisfying.

But, when projects have been achieved with conceptual integrity — when it all hangs together and every part is where it ought to be and fits nicely with all the other parts — well, that’s a situation rare enough to provide an unexpectedly pleasurable experience: “That was wonderful!” or “What a relief!”

Stephen Wolfram described this summer how important conceptual integrity has been in the development of Mathematica, a monumental computational software package:

Throughout everything we’ve followed a principle that I defined right at the beginning: that whatever is added, everything across the whole system must always fit together in a coherent way. Often it’s taken great effort to achieve this. But the payoff has been spectacular. Not only has it kept an ever-larger system easy to learn and use; it’s also made possible a kind of combinatorial growth in the power of the whole system — with each new part routinely able to combine capabilities from every other part.

That’s it! But you can’t hit that mark unless you aim for it.

We can only speculate about the myriad ways in which conceptual integrity was violated for HealthCare.gov. One critical violation, however, is out in the open, as Levin explains (note especially the parts I have bolded):

One key problem, which to date has been the most prominent in public, has to do with a late-in-the-game decision to require users to go through a complex account-creation process before even reaching any coverage options. Administration officials apparently went back and forth several times on this question, and the ultimate decision required the creation of a series of patches over an already developed site in a very short time. Most of the problems people have faced so far are a function of that decision, and have had to do with creating user accounts and so getting through the very first steps involved in purchasing coverage.

Once a project has begun, vacillation on such a scale destroys conceptual integrity: coherence and consistency are gone.

Moreover, conceptual integrity becomes more difficult the bigger the project is. At least 55 contractors were employed for the development of HealthCare.gov, with an aggregate budget upwards of $600,000,000. About such large and complex projects, Brooks wrote this:

Managing large programming projects is qualitatively different from managing small ones, just because of the number of minds involved. Deliberate, and even heroic, management actions are necessary to achieve coherence. (p. 256)

A New York Times article provides some information by which we can tell whether management had what it would take to avoid a train wreck:

In March, Henry Chao, the chief digital architect for the Obama administration’s new online insurance marketplace, told industry executives that he was deeply worried about the Web site’s debut. “Let’s just make sure it’s not a third-world experience,” he told them….

One highly unusual decision, reached early in the project, proved critical: the Medicare and Medicaid agency assumed the role of project quarterback, responsible for making sure each separately designed database and piece of software worked with the others, instead of assigning that task to a lead contractor. Some people intimately involved in the project seriously doubted that the agency had the in-house capability to handle such a mammoth technical task….

The Government Accountability Office, an investigative arm of Congress, warned in June that many challenges had to be overcome before the Oct. 1 rollout…. But Mr. Chao’s superiors at the Department of Health and Human Services told him, in effect, that failure was not an option, according to people who have spoken with him…. Former government officials say the White House, which was calling the shots, feared that any backtracking would further embolden Republican critics who were trying to repeal the health care law.

Thus, we see that the wrong people were put in charge; ultimately, some of the wrong people were “calling the shots” from the White House, where “deliberate” and “even heroic” actions were lacking.

And, as we see, in the real world, failure is indeed an option.

Systems Recapitulate the Real World

I think there is yet a more fundamental problem responsible for The Trainwreck: Obamacare itself hindered, and will continue to hinder, conceptual integrity for federal computerization of the healthcare industry.

The Mythical Man-Month does not discuss this kind of problem much, but Brooks does quote Lars O. Södahl, founder of MYSIGMA Management Consultants, about it:

In my experience most of the complexities which are encountered in systems work are symptoms of organizational malfunctions. Trying to model this reality with equally complex programs is actually to conserve the mess instead of solving the problems. (p. 211)

Translation: software projects will reflect the quality of their real-world counterparts. An organization that attains and maintains its own conceptual integrity will be more likely to produce software systems of consistency and coherence — because they know how to do it in the real world — helping to minimize all the factors that make software projects go awry.

Conversely, organizations that are a mess will be more likely to produce software that is a mess: systems recapitulate the real world.

Why Obamacare, then, is itself a problem for federal computerization of the healthcare industry can best be explained by this chart of the “New Healthcare System” created by the Patient Protection and Affordable Care Act, which is itself a 2,700-page monstrosity, and the Health Care and Education Reconciliation Act.

Getting “150%” from the “A-team” of “the best and the brightest” software experts can do nothing whatever about that.

The Last Word

Three weeks into the launch or, perhaps more accurately, the failure to launch of HealthCare.gov, Fred Brooks deserves the last word:

In the last analysis the customer is the independent auditor. In the merciless light of real use, every flaw will show. (p. 69)

(Thanks to Bill White of Wolfram Research for the Stephen Wolfram quotation.)