On the morning of October 1, 2013, the United States government opened a store, invited the entire country to attend at the same time, and then acted surprised by the crowd. There was a law giving the date. There were speeches giving the date. There were television ads giving the date. It was the most thoroughly announced opening in the history of software, and by lunchtime the store's front door had admitted six customers.
The screens were the story that first day. People sat in kitchens and libraries and break rooms watching pages stall, watching error messages apologize, watching a system explain that it was down for maintenance on the morning it was born. Behind every one of those screens was a person trying to buy health insurance, many of them people who had never been able to buy it before. The comedy in this story is real, but it is aimed where it belongs: at the org chart, not at the applicants. The applicants did everything right. They showed up when they were told to show up.
The administration's own figure for the day was about 4.7 million unique visitors. The internal war room notes, the ones CMS kept for itself and Congress later published, counted the people who actually completed an enrollment on day one. The number was six.
Six. Not six thousand. Six, a number you can seat at a dinner table. Against 4.7 million visitors, that is a conversion rate of roughly one in 780,000, the worst performing store opening in the history of retail, achieved by the largest organization on Earth. By the end of the second day the running total had climbed to 248. Sports stadiums were not built for crowds the website could handle, but a wedding venue would have done it.
Notice which number the government quoted in public and which number it kept in a notebook. The public number was visitors: 4.7 million, a figure that sounds like triumph if you say it fast. The notebook number was enrollments: six. For weeks the administration cited traffic and applications started, anything except the count of people who had actually finished. Every failing launch in history has performed this same substitution, measuring the size of the crowd at the door because the count of customers at the register is too sad to say out loud. The war room notes matter because they prove the true number existed, in writing, inside the building, from the first day. The failure was never that nobody knew. The failure was that knowing was not connected to deciding.
The natural assumption is that the servers melted, that the crowd was simply too big for any machine. The crowd was big, but it was also the most predictable crowd in the history of computing. The launch date had been fixed years in advance by statute. The pent up demand had been measured, debated, and campaigned on. If you run a ticket site, your worst day arrives unannounced when a pop star tweets. HealthCare.gov's worst day arrived with a date printed on it, three years ahead of time, in a law. And still, the crowd was treated as an act of God rather than a load test with a calendar invite.
The crowd was not even the deepest problem. The deepest problem was standing in front of the crowd, wearing a badge, checking IDs.
The front door
Here is the architecture, in the order a visitor met it. Before you could look at a single insurance plan, before you could compare one price, you had to create a secure account. Identity first, shopping second. A late decision in the project had removed the option of browsing plans anonymously: no account, no store.
The account system was a component called the Enterprise Identity Management system, EIDM, built by a contractor named QSSI. In the diagrams the builders later drew for reporters, they called it, without irony, the front door. Everything depended on it, because everything was behind it. The actual marketplace, the application built by the lead contractor CGI Federal, sat beyond the door. Beyond that sat the federal data hub, a switchboard that checked your Social Security number with the Social Security Administration, your citizenship with Homeland Security, your income with the IRS and Equifax, and then assembled your eligibility. It was a genuinely hard system. It also barely got used in week one, because almost nobody could get through the door to reach it.
Administration officials acknowledged within weeks that account creation was the bottleneck: the front door jammed, and a jammed front door looks, from the outside, exactly like a dead website. The reasoning behind the door was not stupid. Subsidies depend on verified identity and income, and an account makes a long application resumable. But there is a difference between checking ID at the checkout and checking ID on the sidewalk. Retail learned this a century ago. You let people wander the aisles. You card them when money moves.
So the first fence writes itself: never put your slowest, hardest, most stateful operation at the entrance to the party, in front of the part everyone came for. The browsing was cheap. The identity check was expensive. The design made every visitor pay the expensive price before receiving any of the cheap value, and it made the expensive component a single point of failure for the entire national program. When the door broke, the store behind it might as well not have existed. In a sense, for six enrollees a day, it did not.
Fifty-five contractors and an empty chair
Why was there no one whose job was to notice this? Because of the most revealing finding in the official reviews, and it is not about code.
The build was divided among 55 contractors. The inspector general's count of the paperwork was 60 contracts spread across 33 companies. CGI Federal held the biggest piece. QSSI built the front door and the data hub plumbing. Others built other pieces. Each contract described a component. No contract described the system.
The role that was missing has a name in this industry: the lead systems integrator, the one party responsible for making the pieces into a whole, for running the whole end to end before the public does, and for saying the word "no" with evidence behind it. The HHS inspector general found that CMS did not select a lead systems integrator until October 2013, which is to say, after the launch it was supposed to integrate. Before that, the agency had apparently assumed CGI Federal had quietly assumed the role on its own. CGI, for its part, believed it had not. Somewhere inside the federal government, the most important job on the project was held by an assumption, and the assumption was never checked by either party who held a copy of it.
Read that again, because it is the whole essay in one sentence. Two organizations each believed the other was integrating the system. Integration is not a task that survives being telepathic. It is meetings, environments, test data, interface contracts, and a person with the authority to stop a launch. Nobody had been given that job, so the job was not done, so the interfaces between 55 contractors were tested for the first time by 4.7 million members of the public.
Testing tells the same story in a different dialect. The GAO found that CMS moved its own readiness assessment from March 2013 to September 2013, weeks before launch, and then launched anyway without verifying that the system met its performance requirements. End to end testing, the kind where a fake citizen walks the entire path from front door to enrollment, happened only in the final stretch and was, by the Secretary's own later admission in testimony, inadequate. The date could not move. The truth about readiness could, and did, quietly, in the direction of the date.
The bill arrived on schedule even if the website did not. The GAO put the obligations at about $840 million through March 2014. The marketplace software alone grew from a $56 million obligation in 2011 to $209 million. The data hub grew from $30 million to $80 million. Cost overruns of this shape are not a billing accident. They are the financial signature of integration debt: every component priced as if the seams were free, and then the seams billed at emergency rates.
Every dashboard was green
On October 24, three weeks after the launch, the contractors were called to Capitol Hill to explain themselves, and their testimony is the most useful document in the whole affair, because nobody at the table was lying. CGI's executive noted that her company's portion had passed its technical reviews, and pointed to changing requirements and to a component built by another contractor as contributors to the collapse. The other firms said versions of the same thing, and each of them was, within the borders of its own contract, substantially correct.
That is what a missing integrator looks like from the inside. It does not look like chaos. It looks like a wall of green dashboards. Every component tested against a simulation of its neighbors and passed. Every milestone review signed. The seams between components, which is where systems actually live, belonged to no one, so the seams were tested by the only party left: the public, at full scale, on live television. A test environment where each contractor mocks the others is a machine for manufacturing confidence, because the mocks are built from the same assumptions as the code. The real neighbor always behaves in the one way the mock did not.
Any engineer who has watched fifty microservices each report healthy while the product is down has sat in that hearing room. The dashboard is not wrong. It is answering a smaller question than the one the user is asking.
The surge, or: hire the integrator
The rescue is the part of the story that should be taught alongside the failure, because the rescue changed exactly one structural variable, and it worked.
In late October the White House brought in Jeffrey Zients to run what everyone called the tech surge. The surge did several sensible things at once: a war room, a single punch list, round the clock work, honest public metrics delivered on weekly calls. But the load bearing decision, the one everything else stood on, was this: QSSI, the contractor that had built the front door, was named general contractor and systems integrator. The empty chair got a person in it, with authority across the other contractors. From that moment, somebody owned the whole system, could test the whole system, and could sequence fixes against the whole system instead of against 55 separate contracts.
The second change was quieter and just as structural: the surge measured in public. Zients reported the site's performance numbers on weekly press calls, good weeks and bad, against targets he had stated in advance. After a month of visitor counts offered as a substitute for enrollments, the rescue ran on the opposite principle: one list of defects, one set of numbers, one person reading them out loud. That is not public relations. That is what an integrator actually is, a single place where the state of the whole system is known, current, and spoken by someone with the authority to act on it.
By December 1, the administration's progress report could say, in Zients's words, that the site was night and day from where it was on October 1. Skepticism was earned by then, but the numbers that followed backed the direction. Open enrollment limped, then walked, then closed in April 2014 with about 8 million people having selected plans, a figure the administration announced as past its revised expectations. The website that enrolled six people on its first day enrolled millions by its deadline. Same country. Same law. Same basic software, patched at speed. The difference was an owner.
There is a temptation to file the whole episode under "government bad at computers," and it should be resisted, because the private sector produces this exact failure on a smaller stage every quarter. Any organization that has ever split a platform across vendors, teams, or repos and let integration be everyone's side job has built a miniature HealthCare.gov and is simply waiting for its October 1. The government's version is just the one with congressional testimony, which is the private sector's postmortem with better lighting and a court reporter.
Our own studio runs on the smaller, survivable version of this rule: a claim is not a fact, and a component is not a system. Every worker's output has to land as a verified artifact, and a piece of work is not done when its author says so, it is done when the whole it belongs to has been exercised end to end and the result can be shown. That rule was not invented in a seminar. It is the scar tissue of watching what happens, in our own logs, when activity substitutes for verification. HealthCare.gov is that lesson at national scale, with the receipts published by two separate watchdog agencies.
So the three fences, each embarrassingly cheap next to $840 million. First: integration is a job, with a name, a contract, and authority, never a hope that the biggest vendor absorbs it by osmosis. If two parties can each believe the other is integrating, you do not have redundancy, you have a vacancy with two witnesses. Second: a launch date fixed by law is still a load test you chose. The date being immovable does not make the demand unforeseeable; it makes the testing schedule the only variable you control, which means testing starts earlier, not later, and readiness reviews do not get moved toward the launch like a dentist appointment nobody wants. Third: browse first, authenticate at checkout. Put the cheap, stateless, cacheable work in front and the expensive identity work at the moment of transaction, so that your hardest component is never the doorman for your easiest one.
HealthCare.gov did not fail because 55 contractors each built a bad piece. It failed because no one's job was the whole, and on launch morning, the whole is the only thing the public ever meets.
Sources, for the record: the day one figures are the administration's 4.7 million unique visitors, cited by CMS Administrator Marilyn Tavenner in her October 29, 2013 testimony to the House Ways and Means Committee, and the CMS war room notes released by the House Oversight Committee (6 enrollments on day one, 248 by the end of day two), as reported by the Associated Press on November 1, 2013. The contracting findings are from the HHS Office of Inspector General's report on the Federal Marketplace acquisition (60 contracts, 33 companies, no lead systems integrator selected until October 2013) and the GAO's July 2014 report and testimony, "Healthcare.gov: Ineffective Planning and Oversight Practices Underscore the Need for Improved Contract Management," which put obligations at about $840 million through March 2014 and found CMS launched without verifying performance requirements after moving its readiness assessment from March to September 2013. The front door account is drawn from NPR's October 24, 2013 diagram of the system based on the builders' own descriptions, including administration officials' acknowledgment that account creation was the bottleneck. The rescue figures are from HHS's December 1, 2013 progress report and the administration's April 2014 announcement of about 8 million plan selections.