At 11:30 in the morning on May 18, 2012, the most anticipated stock debut in the history of markets finally printed its first trade: $42.05 a share, half an hour after it was supposed to open, to a trading floor that had spent the delay doing something traders are not built for, which is waiting in silence with no information. Facebook had priced the night before at $38 a share, raising about $16 billion and valuing itself north of $100 billion. The opening auction was supposed to be the victory lap. Instead it became the day the referee's whistle got stuck in his throat, and then, when he finally blew it, half the players found out two hours later whether they had been in the game at all.
To understand what broke, you have to understand the cross, because the cross is the whole show. An IPO does not open the way a normal stock trades, one buyer meeting one seller at a time. It opens with a single giant auction. For hours before the open, orders pile up on the exchange: buy at $40, sell at $41, buy at any price, sell at the market, hundreds of thousands of instructions, all waiting. At the appointed moment, the exchange's IPO cross application takes the entire pile, runs its checks, and computes one price, the single price at which the largest possible number of shares can change hands. Everyone in the cross gets that price. Then continuous trading begins and the stock is just a stock. The cross is a beautiful piece of machinery, right up until it is the only piece of machinery, which on May 18 it was.
The failure, when the SEC wrote it up a year later, was almost insultingly small. It was not that NASDAQ's systems fell over under the weight of Facebook mania, though the weight was real. The order's language is "design limitation," and the mechanics deserve a slow read. The cross application had a validation step, a sanity check it ran against the orders and cancellations in the book before it would commit to a price. The problem was that the book would not hold still. Orders and cancellations kept arriving while the check ran, and each new arrival invalidated the check and sent the application back to run it again. And again. The machine was not crashing. It was doing its job, faithfully, forever, like a clerk who will not stamp the form because someone in the lobby keeps coughing. The cross fell 19 minutes behind the live order flow while the application rechecked a pile that would not stop growing.
Here is the detail that tells you this was a design failure and not a traffic accident. During the week before the IPO, NASDAQ had tested the cross, prudently, using a dummy test security. It capped those test crosses at 40,000 orders, presumably because nobody wanted the test to be the biggest load the system had ever seen. On the day itself, member firms entered over 496,000 orders into the Facebook cross. The rehearsal was capped at one twelfth of the show. There is a version of this story where the system simply melts under twelve times its tested load, and that version is almost forgivable. But that is not what the SEC found. The system did not melt. It looped. Twelve times the tested volume gave the loop twelve times as many reasons to run one more check, and the loop obliged. The volume was the amplifier. The bug was the song.
At 11:11 a.m., the cross finally produced its price, computed from the orders and cancellations received up to 11:11. By then it was 11:30, and more than 38,000 marketable orders placed in that gap, between 11:11 and 11:30:09, were simply not in the auction they had been sent to. Nobody had told their owners. When continuous trading commenced at 11:30, about 8,000 of those orders were released into the open market like a held breath. The remaining 30,000-plus went nowhere at all. They were stuck: not executed, not cancelled, not confirmed, suspended in a system that had moved on without them. They stayed stuck for more than two hours, in flat violation of NASDAQ's most fundamental rule, the price and time priority rule that says orders are handled in the order and at the price the rules promise. Thirty thousand orders, from firms whose entire business is knowing their position to the share, were in a drawer.
And it was not as if NASDAQ's leadership did not know something was wrong. Immediately before the cross, officials had noticed a discrepancy between the final indicative pricing and volume figures and the totals on their internal systems, a discrepancy that said, in plain terms, some eligible orders are not being handled properly. What happened next is the part of the story that turned a bug into a scandal. Senior leadership convened what NASDAQ calls a Code Blue call. They decided not to delay the start of secondary trading. They believed they had fixed the design limitation by removing a few lines of computer code, live, mid-incident, in the matching engine of an American stock exchange, while the most watched IPO in history sat half opened on it. The SEC's order is dry about this, which somehow makes it worse: they made that decision without understanding the root cause of the problem. The few lines came out. The stock opened. The stuck orders stayed stuck, because the root cause had not been in the lines they removed.
Consider what those two hours felt like from the trading desks, because this is where the story stops being about one exchange's software and starts being about what a market actually is. A market maker sends an order for a client. Normally the exchange answers in microseconds: filled, at this price; or cancelled, confirmed. Confirmation is not paperwork. Confirmation is the trade. Until it arrives, the firm does not know whether it owns the shares or not, which means it does not know its risk, which means every subsequent decision is a guess wearing a suit. On May 18, for over 30,000 orders, the answer did not come for hours. And the desks traded anyway, because the market was open and Facebook was moving and standing still was its own kind of loss. Some firms, receiving no confirmation that their orders had registered, sent the orders again, a perfectly rational response to silence that stacked duplicate exposure on top of the original. UBS later said the malfunctions caused its systems to re-enter orders multiple times and left it holding a huge position of stock it never wanted. When the confirmations finally flushed through, firms discovered positions they had not chosen, in the most volatile new stock in the world, at prices that had already moved. The glitch lasted minutes. The blindness lasted hours. The unwanted inventory was forever, or at least until someone paid to unwind it, which is the financial form of forever.
NASDAQ, meanwhile, had a position of its own, and this is the detail that elevates the affair from engineering failure to governance failure. The mid-incident fix, removing those lines of code, had a known side effect: it created mismatches, and the exchange itself would end up taking the opposite side of them. By the end of the sequence, NASDAQ was short more than three million shares of Facebook in an error account. Its own rules did not permit it to use an error account for any purpose. Not for this purpose, not for any purpose; the SEC's phrasing allows no wiggle room. NASDAQ then covered the short for a profit of approximately $10.8 million, which was also a violation. Sit with that for a moment. The exchange broke the opening, traded anyway, ended up accidentally short three million shares of the stock it had failed to open properly, and made $10.8 million on the cover. The firms stuck in the drawer lost, by their own later estimates, up to about $500 million in aggregate. If you designed a parable about misaligned incentives you would reject this draft as too on the nose.
The after-action took a year and produced numbers with very different levels of truth in them, so let us keep them sorted, because sorting them is half the story. The hardest number is the SEC's: in May 2013, NASDAQ agreed to a $10 million penalty, the largest ever imposed on an exchange at that time, for violating its own rules and the securities laws, along with a censure and a list of undertakings about its systems. The next hardest is the compensation: NASDAQ set up a voluntary plan with a $62 million pool, the SEC approved it, and FINRA processed the claims and calculated the valid ones at $41.6 million, which was duly paid. Then the numbers go soft, on purpose. The estimate that market makers lost up to about $500 million in total is an estimate, assembled by the losers, plausible and unproven in any courtroom. UBS's claimed figure of about $350 million is exactly that, claimed: UBS declined the compensation plan, demanded arbitration, was blocked by a federal judge, appealed, and lost again at the Second Circuit in October 2014. The $350 million was never adjudicated into existence. It remains the largest number in the story and the least verified, which is not a coincidence, because in the aftermath of a systems failure the loudest number is usually a negotiating position with a dollar sign on it.
What did NASDAQ actually change? The order's undertakings pushed the exchange into the unglamorous work it should have done before May 2012: reviewing and hardening the design of the systems that run openings, testing them against the day they are actually for, and putting decision discipline around the choice to trade through a known malfunction. The deeper change was doctrinal, and the SEC said it out loud in the press release, in the voice of its Market Abuse Unit chief: systems disruptions get written off as technical glitches, when it is the design of the systems and the response of the officials that deserve the scrutiny. Glitch is the word an institution uses when it wants the event to be weather. This was not weather. It was a validation loop that could not terminate, a rehearsal capped at a twelfth of the performance, a leadership call that chose the schedule over the diagnosis, and a rulebook that bent the moment it became expensive. Every one of those is a decision, and decisions have authors.
So what is the lesson, for anyone who runs a system that other people depend on at a moment that cannot be rescheduled? Start with the shape of the bug, because the shape is the teaching. The cross did not fail because Facebook was too big. It failed because the validation check had no way to finish: no snapshot it was willing to trust, no bound on how many times it would re-run, no answer to the question "checked as of when?" Every system that validates a changing world has to answer that question, and "keep checking until the world stops" is not an answer, it is an infinite loop with good manners. The fix is almost always the same and almost always taken too late: validate a frozen snapshot, timestamp it, and let the world queue behind the line you drew. NASDAQ eventually priced the cross off the 11:11 snapshot anyway. It just did so by accident, nineteen minutes late, without telling the 38,000 orders that had missed the cut.
Second: your edge case is your main case on the biggest day. The test program capped the rehearsal at 40,000 orders for a debut everyone on earth knew would be historic. Nobody tests at a twelfth of the expected load because they are lazy; they do it because the test environment is expensive, or the calendar is tight, or the cap feels generous relative to ordinary days. Ordinary days are not what the system is for. The system is for the day that makes the news, and on that day the load is not a surprise, it is the entire point of the event. If the rehearsal cannot survive the show, the show becomes the rehearsal, with real money as the test data.
Third, and this is the one that generalizes furthest: never let the incident commander hot-patch the referee. Removing live code from a matching engine mid-incident, without a root cause, on a call whose real agenda was not slipping the schedule, converted a contained defect into rule violations that ran for hours and a short position the rules forbade. The pressure in that room was real, and so is the pressure in every incident room, which is why the discipline has to be built before the room fills: if we do not understand it, we do not open on it. A delayed opening is a headline for a day. An opening that lies to its participants about whether they traded is a federal order, a record penalty, and a permanent footnote in every future prospectus conversation about where to list.
Our own studio runs a small, unfashionable rule that this story keeps vindicating: a claim is not a fact until it lands as something verifiable. A trade without a confirmation is the same object as a task reported done with no artifact on disk, activity mistaken for output. NASDAQ's desks learned in two hours what every operator eventually learns in some currency: the confirmation is not the receipt for the work. The confirmation is the work. Everything before it is a rumor you are trading on.
Facebook, for its part, was fine. The company whose debut broke the exchange went on to become one of the most valuable enterprises on the planet, and the stock eventually made its opening print look cheap. The machine that priced it was not fine, and the reason was embarrassingly human: on the one morning when the whole world was watching the auction, the auction could not stop checking its own work, and the people in charge decided that opening on time mattered more than knowing what they had opened. The biggest debut in history was priced by a loop that could not say done, and blessed by a call that would not say wait.
Sources, for the record: the Securities and Exchange Commission's press release "SEC Charges NASDAQ for Failures During Facebook IPO" and its settled administrative order (May 29, 2013) are the source for the design limitation, the 19-minute lag, the 11:11 a.m. snapshot, the 38,000 excluded orders, the 8,000 released at 11:30 a.m., the 30,000-plus stuck orders and the two-hour delay, the Code Blue decision and the removed lines of code, the three-million-share error account short and its $10.8 million profit on covering, and the $10 million penalty. The opening price figures carry two sources: the $42.05 first trade is the print reported contemporaneously at the 11:30 a.m. open, while NASDAQ’s post-mortem put the cross price at $42.00. The SEC order, as quoted in Prof. Jayanth R. Varma's contemporary analysis, is the source for the 40,000-order test cap and the 496,000 orders entered on the day. Reuters (October 25, 2013) is the source for FINRA's $41.6 million valid-claims figure and for UBS's $350 million claimed losses and arbitration route; Reuters (September 2012 and April 2015) for the $62 million plan, the roughly $500 million aggregate loss estimate, and the appeals court's rejection of UBS's arbitration bid. Figures the firms asserted about their own losses are labeled in the text as estimated or claimed, because that is what they are.