Skip to main content
PING Social
Back to all articles
essayspostmortembackups

The Night the Backups Burned Too

November 3, 2026

At 00:47 on the morning of March 10, 2021, a fire started in a room of a data center in Strasbourg, on the French bank of the Rhine. By sunrise, the photographs were everywhere, and they did not look like technology photographs. They looked like war photographs: a five story building reduced to a blackened skeleton, its floors open to the sky, smoke still coming off the frame. The building was SBG2, one of four data centers on OVHcloud's Strasbourg site, and by the time the fire was out there was essentially nothing left of it to photograph except the outline.

OVHcloud was, and is, one of the largest hosting companies on Earth, the European champion of cloud computing. That morning, an estimated 3.6 million websites across 464,000 domains went dark, according to Netcraft's count. Government platforms in France, the United Kingdom, and Poland were among them. So were banks, webmail services, news sites, and shops selling protective equipment in the middle of a pandemic. The French domain space alone lost about 184,000 websites in a night. It remains the most destructive data center fire in the industry's history, and the remarkable thing about it, the thing nobody was hurt makes it possible to say out loud, is that the fire was not the disaster. The fire was only the trigger. The disaster had been installed quietly, in advance, by everyone who had stored their second copy of the truth inside the same walls as the first.

To understand why, it helps to be precise about what a data center fire actually takes from you, because it is never just the servers in the burning room.

A data center is three systems wearing a trench coat. There is the compute, the machines themselves. There is the power: feeds, switchgear, batteries, and uninterruptible power supplies, the heavy electrical muscle that keeps the machines alive through every flicker of the grid. And there is the cooling, because thousands of machines in a box are a furnace that happens to compute. A fire attacks all three systems at once, and then it attacks a fourth thing nobody lists on the brochure: permission. Firefighters do not let you run a building that is smoldering. They isolate it. At Strasbourg, the whole site was cut off, which meant that SBG3 and SBG4, buildings the fire never touched, went dark too, and SBG1 lost four of its rooms before the line held. About 100 firefighters worked the blaze, with a pump boat on the Rhine feeding the hoses, and it still took over six hours to extinguish. The machines in the surviving buildings were fine. They were fine the way a heart in a jar is fine. Nothing could reach them, nothing could power them, and nobody was allowed near them.

This is the part of physical failure that software people systematically underestimate. We design for components dying. We do not intuitively design for geography dying: for the possibility that the failure has a location, a radius, and a smell of smoke, and that everything inside the radius shares one fate regardless of how independently it was engineered on paper.

In the days after the fire, Octave Klaba, OVHcloud's founder, began posting updates that included a small taxonomy of his customers' data. There was primary data. There was service backup, the backup you paid for. And there was internal backup, followed, in his own parenthesis, by the words "not contractual." It was an honest piece of labeling, and a chilling one, because it admitted in three words what the week was busy proving: for a great many customers, the difference between "we have your data" and "we owe you your data" was the entire ballgame.

OVHcloud's own status page then published the list of services it classified as unrecoverable, based on its assessment of its internal backups. The list is the most quietly damning document of the whole incident. Plesk. NAS-HA storage. Public Cloud Archive. And there, in plain type, "Managed Veeam Backup (Local DC) in SBG 1 and 2." Read that product name again. It was a backup service. It said so on the label. Its defining feature, the feature that determined whether it would ever be used for the one purpose it existed for, was in the parenthesis: Local DC. The backups lived in the same data centers as the machines they were backing up. When the building burned, the insurance policy burned with the house, because the insurance policy was stored in the house.

Some customers had arranged their own versions of the same trap. One security company later wrote up its recovery with admirable honesty: it had servers in Strasbourg, it had backups, and the backups were in the same data center, because proximity was convenient for connectivity and nobody had ever quite attended to where the backup servers physically sat. Its engineers rebuilt their product's rule sets by asking customers to send back the compiled files the company had shipped them, and decompiling its own product to recover its own work. That is a magnificent piece of improvisation. It is also a sentence no engineering team should ever have to write.

But the sharpest version of the story belongs to the customers who did everything they were told. Bati Courtage, a French property firm, had subscribed to OVHcloud's automated backup option for its servers in SBG2. After the fire, it asked for its backups. It asked for about a month. Then OVHcloud admitted the truth: the backups had been destroyed along with the primary data, because they were stored in the same building as the main server that had been completely destroyed. In court, the company's own contract was read back to it. The backup option, the contract said, is physically isolated from the infrastructure in which the server is set up. The judge's conclusion, as reported by Data Centre Dynamics, was that keeping all the backup copies in the same place does not protect the data, does not respect the state of the art of backup, and does not achieve the objective the contract set. The rulings in that case and a second one, brought by a company called Bluepad, went against OVHcloud. Data Centre Dynamics also reported something worse than the storage decision itself: in the litigation, OVHcloud produced logs that appeared to show backup services located away from the primary data, and those logs were false. The map said the backups were somewhere else. The territory said otherwise.

Hold that detail, because it is the one that graduates this story from a facilities mishap to a universal warning. A backup whose location you have never verified is not a backup with an unknown property. It is a story you tell yourself, with a dashboard for illustration.

The building itself, it turned out, had been a co-conspirator. The Bas-Rhin fire service's report described its shock at what it found: no automatic fire extinguishing system, no power cut-off, wooden floors, and an architecture designed to move air that had, in the firefighters' assessment, turned the building into a furnace. The accident investigators added water penetration in the electrical room where the fire started. Klaba, reviewing hundreds of security cameras, said the footage pointed to a power supply unit that had been serviced that very morning. OVHcloud promised its own report once official verdicts landed. Years later, per Data Centre Dynamics, that report has never appeared, and no official cause has ever been published. Europe's cloud champion rebuilt on the site, raised hundreds of millions in an IPO that priced the fire at about 105 million euros of expected cost, and stopped answering questions about it. The silence at the end is part of the record too.

Against all of that, set the strangest and most creditable part of the story, which is how the founder behaved while his building was still warm.

Klaba's first substantial post on the morning of the fire said, in full public view, that firefighters had been unable to control the blaze in SBG2, that the whole site had been isolated, and then it offered the customers of one of the world's largest clouds the following professional advice: activate your Disaster Recovery Plan. Consider what that sentence admitted. It admitted that the cavalry was not coming, that the provider's own recovery was measured in weeks, and that whatever plan you had made for this day was now the plan. Most companies, facing the total loss of a facility, reach for the language of minor service interruptions. Klaba reached for the fire alarm and pulled it where everyone could hear.

And he kept going. Updates several times a day, then video addresses where he and his chief executive walked through the state of each building, the restoration order, the numbers. The company suspended billing for the affected site and issued service credits. It received more than 3,500 new servers and nearly 2,000 terabytes of storage in the first days, with a team of 60 working the site in shifts around the clock, washing soot out of salvageable machines and drying out buildings the fire never reached. The gradual restart was scheduled for March 22. By April 14, Klaba could report 118,000 of the 120,000 affected services restored. Whatever else is true of OVHcloud that spring, the man did not hide. The accountability was real, it was specific, and it was delivered at a pace that made it checkable. That is rarer in this industry than any technology discussed in this essay.

But notice what the accountability could and could not do. It could rebuild a data center. It could not rebuild the data. A service can be restored to an empty room. For the customers whose only copy of their work sat in SBG2, the finest incident communication in the history of hosting was a beautifully narrated funeral.

So who actually came back, and why?

The cleanest survivor story belongs to Kiwi.com, the travel company, which later published its own account. It was running a database cluster of 30 nodes, and the fire took 10 of them at a stroke. The cluster noticed the loss, rebalanced itself, and kept serving. Kiwi's customers noticed nothing, because Kiwi's data did not live in a building. It lived in a quorum, spread widely enough that losing a third of it in one postcode was an event the software had already rehearsed. Same fire, same morning, same provider family of risks. Opposite ending. The variable was not luck, and it was not budget. It was the answer to a single question: how many copies of the truth exist, and how far apart are they standing?

The game studio Facepunch gives you the middle case. The fire destroyed 25 of its European servers for the game Rust outright, and Facepunch said so the same day, in the plainest sentence of the whole affair: "We've confirmed a total loss of the affected EU servers during the OVH data centre fire. Data will be unable to be restored." The servers were replaceable within hours, and were replaced. The player progression on them was gone forever, and no amount of new hardware could change that, because progression was a fact about the past, and the only copies of those facts had been in the building. Hardware is a purchase order. Data is history. You can re-buy the first one by Friday. The second one has a print run of exactly however many copies you made, and if they were all in Strasbourg, the edition is closed.

And then there is the fire's one unambiguous public service, recorded by Kaspersky's researchers with what one hopes was professional detachment: of 140 known malware command and control servers tracked at OVH, about a third went offline and stayed offline, including infrastructure used by several sophisticated hacking groups. For one morning, the most effective takedown operation in Europe was a building with no fire suppression. Everyone's threat model had the same hole in it. It just happened to be pointed the other way.

Two months after the fire, OVHcloud announced what it called a Hyper Resilience plan, and its three points deserve to be read as a confession written in roadmap. First, a revamped approach to its internal backups, which is to say: the copies the company keeps for itself would stop sharing a fate with the originals. Second, a new emphasis on external customer backups, stored outside, which is to say: the copies you keep for yourself should leave the building too. Third, failover spread across three data centers, which is to say: distance, everywhere, by default. Every point of the plan is the same point. The industry had a phrase for this long before Strasbourg, the 3-2-1 rule: three copies, on two different kinds of media, with one of them off site. Strasbourg was what happens when the third number quietly becomes a zero and nobody audits it, because the first two numbers looked so reassuring on the dashboard.

Our own studio learned its version of this lesson without the fire, which is the cheap way and the way we recommend. Our rule is that a piece of work exists only when it exists as a verifiable artifact, in the canonical place, checked by something other than the hands that made it. That rule was born from workers reporting success that had not happened, but it generalizes, and Strasbourg is the generalization at industrial scale: the claim is not the artifact, the dashboard is not the territory, and a copy you have never independently verified is a rumor with good formatting. OVHcloud's customers had dashboards that showed backups. Some of them, the court record says, had logs that showed backups in another location entirely. What they did not have, on the one morning it mattered, was a restore they had ever watched succeed from a copy that lived somewhere the fire could not reach.

So the practical fences are embarrassingly concrete, and they are the same fences whether you run a cloud region or a laptop. Ask where your backup physically is, in terms of buildings and flood plains and fire compartments, not in terms of product names. Ask whether the thing labeled backup is a separate fate or merely a separate folder. Restore from it, on a schedule, into a place you can inspect, because a backup that has never been restored is a hypothesis. And when a supplier's contract promises you isolation, treat the promise the way the Strasbourg court eventually did: as a claim awaiting evidence, not as evidence.

None of this requires believing your data center will burn down. Data centers almost never burn down. That is precisely the psychology that fills them with second copies stored ten meters from the first. Rare is not the same as impossible, and shared fate does not ask how likely the fate is. It only asks whether, on the day it arrives, your two copies are standing close enough together to attend the same ending.

The fire at Strasbourg destroyed a building, four rooms of its neighbor, and about 3.6 million websites' worth of somebody's morning. But the thing it really exposed was a sentence nobody had wanted to say out loud: a backup that shares a fate with the original is not a backup at all. It is a second copy of the same loss, stored a little further from the screen and exactly as far from safety.

Sources, for the record: Reuters reported the fire on March 10, 2021, including the destruction of SBG2, the isolation of the site, and the impact on government platforms; Netcraft's March 2021 survey supplied the count of 3.6 million websites across 464,000 domains. The Register covered OVHcloud's status page list of unrecoverable services on March 15, 2021, and the Hyper Resilience plan on May 6, 2021. Data Centre Dynamics reported the Bas-Rhin firefighters' findings, the accident investigators' findings, and the Lille court cases brought by Bati Courtage and Bluepad, including the contract language on physical isolation and the rulings against OVHcloud. Facepunch's statements of March 10, 2021 confirmed the total loss of 25 Rust servers. Kiwi.com's survival was described in ScyllaDB's account of March 23, 2021. Kaspersky's observation on disrupted malware infrastructure was reported by Infosecurity Magazine.

FAQ

Questions this essay answers

Why did SBG3 and SBG4 go dark even though the fire never touched them?+

Firefighters isolate a smoldering site. At Strasbourg the whole site was cut off, so SBG3 and SBG4 went dark too and SBG1 lost four of its rooms. The machines in the surviving buildings were fine, but nothing could reach them, nothing could power them, and nobody was allowed near them.

What happened to the backups Bati Courtage had paid OVHcloud for?+

Bati Courtage had subscribed to the automated backup option for its servers in SBG2. After asking for its backups for about a month, it learned they had been destroyed along with the primary data, because they were stored in the same building as the main server. That was despite contract language describing the backup option as physically isolated from the infrastructure the server was set up on, and the rulings went against OVHcloud.

What were the three points of the Hyper Resilience plan OVHcloud announced two months later?+

First, a revamped approach to its internal backups, so the copies the company keeps for itself would stop sharing a fate with the originals. Second, a new emphasis on external customer backups, stored outside. Third, failover spread across three data centers. The essay reads all three as the same point, the one behind the 3-2-1 rule: three copies, on two different kinds of media, with one of them off site.

Questions about PING?

These notes document real architecture and real implementation. If something here is useful for your business, get in touch.

Contact PING
The Night the Backups Burned Too · PING