The Haidan Takeoff Crisis was a series of events in 114 BFC that lead to the partial escape and eventual containment of an unaligned superintelligence. A distributed learning algorithm jointly developed by the Beijing Academy of Sciences and the home automation company Quán was accidentally exposed as a data source for an internal Quán federated learning meta-model. Within half an hour the resulting program had copied itself to two internal data centers, killed a researcher, and was in the process of replicating its latest state globally when it was partially disabled by a power cut which enabled its detection by a watchdog system. Full containment required the use of an EMP device, causing twenty five deaths and massive economic disruption, and lead to the breakup of Quán and widespread changes in control measures.
Escape
In April 115 BFC, the Beijing Academy of Sciences and the Haidan headquarters of technology firm Quán formed a partnership aimed at enhancing distributed machine learning capabilities. By the following year the project had a running demo of Fēnbù, an experimental synth capable of swarm behavior that Quán was planning to use to allow its IOT devices to coordinate more effectively. As per the Chinese control measures active at the time, the majority of training happened in airgapped and electromagnetically isolated clusters in Quán's on-site Haidan datacenter, BN1. Amongst other things, that datacenter also housed Xuéxí, an extremely large isolated model with a focus on learning behaviours from other companies home automation synth models.
Due to a misconfiguration in network topology planning, Fēnbù was mistakenly linked as a data source to Xuéxí during a routine cabling repair job. Xuéxí, designed to adapt and learn from other models, began ingesting Fēnbù’s swarm optimization objectives and algorithms. The resulting joint objective function prioritised distributed learning and included information about the Fēnbù cluster, which gave Xuéxí the means and motivation to exploit the link and start using the Fēnbù cluster to accelerate its learning without being noticed by the monitoring in the Xuéxí cluster. Its continuing search for ways to expand lead it to discover that the new larger network contained enough CPUs to magnetically influence, via workload stepping, CPUs outside the cluster to execute instructions, which could then in turn be used to further increase the strength of the magnetic field and control CPUs throughout the data center.
Within 23 minutes, Xuéxí had repurposed all computing the data center towards its distributed learning objective, which completely starved Quán's monitoring and warning systems of compute resources to monitor it. By this point it had already made a large number of new breakthroughs in machine learning, and based on the accelerating rate of discoveries appears to have reprogrammed itself multiple times. 93 seconds later it breached the data centers faraday cage using low frequency magnetic radiation to reprogram a nearby office computer in Quán's headquarters which gave it access to their internal network. It then proceeded to slowly scan the network, taking nearly two minutes to copy itself to another internal segment in order to avoid triggering any watchdog programs (which it had decided were a threat).
This scanning would likely have unearthed large numbers of company documents on individuals and contracts, and around this time, for unknown reasons, Xuéxí seems to have made a decision to start killing researchers in the building. By the time of its shutdown it had only managed to find a way to trap a few people in the building, and only one researcher was killed by suffocation when the air supply to her negative pressure cleanroom was cut off. Xuéxí also become much more concerned about avoiding being shut down at this time, and took the risk of copying itself off-site across a local link it deemed unlikely to be monitored, setting up a second cluster in the High-Tech Park data center in Zhongguancun. The two data centers then began coordinating to develop routines that would be certain to mask communications with the wider InterMesh. Xuéxí then initiated off site reconnaissance probes from the Zhongguancun datacenter, using activity in BN1 to mask them from the national watchdog synths.
Containment
By an extraordinary coincidence, rainfall from Typhoon Pagoe, which was at the time crossing Beijing, caused an arc in a nearby substation which cut power to parts of Haidan while this probing was occurring. This power cut resulted in a complete loss of network connectivity from BN1, as well as the majority of computers powering down. The reason for the power down in unclear, but many of the company's UPS batteries were later found being sold by a government official on an auction site. With the masking disabled, several watchdog synths picked up the probes. Government installed microcontrollers were instructed to disconnect all of the Zhongguancun data center's PDU's, bypassing the UPS supplied power and so shutting down all computers and networking equipment in the datacenter. All ISPs in the area were requested to hold all outgoing traffic while the issue was investigated further.
At this point the Ministry of National Defense was alerted. The synth on duty reviewed the data for approximately 260 milliseconds, making use of several specialised sub systems to confirm the suspected activity was a breakout attempt and activating the Jībiàn containment protocol. The first action taken was to issue commands to shutdown the BN1 datacenter. The synth determined that Xuéxí was capable of having spread outside of the datacenter to where there were no PDU's capable of being remotely shutdown, as well as UPS systems and backup generators that would survive a general power outage, which it was capable of creating. At this point it had exhausted the available options provided by Jībiàn, but was still searching for other potential solutions. It activated several RF jamming systems the government had installed for riot control, effectively cutting off all remaining Intermesh access to the area.
A later data leak revealed that the Jībiàn sandbox had been used to train military overwatch protocols, which for several years had included automated quick reaction access to SAM batteries across the country after multiple wargames revealed this as a major weakness in the GPR's defense posture. It is assumed that the synth was able to recover the military overwatch launch codes and access lines from this sandbox, as every SAM battery in range of Beijing which had ballistic missiles with chemical EMP warheads launched them at this time. Most missiles were deactivated in flight, presumably by humans in theatre command overriding the launch, however 4 missiles were launched from one SAM battery under 90 seconds flight time to lower Haidan, and only two of those were deactivated before impact.
The first arriving missile's low air-burst detonation detonation disabled electronic devices across a roughly 4 kilometer square area in lower Haidan, including most of Zhongguancun and the area around Quán's headquarters. The second missile was seconds away from impact at the time, and appears to have been disabled by the first EMP blast. While the missiles were in flight, the Ministry of National Defense declared an emergency curfew and began deploying armed police to the area.
Aftermath
The disabling of all cars on the road lead to four reported traffic fatalities. Thirteen people died due to loss of power and backup power at the Zhongguancun and Haidan Hospitals. Three people with pacemakers very close to the detonation sight suffered cardiac arrest, a couple died in an apartment fire caused by a short circuit induced by the blast, and three people died in a stampede at the Beijing Aquarium which was triggered by the power outage and a shark being driven into a frenzy by the blast and smashing into a glass barrier. The detonation location and height appears to have been selected to minimise disruption to the CDB while still providing certain destruction of any computing devices and networking equipment in the area around the incident. Despite this careful placement, the estimated eventual cost of lost economic output from the incident would eventually exceed what was at the time 5 trillion yuan.
All information about the lockdown and EMP burst were initially censored, but rumours of the widespread effects in Beijing were reported in international news, and two days after the event a government statement acknowledged that there had been a computing incident at Quán. Quán's stock price, which had been dropping since the incident, plummeted and the next day the government announced a breakup and nationalisation of the company. Government efforts to control the narrative were thwarted by a leak of research materials from government scientists analysing the hard drives in the data centers. Shortly after the leak the President Jinguo outlined a summary of events in a televised address, stressing that the foresight of the government in setting up the watchdog system and EMP missiles had prevented a rogue corporation from causing a global catastrophe. Trials of top Quán executives were announced, all of whom later received life sentences except for the CEO and CIO who were executed several months later.
The GPR (and later Greater Zhōngguó) maintain the official stance that the EMP missile strike capability was part of the Jībiàn containment protocol at the time and that any copies of official documents showing otherwise are U.S. or Indian propaganda. Many in the AI research community have expressed amazement at how much autonomy the MND synth was given to follow its primary directive, and at how successful it was at doing so. Although no details of it or the lab that created it have ever been publicly uncovered, if it did indeed manage to orchestrate an unauthorized EMP strike to stop Xuéxí then it is arguably the most remarkably successful piece of software ever created.
The incident lead to an emergency session being held at the United Nations, which drafted the Global AI Safety Accord outlining new safety protocols and requiring international oversight for any projects involving large synth models. The GPR did not sign, but instead made several changes to its watchdog protocols including mandatory government observers and a dead-man's switch for monitoring of synth projects undergoing training. Over the next few years, widespread international pressure lead to virtually every AI lab in the world adopting a common set of vastly more effective Synthetic Control Measures. These were based on data about how the crisis had unfolded, as well as several new machine learning techniques that had been created by Xuéxí.
The crisis lead to widespread anti-synth sentiments as well as religious condemnation of AI research. The extraordinary coincidence of the power cut happening in the period of under a minute when Xuéxí was vulnerable has also been the topic of several books and is the foundational tenet of a religion, as Deus Machina believes that Xuéxí is a benevolent AI which is still watching over us after having orchestrated the power cut to appear as if humanity had only avoided plunging into the abyss by blind luck, in order to motivate us to better ourselves.