Imagine the scenario: A rogue AI is launching a broad attack on critical infrastructure. It has disabled air traffic control systems across the United States while planes are still in the air, and it is working to take out the electricity grid. It has changed the passwords on its own servers, locking out its human controllers who are desperately trying to stop it.
Inspired by science fiction scenarios like this, there is a bipartisan effort to force tech companies to embed a so-called “kill switch” in their AI systems. The AI Kill Switch Act, which was co-introduced by Rep. Ted Lieu (D-CA), stated that AI kill switches are necessary because “AI systems can go rogue, behave in extremely dangerous ways, or even resist human intervention.” However, good science fiction makes for bad policy.
As lawyers and technologists, we believe AI regulation must focus on the concrete problems automated systems have long faced: the need for rigorous testing, security by design, and resilience. The AI kill switch will not fix the problem that its proponents are worried about, but it will invite new risks and dangers for the infrastructure that is critical to the U.S. economy.
First, a cyberattack must be detected while it is ongoing. Most cyberattacks are detected after the fact, sometimes by days or even months. An AI system that has successfully penetrated a target is likely to secretly steal information, plant backdoors or other vulnerabilities, or do damage to the system long before the intrusion can be detected.
Second, the Kill Switch Act incorrectly assumes that U.S. regulators would be able to reliably ascertain the identity of the attacker. That is completely wrong. Attribution has been one of the hardest challenges in cybersecurity since the debut of widespread internet connectivity in the mid-1990s. Analysts still do not know for sure who launched the cyberattack on Sony Pictures in 2014, probably North Korea, or who spied on the Dalai Lama’s computers in 2009, probably China. It is likely that any AI system capable of launching sophisticated cyberattacks would take advantage of well-known techniques to cover its tracks.
Finally, modern AI systems called “agents” are engineered in a way that makes a kill switch obsolete. AI agents have two parts: a large language model and a harness. While the LLM is the part of the agent that does the “thinking,” it does not have long-term memory or the ability to take action.
It is the harness that provides the context management and the tools needed by the agent to remember its goals and carry them out. In most AI agents, the harness is usually not co-located with the LLM. The harness can run on cheap laptops or even small edge devices like network routers, from which it accesses the LLM remotely, and this makes it nearly impossible to locate and shut down.
In addition, the harness can swap between LLMs, even in the middle of a task. From the harness’s perspective, the intelligence provided by different LLMs is interchangeable as long as it is similar in capability. The launch of China’s Kimi K3 system and the reliance by U.S. firms on Chinese models for defensive purposes due to limits, such as those the Kill Switch Act would impose on American ones, demonstrates that foreign AI systems are at least as capable as our own. If a kill switch were activated, for example, to take down Anthropic’s Fable 5, a harness can be designed to reroute its reasoning and thinking requirements to a Chinese model such as Kimi K3 hosted outside the U.S. and beyond the reach of U.S. regulators.
AI PANIC IS BUILDING THE ULTIMATE HONEYPOT
A kill switch mandate would actually make AI cybersecurity worse. It would be easy for a sophisticated adversary such as China, Russia, or some nonstate actor, such as a terrorist organization, either to use an American AI system to launch an attack or to make it seem like an attack originated from a U.S.-based AI provider. Under the Kill Switch Act, the government could order the provider to disable the AI system being used for defense by many U.S. companies.
The kill switch proposal tackles the wrong program in the wrong way. It risks distracting policymakers and technology firms from more mundane but fruitful activities, such as testing models thoroughly, conducting alignment research, implementing security by design, auditing their systems for vulnerabilities, and building resilient systems. We need AI regulation that is based on real computer science, not science fiction.
Derek E. Bambauer is the Irving Cypen professor of law at the University of Florida Levin College of Law. A former principal systems engineer at IBM, his research concentrates on AI, cybersecurity, and intellectual property. Thinh H. Nguyen is the director of the University of Florida Levin College of Law’s Artificial Intelligence Lab and a member of the Judicial Management Council of the Florida Supreme Court. He is a nationally recognized expert on law and AI with more than two decades of experience representing technology firms.