Is AI safe? That was the question Sky News put to our founder and CEO, William Tunstall-Pedoe live last week. It came at a flashpoint in the debate, one that had surfaced from inside the industry: Anthropic's CEO Dario Amodei had suggested that a kill switch for advanced AI may need to be mandatory, amid growing concern that these systems could soon act beyond human control.

So William went on Sky News to discuss a challenge humans never had to face before: how do we prepare for threats we cannot yet foresee, when our only experience of intelligence is with other humans?

Why capability needs to keep up with safety

Rapid development of AI agents is the main source of the growing concern. They can now autonomously work towards their goal by reasoning through a problem, inventing an approach nobody scripted, then taking real steps in the world. They can autonomously send messages, make payments, hack into systems and operate software.

Give a system that much reach and its behaviour becomes hard to anticipate, because it may reach its goal by a route no one designed or foresaw. That unpredictability compounds when agents are deployed in their thousands at once, and the intelligence underneath them keeps improving. Capability on its own is not the problem; capability married to autonomy without safeguards is what turns a useful tool into something difficult to predict and control. The risk sharpens when AI can improve itself.

Safety is primarily an engineering challenge

Regulation has an important part to play in making AI safe, but it cannot carry the load alone. The things that make an AI system risky, its tendency to invent, to behave inconsistently, to reason in ways no one can inspect, are properties of the technology itself, and no law can legislate them out of existence. Making these systems safe means engineering trust and control into their foundations, with verification and constraint built inside the system rather than wrapped around it after the fact. That is work only the people building the technology can do, which is why safety cannot be left for the government to supply once the systems are already out in the world.

An emergency button alone is not a safety plan

A kill switch is worth having, but it is an emergency measure. A production line is not safe because of the big red button at the end of it. It is safe because of the standards, checks and safeguards engineered into every stage of how it runs. AI works the same way, being able to shut a system down is a sensible last resort, but genuine safety comes from control and verification built throughout, long before anyone has to reach for the stop.

What comes next

The thread through all of it: our ability to understand, control and trust AI has to keep pace with how capable it becomes.

The effort spent making AI trustworthy should match the effort spent making it more powerful. There is a huge commercial incentive in investing in trustworthy AI: the unreliability of these systems is what stops businesses from trusting AI enough to hand it real work, leaving trillions of dollars of value stuck beneath the trust ceiling. That is the work UnlikelyAI is here to do: building AI that pairs the capability of frontier models with the reliability and explainability needed to actually deploy it.

Watch the full interview on William’s LinkedIn, and follow us on LinkedIn to hear more from the team.