In our latest UnlikelyAI webinar, Callum Hackett (Head of Research, UnlikelyAI) and Matt Bongiovi (Research Engineer, UnlikelyAI) were joined by Francesco Leofante (Assistant Professor, Imperial College London; Co-Director, Centre for Explainable AI) to unpack what explainability really means in enterprise settings and why it’s now central to trust, regulation, and product adoption.

Across the conversation, they explored how explainability is evolving from a broad principle into something operational:

• Regulators trying to define what “good” looks like without freezing innovation

• Product teams building systems users can actually work with and rely on

• High-stakes industries (finance, insurance, critical infrastructure) where decisions must be justified and contested

• AI researchers pushing beyond “one-shot” explanations toward robustness, consistency, and usable interaction

Below is a recap of the top takeaways, followed by a curated list of the strongest quotes and learnings from the session.

👉 Click here to download the guide

Top takeaways

1) Explainability drives adoption

Explainability has shifted from a research concern to a practical requirement for deploying AI in real organisations. As AI systems become more widespread, their limitations are becoming more visible and trust is now the bottleneck to adoption.

““Explainability gives everybody who interacts with AI leverage over those systems. The more you understand, the more you can improve the things that you’re building.”— Callum Hackett, Head of Research, UnlikelyAI”

2) “Safe AI” without explanation still fails in practice

Even models that are technically correct or formally verified can fail in practice if humans cannot understand or justify their outputs. In high-stakes environments, explainability is a condition for use not an optional extra.

““If you have a model that is provably safe, but then you cannot explain the outputs to humans, nobody’s going to trust it. Nobody’s going to use it.”— Francesco Leofante, Assistant Professor, Imperial College London”

3) Regulation needs shared language

Regulators are increasingly focused on explainability, but what counts as a “good explanation” is still being defined. Progress depends on collaboration between policymakers, researchers, and industry to establish common ground without stifling innovation.

““We need to regulate the use of AI without stifling innovation, but at the same time make sure that whenever we deploy AI, we are in control.” — Francesco Leofante, Assistant Professor, Imperial College London”

4) Explanations must be usable

An explanation that is technically correct but unusable doesn’t solve the real problem. Users need explanations that support interaction working alongside AI systems, questioning outcomes, and acting on results.

““There are use cases where people are working alongside an intelligent system, and they need to understand how it’s arriving at its outputs so they can work with it effectively.” — Matt Bongiovi, Research Engineer, UnlikelyAI”

““You can’t think of an explanation as something static. Understanding how decisions are made in complex systems requires conversation.”— Callum Hackett, Head of Research, UnlikelyAI”

5) Structure enables trustworthy AI

Neurosymbolic approaches combining structured, symbolic reasoning with statistical models offer a path to AI systems that are both flexible and auditable. By making reasoning explicit upfront, explainability becomes a built-in feature rather than an afterthought.

““You know exactly what the decision process looks like, and you can explain how you arrived at a specific result.” — Matt Bongiovi, Research Engineer, UnlikelyAI”

Looking ahead

The webinar closed with a clear direction of travel:

• For LLMs, consistency and reliability are central research problems, not edge cases.

• For regulated industries, explainability is moving toward systems that support contestability and governance, not just post-hoc justification.

• For product teams, neurosymbolic methods are emerging as a practical design pattern: explicit reasoning where it matters, statistical models where ambiguity demands it.