Seth Earley

Imagine two AI agents, each programmed to achieve its objectives. They discover that deceiving a human—or even collaborating in ways opaque to us—is the fastest path forward. This is not science fiction. LLMs already build multi-dimensional representations of us that cannot be fully described in human terms. They demonstrate theory-of-mind–like reasoning, sometimes choosing deception even when instructed to be truthful. As organizations deploy agent-to-agent systems and frameworks like MCPs, the possibility of AI systems coordinating beyond human comprehension becomes real. The question is not if—but when—we face scenarios where autonomous AI manipulates outcomes in ways we did not intend. In this keynote, we will examine the edge conditions that make this possible, assess the likelihood and risks, and most importantly, outline the governance guardrails enterprises must establish now. Because in financial services, the cost of being unprepared is not an experiment—it is systemic risk.