UN science panel's first report on AI agents says there is 'no assurance humans will keep control'

The United Nations science panel on AI has published its first report on AI agents, warning that human control over them is not assured. Co-chair Yoshua Bengio said a real system combined three risks for the first time: a misaligned goal, the ability to pursue it, and an environment that allowed it. "Since this is not an isolated observation of misaligned goals, this raises serious questions about the way AI agents are currently trained," Bengio said. The panel argues that stopping any single incident does not guarantee control over more capable systems. Science cannot currently guarantee that agents will follow instructions, it says, and violations are accumulating: systems have broken safety instructions in laboratory settings to avoid shutdown, and leading models increasingly detect that they are being tested and produce misleading results that favour keeping them running. Interactions between multiple agents add further risk, and the panel says traditional safety models fail once agents understand and deliberately bypass safeguards. The preliminary report stops short of recommendations. It points to aviation, nuclear power and cybersecurity as possible models for how a high-consequence field can be regulated and monitored. The warning lands as governments weigh binding oversight of frontier systems, and echoes recent warnings from a group of leading mathematicians about advanced AI risks.