As AI agents become more prevalent and powerful, a new paper outlines a framework for better-understanding how to govern and regulate them. From the Future of Being Human Substack.
When OpenAI’s CEO Sam Altman wrote back in January that “in 2025, we may see the first AI agents ‘join the workforce’” he effectively predicted this to be the year of agentic AI.
While much of Altman’s new year’s reflection still reads more as hype than reality, interest in artificial intelligence systems that can make their own decisions and influence the environment they operate in has accelerated over the past few months.
Reflecting this, the latest versions of reasoning models from companies like OpenAI, Anthropic and Google exhibit a growing ability to decide how they’re going to solve problems and achieve the goals they’re set. And the release of the agentic AI Manus just a few weeks ago is shaking up what’s possible when an AI is given access to compute and the internet, and the license to interpret what it’s being asked to do and how it will achieve this.
These technologies increasingly appear to be forerunners of an agentic AI wave that could transform how artificial intelligence is used everywhere from research and teaching to business, healthcare, government and much more. As this wave grows though, there are growing questions around how to ensure AI agents lead to positive outcomes — and don’t cause unnecessary harm.
In effect, the question of how we govern emerging AI technologies that are capable of deciding for themselves what they do to achieve their ends when their actions have very material consequences is becoming increasingly important.
Yet, despite the speed with which agentic AI is developing, there are no easy answers to how to ensure its safe and responsible use. In part this is because we are in new territory here — we’ve never had the ability to create machines that can decide on their own how to solve problems, and then — without human supervision — begin to alter the world around them to do this.
But even before we can grapple with challenge this a problem: there is no current accepted and easy-to-apply definition of an AI agent that is designed to support informed oversight and governance.
In other words, we’re not even sure yet how to formulate the problem of AI agent governance, never mind work out how to address it.
To address this, Atoosa Kasirzadeh from Carnegie Mellon University and Iason Gabriel from Google DeepMind have just published a paper that sets out to develop a framework for informing the oversight and governance of AI agents. And while it’s just one step toward ensuring agentic AI is developed and used responsibly and beneficially, it is an important one …
Related:
US and China Vie for AI Leadership in K-12 Education
Two new national plans promise to overhaul classrooms with AI. Here’s how the US and China differ – and overlap.
What does responsible innovation mean in an age of accelerating AI?
The new AI 2027 scenario suggests artificial intelligence may outpace our ability to develop it responsibly. How seriously should we take this?
Reimagining learning and education in an age of AI
Reflections and provocations from a keynote given at the 2025 Yidan Prize Conference.
More on artificial intelligence at ASU.