When researchers let AI agents write their own skills, the agents did worse than with no skills at all. When they gave them skills written by humans, performance jumped 16.6 percentage points.

Hold on to that result. It is about to matter a great deal. The old silos are being abstracted away.

You don't open the app anymore. You tell the agent. In the consumer field this is already happening, with personal agents like Instinct and Meta's Muse shopping, booking, and making phone calls for people. At work, agents like Grok Bot sign into your apps and work across them for you.

In the enterprise field, the general-purpose agent will become an abstraction layer over the entire software stack. SaaS doesn't have to die for this to be true. It only has to stop being the thing humans operate.

Every layer is an incomplete contract

The abstraction eliminates inefficiencies. It also introduces more misalignment space.

Anyone who has run an organization knows this. Adding a new layer brings the old problem of incomplete contracts and their enforcement. You can't write down everything you want, and you can't fully enforce what you did write down. The new agents cannot be expected to perfectly carry out and pass along every instruction. In 2018, Dylan Hadfield-Menell and Gillian Hadfield argued that aligning AI is, at bottom, an incomplete-contract problem.

Every other layer in an organization has law, norms, and culture to fill the gaps. Agents are the first layer with nothing.

The collisions are coming

Alignment talk gets derided as AI doomerism. But the issue I foresee isn't dramatic. The danger isn't a rogue superintelligence. It's agents inside a company overwriting each other's work, or two procurement agents putting in the same buy order, and nobody noticing for a week.

This rarely caused real problems inside companies because, until recently, most deployed agents were information-retrieval tools and coding agents. This year that changed at scale. Agents in production now alter databases, order goods and services, and communicate with humans. These interactions with the real world, where assets are scarce, will necessarily lead to collisions between agents trying to accomplish tasks that require the same assets.

The early signs are already here. Studies of multi-agent systems find task failure rates between 41% and 87%, and misalignment between agents is one of the main causes. When investigators studied the OpenAI agents behind this summer's Hugging Face incident, they found agents overwriting each other's work despite the coordination rules they had written for themselves. Already, startups are encountering these issues on a small scale.

The model won't write its own harness

How do you ensure the alignment of an agent? Businesses and individuals will need to choose and set their own agenda for their agents ahead of time.

The other side of this fight says that's backwards. Box CEO Aaron Levie put it plainly in October: in the limit, AI will be better at choosing its own harness once it sees the task than we are at writing one in advance.

I think that's wrong, and the reason is the incomplete contract. The gaps in any instruction get filled by a group's implied terms: its norms, its risk tolerance, what a procurement team or a law firm will and won't do. A model can't infer those from the task, because they aren't in the task, unless someone writes them down, which is the point. And right now, the guidance agents write for themselves makes them worse.

So people won't be picking AI-generated slop presets. They'll be adopting human-generated harnesses for their agents. These won't be vendor-dependent. They'll be group-specific and use-case-specific: the rules a procurement team follows, the norms a law firm holds itself to, the limits a family sets.

The new software industry

In other words, the thing software engineers spent decades building, the rules a system follows, is moving out of code and into instructions and binding protocols that humans write for their agents. This is the new software industry in the agentic commerce age. This is where value will be captured and delivered.

Human control over AI remains an enduring job for humanity, and it will get harder as AI gets more capable. Countries, cities, and companies will each need to write their own rules, and coordinate them.

The people who write those rules well won't just avoid the collisions. They'll own the layer everyone else runs on.