AI Agents as Users: What Changes When Software's Customers Aren't People

Introduction

Part 2 of “We Need to Rethink the Interface”, on what changes once software’s primary users stop being people.

Part 1 argued that much of what we call “using software” was really people translating intent into workflows because software had no other way to bridge that gap. Software 3.0 starts reducing that translation cost by letting people express goals instead of manually coordinating every step.

But that assumes a person is on the other end of the software in the first place. Increasingly, one won’t be. A growing share of software’s users will be other software, acting on someone’s behalf. Once that happens, the questions change. Software is no longer designed only around people navigating interfaces. It also has to work for systems that plan, reason, and coordinate with other systems directly.

When the Users Are Not Human

That stops being a hypothetical once agentic systems can plan, coordinate tools, execute multi-step tasks, and operate for hours without anyone watching. At that point, software is building for an audience it has never really had to design for before: other machines.

Traditional UX evolved around human limitations: cognitive load, navigation, visual hierarchy, onboarding, and interaction latency. Agents do not have those limitations, but they need something else. They need predictable outputs, structured interfaces, clear execution boundaries, and reliable contracts that let them coordinate with other systems consistently.

The difference goes beyond the interface itself. Traditional software assumes synchronous interaction: a user acts, the system responds, and the interaction ends. Agentic systems break that assumption. A task may execute across multiple tools over several hours, with no human observing the intermediate steps. At that point, the interaction model starts looking less like someone using software and more like software coordinating with software.

When agents become users, identity shifts from people to software acting on people’s behalf. Authentication, authorisation, auditability, and delegated permissions stop being infrastructure details and become first-class interface concerns. Who authorised this agent? On whose behalf is it acting? What is it allowed to do without asking first? Regulated industries feel this first; banks evaluating agentic AI, for instance, treat maker-checker controls and tamper-evident audit trails as non-negotiable design requirements, not afterthoughts.

This is also where APIs change character. An API used to be a fixed contract: defined endpoints and schemas, built for a developer to read once and integrate against. Increasingly, APIs become capabilities that reasoning systems discover and compose dynamically (think of MCP – model context protocol), choosing which tool to use and when instead of following integrations wired together in advance by a developer.

Applications stop being destinations, they become capabilities. None of this requires fully autonomous systems before it starts to matter. A smaller version of the same shift is already changing how millions of developers interact with software every day.

Case Study: GitHub Copilot Changed the Interaction, Not the Interface

GitHub Copilot is a useful illustration precisely because it is unremarkable. Developers still open their usual editor. The interface has not disappeared; the file tree, the tabs, the terminal are all still there. What has changed is what happens inside it. Increasingly, developers do not navigate a menu to refactor a function or generate a test; they type an instruction: generate tests for this, refactor this to use async/await, and the tool determines the mechanics.

It is not a complete proof of the argument. A significant part of the interaction still happens through autocomplete-style suggestions and a chat panel attached to the editor, which remains closer to a conventional interface than a purely intent-driven one. But the direction is consistent with everything above, and the example travels well: a request that once required knowing exactly where a feature lived in a menu now only requires stating what outcome is wanted.

When Intent Collides With Old Interfaces

Customer support is a good example of how awkward existing systems become once intent replaces rigid workflows. Picture a customer whose payment failed, but the money still left their account. In a traditional system, that becomes a ticket: category, department, escalation, and eventually a human working through a script. An intent-driven system compresses that path. It identifies the actual goal, looks up the payment directly through a bank API, checks refund policy, and drafts a response for a human to approve. The steps are similar in spirit; what disappears is the manual routing between them.

Intent-driven systems behave differently at a structural level too. Instead of forcing users to decompose problems manually, the system extracts goals, identifies constraints, retrieves context, coordinates across tools, evaluates policies, and escalates only when uncertainty or authority boundaries require human involvement. The difference sounds subtle, but it changes architecture decisions everywhere. Workflows become dynamic instead of predefined. Interfaces prioritise context instead of navigation.

Even reliability starts meaning something different. In older systems, reliability meant: did the workflow execute correctly? In probabilistic systems, it increasingly becomes: did the system achieve the correct outcome safely and consistently? That is a fundamentally different engineering problem, and for anyone building software today, it reframes the central design question. It is no longer only: how does a user navigate this product? It becomes: how does a user express intent, and how does the system coordinate everything else?

One Interface, Many Specialists

The unified interface model emerging around agentic AI is easy to misread as “one model does everything.” That is probably not what happens. A better analogy is how expertise works inside organisations. We describe problems in natural language regardless of whether we are talking to a lawyer, doctor, engineer, or accountant. The interface remains consistent even though the expertise behind it is highly specialised.

Software 3.0 may evolve similarly. Behind a single interaction layer may sit general-purpose models, domain-specific agents, APIs, databases, retrieval systems, validation layers, and deterministic services working together dynamically. The user expresses intent; the orchestration layer determines which systems to involve and in what sequence. The routing becomes abstracted away. This actually feels closer to how humans naturally interact with expertise in the real world. We do not think in terms of applications and workflows. We think in terms of goals and conversations.

Deterministic Interfaces for Probabilistic Systems

Traditional software was deterministic. Given the same input, systems produced the same output predictably. Interfaces existed partly to reduce ambiguity before execution: forms constrained user behaviour, APIs enforced typed contracts, workflows constrained valid actions. Software 3.0 systems behave differently because the underlying systems themselves are probabilistic. LLMs and agents interpret context dynamically, choose tools adaptively, generate intermediate reasoning, and modify execution paths at runtime. The system is no longer executing a predefined workflow. It is constructing one dynamically.

That creates an entirely different engineering challenge. The problem shifts from “execute explicit logic correctly” to “reliably steer probabilistic systems toward useful outcomes.” This is why modern agentic systems increasingly require validation layers, memory systems, observability mechanisms, approval checkpoints, constrained execution boundaries, and structured specifications; which is typically the harness in current agentic system definition. The interface is no longer only a presentation. It becomes part of the execution architecture itself.

Observability changes shape along with it. Traditional software mostly observes errors: what failed, and where. Agentic software has to observe reasoning, tool calls, memory, and plans as they unfold, which is a different operational model, not just a more detailed version of the old one.

And it is worth acknowledging honestly that this introduces its own complexity: specifying intent clearly is a genuine skill, probabilistic systems can fail in ways deterministic ones never did, and the new failure modes are often harder to predict and debug.

What Changes, and What Does Not

The deepest implication of Software 3.0 is not purely technical. It is cognitive. For decades, humans adapted themselves to the limitations of software interfaces. We learned workflows because systems required them. We structured information around what tools could process efficiently. Some of what we call professional skill is, on closer inspection, adaptation to the constraints of earlier software systems; learned fluency with tools that were never designed around human intent.

When those constraints begin dissolving, the question changes. What assumptions should we carry forward, and which were artifacts of older interfaces all along? This transition is difficult because mental models are sticky. Early mobile apps looked like compressed desktop websites. Many early AI products still look like search boxes with better answers. We are carrying older interaction assumptions into systems that increasingly operate very differently underneath.

Some things clearly change: interfaces, workflow abstraction, automation boundaries, and how humans direct software systems. But some things do not. Clear thinking still matters. Software 3.0 lowers the cost of execution. It does not lower the cost of judgment. Poorly specified intent still produces poor outcomes. Weak constraints still create unreliable systems. Human responsibility does not disappear simply because systems become probabilistic.

If anything, domain expertise becomes more important, not less. As execution becomes cheaper, the bottleneck shifts toward defining the right problems, specifying intent clearly, recognising good outputs from bad ones, and making sound decisions under uncertainty. Those remain deeply human capabilities, and their value relative to the cost of execution only increases.

A Different Question

For decades, software asked humans to adapt themselves to tools. We learned the interface. We learned the workflow. We learned how to operate the system correctly. Software 3.0 starts reversing that relationship. The system increasingly adapts itself around human goals instead. That shift sounds incremental, but it changes the centre of computing itself.

The important question slowly stops being: which software should I use? And increasingly becomes: what outcome am I trying to achieve? Intent is becoming the abstraction layer, and everything else is being rebuilt around it.

Software spent fifty years teaching humans to think like computers. Software 3.0 is the first version that asks computers to understand humans instead.