--- type: "article" title: "Agent Experience, defined" summary: "There is no Agent Experience without Developer Experience" newsletter: "The Weekly Interface" newsletter_handle: "the-weekly-interface" newsletter_url: "https://usecommune.com/n/the-weekly-interface" author: "Samir Amzani (@samz)" published: "2026-10-23T16:57:27.589Z" canonical_url: "https://usecommune.com/n/the-weekly-interface/a/2026-10-15-wtf-is-agent-experience" markdown_url: "https://usecommune.com/n/the-weekly-interface/a/2026-10-15-wtf-is-agent-experience.md" chat_url: "https://usecommune.com/n/the-weekly-interface/a/2026-10-15-wtf-is-agent-experience/chat" body_source: "native" likes: 1 replies: 1 body_words: 1620 --- # Agent Experience, defined ![](https://fmxorstnlyoqpqdftivs.supabase.co/storage/v1/object/public/article-images/ce4dc54b-7510-4b7c-8a4f-2f24438a73bb/d77d9345-3400-4ff4-9f8d-1060e1b73a25/1792774537256-ybqcbay9.jpg) In February 2024 I wrote that great developer experience means focusing on "your developer's perceptions that trigger their positive emotions" ([How to build an awesome Developer Experience](https://pragmaticapi.substack.com/p/how-to-build-an-awesome-developer)). Today, **70%** of the CLI requests Stripe sees for API resources come from a user that has no emotions at all ([Stripe](https://stripe.com/blog/stripe-projects-adds-new-agents-providers-developer-controls)). That user is a coding agent. Its side of the story now has a name: Agent Experience, or AX. ## What AX means Mathias Biilmann, Netlify's CEO, coined the term in [January 2025](https://biilmann.blog/articles/introducing-ax/): "the holistic experience AI agents will have as the user of a product or platform." He picked "Agent" over "Agentic" on purpose. His argument, [on LinkedIn](https://www.linkedin.com/posts/mathias-biilmann-christensen-a5a3805_ax-agent-or-agentic-when-i-coined-ax-activity-7370875844073803777-bFOz): "We don't talk about Useric Experience. We don't talk about Developerish Experience." We talk about users and developers. Now we talk about agents. A year later he wrote the line I keep coming back to: "All product have an Agent Experience, it's purely a question of whether it is good or bad" ([One Year of AX](https://biilmann.blog/articles/one-year-of-ax/)). Your API already has one, whether you've looked at it or not. The agents are already there. Vercel updated its April post: as of September, more than half of its deployments are started by coding agents ([Vercel](https://vercel.com/blog/agentic-infrastructure)). Cloudflare counts agents at 48% of Wrangler usage, up from 25% in March ([Cloudflare](https://blog.cloudflare.com/cloudflare-cf-cli-launch/)). Coverage of Supabase's October round puts agents and AI tools behind about 70% of its new databases ([Pulse 2.0](https://pulse2.com/supabase-raises-150-million-and-acquires-turso/)). My answer is short. DX is the soul, AX is the body. You can't feel an agent's experience, but you can measure it the way Accelerate measured software delivery, with four numbers. ## DX is the soul, AX is the body I still stand by the 2024 definition. There is no AX without DX, because agents are still orchestrated by humans. A person still picks the agent and reads what it built. Zeno Rocha from Resend, quoted by [Bessemer](https://www.bvp.com/atlas/developer-laws-in-the-ai-era), said it from the other side: "all the changes we made to DX are actually enhancing AX." People spend a lot of words arguing whether AX is a new discipline or rebranded DX. I think the debate is useless. Anything you do to improve your DX is good for AX. The 2024 article also had a practical half. Map the developer journey, flag each step red, amber or green, and fix the red ones first. Then measure three stages: onboarding (time to first "Hello world"), adoption (time to fix an error) and contribution (time to first PR review). The emotional part was always the hard one to measure, and with an agent there's nothing there to measure: it doesn't get frustrated, it burns tokens and retries. What's left is the body, and the body you can count. ## What Accelerate did for delivery We've been here before. For years teams argued about whether they shipped fast or shipped well. Then Nicole Forsgren, Jez Humble and Gene Kim wrote Accelerate and reduced delivery performance to four numbers ([five-minute summary](https://roman.pt/posts/accelerate/)): - **Lead time**: from code committed to running in production. - **Deployment frequency**: how often you deploy. - **Mean time to restore (MTTR)**: how fast you recover from an outage. - **Change fail rate**: what share of deploys break something. High performers deploy on demand, keep lead time and MTTR under an hour, and break things on 0 to 15% of changes. Two lessons from the book carry over to AX. "The research has shown that there's no tradeoff between tempo and stability." And measuring teams "by lines of code (!), velocity, or utilization is a bad idea." Waldek Mastykarz at Microsoft says the same thing about agents: "Best practices are hypotheses until you measure them" ([Microsoft](https://developer.microsoft.com/blog/what-is-agent-experience-ax/)). ## Four AX numbers Each one starts from an Accelerate metric and asks the same question about the agent calling your API. You won't get these from your access logs. You get them the way Microsoft and Supabase do: give agents real tasks against your API, run them many times, and score the results. ### 1. Deployment frequency becomes selection rate This is the loosest of the four: Accelerate counts how often you ship. For AX, count how often an agent picks your API or your tool when it could do the job another way. Microsoft calls this propensity: "Does the agent find your technology and choose it?" Agents choose based on what they read. Supabase's own evals found its Postgres best-practices skill loaded in about one session in ten. After they rewrote the skill's description, the same skill loaded in 60% of sessions ([Supabase Evals](https://supabase.com/blog/introducing-supabase-evals)). They also found that "Claude Code checks our docs in under 40% of scenarios even with skills loaded." Selection also decides how the agent integrates once it has picked you. Google built its Gemini Docs MCP server so that it "nudges agents toward using the SDKs instead of hallucinating their own integrations" ([Speakeasy](https://www.speakeasy.com/blog/partnering-with-google)). ### 2. Lead time becomes time to first working call In 2024 I tracked time to first "Hello world". For an agent, count the turns, tokens and dollars between the task and the first call that works. Nader Cserny calls it cost per successful task ([nader.io](https://nader.io/posts/agent-experience/)). The gaps are large. AXI, a set of design rules for CLIs used by agents, reports a test of the same GitHub tasks run three ways. Its `gh-axi` tool reached 100% success at $0.050 per task. The `gh` CLI reached 86% at $0.054. MCP servers landed at 82 to 87%, for $0.101 to $0.148 ([AXI](https://axi.md/)). That's one model on read-only tasks, run by the people who built `gh-axi`, so read it as a hint. We hit this at Apideck. Our MCP tool definitions cost 55,000 tokens of context before the agent did anything useful ([Apideck](https://www.apideck.com/blog/mcp-server-eating-context-window-cli-alternative)). That number is why we built a CLI. ### 3. Change fail rate becomes silent failure rate Microsoft calls it efficacy: "Once the agent uses your technology, does it use it correctly?" Their example is a scaffolder without a pinned version that "created a project on a version from July 2020 instead of the current one, and the agent took it as a success." The failures are silent. Ant Group's [Alipay-PIBench](https://arxiv.org/abs/2607.14573) tested coding agents on payment integrations. On Basic In-App Payment, agents scored 83.0% on static checks and 38.6% once the integration actually ran. A [TU Darmstadt study](https://arxiv.org/html/2509.20172v6) found GPT-4o wrote fully correct raw HTTP integration code 60% of the time, and that 14 to 39% of the generated URLs were illegal. Michael Carducci has my favourite example. He works on a CRM for professional entertainers. An agent asked the API for all "past gig" events, then told a user about every recent gig except the one he did last night. Carducci's verdict: "it wasn't a hallucination, it was my system giving a poor agent experience" ([LinkedIn](https://www.linkedin.com/posts/michaelcarducci_is-ax-agent-experience-a-thing-i-work-activity-7492754079488802817-9vX6)). Measure this one carefully. Microsoft's new AX playbook warns: "We've seen perfect scores for code that never compiled" ([AX Practitioner Playbook](https://developer.microsoft.com/blog/introducing-the-agent-experience-ax-practitioner-playbook/)). ### 4. MTTR becomes time to recover from an error MTTR is how fast you restore service. For AX, ask how many retries and turns an agent needs to get from an error back to a working call. My 2024 version was time to fix an error. Cserny tracks retry rate. Errors are where you can move this number. Cloudflare now returns RFC 9457 problem details to agents, with fields like `retryable`, `retry_after` and `owner_action_required`. Its rate-limit error went from 14,252 tokens of HTML to 256 tokens of JSON ([Cloudflare](https://blog.cloudflare.com/rfc-9457-agent-error-pages/)). As they put it, "The schema is stable by design, so agents can implement durable control flow." Knut Melvær compares a generic 400 "Invalid request" with a problem details response that lists the valid values and a hint: "Same HTTP status code. Wildly different outcome." The agent, he writes, "goes from spinning to self-correcting" ([knut.fyi](https://www.knut.fyi/blog/2026-03-30/agentic-developer-experience-starts-with-your-system)). This is where Accelerate's lesson shows up again. Better errors cut both lead time and failures. ## Measuring the body can hurt the soul The strongest objection comes from Jim Nielsen. He accepts AX, but ranks it below UX and warns that it may make UX "take a back seat", until "most software will become even more confusing and obtuse to humans" ([AX, DX, UX](https://blog.jim-nielsen.com/2025/thoughts-on-ax/)). There's already a live example. Ben Swerdlow from Freestyle argues that for agents, "Defaults are bad": an agent can read the docs and fill in every field, and "Explicitness is cheap now" ([Designing APIs for Agents](https://www.freestyle.sh/blog/opinion/designing-apis-for-agents)). On [Hacker News](https://news.ycombinator.com/item?id=48894874), developers pushed back: dropping defaults "makes it impossible to quickly debug the API call (e.g. with curl)". Push one of the four numbers on its own and you can trade away the human DX underneath it. The numbers are young, too. Most AX benchmarks today are run by the vendor being measured, like Supabase's evals and AXI. And Microsoft has already seen perfect scores for code that never compiled. My answer comes back to the stance. **There is no AX without DX**. An AX change that breaks the human path fails by definition. Track the four numbers alongside DX, never instead of it. ## Same soul, new body My definition of DX hasn't changed. It's still how your developers feel. What's new is a body you can measure: 1. Selection rate: does the agent pick you? 2. Time to first working call: how many turns, tokens and dollars does it take? 3. Silent failure rate: how often does it look right and break? 4. Time to recover: how fast does it get back on its feet after an error? I'm running a few experiments in this category, and I'll share the first results in the coming weeks. Until then, I'd love to hear from you: What's the worst thing an AI agent has done with your API, and how are you solving it now? *** ## Discussion (1 reply) Numbering shows the reply tree: `2.1` is a reply to `2`. Message text is reproduced verbatim as posted. ### 1. Fran Mendez (@fmvilas) - 2026-10-23T21:54:59.064Z Interesting. I thought there would not be any difference between MCP and a CLI. Why is that? Why are MCP more expensive?