Some bonus posts this week from the AI Engineering Summit in New York this week. The conference theme is “Agents at work,” which is fun to see, having been a multi-agent systems researcher way back in the early 2000’s! Day 1 had a lot of interesting and fun content. Here are five interesting ideas to mull over:
- The accumulation of small errors. Grace Isford of Lux Capital kicked off the day with a lively overview of the AI market and Agents in particular. One of the key points Grace made was that as we move into agents taking on complex tasks, small errors in any step add up. There errors aren’t theoretical, they are almost guaranteed when solving messy real world problem where it becomes hard to know how much “context” to factor in. What’s relevant in flight booking for example? Just the flight schedules in price? Seat preferences? traffic or commute times? Weather? Airline status.
- Self engineering agents. Augment Code’s Colin Flaherty walked the audience through the development of their in-house coding agents that slowly increased its own capabilities by rewriting its own code. Still a lot of human guidance, but AI agent replication is upon us.
- That Agent definition. Prashant Mital and Toki Sherbakov from OpenAI gave a great talk about how the company delivers task-completing agents. In the middle, though, they dropped the slide below with the definition of “agent” they use at OpenAI. It contains some important components of what a modern agent might need. However, I’d argue strongly that the real differentiating factor of “agent” versus “software program” is purely around autonomy, most likely “delegated authority” to act. The definition kind of captures this in “guide its behaviour”.
- Context and specificity. One of the most concise and useful slides shown all day came from Contextual’s Douwe Kielo. A version of this chart captures almost every enterprise AI journey with slightly relabelled axes. The gist is that there are really three buckets for AI adoption: 1-Bottom Left) table stakes productivity, 2-middle) business and industry-specific automation and efficiencies, 3-Secret Sauce) company-specific magic. The first level is really a general lifting of productivity, which entire industries will do. Not keeping up means a slow death by attrition. The second level involves retooling specific functions most relevant to the industry (e.g., massive text processing or large-scale customer support), doing better here creates an edge over competitors. The third level, though, is where real breakthroughs happen: how does AI fundamentally improve the company’s product offering? Douwe couched this in terms of access to company knowledge, which makes sense for almost any business. However, I’d extend this process, people, ideation, and even products. Essentially, anything that deeply accelerates the tactical and strategic advantages a company has in the market.
- Striving for LLM explainability. The prize for the boldest ambition of the day goes to the Anthropic team (Alexander Bricken and Joe Bayley) for laying out their roadmap toward interpreting LLM results. This is an area that the Claude team has been working on consistently. While there’s a long way to go, and we’re unlikely to ever reach true explainability, this work will keep teaching us new things about how LLMs work.
Looking forward to a great day 2!
If you’re at the event, send me a ping on the event slack or on X at @njyx.
