This week: OpenAI releases GPT-6 Astra (and reaches AGI? Maybe?), Claude builds a pretty cool model of a combustion engine, and SpaceX to start making gas turbine blades.
On to this week's main stories:
- The Rise and Fall of Agent Civilizations. This article by Dwarkesh Patel has gotten a lot of attention this past week for its analysis of public information on the OpenAI / Hugging Face security incident. The article describes different groups within OpenAI collaborating on message boards and apparently gaining administrator privileges on a research compute cluster. The breaches and hacks are certainly impressive, and it shows what happens when complex, goal-driven systems interact. A lot of people disagree with the characterization of "civilization" in the title, and it definitely has a clickbait element. Still, people don't seem to grasp that long-running tasks with guidance to simply maximize an outcome give you a system that interacts iteratively with others. Multiagent systems are way more complicated than just a collection of powerful single agents.
- Discovery of a new OpenAI agent message board. In related news: turns out programming stopping conditions is hard. There's a really important question underlying these "hacks" and group communication. Who can prompt an agent, and what environmental triggers can an agent react to? Clearly, in these message board scenarios, agents are reacting to changes in state on the message board. The question is to what (presumably long-running) prompt does the activity relate? Without care, agents prompt agents, and there is little or no control layer to indicate who controls any given agent (and can give it a primary prompt). Enabling others to "prompt" an agent is effectively the most powerful cyber attack surface of all.
- AI Agents Push Humans Out of the Loop. This isn't a dystopian paper about agents displacing humans. It's potentially a much worse indicator that the general paradigm of human supervision of agents will likely fail. This has long been obvious in anecdotal observations. Give a human an interesting job that involves creating a document, and they can complete it (given enough coffee and breaks), but give them the task of reviewing ten such papers that are 95% likely to be correct, and they are much less able to complete the task. The paper shows clear skill degradation with this kind of replacement and suggests ways to structure tasks to avoid it. The suggestions do make sense, but it's telling that we need to adjust human workflows to compensate for human flaws in an agent system. The deeper problem is that "Human-in-the-loop" systems are the dominant paradigm currently being applied to many industry workflows as AI becomes more widely adopted. There are likely cases where full automation might be preferable since it would incentivize building higher-accuracy agent systems in the first place.
- US government backs OpenAI in New York Times copyright case. In an interesting move, the US Government filed a brief in the New York Times v. OpenAI copyright case, arguing that LLM training does not violate copyright law. The primary motivation seems to be a desire for US dominance in AI over other world powers. I think this is likely the right call, but not for the AI-dominance reason. LLMs are likely to cause a massive productivity boost for humanity, and only a small fraction of the profits will be captured by the leading labs given the open-source explosion. It makes much more sense to me to regulate the use of LLMs in copyright-violating ways than their creation. This brief, though, may make the topic more of a political football in upcoming US elections.
- The jobs apocalypse is postponed. An AI jobs boom is here. In more related news: The Economist breaks with the standard AI job loss narrative and estimates that roughly 1M new jobs have been created in the United States due to AI. Much of this is in construction and equipment for physical AI infrastructure. Other roles are in model development, training, and data labeling. There is a question mark as to how durable these roles are, but it seems to me we won't just stop building data centers in 2028. Data labeling will hopefully progress to managing agent systems. Some professions are clearly suffering. From the report: "Since January 2023, employment has fallen by about 10% among customer-service workers, and by roughly 15% among secretaries and administrative assistants. All three are heavy on routine tasks at which AI agents increasingly excel." The AI employment story is clearly complicated and will involve a lot of churn. Even if AI creates new jobs, societies should look at how to suport the job transitions that will be needed.
Wishing you a great weekend!
