While I work to complete my new book “AI-Driven API Design”, I’m keeping my posts here sparse (see below) and light (see my “Approved by Management” comic series). I also apologize in advance for the “instructional tone” this piece has as my brain is in that head space while I work though the new book. I’ll return to my story-telling posts later this year once the book is off my plate. -- mamund
Over the last year or so I’ve been experimenting with a simple model for empowering autonomous agents in constrained environments. I’ve dubbed this work “Goal-Resolution through Affordance-Informed Logic” or GRAIL since one of the key elements in the model is defining and referencing affordances (links and forms) for clients to call.
NOTE : I’ve discussed my GRAIL work here on substack in the past. You can go to my archives page and search for “GRAIL” and see several articles. Feel free to refer to the previous articles at any time.
Recently I created some experiments exploring how the system holds up when the service’s preconditions are presented to the client in random order. Does this model hold up even when the possible “next steps” are presented in random order?
The GRAIL Algorithm
As a quick refresher, the GRAIL algorithm is very small and, so far, very effective. The code for a fully operational GRAIL client is less than 50 lines of NodeJS. That includes console.log statements peppered through the instrumented client.
The algorithm itself is straightforward:
1. Start with a goal
2. Attempt the action associated with that goal
if SUCCESS then EXIT
if BLOCKED by an unmet precondition,
follow an available action that can satisfy the unmet condition
then
go to step 2
Continue until the original goal succeeds or there is no available path forward.The trick is to start at the end. Try the end goal first. This algorithm also depends on services knowing their own preconditions (you must be logged in first, you must have a non-zero balance, etc.) and exposing those preconditions to the client when they are unmet. Of course, this is all recursive so attempting to execute on an unmet precondition might also return a BLOCKED message with one or more unmet preconditions for that precondition; and so forth.
Visually, it looks like this:
At the heart of all this is to a registry of affordances (actions) arranged as pre-conditions for a particular goal (onboardCustomer, computeNetPresentValue, produceOverdueAccountsReport, and so forth). The registry is, essentially, the environment in which the agent exists and interacts with affordances.
Let’s Randomize it
A set of actions lives in an environment and that environment is expressed in a registry.json file. Each entry lists all the details needed to successfully execute the action such as name, associated function, inputs, expected outputs, side effects, and (if any) preconditions. The client is built to understand this standardized representation and uses the affordance collection to complete its assigned goal.
In some cases, the affordance has quite a few preconditions that need to be met before the client can execute the selected action. For example, the registry entry below shows an affordance with eight preconditions. And, as we’ve already stated, it may be true that any one of these eight preconditions have their own list of additional preconditions (and so forth).
The assertion I set out to test is whether the GRAIL system requires execution of preconditions in a specific order (e.g. in listed order). Since my current examples are all run as CLI utilities (there is no MCP/LLM needed), I needed to add a bit of randomization to the unmet precondition routine. This would approximate the possibility that a client app might not execute the preconditions in a fixed (and repeatable) order.
All it took for that was to add a single line of code that randomly selected one of the unmet preconditions in the runtime registry:
if (unmetPreconditions.length > 0) {
const pre = unmetPreconditions[
Math.floor(Math.random() * unmetPreconditions.length)
];Now, when I run the test, each trace has the potential to be slightly different. The questions is:
Does the client consistently complete the goal, no matter the execution order of the preconditions?
Tracing in GRAIL
The GRAIL test harness has the ability to produce a trace file that shows the path of the client through the provided environment. With my randomizer in place, I ran several traces and happily discovered that first, each traced path is different and second, all traced runs reached the goal successfully.
Below are screenshots of two example traces. Note the step-by-step order in the numerals next to each navigation line in the diagram.
In this first example (below), you can see that the first precondition executed (after attempting the goal directly) is startCustomerOnboarding. Then there’s a series of back-and-forth actions to resolve the remaining preconditions before returning to execute the associated goal action.
However, in the second run below (see below), the client followed a different path in attempts to resolve most of the minor details (starting with verifyCustomerPhonenumber before actually finding the startCustomerOnboarding action.
An interesting side note that I did not have time to explore is that, while each trace shows a unique navigation order, both traces took the same number of steps (16) to complete. In fact, all the traces I ran took 16 steps. This leads me to assume that, while the navigation path may vary, the differences in path order do not impose any additional execution cost. There is no “cheaper” path in these GRAIL models.
Different Routes, Same Destination
In most workflow type systems, we’re lead into authoring a fixed series of steps in order to reach a goal. for example, an approximate rendering of my above onboarding example looks like this in Arazzo (see specs here). And it is important to note the the example below is a fixed set of steps in order.
arazzo: 1.0.1
info:
title: Customer Onboarding Workflow
version: 1.0.0
sourceDescriptions:
- name: customerApi
url: ./openapi.yaml
type: openapi
workflows:
- workflowId: onboardCustomer
summary: Satisfy customer onboarding requirements and onboard the customer
steps:
- stepId: onboardingStarted
operationId: startOnboarding
- stepId: customerProfileCollected
operationId: collectCustomerProfile
- stepId: customerEmailSet
operationId: setCustomerEmail
- stepId: customerEmailVerified
operationId: verifyCustomerEmail
- stepId: customerPhoneNumberSet
operationId: setCustomerPhoneNumber
- stepId: customerPhoneNumberVerified
operationId: verifyCustomerPhoneNumber
- stepId: customerAddressSet
operationId: setCustomerAddress
- stepId: customerTermsAccepted
operationId: acceptCustomerTerms
- stepId: onboardCustomer
operationId: onboardCustomer
successCriteria:
- condition: $statusCode == 200I’m even seeing people working on creating agents that do this kind of planning before they attempt to reach the goal. I think this is not needed and makes the work of designers and engineers harder than it needs to be.
The GRAIL model is built in a way that agents are free to take more than one particular path through the environment (as defined by the registry.json file) and still meet the intended goal. And they can do that without any hints or set up from a human. They just need the end goal and access to the environment.
Conclusion
GRAIL shows us we can create systems where we don’t need to know exactly what that path will be before starting a run. And we don’t need to guarantee the same path is traced each time the agent attempts to reach the goal.
Essential guidance for GRAIL authors would be:
Define the environment, not the path.
If this kind of thing is interesting to you, feel free to check out the public GRAIL repo where some other experiments will be appearing in the near future.
