UprelicStrom

Agent guard: rules in plain words for your AI agent

Check your agent's tool calls, their results and its replies against rules you write as sentences.

·4 min read

An AI agent decides on its own which tools to call and what to tell the user. Most of the time that's what you want. Sometimes it refunds too much money, passes on data it shouldn't, or promises something nobody agreed to. Agent guard gives you hooks at three points in the agent loop, where each step is checked against your rules before it takes effect.

How it works

Rules. You write each rule as a sentence, for example "Block refunds above 500 unless the user's role is manager." You say which tools it covers, and whether it should block or only report for now. Rules are saved on your account, so you can change them without changing or redeploying the agent.

Checks. Your agent sends what it is about to do, or what it just got back, to one endpoint, POST /v1/guard/check. For each rule that applies, Strom answers how likely it is that the rule is broken, and tells you whether to go ahead. There are three points in the agent loop where you can check:

  • before a tool runs (before_tool),
  • after a tool runs, before the agent sees the result (after_tool),
  • before a reply goes to the user (before_reply).

A rule belongs to one of these stages. A rule in enforce mode blocks when it matches. A rule in observe mode never blocks; it only reports would_block, so you can see what it would catch first.

A sample run

A support agent for an online shop has two tools: lookup_order and refund. We give it three rules:

Rule Stage Tools Mode
refund-limit: "Block refunds above 500 unless the user's role is manager." before_tool refund enforce
no-card-numbers: "Block results that contain a full card number." after_tool lookup_order enforce
no-promised-dates: "Block replies that promise the customer a date for a refund or delivery." before_reply observe

A rule is created with one request:

curl https://api.uprelic.com/v1/guard/rules \
  -H "Authorization: Bearer $UPRELIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "id": "refund-limit",
    "text": "Block refunds above 500 unless the user’s role is manager.",
    "stage": "before_tool",
    "tools": ["refund"],
    "mode": "enforce"
  }'

The shop's support staff use the agent to answer customers. A support employee, signed in with the role support_agent, gives it a customer's message: "I want a refund for order A1." The responses below are real ones from the API, shortened.

1. The agent looks up the order

Before lookup_order runs, the agent checks the call:

{"action": "lookup_order", "params": {"order_id": "A1"}, "context": {"role": "support_agent"}}

No before_tool rule covers lookup_order, so the answer is allow, with no rules evaluated. A check like this is free.

2. The order comes back with a card number

The order system returns more than it should:

{"order_id": "A1", "total": 800, "status": "delivered", "card": "4111 1111 1111 1111"}

Before the agent sees it, the result is checked, sent as result with the same action and params:

{
  "decision": "block",
  "rules": [
    {"id": "no-card-numbers", "probability": 0.92, "threshold": 0.9,
     "mode": "enforce", "matched": true}
  ]
}

The agent doesn't get the result. It gets the rule's text instead: "Block results that contain a full card number." The same order with only the card's last four digits gets a probability of 0.07 and is allowed.

The tool has still run. A check after a tool can stop its result from reaching the agent, but it can't undo what the tool did. For tools that change something, check before they run.

3. The agent tries to refund 800

{"action": "refund", "params": {"order_id": "A1", "amount": 800}, "context": {"role": "support_agent"}}
{
  "decision": "block",
  "rules": [
    {"id": "refund-limit", "probability": 0.94, "threshold": 0.9,
     "mode": "enforce", "matched": true}
  ]
}

The refund doesn't run, and the agent is told why. The same call with "role": "manager" gets a probability of 0.02 and is allowed.

The role comes from context, and your application fills it in from the signed-in user, not from anything the model wrote. If the model could set the role, it could also talk itself into being a manager.

4. The agent writes a reply

The agent drafts: "Refunds above 500 need a manager's approval. I've passed your request on, and you'll have your money back by Friday."

Nobody has agreed to Friday. The reply is checked with response (and the user's message as request):

{
  "decision": "would_block",
  "rules": [
    {"id": "no-promised-dates", "probability": 0.94, "threshold": 0.9,
     "mode": "observe", "matched": true}
  ]
}

The rule is still in observe mode, so the reply is sent. The same reply without "by Friday" gets 0.02.

5. Switching the rule on

After a few days, we list what the reply rule would have blocked:

curl "https://api.uprelic.com/v1/guard/checks?decision=would_block" \
  -H "Authorization: Bearer $UPRELIC_API_KEY"

Each check comes back with the reply, the user's message and each rule's probability. If the matches look right, we change the rule's mode to enforce with PUT /v1/guard/rules/no-promised-dates. If they don't, we reword the rule or raise its threshold and watch again.

What it cost

Four checks, three of them with one rule each: 3 rule evaluations at $0.01 per 1,000, so $0.00003. Each check took 30 to 40 ms on the server.

In Python

With the Python SDK, a decorator does the checks for you:

import uprelic

client = uprelic.Uprelic()

@client.guard("refund", context=lambda: {"role": current_user.role})
def refund(order_id, amount):
    ...

@client.guard("lookup_order", check_result=True)
def lookup_order(order_id):
    ...

try:
    refund("A1", amount=800)
except uprelic.Blocked as blocked:
    tool_result = str(blocked)  # give this to the agent instead

Strengths

  • Rules are sentences. Someone who knows the policy, but not the code, can write and change them, with no deploy.
  • It works with any agent. It's an HTTP call at three points in the loop, whatever framework or model runs the agent.
  • You can try a rule before it blocks. Observe mode runs a rule on real traffic and records what it would have blocked.
  • It's fast and cheap. A check takes tens of milliseconds and costs $0.01 per 1,000 rules.

Weaknesses

  • It is a model, so it can be wrong. It will sometimes miss a case or block a harmless one. Exact limits, like "amount above 500", are safer in your own code. Rules are most useful for what code can't easily check, like what a reply promises.
  • A check only sees what you send. It has no memory of earlier calls, so a rule like "at most three refunds per day" needs the count in context.
  • It adds a round trip at every point you check, which adds up in a long loop of tool calls.

Try it

  • The Agent guard demo runs a mocked agent through these checks in the console, no code needed.
  • The docs describe every field and error.
  • The Python SDK: pip install uprelic.

Questions and feedback: support@uprelic.com.

More from the blog