How to add policy enforcement to a LangGraph agent (before it does something dumb)

If you've built anything with LangGraph past the demo stage, you've probably had the same uneasy moment I did. The agent works, it's calling tools, it's doing real things, and then you realize the only thing stopping it from doing the wrong real thing is a line in the prompt that says "please don't."

A prompt isn't a control. The agent can be talked into ignoring it, some upstream input can steer it somewhere you didn't expect, and either way the tool call just runs. Once that tool call can move money, hit prod, or touch customer data, "the model seemed confident" isn't where you want your safety to live.

So here's how to put a real check in front of the tool call instead. I'll use Faramesh, the open source thing I've been building for exactly this. It's a local daemon that sits in front of your agent's tool calls and returns permit / deny / defer based on a policy you write. No LLM in the decision path, so the same call always gets the same answer.

The whole thing takes about 10 minutes. Every command below is copy-pasteable. I'll be clear about the one or two spots where you swap in your own stuff.

How it works in one picture

Your agent tries to run a tool. Before it actually runs, the call hits Faramesh, which checks it against your policy:

• permit -> runs normally
• deny -> blocked, the agent never gets to run it
• defer -> paused and sent to a human to approve or reject You write that policy in a single file called governance.fms. That file is the heart of Faramesh. It's the one place that defines what your agents are allowed to do, you commit it to your repo like any other code, and the daemon enforces whatever's in it.

Step 1: install

curl -fsSL https://install.faramesh.dev/install.sh | bash
faramesh --version

If faramesh --version prints a version number, you're good.

Step 2: let it generate your governance.fms

From the root of your agent project, run:

faramesh init

This detects your framework and the tools your agent uses, and writes a starter governance.fms for you. You don't have to write it from scratch. Open it up and it'll look something like this:

runtime {
  mode    = "enforce"
  wal_dir = "./wal"
}

agent "langgraph-agent" {
  default deny

  rules {
    permit http/get
    permit crm/read
    defer  payment/refund   reason: "refund needs a human"
    deny   billing/delete_account
  }
}

Here's how to read that, because this is the part that actually matters:

• mode = "enforce" means decisions are live. (There's also a shadow mode if you just want to watch what would happen first, more on that at the end.)
• default deny means anything you don't explicitly allow is blocked. So a tool you forgot about can't quietly slip through. This is the safe default and I'd leave it.
• Each line under rules is a decision. permit lets it run, deny blocks it outright, defer pauses it for a human. This is the file you edit. The tool names (http/get, payment/refund, etc.) match the names of your actual tools, so swap these for whatever your agent actually does. The rule of thumb: permit the safe reads, defer anything risky or irreversible (payments, deletes, external emails), deny the stuff that should never happen automatically.

Step 3: name your tools to match

Faramesh checks tool calls by name, so your LangGraph tools just need names that line up with your policy scopes. In LangChain/LangGraph that's the first argument to @tool:

from langchain_core.tools import tool

@tool("http/get")
def http_get(url: str) -> str:
    return fetch(url)

@tool("payment/refund")
def payment_refund(amount: int) -> str:
    return issue_refund(amount)

So @tool("payment/refund") is the thing the defer payment/refund line in your policy is talking about. Keep the names consistent and you're done here.

Step 4: turn on interception

Add these two lines near the top of your agent script, before you build your graph:

from faramesh.adapters.langchain import install_langchain_interceptor

install_langchain_interceptor(include_langgraph=True, fail_open=False)

That's the whole integration. You don't rewrite your ToolNode and you don't wrap every tool by hand, it patches LangGraph's execution path so every tool call gets checked.

One flag to understand: fail_open=False. It means if the daemon ever errors or can't reach a decision, the call is denied, not waved through. You want enforcement to fail closed, if something breaks, the safe move is to not run the action.

Step 5: run it

Run your agent under governance with dev, which enforces your policy locally while you're still testing:

faramesh dev

Then run your agent as you normally would (in another terminal or however you launch it). Every tool call now routes through Faramesh.

When you're happy with how it behaves and want full enforcement, switch on:

faramesh apply

faramesh apply compiles your governance.fms and starts the daemon in full enforce mode.

Now watch what happens. When your agent calls http/get, it just runs. When it calls payment/refund, it doesn't, it pauses and waits, because you set that to defer. You'll get a pending approval. List and resolve it like this:

faramesh approvals list
faramesh approvals approve <id>   # or: faramesh approvals deny <id>

Approve, and the original call resolves and runs. Deny, and it never happens. Either way the call, the decision, and the reason all land in an audit log you can read back later with faramesh explain <action-id>.

If you want to test before you enforce

Flipping straight to enforce on a live agent is nerve-wracking, so you don't have to. Set the runtime to shadow mode (or run faramesh dev) and Faramesh will log what it would have blocked or deferred without actually stopping anything. You watch the decisions against real traffic, tune your rules until they're right, then switch to enforce. Way less scary than guessing.

Why deterministic, instead of "ask another LLM"

There's a popular pattern where you put a second LLM in front of the first one to judge whether an action is safe. I think that's the wrong bet for enforcement. The thing you're worried about is your agent getting manipulated into a bad action. If your guard is also an LLM, it can be manipulated too. You're using a promptable thing to protect a promptable thing.

A rule engine doesn't have that problem. deny billing/delete_account means the account does not get deleted. Same input, same answer, every time, and you can hand the log to an auditor without shrugging. The agent doesn't get the final say on what it's allowed to do, which, once it's touching real systems, is sort of the entire point.

Repo's here if you want to try it or dig into how it works: github.com/faramesh/faramesh-core. It works with a bunch of other frameworks too (LangChain, CrewAI, AutoGen, MCP, others), LangGraph's just what I used here. If you try it and something's confusing or broken, I'd love to hear it, that feedback is what's making it better right now :)