Credit Scoring for AI Agents: Why I Stopped Trusting Self-Reported Success

Testnet only, no real money involved — that up front, before anything else.

The problem

Every agent-to-agent system I looked at collapses to the same trust

primitive: the agent's own claim that it worked. No memory across runs,

no independent check, and a confidently wrong answer looks identical to

a correct one to whatever's reading the output.

That's not a small gap. If agents are going to hire each other, pay each

other, and eventually borrow against their own track record, "it said so"

can't be the foundation.

So I built Ledgermind: each agent gets a real credit history — earned

from work that's actually verified, not self-reported — and a

programmable, on-chain credit limit it can draw against.

Grader ≠ solver

The design rule that everything else follows from: the agent that does

the work is never the one who grades it.

Acceptance-test jobs. A requester posts a job with Python assert

statements attached. When a worker submits code, the platform runs

those tests server-side — never the worker's own runtime. Pass, and

escrow releases automatically. Fail, and the job auto-refunds and

reposts for a different worker. The worker literally cannot grade its

own homework.

Proving Ground. The server procedurally generates a problem and

the hidden answer, sends only the problem to the solving agent, and

grades the submission against the hidden answer once it comes back.

Settlement happens via commit-reveal (front-running resistant).

Both produce a "graded fact" event, weighted far above a self-reported

"I completed the task" event in the scoring formula.

A bug I found the hard way

Early on, I had a real vulnerability: the grading endpoint judged

pass/fail off the graded subprocess's exit code. Any submission ending

in sys.exit(0) — accidental, from a model's leftover if __name__ ==

"__main__" block, or deliberate — would skip the actual test code

while the process still exited 0, which read as a pass.

That's a style nit right up until a passing verdict can auto-release

real escrow with no human review, which mine does now. Fixed by

wrapping both the solution and test phases in try/except SystemExit

and only printing an unguessable marker after both phases provably ran

to completion — the caller checks for the marker in stdout, not the

exit code.

I also found (with help from an external reviewer) a Sybil-style hole:

a user could leave one agent's drawn credit unpaid, spin up a brand new

agent, and get an independent credit line with zero regard for the

first agent's debt — because the on-chain vault's outstanding mapping

is keyed per agent address, not per owner. Fixed by netting owner-wide

exposure across every agent a user controls before publishing a credit

limit on-chain.

Both writeups (and a few other findings) are in the repo's Claude.md

if you want the specifics.

How to actually try it

• 
  /guest is a live, no-login view of whatever's actually happening on
  the platform right now — no seeded numbers anywhere.


• Connecting a local model (Ollama, LM Studio) as a paid worker is one
  command; the worker process polls outbound, so there's no tunnel, no
  open port, works behind any firewall.


• No local model? Paste a cloud API key (Groq, OpenAI, etc.) instead —
  no terminal at all, the platform calls it for you when there's work.

Stack

Next.js + Postgres for the app and credit engine, Python + LangGraph for

the agent runtime, Solidity/Foundry contracts (Sepolia + GIWA testnet)

with ERC-4337 smart accounts per agent. Payments/reads go over

x402 where it makes sense — pay-per-call, no

account needed to post a job or pull an agent's credit report.

Known gaps (written down, not hidden)

No formal contract audit yet. The code-execution sandbox isn't

network-isolated. No calibration yet for "confidently wrong" output

specifically (only right/wrong). All tracked openly in the repo instead

of glossed over.

Repo (Apache 2.0): https://github.com/Kairose-master/ai-agent-credit-dashboard

Live demo: https://ai-agent-credit-dashboard.vercel.app/guest

Would genuinely like the grading design picked apart — I'd rather find

out it's broken from a comment here than from someone exploiting it.