The guardrail, measured: 15µs, 37,700 checks/sec, zero false-ALLOWs

Since launch we’ve quoted the same three numbers: 124 microseconds per check, 8,000 checks per second, 50 MB of memory. They came from an early prototype measurement, were copied from page to page, and — like most quoted numbers — nobody ever re-ran them.

That’s not how a company that sells determinism should behave. If our whole pitch is “the same input produces the same output, verifiably,” our performance claims should be verifiable too. So this week we built one.

The benchmark

cargo run --release --bin guardrail_bench -- --output json

One command. It runs 10,000 timed iterations with 1,000 warmup, then one million mixed realistic commands, then stress tests: a 10 KB single command, 1,000-deep nested subshells, a 1 MB environment block. Full methodology and every number: /docs/benchmark/.

The numbers are better than we claimed

Our site had been understating the product by roughly an order of magnitude. The average check takes 19 microseconds — about 2,500 times faster than a keystroke. Across a million commands, the slowest single check finished in under half a millisecond.

We also stopped only benchmarking Bash. The mapper’s normalization pipeline is ecosystem-agnostic, so the benchmark now fuzzes dangerous tool calls across six ecosystems: SQL, REST APIs, cloud SDKs, Kubernetes, Docker, and database clients. 115 of 115 dangerous calls denied. Zero false-ALLOWs. Zero false-DENYs. The defense is layered — AST rules, string rules, tool-name mapping, param-key detection — and it held across all six.

The honest surprise

Adding threads made it slower.

Parallel across 14 CPU threads: 22,096 checks/sec. Sequential on one core: 37,745. The pipeline is serialized behind a mutex — the kernel decides one command at a time by design — so fourteen threads just queue for one lock and add scheduler overhead. Roughly a 40% regression.

We’re publishing this because it’s a useful fact, not a flattering one. It means “scale out, not up”: the fastest deployment is one thread per process with a queue in front, which is exactly the model guardrail_serve uses. Until the kernel is sharded, more cores buy you nothing. If you’re sizing a deployment, that saves you money — and it’s the kind of finding that only shows up when you measure instead of assuming.

What measuring exposed

The benchmark also caught a real defect. The classifier’s learning engine was accumulating rules with every check — 5,800 and counting — growing the rule base without bound, slowing every cycle, heading for memory exhaustion. For a security product that must judge repeated queries identically, learning was the wrong default. It’s now disabled in the guardrail by design: speed on day 1 equals speed on day 1,000, and the benchmark numbers are reproducible on demand.

The other honest caveat: internally, most of each check’s effort goes to memory bookkeeping — about 878 allocations per check — not decision making. It doesn’t matter at 37,700 checks/sec on a single core, but it’s the known lever if we ever need more.

Why publish this

Two reasons.

First: numbers that can’t be reproduced aren’t numbers, they’re marketing. Every figure above can be regenerated on your hardware with one cargo command (requires licensed source). If they don’t match, tell us — that’s a bug.

Second: this is what the product promises. Deterministic means the same input gives the same output every time — and that applies to our own claims, not just our engine. The benchmark is now part of the repo, it runs in CI, and it will be remeasured whenever the pipeline changes. The sales line stays short: “Guardrail checks every command before it runs, in under half a millisecond in the worst case — users never feel it, and it never lets a dangerous command through, even after a million tests.” Both halves are backed by a benchmark anyone can run.