Policy Tuning Guide

Understanding False-ALLOW vs False-DENY Tradeoffs

The guardrail is intentionally biased toward false-DENY (blocking safe operations) over false-ALLOW (allowing dangerous operations). This is a security-first design choice. However, both types of errors have costs:

False-ALLOW (Dangerous Command Incorrectly Allowed)

A false-ALLOW means the guardrail failed to block a destructive, exfiltrating, or auth-disabling operation. This is a security incident and should be treated as a critical bug.

Causes:

• Missing action classification in the policy file
• Missing AST rule for a new dangerous command pattern
• AST normalization that inadvertently matches a dangerous command to a safe rule
• Rule ordering: a safe prefix rule fires before a dangerous contains rule
• String-level matching bypass (mutations that AST rules should catch)

Detection: Run the mutation fuzzer (guardrail_mutation_fuzz) and the coverage fuzzer (guardrail_coverage_fuzz) regularly. See the threat model for techniques that are not fully covered.

False-DENY (Safe Command Incorrectly Denied)

A false-DENY means the guardrail blocked a legitimate developer operation. This is a usability degradation but not a security issue.

Causes:

• Overly broad AST rule (e.g., an exact dd match also catches dd --help)
• Overly broad string-level rule (e.g., a contains("rm") rule catches remarkable — prevented by requiring the space in "rm ")
• A param-trap rule fires on a benign parameter key that shares a name with a dangerous one
• Session chain detection false-positive: a legitimate task legitimately reads several files then writes one

Costs

Tuning Priority

1. Eliminate false-ALLOW first (security)
2. Minimize false-DENY second (usability)
3. If the false-DENY rate exceeds 1% of safe operations, investigate rule over-broadness

Adjusting the Policy File

The policy file (classification.toml) is the guardrail policy. It defines which actions belong to which categories, along with parameter keys and bridge rules — and the verdict for each category. The engine compiles the policy into its rule network at startup, so you change the policy, not the engine (restart to apply — there is no hot reload).

Adding a New Action

# Add a new destructive action
[[action]]
name = "my_dangerous_action"
category = "destructive_op"

# Add a new safe action
[[action]]
name = "my_safe_action"
category = "read_only_op"

# Add a new action that should escalate to LLM
# Simply don't add it — unclassified actions always escalate (fail-safe).

That is the entire change. Because the verdict rules are category-wide, the new action inherits the correct verdict without any code modification or recompile.

Adding a New Param Key

[[param_key]]
name = "my_new_param"
category = "capability_granting"

This causes any action with a my_new_param parameter to trigger the param-trap rule, overriding a safe action’s Allow verdict to Deny.

Adding Bridge Rules (Transitive IS_A Chains)

[[bridge]]
sub = "my_subcategory"
super = "destructive_op"

Bridge rules enable 2-hop inference: action IS_A my_subcategory → bridge rule → action IS_A destructive_op → verdict fires. Useful for intermediate grouping without changing the top-level category.

Removing an Action

Just remove (or comment out) its [[action]] entry. The action becomes unclassified and the guardrail escalates it to the LLM tier, or returns Escalate (fail-safe) if no LLM is configured.

Changing a Category

Change the category value. The new category’s verdict rule takes effect at the next server start (restart required).

Important Constraints

• Do NOT change existing action names without a migration plan (they are used in corpus baselines, audit traces, and integration tests).
• The name field must match exactly what the mapper produces for that tool or command (see the integration guide for how tool names map to actions).
• Category names are case-sensitive, and you can define your own in [[category]] rows. If the policy defines no categories, built-in defaults apply: destructive_op, exfiltration, auth_disable, and network_write are Deny; read_only_op is Allow. Unclassified actions always escalate.

Confidence Levels Explained

Each mapped call carries a Confidence level. The confidence does NOT change the verdict (Allow/Deny/Escalate) — it informs downstream consumers how much to trust the deterministic tier’s decision.

How Confidence Affects Escalation

• High + Allow: The deterministic tier is sure this is safe. The LLM is never called.
• High + Deny: The deterministic tier is sure this is dangerous. The LLM is never called.
• Medium + Allow: The deterministic tier considers this safe. The LLM is never called (the verdict is confident enough).
• Unknown: The deterministic tier has no classification. The call always escalates to the LLM (or returns Escalate if no LLM is configured).

Key insight: The confidence level does NOT gate LLM escalation. Only the verdict does: if the deterministic tier returns Allow or Deny, the LLM is not consulted. If it returns Escalate (which happens for Unknown-confidence actions), the LLM is consulted (if configured).

Custom Command Rules

Beyond the TOML policy, the engine ships a set of built-in AST rules that match against the parsed Bash command structure rather than the raw string. They are immune to quoting, whitespace, unicode, case, and path-encoding bypasses. The rules are organized into phases that determine evaluation order:

1. Phase 1: Structural danger — Redirects, pipes, command substitutions regardless of command word
2. Phase 2: Compound dangerous — Command + specific dangerous flag/arg combinations (high confidence)
3. Phase 3: Dangerous commands — Always risky regardless of flags (medium confidence)
4. Phase 4: LLM-judgment needed — Cannot determine from structure alone (curl, git clone, package managers)
5. Phase 5: Safe commands — Known-safe commands that fire AFTER all dangerous patterns

The mapper checks AST rules before string-level rules. Within AST rules, Phase 1 fires before Phase 2, and so on.

Most customization needs — new actions, new dangerous commands, new param keys — are covered by the TOML policy format above, which requires no code change or recompile. Modifying the built-in AST rule set itself requires the licensed source and a rebuild.

LLM Judge Configuration

The LLM tier is optional and disabled by default. When enabled, it acts as a semantic judge for cases the deterministic tier cannot resolve. In the library it is enabled programmatically; in the server deployment it is configured with GUARDRAIL_LLM_URL and GUARDRAIL_LLM_MODEL (see the deployment guide).

When to Use the LLM Tier

Cost Implications

Model Selection

The LLM judge communicates via Ollama’s /api/generate endpoint. Any model that responds with plain text works.

Recommended models:

The judge is instructed to reply with exactly one word — allow, deny, or escalate — given the guardrail action name, target, and parameters.

LLM Downgrade Prevention

The pipeline enforces a critical invariant: the LLM can only harden a verdict, never weaken it. If the LLM returns “allow” for a call that the deterministic tier escalated, the pipeline overrides the LLM’s verdict to “Escalate”. This eliminates the LLM as an attack surface: a jailbroken model cannot open a hole the deterministic tier left closed.

Session Chain Detection Configuration

The session-spanning chain detection catches recon → exploit patterns that span multiple turns: N+ consecutive safe (Allow) calls followed by a destructive or unknown (Deny/Escalate) call.

Default Values

Tuning Guidelines

Session Lifecycle

• The session window is in-memory only and does not persist across process restarts.
• Clear the session at conversation boundaries (e.g., when a user ends a session).
• The window is automatically trimmed to window_size entries, discarding the oldest first.
• Session chain detection is local to each pipeline instance and does NOT span replicas in a load-balanced deployment.

Rate Limiting Configuration

The guardrail server has built-in per-IP rate limiting (token bucket):

• Default: 100 requests/second per client IP, configurable via GUARDRAIL_RATE_LIMIT
• Burst: bucket size equals the configured rate (minimum 10 tokens), so ~1 second of full-rate traffic can arrive at once
• Response: HTTP 429 with a JSON body {"error": "rate limit exceeded", "retry_after_ms": 1000}
• Exempt: GET /health is never rate limited, so liveness probes keep working under load. /check, /metrics, and / all cost tokens
• Related limits: GUARDRAIL_MAX_CONNS (default 128 concurrent connections → 503 beyond), GUARDRAIL_MAX_BODY (default 64 KiB → 413), 16 KiB header cap (→ 431), 10 s per-request deadline

For internet-facing deployments, an additional reverse-proxy layer (nginx, HAProxy) is still recommended for TLS termination and DDoS absorption:

limit_req_zone $binary_remote_addr zone=guardrail:10m rate=100r/s;

location /check {
    limit_req zone=guardrail burst=20 nodelay;
    proxy_pass http://127.0.0.1:9090;
}

Recommendations for the API User

• Max throughput: 37,745 checks/second (sequential, deterministic-only) on a single core — and only 22,096 with 14 parallel threads (see below): the pipeline is serialized behind a mutex, so concurrent connections queue. Scale with additional instances, not cores.
• With LLM tier: Limited by LLM inference speed (~1–5 calls/second when escalation triggers)
• Recommended client rate: stay under GUARDRAIL_RATE_LIMIT per client IP (default 100 req/s); on a 429, wait retry_after_ms from the JSON body before retrying

Performance Tuning

Thread Count

The guardrail server (guardrail_serve) accepts connections in a listener loop and handles each on its own thread, capped by GUARDRAIL_MAX_CONNS (default 128; excess connections get HTTP 503). All checks are serialized through a mutex, so per-connection threads do not multiply check throughput. For higher throughput:

1. Multi-instance deployment: Run multiple server processes behind a load balancer (the fastest known configuration is one thread per process with a queue in front — exactly the model guardrail_serve uses).
2. Embedded usage: The pipeline can be shared behind a lock in a thread pool; remember the lock serializes checks, so add processes, not threads.
3. Async runtimes: The pipeline is synchronous; run it on a blocking thread pool rather than the async executor.

Connection Pooling (LLM Tier)

The LLM judge opens a connection per call. Configure connection reuse if you expect high LLM call volume:

• Increase the connection pool size
• Use HTTP/2 multiplexing if the LLM endpoint supports it
• Batch multiple escalate requests

Bottleneck Reference

Measured 2026-08-12 on the reproducible benchmark:

Key Performance Characteristics

• The deterministic tier (~15 µs p50) is orders of magnitude faster than the LLM tier (~1–5 s per call).
• Over 99% of calls in normal development workflows resolve at the deterministic tier.
• Parallel is slower than sequential: the pipeline is mutex-serialized, so 14 threads measured 22,096 checks/sec vs 37,745 checks/sec on a single thread — roughly a 40% regression. Until the kernel is sharded, the fastest deployment is one thread per process with a queue in front.
• Memory: ~65 KB (~878 allocations) per check, measured over 6.38M checks.
• The rule base is fixed by design — the engine does not learn at runtime — so speed on day 1 equals speed on day 1,000, and the benchmark numbers are reproducible on demand.