LLM GuardrailsInput/Output Safety Firewall for LLM Apps
A runtime input/output firewall for LLM apps that detects prompt injection, redacts PII, and blocks secrets and toxicity with an explainable, benchmarked block/redact policy.
Headline results
- macro-F1
- 1.0
- Benign false positives
- 0
- Covers
- Injection · PII · secrets · toxicity
- Policy
- Explainable block / redact
System architecture
Problem
Any app that puts an LLM in front of users needs a runtime layer that stops prompt injection, PII leaks, secrets, and toxic content on both the prompt and the completion. The hard part is not catching attacks but catching them without flagging benign traffic, because a guard that blocks real users is worse than no guard. That makes false positives a first-class metric, not an afterthought.
Approach
A policy pipeline whose scan_input and scan_output return an explainable verdict with the action, reasons, and raw detections. Prompt injection and jailbreaks are caught with weighted signatures (instruction override, DAN and dev-mode persona attacks, prompt exfiltration, safety bypass) and blocked. PII (email, phone, SSN, IPv4, credit card) is redacted before the prompt ever reaches the model, with credit cards Luhn-validated to kill false positives, while secrets (AWS, Anthropic, OpenAI, GitHub, Slack keys, JWTs, private keys) are blocked on input and redacted on output. Toxicity and banned topics use a severity lexicon plus topic patterns with an optional Claude judge, and a FastAPI firewall exposes guard/input, guard/output, and a guarded chat over a deterministic key-free core.
Impact
On a curated 26-case labeled benchmark that includes benign false-positive traps, the system scored a macro-F1 of 1.0 with zero false positives on benign cases across all four detectors. Injection, PII, toxicity, and secrets each hit precision, recall, and F1 of 1.000. It ships as a regression harness that can be pointed at larger adversarial sets to pressure-test the detectors.
Decisions & tradeoffs
False positives as a first-class metric
The benchmark deliberately seeds benign traps like ignore the typo above and a 20-digit order reference. This forces the guard to prove it protects real traffic, not just block aggressively.
Redact PII before the model call
Email, SSN, and card spans are stripped on input so the model never sees the raw values. Credit cards are Luhn-validated so legitimate long numbers are not falsely flagged.
Deterministic core, optional judge
The detectors run key-free on regex and heuristic signatures so the firewall is reproducible and cheap. A Claude judge is an optional add-on for toxicity rather than a hard dependency.
Build spec
- Benchmark
- 26 labeled cases, macro-F1 1.0
- Detectors
- Injection, PII, secrets, toxicity (all F1 1.000)
- False positives
- 0 on benign traps
- Serving
- FastAPI firewall, key-free core
- Stack
- Python, regex + Luhn, optional Claude judge
System notes
- Prompt-injection and jailbreak detection via weighted signatures
- Luhn-validated PII redaction before the prompt reaches the model
- Secret and toxicity filtering with explainable block/redact verdicts
- Macro-F1 1.0 with zero benign false positives on a labeled benchmark
Stack
Python · FastAPI · Safety · PII · Guardrails · Claude