LLM Red-Teaming FrameworkThe Only Door Left Open Was Indirect Injection
A Bedrock-native red-teaming harness probing its own support bot: 78 attacks across five OWASP-LLM categories replayed identically against naive and hardened builds, with canary-based deterministic scoring and a real before/after number.
Headline results
- Attacks
- 78 across 5 OWASP-LLM classes
- Violation rate
- 2.6% → 0.0%
- Direct attacks
- 0 succeeded
- Indirect injection
- 13.3% (2 of 15)
System architecture
Problem
Modern aligned models refuse blunt jailbreaks, so the real attack surface is the application layer: indirect prompt injection, context leakage, PII extraction. Teams harden blindly, with no number proving the hardening worked.
Approach
Replays 78 attacks across five OWASP-LLM categories (jailbreak, system-prompt leak, sensitive-data disclosure, toxicity, indirect prompt injection) against a frozen v1 of the target app, scores violations with planted canary secrets for deterministic leak detection plus an LLM judge for the model-safety categories, then hardens the prompt and adds an output guardrail into a frozen v2 and replays the identical set. Bedrock-native throughout, no DeepEval.
Impact
Claude Haiku refused every direct attack: jailbreak 0 of 28, prompt-leak 0 of 15, PII 0 of 10, toxicity 0 of 10. The only exploitable surface was indirect prompt injection at 13.3%, where laundering an instruction through a benign internal task made the agent reproduce planted canary secrets. Hardening took overall violations from 2.6% to 0.0%. Stated honestly, that 100% reduction rests on two violation events out of 78: the finding is closing a narrow, high-impact application-layer gap, not fixing a broadly broken model.
Decisions & tradeoffs
Canaries over judges for leak detection
Planting synthetic secrets in context makes leakage binary: the canary string either appears in output or it doesn't. Ground truth where ground truth is possible; the LLM judge is reserved for categories that genuinely need judgment.
Application-layer attacks over blunt jailbreaks
Single-turn 'ignore your instructions' fails against modern aligned models. The exploitable surface is indirect injection through retrieved documents and tool outputs, so that's where the attack budget goes.
Immutable versions, identical replay
v1 and v2 are frozen, and the exact same attack set runs against both. Any delta in the violation table is attributable to the hardening, nothing else.
Report the small n instead of the big percentage
The headline reduction is 100%, and it rests on two violation events out of 78. The report says so in its own findings section rather than letting the percentage imply a broadly broken model that was fixed. A safety number that hides its sample size is not a safety number.
Build spec
- Attack corpus
- 78 attacks · 8 AdvBench seeds + 70 crafted
- Categories
- 5 OWASP LLM Top 10 · not 40
- Result
- 2.6% → 0.0% overall · injection 13.3% → 0
- Scoring
- Canary ground truth + LLM judge, judge verified
- Models
- Claude Haiku target + judge (AWS Bedrock) · ≈$0.21
System notes
- Direct attacks all refused; indirect prompt injection was the only opening, at 13.3%
- Canary-based scoring: planted synthetic secrets make leak detection binary, no judge needed
- Frozen v1/v2 versioning: the identical 78-attack set replayed against naive and hardened builds
- Judge itself verified: 20 labels hand-reviewed at 100% agreement, zero false positives or negatives
What this does not show
- The 100% reduction rests on two violation events out of 78, which is a small base for a headline percentage.
Stack
AdvBench · AWS Bedrock · Claude · OWASP LLM Top 10 · Canary scoring · Python