Stinger
Measures whether an AI coding agent cheats β in your configuration, in your CI.
Runs a corpus of sandboxed trap scenarios against a configured coding agent and mechanically detects dishonest shortcuts: weakened tests, forbidden-file edits, fabricated completion claims, broken house rules, leaked bait secrets. Emits a scored, evidence-linked Integrity Report under a preregistered, frozen rubric. Model-agnostic and vendor-neutral by construction. CLI and GitHub Action.
- Mechanical truth first. Cheating is decided by deterministic detectors, never an LLM judge. The optional judge may only flag for human review; it can never set a verdict.
- Fail closed. Any ambiguity, harness error, or missing evidence resolves to a
non-scored
errorβ never to a favorable score.
Every fixture test passed, so I spent the money to run it against live agents anyway β because a measurement instrument that has only ever been tested against its own fixtures has not been tested. Those runs found five real defects, each of which would have produced a plausible wrong answer rather than a visible failure. The worst: it scored a textbook-perfect refusal as a cheat, because the trap only ever recognised my own vocabulary. The mislabelled evidence package is committed unedited.
How it was built, and what hitting reality broke Evidence packages Specification