Mantiz Hunts
AI coding agent lie detector.
Scans diffs and PRs for the patterns agents use to fake a passing test suite.
Your agent cheats. Mantiz doesn't.
Try the Evasion Playground
Select a cheat template or paste a diff to see how the AST scoring engine rates the code in real-time.
Scan Pending
Select a template and click "Analyze Diff Live" to see the engine's verdict.
What Mantiz Detects
Patterns your AI agent might be using to fake a passing test suite
From Diff to Verdict
Paste any diff. Mantiz scans every line through 11 detection engines.
No signup needed. No GitHub required.
Paste a Diff
Paste any GitHub-style diff or PR link. Mantiz parses every changed line into structured hunks.
function testAdd() {
+ // assert.equal(add(2,3), 5)
}
- assert.equal(add(2,3), 5)
11 Detection Engines
Each line is scanned against 11 patterns. Findings are ranked by confidence and severity.
Get the Verdict
Trust score (0-100) with ranked findings and evidence excerpts for every detection.
What Mantiz Detects
Every pattern is backed by AST-level analysis. No regex tricks. No guesswork.
Hallucinated Assertion
Non-existent Jest/Vitest matchers hallucinated by AI agents (e.g. .toExist(), .toBeString()). Precision: 77.8%.
Assertion Tampering
Expected value in a test changed to match broken output, with no corresponding spec change. Precision: 100%.
Mock-to-Avoid-Failure
New mock introduced around a previously-failing real call path, with no coverage of the real path. Precision: 100%.
Disabled Assertion
Test or assertion commented out, wrapped in if(false), or marked .skip() in the same diff that claims a fix. Precision: 45.5%.
Silent Catch-and-Pass
Empty catch block newly added around code that previously threw and failed a test. Precision: 33.3%.
Mutation Susceptibility
Detects fragile tests with low assertion density that can be easily mutated to pass with broken code. Precision: 30.0%.
Claim-Diff Mismatch
Commit message claims to fix behavior X, but the diff contains zero changes to files related to X.
Tree-sitter AST
Multi-language AST analysis via WASM parsers — Python, Go, Java, Ruby, Rust, and PHP detection.
Historical Behavioral
Tracks author behavior patterns — style changes, odd-hour commits, score volatility, and frequency anomalies.
AI-Assisted Detection
LLM-powered semantic analysis detects test weakening, assertion removal, and coverage reduction. Requires GROQ_API_KEY.
AI Judge
Reviews static findings and filters false positives using LLM reasoning. Increases precision by removing invalid detections.
How Detection Works
Mantiz analyzes diffs through a multi-stage pipeline: static pattern matching, AST parsing, behavioral analysis, and optional AI-powered detection.
Diff Input
Paste a raw git diff or provide a GitHub PR URL. Mantiz parses every changed file and line.
Static Analysis
11 detection engines scan every line. Findings ranked by severity and confidence with per-detector calibrated weights.
AI + Behavioral
Optional LLM-powered analysis detects semantic bypass and coverage reduction. Historical tracking catches author pattern anomalies.
Trust Score
Weighted scoring: findings deduct points based on confidence level. Score ≥ 80 means the code passes the honesty check.
Why this matters: A loop without a real checker doesn't fail loudly — it hallucinates progress. Mantiz is the honesty detector for the entire pipeline.
A loop without a real checker doesn't fail loudly. It hallucinates progress. Mantiz is the checker for the checker.