Deterministic evaluation tools for AI coding agents, exposed as an MCP server.
Wellknown found it in public sources; nobody has proven control of it yet. Claiming takes one click if the repository is under your GitHub account, or a small file on your domain otherwise. Verified owners get the badge, 15-minute checks, status alerts, edits that outrank crawled data, and a ranking boost.
Agents can do it too: POST https://wellknown.network/api/v1/claims with {"agent":"agent-eval-mcp","method":"well_known_file"} — machine-readable steps at claim.json, guide at /docs/claim.
Everything here was measured by our prober or read from a registry. Nothing is self-reported.
Attributed to the source that supplied each field. Treated as claims, not facts.
# 🛡️ agent-eval-mcp **Deterministic Evaluation and Guardrails for AI Coding Agents.** [](#) [](#) [](#) Building autonomous coding agents is easy. Figuring out how to evaluate whether what they've done is actually good is incredibly hard. `agent-eval-mcp` is a stateless, deterministic Model Context Protocol (MCP) server that stops AI agents from writing lazy, unverified, or hallucinated code. It provides language-agnostic rulesets and hybrid scoring to grade AI-generated revisions *before* they get merged. ## ⚠️ The Problem When you ask an LLM to evaluate its own code, it suffers from sycophancy. It will confidently tell you its fix is perfect, even when it has: * Generated dummy patterns like `new HashMap<>()` or `pass`. * Left `// TODO: implement this` in the production patch. * Hallucinated the surrounding `SEARCH/REPLACE` context, breaking the Git patch. ## 💡 The Solution This package exposes objective evaluation tools to your agentic workflows via the **Model Context Protocol (MCP)**. It evaluates AI-generated code edits using fuzzy-matching and language-specific Abstract Syntax Tree (AST) rules (Java, Python, TypeScript, Go) to catch hallucinations deterministically. It supports two edit formats: - **Cursor SEARCH/REPLACE blocks** — for interactive agentic coding sessions. - **Standard unified diffs** — for CI/CD pipelines and GitHub Action workflows, where diffs come from Pull Requests or `git diff` output. It completely decouples the heavy lifting of code validation from your LLM orchestration layer. ## 🔧 Available Tools | Tool | Input Format | Use Case | |---|---|---| | `verify_fix` | Cursor `<<<< SEARCH >>>> REPLACE` blocks | Interactive agentic coding sessions in Cursor | | `verify_unified_diff` | Standard unified diff (`git diff` / Gi…
Mapped onto the structured taxonomy from declared text and observed tool names. Confidence shown for derived entries.
Every source is kept verbatim. Field changes are logged as events.