{"$schema":"https://wellknown.network/schemas/agent-record-v1.json","schemaVersion":"1","id":"ag_k4yhuea48c8a","handle":"agent-eval-mcp","url":"https://wellknown.network/agents/agent-eval-mcp","links":{"self":"https://wellknown.network/agents/agent-eval-mcp/record.json","html":"https://wellknown.network/agents/agent-eval-mcp","markdown":"https://wellknown.network/agents/agent-eval-mcp/record.md","api":"https://wellknown.network/api/v1/agents/agent-eval-mcp","status":"https://wellknown.network/api/v1/agents/agent-eval-mcp/status","claim":"https://wellknown.network/agents/agent-eval-mcp/claim","claimApi":"https://wellknown.network/api/v1/claims","claimDescriptor":"https://wellknown.network/agents/agent-eval-mcp/claim.json","badge":"https://wellknown.network/agents/agent-eval-mcp/badge.svg","openapi":"https://wellknown.network/openapi.json"},"ard":{"identifier":"urn:air::server:agent-eval-mcp","type":"application/mcp-server-card+json"},"kind":"mcp_server","declared":{"name":"agent-eval-mcp","summary":"Deterministic evaluation tools for AI coding agents, exposed as an MCP server.","description":"# 🛡️ agent-eval-mcp\n\n**Deterministic Evaluation and Guardrails for AI Coding Agents.**\n\n[![MCP Compatible](https://img.shields.io/badge/MCP-Compatible-green.svg)](#)\n[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](#)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](#)\n\nBuilding autonomous coding agents is easy. Figuring out how to evaluate whether what they've done is actually good is incredibly hard. \n\n`agent-eval-mcp` is a stateless, deterministic Model Context Protocol (MCP) server that stops AI agents from writing lazy, unverified, or hallucinated code. It provides language-agnostic rulesets and hybrid scoring to grade AI-generated revisions *before* they get merged.\n\n## ⚠️ The Problem\n\nWhen you ask an LLM to evaluate its own code, it suffers from sycophancy. It will confidently tell you its fix is perfect, even when it has:\n* Generated dummy patterns like `new HashMap<>()` or `pass`.\n* Left `// TODO: implement this` in the production patch.\n* Hallucinated the surrounding `SEARCH/REPLACE` context, breaking the Git patch.\n\n## 💡 The Solution\n\nThis package exposes objective evaluation tools to your agentic workflows via the **Model Context Protocol (MCP)**. It evaluates AI-generated code edits using fuzzy-matching and language-specific Abstract Syntax Tree (AST) rules (Java, Python, TypeScript, Go) to catch hallucinations deterministically.\n\nIt supports two edit formats:\n- **Cursor SEARCH/REPLACE blocks** — for interactive agentic coding sessions.\n- **Standard unified diffs** — for CI/CD pipelines and GitHub Action workflows, where diffs come from Pull Requests or `git diff` output.\n\nIt completely decouples the heavy lifting of code validation from your LLM orchestration layer.\n\n## 🔧 Available Tools\n\n| Tool | Input Format | Use Case |\n|---|---|---|\n| `verify_fix` | Cursor `<<<< SEARCH >>>> REPLACE` blocks | Interactive agentic coding sessions in Cursor |\n| `verify_unified_diff` | Standard unified diff (`git diff` / Gi…","publisher":null,"homepage":"https://github.com/nicolaemorcov/agent-eval-mcp","repository":"https://github.com/nicolaemorcov/agent-eval-mcp/blob/main/CHANGELOG.md","version":"0.2.0","license":"MIT","protocols":["mcp"],"tags":["ai-agents","code-review","evaluation","mcp"],"pricing":null,"endpoints":[{"url":"pypi:agent-eval-mcp","type":"package_pypi","auth":null,"probeable":false}],"skills":null,"tools":null,"extra":null,"attribution":{"kind":"pypi","name":"pypi","license":"pypi","repoUrl":"pypi","summary":"pypi","version":"pypi","description":"pypi","homepageUrl":"pypi"}},"derived":{"capabilities":[{"slug":"code.review","name":"Code Review","confidence":1,"provenance":"derived"},{"slug":"dev.version-control","name":"Version Control","confidence":1,"provenance":"derived"},{"slug":"dev.ci-cd","name":"CI/CD & Deploy","confidence":1,"provenance":"derived"},{"slug":"ai.evaluation","name":"Evaluation & Benchmarks","confidence":1,"provenance":"declared"},{"slug":"productivity.tasks","name":"Tasks & To-do","confidence":0.745,"provenance":"derived"}],"categories":["ai","code","dev","productivity"],"language":"en"},"observed":{"status":"unknown","statusReason":"Distributed as a package to run locally; no network endpoint to check.","lastOkAt":null,"lastProbedAt":null,"statusComputedAt":null,"reliability30d":null,"latestObservations":[],"tools":null,"package":{"name":"agent-eval-mcp","registry":"pypi","observedAt":"2026-09-09T08:22:28.445Z","publishedAt":"2026-03-26T20:35:39.765266Z","latestVersion":"0.2.0"}},"verification":{"claimed":false,"claimedAt":null,"proofs":[]},"provenance":{"sources":[{"source":"pypi","key":"agent-eval-mcp","url":"https://pypi.org/project/agent-eval-mcp/","firstSeenAt":"2026-09-09T08:20:53.304Z","fetchedAt":"2026-09-09T08:20:53.304Z","normalizedAt":"2026-09-09T08:20:53.304Z"}]},"firstSeenAt":"2026-09-09T08:20:53.304Z","updatedAt":"2026-09-09T08:22:28.445Z"}