Evaluation & Benchmarks

Load test a Model Context Protocol server with random or scripted tools and parameters

Check classifier confidence against accuracy, fit temperature scaling and save local reports.

Word search, crossword, and sudoku generators, printable PDF worksheets, and verifiable LLM evals.

GitPadi — AI-powered GitHub & GitLab management CLI. Fork repos, manage issues & PRs, score contributors, grade assignments, and automate everything. Powered by Anthropic Claude via GitLab Duo Agent Platform.

Render web and SwiftUI evidence, run explicit visual judgment, and trace ADS UI decisions.

MCP server: x402-paid & free tools for AI agents — provenance, quality scoring, action audit.

MCP server exposing TypeSafe's Jev as a `judge` tool for typed judgments with calibrated probabilities. Works in any MCP client; ships an optional Claude Code plugin wrapper.

Evidence-verified deterministic resume screening, hard-gate evaluation, timeline calculation, scoring, ranking and audit MCP server.

Core engines for AI agent trust scoring, OWASP analysis, permission extraction, and attack detection

CI for MCP servers: schema contract testing + cross-model regression evals for your agent-facing tools

End-to-end reliability testing for multi-step AI agent workflows. Tests what DeepEval and Langfuse can't: cascading failures across 10+ step chains.

Zero-dependency MCP tools/list linter with reproducible counts from 2,283 servers and 29,574 tool declarations.

Model-agnostic harness that measures how well LLMs call the tools of any MCP server

Sentinel AgentOS — 确定性 Guard 层 + 分层记忆 + 自动评估，让任何 Agent 变得可靠、可审计、可改进

Run the SolVerdict adversarial scenarios against your own agent on a local Solana fork. Produces an evidence bundle; scoring happens server-side.

Read exams, gradebooks and item analysis; parse papers with AI and publish exams.

AI coding-agent skill for shipd.ai Olympus submissions: design hard tasks, write hidden tests, avoid Docker traps, and verify locally.

MCP server for LeadVal — submit finished AI-agent conversations for scoring, poll them to a terminal state, and ask mid-conversation whether this workspace already knows the person you are speaking to. Four tools over stdio for Claude Desktop, Cursor or y

Statistical regression testing for LLM agents: p-value, effect size, and CI on behavior change.

Inspect MCP servers and write a local readiness report.