{"$schema":"https://wellknown.network/schemas/agent-record-v1.json","schemaVersion":"1","id":"ag_2pcbv5m886nn","handle":"mcp-data-check","url":"https://wellknown.network/agents/mcp-data-check","links":{"self":"https://wellknown.network/agents/mcp-data-check/record.json","html":"https://wellknown.network/agents/mcp-data-check","markdown":"https://wellknown.network/agents/mcp-data-check/record.md","api":"https://wellknown.network/api/v1/agents/mcp-data-check","status":"https://wellknown.network/api/v1/agents/mcp-data-check/status","claim":"https://wellknown.network/agents/mcp-data-check/claim","claimApi":"https://wellknown.network/api/v1/claims","claimDescriptor":"https://wellknown.network/agents/mcp-data-check/claim.json","badge":"https://wellknown.network/agents/mcp-data-check/badge.svg","openapi":"https://wellknown.network/openapi.json","history":"https://wellknown.network/api/v1/agents/mcp-data-check/history","tools":"https://wellknown.network/api/v1/agents/mcp-data-check/tools"},"ard":{"identifier":"urn:air::server:mcp-data-check","type":"application/mcp-server-card+json"},"kind":"mcp_server","declared":{"name":"mcp-data-check","summary":"Evaluate MCP server accuracy against known questions and answers","description":"# mcp-data-check\n\nEvaluate MCP server accuracy against known questions and answers.\n\n## Installation\n\n```bash\npip install mcp-data-check\n```\n\nOr install from source:\n\n```bash\npip install -e .\n```\n\n## Usage\n\n### Python API\n\n**Anthropic (default)**\n\n```python\nfrom mcp_data_check import run_evaluation\n\nresults = run_evaluation(\n    questions_filepath=\"questions.csv\",\n    api_key=\"sk-ant-...\",\n    server_url=\"https://mcp.example.com/sse\"\n)\n\nprint(f\"Pass rate: {results['summary']['pass_rate']:.1%}\")\nprint(f\"Passed: {results['summary']['passed']}/{results['summary']['total']}\")\n```\n\n**OpenAI**\n\n```python\nfrom mcp_data_check import run_evaluation\n\nresults = run_evaluation(\n    questions_filepath=\"questions.csv\",\n    api_key=\"sk-...\",\n    server_url=\"https://mcp.example.com/sse\",\n    provider=\"openai\",\n    model=\"gpt-4o\"\n)\n```\n\n### Command Line\n\n**Anthropic (default)**\n\n```bash\nmcp-data-check https://mcp.example.com/sse -q questions.csv -k YOUR_API_KEY\n```\n\n**OpenAI**\n\n```bash\nmcp-data-check https://mcp.example.com/sse -q questions.csv -p openai -m gpt-4o -k YOUR_API_KEY\n```\n\n**Baseline comparison (MCP vs no tools)**\n\n```bash\n# Anthropic\nmcp-data-check https://mcp.example.com/sse -q questions.csv --compare\n\n# OpenAI\nmcp-data-check https://mcp.example.com/sse -q questions.csv -p openai -m gpt-4o --compare\n```\n\nEach question is run `--repeats` times per mode (default 5). Pass/fail is determined by majority vote. Results are saved to `./results/comparison_<timestamp>.json`.\n\nOptions:\n- `-q, --questions`: Path to questions CSV file (required)\n- `-p, --provider`: LLM provider to use: `anthropic` (default) or `openai`\n- `-k, --api-key`: API key for the chosen provider (defaults to `ANTHROPIC_API_KEY` or `OPENAI_API_KEY` env var)\n- `-o, --output`: Output directory for results (default: `./results`)\n- `-m, --model`: Model to use for evaluation (default: `claude-sonnet-4-20250514`; use e.g. `gpt-4o` for OpenAI)\n- `-n, --server-name`: Name for the MCP server (default: `mcp-server`)\n…","publisher":null,"homepage":"https://github.com/GSA-TTS/mcp-data-check","repository":"https://github.com/GSA-TTS/mcp-data-check","version":"0.6.1","license":"MIT","protocols":["mcp"],"tags":["anthropic","claude","evaluation","mcp","testing"],"pricing":null,"endpoints":[{"url":"pypi:mcp-data-check","type":"package_pypi","auth":null,"probeable":false}],"skills":null,"tools":null,"extra":null,"attribution":{"kind":"pypi","name":"pypi","license":"pypi","repoUrl":"pypi","summary":"pypi","version":"pypi","description":"pypi","homepageUrl":"pypi"}},"derived":{"capabilities":[{"slug":"ai.evaluation","name":"Evaluation & Benchmarks","confidence":1,"provenance":"declared"},{"slug":"code.testing","name":"Testing","confidence":1,"provenance":"declared"},{"slug":"dev.terminal","name":"Terminal & Shell","confidence":0.825,"provenance":"derived"}],"categories":["ai","code","dev"],"language":"en"},"observed":{"status":"unknown","statusReason":"Distributed as a package to run locally; no network endpoint to check.","lastOkAt":null,"lastProbedAt":null,"statusComputedAt":null,"reliability30d":null,"latestObservations":[],"tools":null,"package":{"name":"mcp-data-check","registry":"pypi","observedAt":"2026-10-28T10:24:36.307Z","publishedAt":"2026-04-29T18:56:30.105833Z","latestVersion":"0.6.1"},"toolSurface":null,"endpointFacts":[]},"verification":{"claimed":false,"claimedAt":null,"proofs":[]},"provenance":{"sources":[{"source":"pypi","key":"mcp-data-check","url":"https://pypi.org/project/mcp-data-check/","firstSeenAt":"2026-09-10T01:19:54.916Z","fetchedAt":"2026-10-28T10:24:00.392Z","normalizedAt":"2026-10-28T10:24:00.392Z"}]},"firstSeenAt":"2026-09-10T01:19:54.916Z","updatedAt":"2026-10-28T10:24:36.307Z"}