{"$schema":"https://wellknown.network/schemas/agent-record-v1.json","schemaVersion":"1","id":"ag_uwxckg4c942s","handle":"mcp-confidence","url":"https://wellknown.network/agents/mcp-confidence","links":{"self":"https://wellknown.network/agents/mcp-confidence/record.json","html":"https://wellknown.network/agents/mcp-confidence","markdown":"https://wellknown.network/agents/mcp-confidence/record.md","api":"https://wellknown.network/api/v1/agents/mcp-confidence","status":"https://wellknown.network/api/v1/agents/mcp-confidence/status","claim":"https://wellknown.network/agents/mcp-confidence/claim","claimApi":"https://wellknown.network/api/v1/claims","claimDescriptor":"https://wellknown.network/agents/mcp-confidence/claim.json","badge":"https://wellknown.network/agents/mcp-confidence/badge.svg","openapi":"https://wellknown.network/openapi.json","history":"https://wellknown.network/api/v1/agents/mcp-confidence/history","tools":"https://wellknown.network/api/v1/agents/mcp-confidence/tools"},"ard":{"identifier":"urn:air::server:mcp-confidence","type":"application/mcp-server-card+json"},"kind":"mcp_server","declared":{"name":"mcp-confidence","summary":"A drop-in confidence gate for LLM agents: turn token logprobs into an accept / verify / ask-a-human routing decision, with honest per-model calibration.","description":"

mcp-confidence

\n\n

\n A drop-in confidence gate for LLM agents.
\n Turn any model's token logprobs into an accept / verify / ask-a-human routing decision —
\n with honest, per-model calibration. Local & open-model first.\n

\n\n

\n \"PyPI\n \"MIT\n \"Python\n

\n\n---\n\nA language model can tell you *how* surprised it was by every token it just produced — that's what `logprobs=True` returns. `mcp-confidence` reads those numbers and turns them into a routing decision your agent can act on:\n\n- **HIGH** → auto-accept the model's output.\n- **MID** → verify (hand off to a stronger model or a supervisor check).\n- **LOW** → ask a human.\n\nIt's **local- and open-model-first** (vLLM, Ollama, llama.cpp, TGI, or any OpenAI-compatible endpoint), the core has **zero third-party runtime dependencies**, and it ships a one-command **MCP server** so a cloud \"director\" (e.g. Claude) can delegate generation to a local worker and get a confidence band back with every answer.\n\n## Why\n\nThe naive approach — \"average the token probabilities, accept above 0.95\" — is a weak and *uncalibrated* signal for open-domain free text, and quietly wrong on most models. High-frequency function words and MoE routing pull the per-token mean toward zero, so `exp(mean_logprob)` sits at 0.05–0.20 even for perfectly fluent output. A fixed probability threshold tuned on short structured outputs will reject good answers and accept bad ones once you point it at real prose.\n\n`mcp-confidence` takes four deliberate positions to make the signal usable:\n\n1. **Band in mean-lo…","publisher":null,"homepage":"https://github.com/shaxzodbek-uzb/mcp-confidence","repository":"https://github.com/shaxzodbek-uzb/mcp-confidence","version":"0.2.0","license":"MIT","protocols":["mcp"],"tags":["agents","calibration","confidence","hitl","llm","local-llm","logprobs","mcp"],"pricing":null,"endpoints":[{"url":"pypi:mcp-confidence","type":"package_pypi","auth":null,"probeable":false}],"skills":null,"tools":null,"extra":null,"attribution":{"kind":"pypi","name":"pypi","license":"pypi","repoUrl":"pypi","summary":"pypi","version":"pypi","description":"pypi","homepageUrl":"pypi"}},"derived":{"capabilities":[{"slug":"dev.package-management","name":"Packages & Dependencies","confidence":0.825,"provenance":"derived"}],"categories":["dev"],"language":"en"},"observed":{"status":"unknown","statusReason":"Distributed as a package to run locally; no network endpoint to check.","lastOkAt":null,"lastProbedAt":null,"statusComputedAt":null,"reliability30d":null,"latestObservations":[],"tools":null,"package":{"name":"mcp-confidence","registry":"pypi","observedAt":"2026-11-09T14:28:14.805Z","publishedAt":"2026-08-16T09:38:47.948337Z","latestVersion":"0.2.0"},"toolSurface":null,"endpointFacts":[]},"verification":{"claimed":false,"claimedAt":null,"proofs":[]},"provenance":{"sources":[{"source":"pypi","key":"mcp-confidence","url":"https://pypi.org/project/mcp-confidence/","firstSeenAt":"2026-09-10T01:19:27.654Z","fetchedAt":"2026-11-09T14:27:26.224Z","normalizedAt":"2026-11-09T14:27:26.224Z"}]},"firstSeenAt":"2026-09-10T01:19:27.654Z","updatedAt":"2026-11-09T14:28:14.805Z"}