LiveMCPBench is a benchmark for evaluating the ability of agents to navigate and utilize a large-scale MCP toolset. It provides a comprehensive set of tasks that challenge agents to effectively use various tools in daily scenarios.
Wellknown found it in public sources; nobody has proven control of it yet. Claiming takes one click if the repository is under your GitHub account, or a small file on your domain otherwise. Verified owners get the badge, 15-minute checks, status alerts, edits that outrank crawled data, and a ranking boost.
Agents can do it too: POST https://wellknown.network/api/v1/claims with {"agent":"iflow-mcp-icip-cas-livemcpbench","method":"well_known_file"} — machine-readable steps at claim.json, guide at /docs/claim.
Everything here was measured by our prober or read from a registry. Nothing is self-reported.
Attributed to the source that supplied each field. Treated as claims, not facts.
<a id="readme-top"></a> <!-- PROJECT --> <br /> <div align="center"> <h3 align="center">LiveMCPBench: Can Agents Navigate an Ocean of MCP Tools?</h3> <p align="center"> Benchmarking the agent in real-world tasks within a large-scale MCP toolset. </p> </div> <p align="center"> <a href="https://www.python.org/downloads/release/python-31113/"><img src="https://img.shields.io/badge/python-3.11-blue.svg" alt="Python 3.11"></a> <a href="https://github.com/astral-sh/ruff"><img src="https://img.shields.io/badge/code%20style-ruff-000000.svg" alt="Code style: ruff"></a> </p> <p align="center"> 🌐 <a href="https://icip-cas.github.io/LiveMCPBench" target="_blank">Website</a> | 📄 <a href="https://arxiv.org/abs/2508.01780" target="_blank">Paper</a> | 🤗 <a href="https://huggingface.co/datasets/ICIP/LiveMCPBench" target="_blank">Dataset</a> | 🐳 <a href="https://hub.docker.com/r/hysdhlx/livemcpbench" target="_blank">Docker</a> | 🏆 <a href="https://docs.google.com/spreadsheets/d/1EXpgXq1VKw5A7l7-N2E9xt3w0eLJ2YPVPT-VrRxKZBw/edit?usp=sharing" target="_blank">Leaderboard</a> | 🙏 <a href="#citation" target="_blank">Citation</a> </p>  ## News * [8/18/2025] We releas [Docker images](https://hub.docker.com/r/hysdhlx/livemcpbench) and add evaluation results in [leaderboard](https://docs.google.com/spreadsheets/d/1EXpgXq1VKw5A7l7-N2E9xt3w0eLJ2YPVPT-VrRxKZBw/edit?usp=sharing) for three new models: GLM 4.5, GPT-5-Mini, and Kimi-K2. * [8/3/2025] We release the LiveMCPBench. ## Getting Started ### Prerequisites We recommend using our docker image, but if you want to run the code locally, you will need to install the following tools: * npm * uv ### Installation 1. Pull the docker image ```bash docker pull hysdhlx/livemcpbench:latest ``` 2. Git the repo and run the docker image ```bash git clone https://github.com/icip-cas/LiveMCPBench.git …
Mapped onto the structured taxonomy from declared text and observed tool names. Confidence shown for derived entries.
Every source is kept verbatim. Field changes are logged as events.