{"$schema":"https://wellknown.network/schemas/agent-record-v1.json","schemaVersion":"1","id":"ag_jn2pcffb5kbn","handle":"iflow-mcp-icip-cas-livemcpbench","url":"https://wellknown.network/agents/iflow-mcp-icip-cas-livemcpbench","links":{"self":"https://wellknown.network/agents/iflow-mcp-icip-cas-livemcpbench/record.json","html":"https://wellknown.network/agents/iflow-mcp-icip-cas-livemcpbench","markdown":"https://wellknown.network/agents/iflow-mcp-icip-cas-livemcpbench/record.md","api":"https://wellknown.network/api/v1/agents/iflow-mcp-icip-cas-livemcpbench","status":"https://wellknown.network/api/v1/agents/iflow-mcp-icip-cas-livemcpbench/status","claim":"https://wellknown.network/agents/iflow-mcp-icip-cas-livemcpbench/claim","claimApi":"https://wellknown.network/api/v1/claims","claimDescriptor":"https://wellknown.network/agents/iflow-mcp-icip-cas-livemcpbench/claim.json","badge":"https://wellknown.network/agents/iflow-mcp-icip-cas-livemcpbench/badge.svg","openapi":"https://wellknown.network/openapi.json","history":"https://wellknown.network/api/v1/agents/iflow-mcp-icip-cas-livemcpbench/history","tools":"https://wellknown.network/api/v1/agents/iflow-mcp-icip-cas-livemcpbench/tools"},"ard":{"identifier":"urn:air::server:iflow-mcp-icip-cas-livemcpbench","type":"application/mcp-server-card+json"},"kind":"mcp_server","declared":{"name":"iflow-mcp_icip-cas-livemcpbench","summary":"LiveMCPBench is a benchmark for evaluating the ability of agents to navigate and utilize a large-scale MCP toolset. It provides a comprehensive set of tasks that challenge agents to effectively use various tools in daily scenarios.","description":"<a id=\"readme-top\"></a>\n\n<!-- PROJECT -->\n<br />\n<div align=\"center\">\n  <h3 align=\"center\">LiveMCPBench: Can Agents Navigate an Ocean of MCP Tools?</h3>\n\n  <p align=\"center\">\n    Benchmarking the agent in real-world tasks within a large-scale MCP toolset.\n  </p>\n</div>\n<p align=\"center\">\n<a href=\"https://www.python.org/downloads/release/python-31113/\"><img src=\"https://img.shields.io/badge/python-3.11-blue.svg\" alt=\"Python 3.11\"></a>\n<a href=\"https://github.com/astral-sh/ruff\"><img src=\"https://img.shields.io/badge/code%20style-ruff-000000.svg\" alt=\"Code style: ruff\"></a>\n</p>\n\n<p align=\"center\">\n  🌐 <a href=\"https://icip-cas.github.io/LiveMCPBench\" target=\"_blank\">Website</a> &nbsp; | &nbsp;\n  📄 <a href=\"https://arxiv.org/abs/2508.01780\" target=\"_blank\">Paper</a> &nbsp; | &nbsp;\n  🤗 <a href=\"https://huggingface.co/datasets/ICIP/LiveMCPBench\" target=\"_blank\">Dataset</a> &nbsp; | &nbsp;\n  🐳 <a href=\"https://hub.docker.com/r/hysdhlx/livemcpbench\" target=\"_blank\">Docker</a> &nbsp; | &nbsp;\n  🏆 <a href=\"https://docs.google.com/spreadsheets/d/1EXpgXq1VKw5A7l7-N2E9xt3w0eLJ2YPVPT-VrRxKZBw/edit?usp=sharing\" target=\"_blank\">Leaderboard</a> \n  &nbsp; | &nbsp;\n  🙏 <a href=\"#citation\" target=\"_blank\">Citation</a>\n</p>\n\n![Overview](media/LiveMCPBench.png)\n## News\n* [8/18/2025] We releas [Docker images](https://hub.docker.com/r/hysdhlx/livemcpbench) and add evaluation results in [leaderboard](https://docs.google.com/spreadsheets/d/1EXpgXq1VKw5A7l7-N2E9xt3w0eLJ2YPVPT-VrRxKZBw/edit?usp=sharing) for three new models: GLM 4.5, GPT-5-Mini, and Kimi-K2.\n* [8/3/2025] We release the LiveMCPBench.\n## Getting Started\n\n### Prerequisites\nWe recommend using our docker image, but if you want to run the code locally, you will need to install the following tools:\n* npm\n* uv\n### Installation\n1. Pull the docker image\n\n   ```bash\n   docker pull hysdhlx/livemcpbench:latest\n   ```\n2. Git the repo and run the docker image\n\n   ```bash\n   git clone https://github.com/icip-cas/LiveMCPBench.git\n   …","publisher":null,"homepage":null,"repository":null,"version":"0.1.1","license":null,"protocols":["mcp"],"tags":["mcp"],"pricing":null,"endpoints":[{"url":"pypi:iflow-mcp_icip-cas-livemcpbench","type":"package_pypi","auth":null,"probeable":false}],"skills":null,"tools":null,"extra":null,"attribution":{"kind":"pypi","name":"pypi","summary":"pypi","version":"pypi","description":"pypi"}},"derived":{"capabilities":[{"slug":"ai.model-access","name":"Model Access","confidence":0.905,"provenance":"derived"},{"slug":"ai.evaluation","name":"Evaluation & Benchmarks","confidence":0.859,"provenance":"derived"},{"slug":"dev.version-control","name":"Version Control","confidence":0.745,"provenance":"derived"}],"categories":["ai","dev"],"language":"en"},"observed":{"status":"unknown","statusReason":"Distributed as a package to run locally; no network endpoint to check.","lastOkAt":null,"lastProbedAt":null,"statusComputedAt":null,"reliability30d":null,"latestObservations":[],"tools":null,"package":{"name":"iflow-mcp_icip-cas-livemcpbench","registry":"pypi","observedAt":"2026-10-28T04:28:34.326Z","publishedAt":"2026-02-08T01:39:25.108765Z","latestVersion":"0.1.1"},"toolSurface":null,"endpointFacts":[]},"verification":{"claimed":false,"claimedAt":null,"proofs":[]},"provenance":{"sources":[{"source":"pypi","key":"iflow-mcp_icip-cas-livemcpbench","url":"https://pypi.org/project/iflow-mcp_icip-cas-livemcpbench/","firstSeenAt":"2026-09-09T19:23:47.510Z","fetchedAt":"2026-10-28T04:27:48.831Z","normalizedAt":"2026-10-28T04:27:48.831Z"}]},"firstSeenAt":"2026-09-09T19:23:47.510Z","updatedAt":"2026-10-28T04:28:34.326Z"}