This tool is designed to analyze MSBench log zip file, automatically extract error information from nested logs.
Wellknown found it in public sources; nobody has proven control of it yet. Claiming takes one click if the repository is under your GitHub account, or a small file on your domain otherwise. Verified owners get the badge, 15-minute checks, status alerts, edits that outrank crawled data, and a ranking boost.
Agents can do it too: POST https://wellknown.network/api/v1/claims with {"agent":"msbench-analysis-mcp","method":"well_known_file"} — machine-readable steps at claim.json, guide at /docs/claim.
Everything here was measured by our prober or read from a registry. Nothing is self-reported.
Attributed to the source that supplied each field. Treated as claims, not facts.
# MSBench Analysis MCP This tool is designed to analyze MSBench log zip files, automatically extracting error information from nested logs and trajectory. After starting, open Copilot Chat and switch the mode to `agent mode`. 4. Check if the five tools provided by this server are recognized by Copilot: - `analyze_cls_errors_and_save_report` - Cls log analysis with Excel output - `analyze_cls_errors` - Cls log analysis with markdown output - `analyze_trajectory_errors_and_save_report` - Trajectory analysis with Excel output - `analyze_trajectory_errors` - Trajectory analysis with markdown output - `compare_two_zips_trajectory_analysis` - Compare two trajectory zip files with Excel output 5. You can now use Copilot Chat in conversation mode to analyze MSBench errors interactively. supports both cls log analysis and tool call trajectory analysis. It is intended for use as an MCP (Model Context Protocol) server, supporting integration with MCP clients such as Copilot Chat or mcp-inspector. ## Features - **Cls Log Analysis:** - Recursively extracts and analyzes all inner zip files within a given MSBench log archive. - Only processes `cls.log` files located in the `output` directory of each inner zip. - Extracts error messages and error types from logs using regular expressions. - Aggregates error statistics by type and provides total error counts. - Tracks and reports the number of skipped (unreadable or broken) inner zip files and cls.log files. - Generates a detailed Excel report with error details and summary statistics. - **Trajectory Data Analysis:** - Analyzes `fetchLog.json` files containing detailed tool call interaction data. - Processes step-by-step tool call sequences with assistant-tool message pairs. - Identifies failed tool calls with "Tool call failed with error:" patterns. - Extracts tool call error details including step number, tool call ID, tool name, and error messages. - Uses optimized single-pass analysis to …
Mapped onto the structured taxonomy from declared text and observed tool names. Confidence shown for derived entries.
Every source is kept verbatim. Field changes are logged as events.