MCP server that tells flaky tests from genuinely broken ones using real CI run history, and generates synthetic test data / API contract diffs for QA automation.
Listed on its own since 2026-09-24. Until then it was part of triagekit-mcp-core, where its earlier checks remain.
Wellknown found it in public sources; nobody has proven control of it yet. Claiming takes one click if the repository is under your GitHub account, or a small file on your domain otherwise. Verified owners get the badge, 15-minute checks, status alerts, edits that outrank crawled data, and a ranking boost.
Agents can do it too: POST https://wellknown.network/api/v1/claims with {"agent":"triagekit-mcp-flaky-test-detective","method":"well_known_file"} — machine-readable steps at claim.json, guide at /docs/claim.
Everything here was measured by our prober or read from a registry. Nothing is self-reported.
Attributed to the source that supplied each field. Treated as claims, not facts.
# triagekit-mcp-flaky-test-detective Part of the [TriageKit](../../README.md) MCP server suite. Tells flaky tests from genuinely broken ones using real CI run history — instead of an engineer guessing "probably flaky, re-run it" or a broken test getting rubber-stamped past review because nobody has time to check. ## Why CI red gets treated one of two bad ways: blindly re-run until green (masking a real bug), or dismissed as "that test is always flaky" (hiding a regression). This server pulls a test's actual pass/fail history from GitHub Actions, classifies the pattern deterministically, clusters failures by root cause, and checks for infra correlation — so the decision is based on evidence, not vibes. ## Tools | Tool | What it does | |---|---| | `classify_test_failure` | Classifies a test's recent history as consistently-failing / consistently-passing / intermittent (flaky) / insufficient-data, with run-by-run evidence | | `get_flaky_test_leaderboard` | Ranks all tests in a repo's recent CI history by computed flake rate | | `diff_failure_stack_traces` | Clusters a test's failures by stack-trace similarity — is "flaky" really 2-3 distinct root causes? | | `correlate_flake_with_infra` | Checks whether flake rate correlates with the CI runner — infra contention vs. genuine non-determinism | | `suggest_quarantine_candidates` | Returns genuinely intermittent tests above a flake-rate threshold | | `file_flaky_test_ticket` | Opens a Jira ticket pre-filled with flake evidence (optional — requires Jira config) | | `generate_synthetic_test_data` | Generates synthetic fixtures from a JSON-Schema fragment or example payload | | `diff_api_contract` | Diffs two OpenAPI specs and classifies each change as breaking or non-breaking | ## Install > **Not yet published.** This package is not on PyPI or GHCR yet (alpha, pre-release) - the > commands below are the intended install path once `triagekit-mcp-core` and this package are > published (core must publish first; see [CON…
Mapped onto the structured taxonomy from declared text and observed tool names. Confidence shown for derived entries.
Every source is kept verbatim. Field changes are logged as events.