GitScribe: Web Scraping RAG MCP Server for Git-based Documentation
Wellknown found it in public sources; nobody has proven control of it yet. Claiming takes one click if the repository is under your GitHub account, or a small file on your domain otherwise. Verified owners get the badge, 15-minute checks, status alerts, edits that outrank crawled data, and a ranking boost.
Agents can do it too: POST https://wellknown.network/api/v1/claims with {"agent":"gitscribe-mcp","method":"well_known_file"} — machine-readable steps at claim.json, guide at /docs/claim.
Everything here was measured by our prober or read from a registry. Nothing is self-reported.
Attributed to the source that supplied each field. Treated as claims, not facts.
# GitScribe 📜 > *Scribing knowledge from the Git universe* GitScribe is a powerful Model Context Protocol (MCP) server that enables intelligent web scraping of Git-based documentation with Retrieval Augmented Generation (RAG) capabilities. This tool helps code assistants and developers efficiently extract, process, and retrieve information from documentation websites, GitHub repositories, and other Git-based resources to accelerate application development. ## ✨ Features - **🌐 Universal Git Support**: Works with GitHub, GitLab, Bitbucket, and Azure DevOps - **🧠 Intelligent RAG System**: ChromaDB + Sentence Transformers for semantic search - **📄 Multi-Format Parsing**: Markdown, HTML, reStructuredText, and source code files - **⚡ High Performance**: Async scraping with intelligent rate limiting - **🔧 MCP Integration**: Full Model Context Protocol compliance for AI assistants - **📊 Rich CLI**: Command-line interface for testing and management - **🎯 Smart Filtering**: Automatic content filtering and relevance scoring ## 🚀 Quick Start ### Installation ```bash # Install from PyPI (recommended) pip install gitscribe-mcp # Or install with uv (recommended for development) uv sync # Or install with pip for development pip install -e . # Or install dependencies manually pip install -r requirements-gitscribe.txt ``` ### Verify Installation ```bash # Check if installation was successful gitscribe-mcp --help # Test the server (should start without errors) gitscribe-mcp server --help ``` ### Basic Usage #### 1. Start the MCP Server ```bash # Start the server for use with AI assistants gitscribe-mcp server # Or run directly with uv uv run gitscribe-mcp server ``` #### 2. Scrape Documentation ```bash # Scrape Python documentation gitscribe-mcp scrape https://docs.python.org --depth 2 --output python_docs.json # Scrape a GitHub repository gitscribe-mcp scrape https://github.com/microsoft/vscode --formats md html rst ``` #### 3. Index Documents ```bash # In…
Mapped onto the structured taxonomy from declared text and observed tool names. Confidence shown for derived entries.
Every source is kept verbatim. Field changes are logged as events.