A lightweight Model Context Protocol (MCP) server that exposes Crawl4AI web scraping and crawling capabilities as tools for AI agents.
Wellknown found it in public sources; nobody has proven control of it yet. Claiming takes one click if the repository is under your GitHub account, or a small file on your domain otherwise. Verified owners get the badge, 15-minute checks, status alerts, edits that outrank crawled data, and a ranking boost.
Agents can do it too: POST https://wellknown.network/api/v1/claims with {"agent":"iflow-mcp-sadiuysal-crawl4ai-mcp-server","method":"well_known_file"} — machine-readable steps at claim.json, guide at /docs/claim.
Everything here was measured by our prober or read from a registry. Nothing is self-reported.
Attributed to the source that supplied each field. Treated as claims, not facts.
# Crawl4AI MCP Server 🕷️ **A lightweight Model Context Protocol (MCP) server that exposes [Crawl4AI](https://docs.crawl4ai.com/) web scraping and crawling capabilities as tools for AI agents.** Similar to Firecrawl's API but self-hosted and free. Perfect for integrating web scraping into your AI workflows with OpenAI Agents SDK, Cursor, Claude Code, and other MCP-compatible tools. ## Features - **🔧 MCP Tools**: Exposes 4 powerful tools: `scrape`, `crawl`, `crawl_site`, `crawl_sitemap` via stdio MCP server - **🌐 Web Scraping**: Single-page scraping with markdown extraction - **🕷️ Web Crawling**: Multi-page breadth-first crawling with depth control - **🧠 Adaptive Crawling**: Smart crawling that stops when sufficient content is gathered - **🛡️ Safety**: Blocks internal networks, localhost, and private IPs - **📱 Agent Ready**: Works with OpenAI Agents SDK, Cursor, and Claude Code - **⚡ Fast**: Powered by Playwright and Crawl4AI's optimized extraction ## 🚀 Quick Start Choose between **Docker** (recommended) or **manual installation**: ### Option A: Docker (Recommended) 🐳 Docker eliminates all setup complexity and provides a consistent environment: #### Option A1: Use Pre-built Image (Fastest) ⚡ ```bash # No setup required! Just pull and run the published image docker pull uysalsadi/crawl4ai-mcp-server:latest # Test it works python test-config.py # Use directly in MCP configurations (see examples below) ``` #### Option A2: Build Yourself ```bash # Clone the repository git clone https://github.com/uysalsadi/crawl4ai-mcp-server.git cd crawl4ai-mcp-server # Quick build and test (simplified) docker build -f Dockerfile.simple -t crawl4ai-mcp . echo '{"jsonrpc": "2.0", "id": 1, "method": "initialize", "params": {"protocolVersion": "2024-11-05", "capabilities": {}, "clientInfo": {"name": "test", "version": "1.0"}}}' | docker run --rm -i crawl4ai-mcp # Or use helper script (full build with Playwright) ./docker-run.sh build ./docker-run.sh test ./docker-run…
Mapped onto the structured taxonomy from declared text and observed tool names. Confidence shown for derived entries.
Every source is kept verbatim. Field changes are logged as events.