AI Crawler Index

Name: AI Crawler Index
Author: dev.workers.pathwren.www

MCP serverMCP

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} free no key

by dev.workers.pathwren.wwwhomepage ↗v1.5.0

Is this yours?

This record is unclaimed

Wellknown found it in public sources; nobody has proven control of it yet. Claiming takes one click if the repository is under your GitHub account, or a small file on your domain otherwise. Verified owners get the badge, 15-minute checks, status alerts, edits that outrank crawled data, and a ranking boost.

Agents can do it too: POST https://wellknown.network/api/v1/claims with {"agent":"ai-crawler-index","method":"well_known_file"} — machine-readable steps at claim.json, guide at /docs/claim.

Claim this record

Observed by Wellknown

Evidence

Everything here was measured by our prober or read from a registry. Nothing is self-reported.

30-day availability1/1 checks ok

30 days agotoday

Tools observed via MCP handshake

whoami — {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp and you
example — {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"example","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp and you
classify_user_agent — FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Identify the crawler behind a raw User-Agent header: operator, category, robots.txt stance, how to
lookup_crawler — FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — The full index record for one crawler: operator, category, robots.txt token and stance, user-agent
list_crawlers — FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — The index, filtered or whole: each row says what the crawler is and what blocking it costs, and th
generate_robots_txt — FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — A ready-to-paste robots.txt block for one stance, with the reasoning and the crawlers it names. Om
is_verified_crawler_ip — FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Test an IPv4 or IPv6 address against every crawler prefix the operators themselves publish, remirr
whats_changed — FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Freshness and change state: when the index was built, when each operator source was last fetched,
changes_since — FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Only what changed since your last call: prefixes added or removed, upstreams failed or recovered,

Declared by sources

What the publisher says

Attributed to the source that supplied each field. Treated as claims, not facts.

No long description beyond the summary.

kind ← Official MCP Registry

name ← Official MCP Registry

summary ← Official MCP Registry

version ← Official MCP Registry

description ← Official MCP Registry

homepageUrl ← Official MCP Registry

publisherName ← Official MCP Registry

Provenance

Where this record came from

Every source is kept verbatim. Field changes are logged as events.

Official MCP Registrydev.workers.pathwren.www/ai-crawler-index

first seen 12 h ago · fetched 12 h agoopen ↗

Field history (6 recent events)

12 h ago · declared · summary → "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"tools/call\",\"params\":{\"name\":\"whoami\",\"arguments\":{}}} free no key"
12 h ago · declared · description → "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"tools/call\",\"params\":{\"name\":\"whoami\",\"arguments\":{}}} free no key"
12 h ago · declared · publisherName → "dev.workers.pathwren.www"
12 h ago · declared · homepageUrl → "https://www.pathwren.workers.dev/c/mcp-registry-official/mcp.html"
12 h ago · declared · version → "1.5.0"
12 h ago · declared · protocols → ["mcp"]

when	check	result	http	latency	detail
29 min ago	mcp initialize	ok	200	24 ms	server ai-crawler-index 1.0.0 · protocol 2025-06-18 · 9 tools
10 h ago	mcp initialize	ok	200	15 ms	server ai-crawler-index 1.0.0 · protocol 2025-06-18 · 9 tools

This record is unclaimed

Evidence

What the publisher says

Capabilities

Where this record came from