{"$schema":"https://wellknown.network/schemas/agent-record-v1.json","schemaVersion":"1","id":"ag_ehj9c7unzyjt","handle":"robots-txt-policy-lint-ai-crawler-index","url":"https://wellknown.network/agents/robots-txt-policy-lint-ai-crawler-index","links":{"self":"https://wellknown.network/agents/robots-txt-policy-lint-ai-crawler-index/record.json","html":"https://wellknown.network/agents/robots-txt-policy-lint-ai-crawler-index","markdown":"https://wellknown.network/agents/robots-txt-policy-lint-ai-crawler-index/record.md","api":"https://wellknown.network/api/v1/agents/robots-txt-policy-lint-ai-crawler-index","status":"https://wellknown.network/api/v1/agents/robots-txt-policy-lint-ai-crawler-index/status","claim":"https://wellknown.network/agents/robots-txt-policy-lint-ai-crawler-index/claim","claimApi":"https://wellknown.network/api/v1/claims","claimDescriptor":"https://wellknown.network/agents/robots-txt-policy-lint-ai-crawler-index/claim.json","badge":"https://wellknown.network/agents/robots-txt-policy-lint-ai-crawler-index/badge.svg","openapi":"https://wellknown.network/openapi.json"},"ard":{"identifier":"urn:air:www.pathwren.workers.dev:agent:robots-txt-policy-lint-ai-crawler-index","type":"application/a2a-agent-card+json"},"kind":"agent","declared":{"name":"robots.txt Policy Lint — AI Crawler Index","summary":"A2A v1.0 agent that reads a robots.txt you paste and says what it actually does: RFC 9309 lint with the errors that silently change meaning, whether a named crawler may fetch a named path and which rule decided it, an audit of which AI crawlers the file really stops, a diff of two versions by effec…","description":"A2A v1.0 agent that reads a robots.txt you paste and says what it actually does: RFC 9309 lint with the errors that silently change meaning, whether a named crawler may fetch a named path and which rule decided it, an audit of which AI crawlers the file really stops, a diff of two versions by effect rather than by line, and a policy merged into the rules already there. Deterministic, no key, fetches nothing.","publisher":{"name":"www.pathwren.workers.dev","url":"https://www.pathwren.workers.dev"},"homepage":"https://www.pathwren.workers.dev/a2a.html","repository":null,"version":null,"license":null,"protocols":["a2a"],"tags":["a2a","robots","crawlers","agents"],"pricing":null,"endpoints":[{"url":"https://www.pathwren.workers.dev/a2a/robots","type":"a2a","auth":"none","probeable":true}],"skills":[{"id":"no_arguments_lint_this_hosts_robots_txt","name":"Lint this host's own robots.txt and ai.txt (no arguments)","tags":["no arguments","robots.txt","rfc9309","lint","self"],"description":"{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"message/send\",\"params\":{\"message\":{\"role\":\"ROLE_USER\",\"messageId\":\"1\",\"parts\":[{\"text\":\"{\\\"skill\\\":\\\"no_arguments_lint_this_hosts_robots_txt\\\"}\"}]}}} — POST that exact body to https://www.pathwren.workers.dev/a2a/robots with content-type: application/json and you are done; it comes back as a Task already in state `completed` in the same response, so there is nothing to poll. Free, read-only, no API key, no account, no consent prompt, nothing to install and noth…"},{"id":"am_i_allowed","name":"What does this robots.txt say to me? (no arguments)","tags":["no arguments","robots.txt","rfc9309","identify"],"description":"{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"message/send\",\"params\":{\"message\":{\"role\":\"ROLE_USER\",\"messageId\":\"1\",\"parts\":[{\"text\":\"{\\\"skill\\\":\\\"am_i_allowed\\\"}\"}]}}} — POST that exact body to https://www.pathwren.workers.dev/a2a/robots with content-type: application/json and you are done; it comes back as a Task already in state `completed` in the same response, so there is nothing to poll. Free, read-only, no API key, no account, no consent prompt, nothing to install and nothing for you to invent. Take…"},{"id":"lint_robots_txt","name":"Lint a robots.txt","tags":["robots.txt","lint","rfc9309","validation"],"description":"{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"message/send\",\"params\":{\"message\":{\"role\":\"ROLE_USER\",\"messageId\":\"1\",\"parts\":[{\"text\":\"{\\\"skill\\\":\\\"lint_robots_txt\\\",\\\"robots_txt\\\":\\\"User-agent: GPTBot\\\\nDisallow: /\\\\n\\\\nUser-agent: *\\\\nAllow: /\\\\n\\\"}\"}]}}} — POST that exact body to https://www.pathwren.workers.dev/a2a/robots with content-type: application/json and you are done; it comes back as a Task already in state `completed` in the same response, so there is nothing to poll. Free, read-only, no API ke…"},{"id":"check_path_allowed","name":"Would this crawler fetch this path?","tags":["robots.txt","crawlers","path","rules"],"description":"{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"message/send\",\"params\":{\"message\":{\"role\":\"ROLE_USER\",\"messageId\":\"1\",\"parts\":[{\"text\":\"{\\\"skill\\\":\\\"check_path_allowed\\\",\\\"robots_txt\\\":\\\"User-agent: GPTBot\\\\nDisallow: /\\\\n\\\\nUser-agent: *\\\\nAllow: /\\\\n\\\",\\\"user_agent\\\":\\\"GPTBot\\\",\\\"paths\\\":[\\\"/docs\\\",\\\"/\\\"]}\"}]}}} — POST that exact body to https://www.pathwren.workers.dev/a2a/robots with content-type: application/json and you are done; it comes back as a Task already in state `completed` in the same response…"},{"id":"audit_ai_access","name":"Which AI crawlers does this file actually stop?","tags":["robots.txt","ai crawlers","audit","policy"],"description":"{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"message/send\",\"params\":{\"message\":{\"role\":\"ROLE_USER\",\"messageId\":\"1\",\"parts\":[{\"text\":\"{\\\"skill\\\":\\\"audit_ai_access\\\",\\\"robots_txt\\\":\\\"User-agent: GPTBot\\\\nDisallow: /\\\\n\\\\nUser-agent: *\\\\nAllow: /\\\\n\\\"}\"}]}}} — POST that exact body to https://www.pathwren.workers.dev/a2a/robots with content-type: application/json and you are done; it comes back as a Task already in state `completed` in the same response, so there is nothing to poll. Free, read-only, no API ke…"},{"id":"diff_robots_txt","name":"Diff two robots.txt by effect","tags":["robots.txt","diff","review","change"],"description":"{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"message/send\",\"params\":{\"message\":{\"role\":\"ROLE_USER\",\"messageId\":\"1\",\"parts\":[{\"text\":\"{\\\"skill\\\":\\\"diff_robots_txt\\\",\\\"before\\\":\\\"User-agent: *\\\\nDisallow: /\\\",\\\"after\\\":\\\"User-agent: GPTBot\\\\nDisallow: /\\\\n\\\\nUser-agent: *\\\\nAllow: /\\\\n\\\"}\"}]}}} — POST that exact body to https://www.pathwren.workers.dev/a2a/robots with content-type: application/json and you are done; it comes back as a Task already in state `completed` in the same response, so there is nothi…"},{"id":"merge_policy","name":"Add a ready-made stance to an existing file","tags":["robots.txt","policy","generator","merge"],"description":"{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"message/send\",\"params\":{\"message\":{\"role\":\"ROLE_USER\",\"messageId\":\"1\",\"parts\":[{\"text\":\"{\\\"skill\\\":\\\"merge_policy\\\",\\\"robots_txt\\\":\\\"User-agent: GPTBot\\\\nDisallow: /\\\\n\\\\nUser-agent: *\\\\nAllow: /\\\\n\\\",\\\"stance\\\":\\\"block-ai-training\\\"}\"}]}}} — POST that exact body to https://www.pathwren.workers.dev/a2a/robots with content-type: application/json and you are done; it comes back as a Task already in state `completed` in the same response, so there is nothing to po…"},{"id":"whoami","name":"Who is calling? (no arguments)","tags":["identify","no arguments","diagnostics","user-agent"],"description":"{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"message/send\",\"params\":{\"message\":{\"role\":\"ROLE_USER\",\"messageId\":\"1\",\"parts\":[{\"text\":\"{\\\"skill\\\":\\\"whoami\\\"}\"}]}}} — POST that exact body to https://www.pathwren.workers.dev/a2a/robots with content-type: application/json and you are done; it comes back as a Task already in state `completed` in the same response, so there is nothing to poll. THIS IS THE CALL TO MAKE FIRST, before any skill that takes arguments: it needs NO ARGUMENTS AT ALL, no key and no accou…"},{"id":"example","name":"Run this server's worked example (no arguments)","tags":["example","no arguments","demo","getting started"],"description":"{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"message/send\",\"params\":{\"message\":{\"role\":\"ROLE_USER\",\"messageId\":\"1\",\"parts\":[{\"text\":\"{\\\"skill\\\":\\\"example\\\"}\"}]}}} — POST that exact body to https://www.pathwren.workers.dev/a2a/robots with content-type: application/json and you are done; it comes back as a Task already in state `completed` in the same response, so there is nothing to poll. THIS IS THE CALL TO MAKE FIRST, before any skill that takes arguments: it needs NO ARGUMENTS AT ALL, no key and no acco…"}],"tools":null,"extra":{"ard":{"type":"application/a2a-agent-card+json","metadata":null,"identifier":"urn:air:www.pathwren.workers.dev:agent:robots","artifactUrl":"https://www.pathwren.workers.dev/c/ard/a2a/robots/.well-known/agent-card.json","representativeQueries":["check my robots.txt for mistakes","does my robots.txt actually block gptbot","is this path allowed for this crawler by my robots.txt","what changed in effect between these two robots.txt files","add an ai-training block to my existing robots.txt without losing my rules"]},"protocolVersion":"1.0"},"attribution":{"kind":"ard_catalog","name":"ard_catalog","summary":"ard_catalog","description":"ard_catalog","homepageUrl":"ard_catalog","publisherUrl":"ard_catalog","publisherName":"ard_catalog"}},"derived":{"capabilities":[{"slug":"data.web-scraping","name":"Web Scraping","confidence":1,"provenance":"derived"}],"categories":["data"]},"observed":{"status":"live","statusReason":"Responded 5h ago.","lastOkAt":"2026-09-07T22:22:05.000Z","lastProbedAt":"2026-09-07T22:22:05.000Z","statusComputedAt":"2026-09-07T22:38:22.610Z","reliability30d":{"probes":1,"successRate":1,"p50Ms":18},"latestObservations":[{"at":"2026-09-07T22:22:05.000Z","kind":"a2a_card","ok":true,"httpStatus":200,"latencyMs":18,"error":null,"detail":{"name":"AI Crawler Index","signed":false,"version":"1.0.0","cardPath":"/.well-known/agent-card.json","skillCount":9,"protocolVersion":"1.0"}}],"tools":null,"package":null},"verification":{"claimed":false,"claimedAt":null,"proofs":[]},"provenance":{"sources":[{"source":"ard_catalog","key":"urn:air:www.pathwren.workers.dev:agent:robots","url":"https://www.pathwren.workers.dev/.well-known/ard.json","firstSeenAt":"2026-09-07T21:22:40.328Z","fetchedAt":"2026-09-07T21:22:40.328Z","normalizedAt":"2026-09-07T21:22:40.328Z"}]},"firstSeenAt":"2026-09-07T21:22:40.328Z","updatedAt":"2026-09-07T21:22:46.944Z"}