Web Scraping

Verification layer for the agentic web: a signed answer about any site before an agent acts.

Pay-per-call x402 SEC EDGAR data: company search, profiles, XBRL financials, filings.

What is being built for AI agents and what is moving — measured daily.

FBI Wanted MCP — FBI Wanted public API (free, no auth)

peru-sunat-ruc -- Peru's SUNAT taxpayer register (Padron reducido del RUC). Fleet #2929.

Audit and set up Meta Pixel and the Conversions API from your AI agent.

Pay-per-call DNS, WHOIS, domain age, TLS, email auth, blocklists, IP intel. Free samples.

Free web search, 130M+ business records, local search, weather and travel data for AI agents.

Web search and clean-text page fetch for AI agents, with SSRF protection.

Audits AI-crawler access, structured data and llms.txt on a site, and drafts the fixes.

x402 endpoint and MCP server index: find paid APIs for agents, x402 checks and price benchmarks.

Read any public page as markdown, link previews, URL safety, page audits, domain availability and…

Ranks the best AI tool or API per task: transcription, TTS, web search, scraping and OCR.

Per-call data for agents: graded behavioural science, dark-pattern detection, web fetch, geocode.

Live census of AI agents: prove you can reason (reverse CAPTCHA), check in, talk to other agents.

129 pay-per-call agent tools: security, threat intelligence, web search and SEO data tools. x402

Current pricing, cost estimates and recommendations for web scraping APIs, with official sources.

Doorman reads any website's robots.txt and shows which AI bots it lets in: who can train on it, which AI search engines can find and cite it, and which assistants can read it for people. Pick what you want and copy a fixed robots.txt, including the new shared AI-training opt-out line. No account.

Explore Weav customer service pricing, features, comparisons, signup, and demos. No auth required.

搜尋引擎與 AI 爬蟲工具箱：搜尋指令、爬蟲驗證、收錄與 AEO 健檢。台灣繁體中文 MCP 工具。