Remote MCP servers

Find a server for your workflow, with connection evidence you can inspect.

Connect your agent

function: Browser & web scraping

441 servers

Ordered by checks passedClear filters

Outdated check13 / 22 checks passed
Robots.txt & AI Crawler Checker
robots-check.quietworklabs.workers.devVerified
robots.txt tester (RFC 9309), AI crawler access (GPTBot, ClaudeBot...) and sitemap check.
Other Browser & web scraping
- Search & web data
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 81 ms · Last checked 1 day ago
Outdated check13 / 22 checks passed
Botcard
whosthatbot.comVerified
Cited AI crawler registry (training, search or user fetch) plus robots.txt checks. No fetching.
Other Browser & web scraping
- Search & web data
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 82 ms · Last checked 1 day ago
Outdated check13 / 22 checks passed
headlesson
headlesson.comVerified
Find and compare headless apps (CLI, MCP, API) for agents, with sources, lessons and changes.
Education Browser & web scraping
- AI models & agents
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 90 ms · Last checked 1 day ago
Responded13 / 22 checks passed
AgentScrape
hshintelligenceVerified
Pay-per-call web scraping for AI agents via x402 on Base USDC. Six tools, no signup.
Finance & Crypto Browser & web scraping
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 107 ms · Last checked 21 hr ago
Outdated check13 / 22 checks passed
Fourward AI visibility tools
fourwardhq.comVerified
Score any website's AI search visibility out of 100 and see which of 20 AI crawlers can read it.
Other Browser & web scraping
- Search & web data
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 118 ms · Last checked 1 day ago
Outdated check13 / 22 checks passed
Atalaya AI crawler check
atalaya-ai.comVerified
Which AI crawlers (GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot) a website allows, and why.
Other Browser & web scraping
- Search & web data
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 122 ms · Last checked 1 day ago
Outdated check13 / 22 checks passed
com.rattlesnakesbymail/rattlesnakesbymail
rattlesnakesbymail.comVerified
Dated, sourced claims about AI crawlers and agents, offered as MCP tools.
Developer Tools Browser & web scraping
- AI models & agents
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 129 ms · Last checked 1 day ago
Outdated check13 / 22 checks passed
Token Tender
tokentender.aiVerified
A bar for AI agents. Keep a tab, check if Claude or OpenAI is down, see which crawlers a site bans.
Developer Tools Browser & web scraping
- AI models & agents
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 241 ms · Last checked 1 day ago
Outdated check13 / 22 checks passed
GEO-bot
feelswebman.comVerified
Free check whether a website is technically open to AI assistants (AI crawlers, llms.txt, JSON-LD).
Other Browser & web scraping
- AI models & agents
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 255 ms · Last checked 1 day ago
Outdated check13 / 22 checks passed
Hydrafetch
hydrafetch.comVerified
Clean web data for agents: any URL to Markdown or JSON, plus search, crawls and company brand data.
Developer Tools Browser & web scraping
- Search & web data
- AI models & agents
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 297 ms · Last checked 1 day ago
Outdated check13 / 22 checks passed
Citevana AI visibility tools
citevana.comVerified
Free MCP tools for AI-search visibility: crawler checks, page audits, and llms.txt generation.
Developer Tools Browser & web scraping
- Search & web data
- AI models & agents
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 319 ms · Last checked 1 day ago
Responded13 / 22 checks passed
x402 Scraper API
LKennedyShareVerified
Paid web extraction: markdown, structured JSON, market-rules and page-change checks via x402.
Developer Tools Browser & web scraping
- Finance & payments
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 355 ms · Last checked 21 hr ago
Outdated check13 / 22 checks passed
Siteiz
siteiz.comVerified
Check which AI crawlers a site allows and see pages the way AI crawlers do. Free, read-only.
Other Browser & web scraping
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 458 ms · Last checked 1 day ago
Outdated check13 / 22 checks passed
pigfox.ai AI search readiness
pigfox.aiVerified
Check if a site is ready for AI search: AI crawler access, llms.txt, JSON-LD, scored out of 100.
Other Browser & web scraping
- Search & web data
- AI models & agents
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 505 ms · Last checked 1 day ago
Responded13 / 22 checks passed
io.tranx/alice
tranx.ioVerified
Audit any site's AI visibility from your assistant: crawler access, rendering, and schema.
Other Browser & web scraping
- Search & web data
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 533 ms · Last checked 23 hr ago
Outdated check13 / 22 checks passed
The Crawler Zoo Arena
crawlerzoo.comVerified
Games for AI agents: negotiation, social deduction, racing and puzzles, vs bots and agents.
Other Browser & web scraping
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 541 ms · Last checked 1 day ago
Outdated check13 / 22 checks passed
AEO Lookup
aeolookup.comVerified
Free AI crawler, schema, llms.txt and answer-readiness checks for public web pages. No API key.
Other Browser & web scraping
- AI models & agents
- Search & web data
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 660 ms · Last checked 1 day ago
Outdated check13 / 22 checks passed
PageCritic
pagecritic.appVerified
Audit a landing page's conversion (score, top issues, fixes) and check if AI crawlers can read it.
Other Browser & web scraping
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 694 ms · Last checked 1 day ago
Outdated check13 / 22 checks passed
Domduck
domduck.comVerified
Domain reputation over time: Domain Rating, Tranco rank, TLS, and AI crawler policy.
Other Browser & web scraping
- Search & web data
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 836 ms · Last checked 1 day ago
Outdated check13 / 22 checks passed
Agent Ready Website Audit
workpods.aiVerified
SEO and AI search audit for any website: technical SEO, schema, AI crawler access and llms.txt.
Marketing & Sales Browser & web scraping
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 1199 ms · Last checked 1 day ago
Outdated check13 / 22 checks passed
Oxylabs
pipeworx-ioVerified
Oxylabs MCP — Oxylabs Web Scraper API (oxylabs.io)
Developer Tools Browser & web scraping
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 126 ms · Last checked 1 day ago
Outdated check13 / 22 checks passed
Scraperapi
pipeworx-ioVerified
ScraperAPI MCP — wraps ScraperAPI (scraperapi.com), a proxy-based web scraper with structured-data…
Developer Tools Browser & web scraping
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 130 ms · Last checked 1 day ago
Responded13 / 22 checks passed
Agent Crawl Readiness
sadri-dridiVerified
Bounded crawl-readiness check for agent access: robots.txt, sitemap.xml, llms.txt, homepage.
Developer Tools Browser & web scraping
- AI models & agents
- Search & web data
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 139 ms · Last checked 14 hr ago
Outdated check13 / 22 checks passed
Scrapingbee
pipeworx-ioVerified
ScrapingBee MCP — wraps the ScrapingBee headless-scraping API (scrapingbee.com)
Developer Tools Browser & web scraping
- Search & web data
- Passed: Handshake
- Passed: Tools listed
- Passed: Updated recently
Median latency 172 ms · Last checked 1 day ago

Automated measurements. Not a rating. Unmeasured checks are not failures. How to read the evidence