Fetch
Free foreveranthropic · Dev Tools
200k installs
The Fetch MCP server gives any AI agent a reliable, standards-compliant way to retrieve live web content and convert it into clean markdown for downstream reasoning. Instead of writing custom scraping logic or wiring together HTTP clients, HTML parsers, and robots.txt checkers for every project, teams can expose a single fetch tool that handles retrieval, respects site crawling rules, and returns content in a format large language models can actually use. This makes it a natural fit for research agents, competitive intelligence workflows, RAG pipelines, and any use case where an AI needs current information beyond its training data.
Because Fetch honors robots.txt compliance out of the box, it's a safer default for teams that need to pull public web content without risking policy violations or getting blocked. The markdown conversion step strips out navigation clutter, ads, and boilerplate HTML so agents spend their context window on substance rather than markup. That combination — polite crawling plus LLM-ready output — is why Fetch is frequently the first web-access tool teams add when building AI agents that need to browse, summarize, or fact-check against live pages.
On BusinessMCP, Fetch is deployed as part of your unified, hosted MCP server rather than a standalone integration you have to babysit. Connect it once through the dashboard and it becomes immediately available to Claude, GPT, Gemini, or any other model-agnostic agent calling your /api/mcp endpoint with a Bearer mcph_* key. There's no separate server to run, no robots.txt logic to maintain, and no per-model reconfiguration — the same fetch tool works identically regardless of which AI model your team or your customers are using that week.
Because BusinessMCP consolidates every tool, database, and ad platform behind one MCP endpoint, Fetch content retrieval sits alongside your other data sources in the same business-intelligence dashboard. That means a fetched web page can feed straight into a workflow that also queries your PostgreSQL database, checks GitHub issues, or pulls ad performance data — all orchestrated by a single agent through a single connection. For teams building internal research assistants, content monitoring bots, or lightweight competitive-analysis tools, this turns
Just say it in a thread
No configs, no docs. Once connected, these are the kinds of messages your agents act on.
"Retrieve the raw content of a given url, checking robots.txt before requesting — and give me the highlights."
"Fetch a web page and return its main content converted into clean markdown for me, then post a summary in the thread."
"Check whether a given url is allowed to be crawled according to the site's robots.txt and flag anything that needs my approval."
What teams use it for
- Feed an AI research assistant live web pages converted to clean markdown for summarization
- Monitor competitor or news pages and pass fetched content into a downstream reporting workflow
- Support a RAG pipeline that needs fresh, non-indexed web content alongside stored documents
- Fact-check or enrich chatbot responses with up-to-date information pulled directly from public URLs
- Build a content-aggregation agent that respects site crawling rules while collecting article text
Agent-callable tools
fetch_url
Retrieve the raw content of a given URL, checking robots.txt before requesting.
fetch_as_markdown
Fetch a web page and return its main content converted into clean markdown.
check_robots_permission
Check whether a given URL is allowed to be crawled according to the site's robots.txt.
fetch_batch_urls
Retrieve and convert multiple URLs in a single call for bulk content gathering.
extract_page_metadata
Pull title, description, and other metadata from a fetched web page.
fetch_with_headers
Fetch a URL while specifying custom request headers such as user agent or accept-language.
Your data stays yours
Credentials live in your vault. We route requests — we never store, log, or train on your data.
Works with every AI
Connect once — portable across Claude, GPT, Gemini, and every local agent you run.
Pairs well with
Best Dev Tools MCP servers
Puppeteer
anthropic
Browser automation and web scraping using Puppeteer. Navigate pages, take screenshots, and interact with web elements.
Firecrawl
firecrawl
Advanced web scraping that handles JavaScript-rendered pages, removes boilerplate, and outputs clean markdown.
Jina AI Reader
jina-ai
Convert any URL to LLM-friendly text. Extract clean content from web pages, PDFs, and documents.
Playwright MCP
microsoft
Browser automation with Playwright. Navigate pages, fill forms, click elements with cross-browser support.
Brave Search (Official)
anthropic
Official Anthropic Brave Search MCP. Web search with rich, structured results for AI consumption.
Frequently asked questions
Does Fetch respect a website's crawling rules?
Yes, Fetch checks robots.txt before retrieving a page so agents avoid pulling content from paths that site owners have disallowed.
What format does Fetch return web content in?
Fetch converts raw HTML into clean markdown, stripping navigation and boilerplate so the output is easier for AI agents to read and reason over.
How does Fetch work with other tools on BusinessMCP?
Since it runs inside your single hosted MCP server, any agent calling your /api/mcp endpoint can combine Fetch with your databases, GitHub, or other connected tools in the same workflow.
Keep exploring
Give your AI team the Fetch skill
Free forever plan, no credit card. Connected and working in under five minutes.
Connect Fetch free