Open-Source Web Search for AI Agents Now Costs $0
Open-source web search for AI agents has a $0 base layer now. Proof's sitting on GitHub's web-search topic page: a project advertising $0-per-query web access. Local-first search, fetch, crawl and research over MCP, no API keys, no cloud, public beta.
Tuesday night, three of my automations running behind me, all paying for lookups that didn't need a paid API.
Flat sentence, biggest deal in the piece: the entry price per query is zero.
And open-source MCP servers now hand AI agents direct web access through one unified search tool.
The wet MCP server shows the shape of the thing. One server, three jobs. Web search, content extraction, library documentation search.
Search runs on embedded SearXNG metasearch across Google, Bing, DuckDuckGo and Brave.
Paid backends. Tavily, Brave, Exa, Kagi, OpenRouter, Firecrawl. Sit behind it as an optional fallback chain, not a requirement. Been paying per query for generic lookups? That's the line item this replaces.
Open-Source Web Search for AI Agents: What GitHub's Topic Page Shows
Two listings on the web-search topic page matter more than they first appear.
One is that local-first project, built entirely on MCP, no API keys, no cloud services.
The other sells private web search through SearXNG and lists its supported clients as "Claude, Cursor, and any MCP client."
That second line is the actual news.
"Any MCP client" means the search tool gets written once and runs everywhere the protocol does. Your Claude setup and your Cursor setup share one web access layer instead of each carrying its own integration. This isn't a vendor bundling products for a price hike. It's a protocol making bundling cheap. That structural difference is why this wave won't collapse the way the API-per-service land grab did.
Six Search Surfaces, One MCP Server
wet's documentation lists six search capabilities for AI agents:
- web search - news - images - academic research - library documentation - similar-page discovery
Underneath sits SearXNG metasearch with query expansion and standardized citations.
A TTL cache holds 1 hour for general queries, 5 minutes for time-sensitive ones. Snippets cap at 200 tokens. Retrieval runs on a built-in local reference embedding plus reranking through fastretrieval, which is why the whole thing operates with no API keys at all.
Read those numbers as agent-first design. Human users never complained about long snippets; AI agents choke on context bloat. So the 200-token cap exists for them. An answer you can't verify is worthless in production, so citations are standardized. The split cache exists because a 1-hour cache poisons anything time-sensitive. Those queries expire on the 5-minute clock instead.
Free Web Search Hits the Extraction Arms Race
Content extraction is where free gets honest. wet runs a 5-strategy escalation chain:
1. basic_http first 2. tls_spoof next 3. render backends. Native, browserless, or cf-browser-rendering 4. an optional key-gated captcha layer at the top
A markitdown bridge handles low-tier HTML and markdown fallback. Output arrives as structured chunks carrying clean text, markdown, JSON-LD, code blocks and metadata. Batch processing goes up to 50 URLs, plus deep crawling and site mapping.
Read that chain as a confession. Era of just fetching the URL is long over. Extraction is an arms race where every step costs more than the last, ordering is free-first. And the top layer is marked key-gated. Which tells you exactly where money reappears: the hard cases. Disclosed in the architecture diagram beats discovered at 2 a.m. during a production incident.
I'd take that trade every time.
Is Free MCP Web Search Enough? Limits, Caches and Paid Fallbacks
Economics change first. When the search layer costs $0 per query, an AI agent pulling extra sources before answering stops being a cost decision and becomes a quality decision. You stop rationing verification. That matters. Verification is one of the few behaviors that actually reduces confident nonsense coming out of your automations.
The fallback chain is the right architecture for a small budget:
- Start on the free SearXNG layer. - If a specific workload consistently fails there, switch on one paid backend for that workload only. Tavily, Brave, Exa, Kagi, OpenRouter or Firecrawl. - Leave everything else on $0.
Paid search becomes a scalpel instead of a subscription.
Two cautions before you rebuild anything. SearXNG aggregates Google, Bing, DuckDuckGo and Brave, so your free layer depends on upstream engines tolerating metasearch traffic. I wouldn't put a revenue-critical pipeline on that dependency without watching it fail at least once. Also note $0 per query covers the search layer, not the key-gated extraction cases. Budget for the hard scrapes before promising anyone a fully free pipeline.
This week, keep the evaluation small.
Pick the MCP client you already live in.
Claude or Cursor.
Wire up one of these servers, and test three behaviors:
- Can the agent find the right page? - Can it extract the content? - Does the citation point where it claims?
A handful of real queries will tell you more than any feature list will.
FAQ: Free Web Search for AI Agents
Does free MCP web search need API keys?
No. wet runs on embedded SearXNG plus a built-in local reference embedding with fastretrieval reranking.
No keys, no cloud.
Keys only appear at the optional key-gated captcha layer, for the hardest extraction cases.
Is SearXNG-free search reliable at scale?
Depends on upstream engines.
SearXNG aggregates Google, Bing, DuckDuckGo and Brave, so reliability rides on those engines tolerating metasearch traffic. Watch it fail once before you trust a pipeline to it.
When should you add a paid backend?
When a specific workload consistently fails on the free SearXNG layer.
One backend, that workload only — Tavily, Brave, Exa, Kagi, OpenRouter and Firecrawl all sit in the fallback chain for exactly this.
What does $0 per query not cover?
The key-gated extraction cases at the top of the 5-strategy chain. Hard scrapes cost money no matter who's serving them.
Comments ()