Blog
Learn how to scrape the web and create tools with public data.
Giving a Mastra Agent Live Web Search
Mastra's tool API is Zod-first and TypeScript-native. Here's a search tool that actually type-checks against a real SERP response.
Wiring Live Search into n8n Without Writing a Custom Node
n8n's HTTP Request node is enough to give any workflow real search results. The tricky parts are the path format and the two response shapes.
Adding a Freshness Layer to an Existing RAG Pipeline
Your vector store is right about your documents and wrong about the world. A routing layer fixes that without rebuilding anything.
A Fact-Checking Agent That Admits When It Can't Tell
Most fact-checking agents return true or false for everything. The useful ones have a third answer, and use it often.
A Video Research Agent That Doesn't Pretend to Watch
Video search results are titles and links, nothing more. Building an honest video agent means being clear about what you actually know.
A Product Research Agent That Reads Ratings Honestly
Product listings come with ratings, review counts, and badges. Most shopping agents misuse all three.
Cross-Checking Two Search Engines to Catch Agent Errors
When Google and Bing disagree about the top results for a factual query, that disagreement is signal your agent can use.
Fanning Out Search Calls Without Melting Your Quota
Sequential search in an agent loop is the main source of latency. Here's how to parallelize it with asyncio and still respect a rate limit.
Testing Agents That Search, Without Hitting the API
Recorded fixtures, deterministic replay, and the failure cases you should be simulating but probably aren't.
Why Your AI Agent Needs a Scraper, Not Just a Search API
Search results give an agent titles and snippets. Most tasks need the actual page. Here's why request-based scraping belongs next to search in your stack.