Streaming Web Search in the Vercel AI SDK

• The tool definition
• A reader tool
• The route handler
• Rendering sources as they stream
• Caching

The AI SDK’s streamText handles the hard parts of tool-calling in a web app: it runs the tool, feeds the result back, and keeps streaming without you managing the loop. What it doesn’t do is give the model anything to look up. That part is yours.

The tool definition

import { tool } from 'ai'
import { z } from 'zod'

const API_KEY = process.env.SERPLY_API_KEY!

export const webSearch = tool({
  description:
    'Search the live web. Use this for anything current, factual, or ' +
    'that you are not certain about from memory.',
  parameters: z.object({
    query: z.string().describe('The search query, phrased as a Google search'),
    num: z.number().min(1).max(50).default(10).describe('How many results'),
  }),
  execute: async ({ query, num }) => {
    const q = encodeURIComponent(query)
    const res = await fetch(
      `https://api.serply.io/v1/search/q=${q}&num=${num}`,
      { headers: { 'X-Api-Key': API_KEY, 'X-Proxy-Location': 'US' } },
    )

    if (!res.ok) {
      if (res.status === 429) {
        return { error: 'Rate limited. Answer with what you already have.' }
      }
      return { error: `Search failed with status ${res.status}` }
    }

    const data = await res.json()
    return {
      results: (data.results ?? []).map((r: any) => ({
        title: r.title,
        url: r.link,
        snippet: r.description ?? '',
        position: r.position,
      })),
    }
  },
})

Serply puts the Google-style query string in the path rather than as query params, so the URL is /v1/search/q=.... The organic results land in results, each with title, link, description, and position.

Returning an error object instead of throwing is deliberate. A thrown error aborts the stream; a returned error becomes a tool result the model can read and route around. For a chat UI, degrading to “I couldn’t search, here’s what I know” beats a dead connection.

A reader tool

Snippets are often too thin to answer with. Add a second tool that pulls the actual page:

export const readPage = tool({
  description: 'Read the full text of a web page. Use after webSearch when a snippet is not enough.',
  parameters: z.object({
    url: z.string().url().describe('The page URL to read'),
  }),
  execute: async ({ url }) => {
    const res = await fetch('https://api.serply.io/v1/request', {
      method: 'POST',
      headers: { 'X-Api-Key': API_KEY, 'Content-Type': 'application/json' },
      body: JSON.stringify({ url, response_type: 'markdown' }),
    })
    if (!res.ok) return { error: `Could not read page (${res.status})` }
    const text = await res.text()
    return { url, content: text.slice(0, 15000) }
  },
})

The markdown mode returns the converted text as the response body, so res.text() is correct here — calling res.json() will throw. If you want raw HTML instead, send response_type: 'full' and read data off the JSON object that comes back.

The route handler

import { streamText, convertToModelMessages } from 'ai'
import { openai } from '@ai-sdk/openai'

export const maxDuration = 60

export async function POST(req: Request) {
  const { messages } = await req.json()

  const result = streamText({
    model: openai('gpt-4o'),
    system:
      'You answer with live information. Search before making factual claims ' +
      'about current events, prices, or anything after your training cutoff. ' +
      'Cite sources as markdown links.',
    messages: convertToModelMessages(messages),
    tools: { webSearch, readPage },
    stopWhen: ({ steps }) => steps.length >= 6,
  })

  return result.toUIMessageStreamResponse()
}

The stopWhen cap is not optional in production. Without it a model that keeps deciding it needs one more search will keep getting one, and a single chat turn can quietly issue a few dozen API calls.

Rendering sources as they stream

The client gets tool calls and results in the message parts, so you can show sources the moment the search returns — before the model has written a word about them:

'use client'
import { useChat } from '@ai-sdk/react'

export default function Chat() {
  const { messages, sendMessage } = useChat()

  return (
    <div>
      {messages.map((m) => (
        <div key={m.id}>
          {m.parts.map((part, i) => {
            if (part.type === 'text') return <p key={i}>{part.text}</p>

            if (part.type === 'tool-webSearch' && part.state === 'output-available') {
              const out = part.output as { results?: { title: string; url: string }[] }
              return (
                <ul key={i} className="sources">
                  {out.results?.slice(0, 5).map((r) => (
                    <li key={r.url}>
                      <a href={r.url} target="_blank" rel="noreferrer">{r.title}</a>
                    </li>
                  ))}
                </ul>
              )
            }

            if (part.type === 'tool-webSearch' && part.state === 'input-available') {
              return <p key={i} className="status">Searching…</p>
            }

            return null
          })}
        </div>
      ))}
    </div>
  )
}

Showing the query while it’s in flight and the links the moment they land does more for perceived quality than any amount of prompt tuning. Users forgive a slow answer when they can see it working.

Caching

Chat apps re-ask the same things constantly. A short-lived cache in front of the search call pays for itself immediately:

const cache = new Map<string, { at: number; value: unknown }>()
const TTL = 5 * 60 * 1000

function cached<T>(key: string, fn: () => Promise<T>): Promise<T> {
  const hit = cache.get(key)
  if (hit && Date.now() - hit.at < TTL) return Promise.resolve(hit.value as T)
  return fn().then((value) => {
    cache.set(key, { at: Date.now(), value })
    return value
  })
}

Five minutes is a reasonable default for general queries. Drop it to under a minute for anything price- or news-related, where staleness is the entire problem you were solving.