An SEO Audit Agent That Looks at the Whole SERP

Rank tracking tells you position 4. It doesn't tell you that positions 1 through 3 are below an answer box, a video carousel, and four ads.

Profile picture of Serply
Serply
A search results page annotated with its feature blocks

Position 4 on a clean page and position 4 below an answer box, a video carousel, and a block of shopping ads are completely different outcomes. Rank trackers report the same number for both.

The search response carries the feature blocks alongside the organic results, which makes a fuller audit a parsing problem rather than a scraping one.

What comes back

import os
import requests
from urllib.parse import quote_plus

API_KEY = os.environ["SERPLY_API_KEY"]


def serp(query: str, num: int = 20, location: str = "US",
         device: str = "desktop") -> dict:
    resp = requests.get(
        f"https://api.serply.io/v1/search/q={quote_plus(query)}&num={num}",
        headers={
            "X-Api-Key": API_KEY,
            "X-Proxy-Location": location,
            "X-User-Agent": device,
        },
        timeout=30,
    )
    resp.raise_for_status()
    return resp.json()

Beyond results, the response includes ads and ads_count, answers, knowledge_graph, related_questions, related_searches, image_results, shopping_ads, carousel and carousel_count, places, local_businesses, and company. Each is populated only when the page actually had that block, so their presence is itself the signal.

Profiling the page

def profile(data: dict) -> dict:
    def size(key):
        v = data.get(key)
        if isinstance(v, list):
            return len(v)
        return 1 if v else 0

    return {
        "query": data.get("query"),
        "device": data.get("device_type"),
        "region": data.get("device_region"),
        "organic": len(data.get("results") or []),
        "ads": data.get("ads_count") or size("ads"),
        "shopping_ads": size("shopping_ads"),
        "answer_box": size("answers") > 0,
        "knowledge_graph": bool(data.get("knowledge_graph")),
        "paa": size("related_questions"),
        "videos": size("carousel") or (data.get("carousel_count") or 0),
        "images": size("image_results"),
        "local_pack": size("places") or size("local_businesses"),
    }

Read that profile as intent. A knowledge graph plus an answer box means Google thinks the query has one factual answer — a blog post competing there will get impressions and very few clicks. A local pack means the query is treated as geographic whether or not you wrote it that way. Heavy shopping ads mean commercial intent, and an informational article is the wrong asset.

Position versus real position

Each organic result carries both position (rank among organic results) and realPosition (where it actually sits on the page, counting features above it). The gap is the number that matters:

def displacement(data: dict) -> list[dict]:
    out = []
    for r in data.get("results", []):
        pos, real = r.get("position"), r.get("realPosition")
        out.append({
            "title": r.get("title"),
            "url": r.get("link"),
            "organic_rank": pos,
            "page_rank": real,
            "pushed_down_by": (real - pos) if (pos and real) else None,
        })
    return out

A pushed_down_by of 5 at organic position 2 means you’re nominally near the top and functionally below the fold. That’s the finding rank tracking can’t produce, and it’s usually the explanation for a “we rank well but get no traffic” complaint.

Desktop and mobile are different pages

from concurrent.futures import ThreadPoolExecutor


def both_devices(query: str, location: str = "US") -> dict:
    with ThreadPoolExecutor(max_workers=2) as pool:
        d = pool.submit(serp, query, 20, location, "desktop")
        m = pool.submit(serp, query, 20, location, "mobile")
        desktop, mobile = d.result(), m.result()

    return {
        "desktop": profile(desktop),
        "mobile": profile(mobile),
        "desktop_top": [r.get("link") for r in (desktop.get("results") or [])[:5]],
        "mobile_top": [r.get("link") for r in (mobile.get("results") or [])[:5]],
    }

The X-User-Agent header switches between them. Mobile SERPs typically carry more features above the first organic result, so a page that looks like position 3 on desktop can be effectively position 8 on mobile — and mobile is where most of the traffic is.

Finding the content gaps

related_questions is the most directly actionable field in the whole response. It’s Google telling you what else people ask about this topic:

def question_gap(query: str, your_domain: str) -> dict:
    data = serp(query, num=20)
    questions = [
        q.get("question") if isinstance(q, dict) else q
        for q in (data.get("related_questions") or [])
    ]
    ranked = [
        r for r in (data.get("results") or [])
        if your_domain in (r.get("link") or "")
    ]
    return {
        "questions": [q for q in questions if q],
        "you_rank": bool(ranked),
        "your_position": ranked[0].get("realPosition") if ranked else None,
        "related_searches": data.get("related_searches") or [],
    }

Run this across your target keyword set and the questions that appear repeatedly, with no page of yours ranking, are your content backlog — sorted by demand, without a keyword tool.

Letting a model read the profile

The numbers are mechanical; the recommendation isn’t:

import json
from anthropic import Anthropic

client = Anthropic()

SYSTEM = """You audit search results pages for SEO strategy.

Given a SERP feature profile, assess:
1. Search intent — informational, commercial, navigational, or local — and what
   the feature mix implies about it.
2. Realistic organic opportunity. Say plainly when a page is so feature-heavy
   that organic clicks will be low regardless of rank.
3. Which feature is worth targeting (answer box, People Also Ask, video) and
   what content format that requires.

Be concrete and be willing to say a keyword is not worth pursuing. Do not
invent search volume, difficulty scores, or traffic estimates — you do not
have that data."""


def audit(query: str, domain: str) -> str:
    data = serp(query, num=20)
    payload = {
        "profile": profile(data),
        "displacement": displacement(data)[:10],
        "questions": [
            q.get("question") if isinstance(q, dict) else q
            for q in (data.get("related_questions") or [])
        ],
        "your_result": next(
            (r for r in data.get("results", []) if domain in (r.get("link") or "")),
            None,
        ),
    }
    msg = client.messages.create(
        model="claude-sonnet-4-5", max_tokens=2000, system=SYSTEM,
        messages=[{"role": "user", "content": json.dumps(payload, indent=2)}],
    )
    return msg.content[0].text

The “do not invent volume or difficulty” line is load-bearing. Models will happily produce a keyword difficulty score of 47 out of nothing, and that number will end up in a client deck.