How to evaluate an AI search optimization agency

A good AI search optimization agency does solid SEO, measures AI visibility with methods you can check, and can explain every tactic in terms Google’s own documentation would recognize. A weak one sells a secret method, promises placements inside ChatGPT, or reports a score you can’t trace back to real answers. You can tell the two apart in one sales call if you ask the right questions, and most of those questions come straight from what Google has published.

I run one of these agencies, so read this with that in mind. I’ve written it so you can run us through the same test.

What AI search optimization should include

Google’s guide to optimizing for generative AI features says “optimizing for generative AI search is optimizing for the search experience, and thus still SEO.” So an honest scope looks a lot like good SEO, with AI-specific measurement and brand-fact work added on top.

For comparison, our own AI search optimization service lists four deliverables: an AI visibility audit, a technical SEO audit with a ranked fix list, a competitor gap analysis with a content calendar built from it, and schema and on-page recommendations for each priority page. Any agency you talk to should be able to name its deliverables that plainly.

Check the agency’s own visibility

An agency that sells AI visibility should have some. Before the first call, spend 20 minutes checking:

1. Ask ChatGPT, Google AI Mode, Gemini, Perplexity and Copilot the questions a buyer would ask about this kind of agency in your region or industry. Note whether the agency comes up and what’s said about it.
2. Ask each assistant directly about the agency. Check whether the facts are right: services, location, leadership.
3. Read three of the agency’s own articles. Look for original data, first-hand examples and named authors with real background. If their blog reads like it could be on any agency’s site, that’s what they’ll produce for you.
4. Look at how much they publish. A blog with hundreds of near-identical posts suggests a content machine, which is the opposite of what Google rewards.
5. Look for case studies with specifics: what was done, with which tools, by whom, and what the output was. Growth percentages with no method behind them tell you nothing.

Visibility for an agency’s own brand isn’t everything. A new agency may have good people and little visibility yet. But if they can’t explain why they don’t show up, that tells you something too.

Reporting to expect

The monthly report should show what’s happening in AI answers and whether it’s turning into business. At minimum:

• Share of answers and citation rate for a fixed prompt panel, with the full question list and raw answers available to you.
• Factual errors found in AI answers about your brand, and what was done about each.
• Google AI impressions from Search Console’s generative AI performance report, and citations from Bing Webmaster Tools’ AI Performance.
• Sessions, key events and revenue from the AI Assistant channel in GA4.
• Organic rankings, traffic and leads, because the SEO underneath still does most of the work.
• A written read on what changed, why they think it changed, and what they’ll do next.

One thing should never appear: a “rank” inside an AI answer. When SparkToro had volunteers run 12 brand-recommendation prompts a combined 2,961 times, it found less than a 1 in 100 chance that ChatGPT or Google’s AI would return the same list twice in 100 runs. A position number built on answers that change every time isn’t a measurement.

Questions that separate real work from rebranded SEO

Google itself says AI search optimization is still SEO, so an agency that calls its work SEO is being accurate. What you’re testing for is whether the SEO is any good, and whether the AI-specific claims hold up against what Google has published.

The “what does AI do well and where does it fail” question is one I’d ask any marketing hire. Most teams don’t have anybody who can answer it with confidence, and an agency that sells AI services should.

What strong process looks like

Ask for a walkthrough of a real project. On a 20,000-page SEO audit we ran, Semrush crawled every page and pulled technical, ranking and backlink data. Claude read the full export and grouped issues by pattern. Our SEO team reviewed the groupings, discarded the false alarms and ranked the fixes by impact and effort. That’s the shape to listen for: AI on the volume work, named people on the decisions. As I say in my talks, “Capability is now free. Judgement is the premium.” You’re paying an agency for the judgement.

Pricing models

Agencies price AI search work in a handful of ways. Each one rewards a different behavior, so it’s worth knowing which incentive you’re buying.

For our own work, the pricing page lists no fixed prices. It describes a readiness assessment (two to three weeks), a strategy and roadmap project (four to six weeks), a month-to-month ongoing partnership that includes AI search work, and training priced per session or per team. Every engagement gets a written proposal with a fixed scope and price, and no long contract is required. Price depends on team size, the number of tools and channels, how much of the work we do versus your team, and how fast you need it. Whatever agency you choose, those are fair questions to ask about their pricing too.

A scorecard

Score each agency 0 (no), 1 (partly) or 2 (yes) on each line, out of 24. A zero on guarantees, honest mentions or raw answers is reason enough to walk away, whatever the total.

Use the scorecard on us too. One more line from my talks: “AI doesn’t fix a bad strategy. It executes it faster.” That applies to agencies as much as to tools.

Next step

Bring the scorecard to your next agency call and ask the questions in the table. If you’d like to see how we’d answer them for your site, book a strategy call or start with an AI visibility audit through our AI search optimization service.