SEO and AI search
What actually works in search, and what gets sold anyway.
Most of what is sold as SEO has no evidence behind it, and some of it breaks Google's published policies. This page sorts what Google and the AI companies confirm in their own documentation from what is unproven, and from what gets sold anyway. Every claim is graded, and linked to its source.
Start here
Why you cannot tell good SEO from bad.
Because almost everything written about it is written by someone selling it. Search for research on AI citations and you land in a ring of sites citing each other, re-citing the same few studies with the numbers changed, and no method you can check.
Google's own warning signs, quoted
- "No one can guarantee a #1 ranking on Google." And: "If they guarantee you that their changes will give you first place in search results, find someone else."
- "Be wary of SEO firms and web consultants or agencies that email you out of the blue."
- "Be careful if a company is secretive or won't clearly explain what they intend to do."
- "You should never have to link to an SEO."
- "Be wary of tools that claim to be 'acceptable' or 'approved' by Google Search."
- Give an auditor read access to Search Console, never write access.
The same guide gives you two questions to ask whoever is selling to you. "Do they cite official Google documentation as supporting evidence for their recommendations?" And, about AEO and GEO sellers specifically, "is their advice aligned with Google Search's official guidance on optimizing for generative AI features?" Everything below is graded and linked so you can answer both about me. Google, Do you need an SEO?
The evidence
What Google actually confirms.
Three requirements make a page eligible at all: Googlebot is not blocked, the page returns a 200, and it has indexable content. In Google's words, "there are actually very few technical things you need to do to a web page." Everything else is weaker than it is sold as.
Crawlable, reachable, made of text
Those three are the whole list. One trap: robots.txt stops crawling, but noindex only works if the page stays crawlable.
Confirmed Google, Technical requirements
Real links, with a real href
Google "can't reliably extract URLs from <a> elements that don't have an href attribute." Every page you care about needs a link from at least one other.
Confirmed Google, Link best practices
The mobile page is the page
"Only the content shown on the mobile site is used for indexing." Anything hidden on small screens is not demoted. It is not indexed.
Confirmed Google, Mobile-first indexing
A title, and a heading that agrees with it
Google builds the blue link from your title, your main heading and prominent text, and rewrites yours when it does not fit. You cannot request a correction.
Confirmed Google, Title links
Canonicals and redirects, not panic
The two signals Google calls strong. And it gets sold as an emergency: duplicate content is not a penalty. Google says not to fret about it.
Confirmed Google, Consolidate duplicate URLs
Speed counts, and relevance still wins
"Core Web Vitals are used by our ranking systems," but also "there is no single signal," and Search shows the most relevant content "even if the page experience is sub-par." Fast is a floor, not a lever.
Confirmed, subordinate Google, Page experience
HTTPS, and how small it really is
Confirmed, but Google called it "a very lightweight signal" affecting "fewer than 1% of global queries." That was 2014, never updated since. Have it, and stop paying for it.
Confirmed, minor Google, HTTPS as a ranking signal
Structured data buys rich results, not citations
It makes a page eligible for the extra detail in a Google result. But Google is explicit: "structured data isn't required for generative AI search." I write it everywhere, and never sell it as why an assistant quotes you.
Confirmed, not for AI Google, AI features and your site
Quotable writing is the one tested tactic
A peer-reviewed 2024 KDD experiment tested content changes against generative engines. Direct quotations raised visibility most, statistics and cited sources close behind, and keyword stuffing scored below making no change at all. It ran on 2023-era models.
Peer reviewed Aggarwal and others, GEO, KDD 2024
The machines
How AI assistants actually find you.
For Google's own answers there is no second job. To appear in AI Overviews a page "must be indexed and eligible to be shown in Google Search with a snippet," and "there are no additional technical requirements." The search work is the AI work.
Two crawlers each, and people block the wrong one
OpenAI
GPTBot trains. OAI-SearchBot decides whether you appear in ChatGPT's search answers. Each setting is "independent of the others."
Anthropic
ClaudeBot collects for models. Claude-SearchBot works on search quality. A robots rule matching every agent containing "Claude" removes you from the answers too.
Perplexity
PerplexityBot surfaces and links sites in results. Perplexity states it is not used to crawl for foundation models, because it does not build them.
Google-Extended governs Gemini training and grounding. It "does not impact a site's inclusion in Google Search," so blocking it does not remove you from AI Overviews.
Confirmed Each company documents this itself: OpenAI, Anthropic, Perplexity, Google. Setting these four on purpose is an afternoon of work most sites have never done.
Most assistants do not run your JavaScript
Vercel and MERJ measured this in December 2024: GPTBot, ClaudeBot and PerplexityBot fetch JavaScript but do not execute it, while Gemini and Applebot render. Text that only exists after a script runs was, on that evidence, invisible to three of them.
One study, dated Vercel and MERJ, December 2024
A citation is presence, not traffic
Pew Research Center tracked real browsing: people clicked a result on about 8% of visits where an AI summary appeared, against about 15% where none did, and clicked a link inside the summary on roughly 1%. Being in the answer is worth having, but it is not a traffic channel, and anyone selling you AI traffic is selling what the best independent data does not support.
Independent Pew Research Center, July 2025
The honest part
What nobody knows yet, including me.
AI search has existed at scale for about two years. Anyone describing it as a settled discipline is describing something that does not exist. Here is what is genuinely unknown, from me rather than later.
The same question gives different answers
Published testing found models served over an API matched their own earlier output only about a fifth of the time, even at a fixed temperature and seed. So one check proves little, and a monthly score built from one run per question is measuring noise. Ask any vendor how many runs they average.
Unknown Reproducibility is still being characterised
One controlled experiment exists
The KDD study above is the only peer-reviewed controlled test of content tactics I could find, and it ran on 2023-era models. Everything else is correlation, mostly on data owned by whoever sells the tool that measured it.
Unknown No causal evidence on current models
Nobody outside these companies knows how retrieval works
ChatGPT's mix of its own index and third-party providers is unpublished. Anthropic does not name a search provider. Perplexity's third-party crawlers are under agreement. Advice built on a guessed architecture inherits the guess.
Unknown Architectures are undisclosed, and change without notice
Attribution is partly broken
Traffic from Claude's own app carries no referrer, so ordinary analytics cannot attribute it. Anyone promising clean reporting on AI-driven visits is promising what the plumbing does not support.
Unknown The gaps are structural, not a tooling problem
Cause and effect may run backwards
The strongest published correlate of being cited is how often a brand is mentioned elsewhere. But brands mentioned often were already well known. "Get more mentions" may describe success rather than cause it.
Unknown The direction of causation is unestablished
One disclosure, since this page asks you to check sources. I sell web work, so "hand-written HTML with the text in the source" is a conclusion that happens to favour what I build. The study is linked above, dated December 2024, never replicated at scale, and it does not apply to Google. Read it and decide for yourself.
Still being sold
What does not work, and gets sold anyway.
Each is named in Google's spam policies, contradicted by Google's own documentation, or measured and found not to work. If one is in a proposal you have been sent, ask the person to show you the policy page.
- Buying links, packages or placements, including to raise "Domain Authority". Google's link spam policy names "buying or selling links for ranking purposes." A disclosed sponsored link is fine; one that passes ranking credit is not. Since 2022 there is usually no penalty notice either: the links are quietly neutralised, so you paid for nothing. Domain Authority is a Moz metric anyway, not a number Google uses.
- Guaranteed rankings, or guaranteed AI mentions. Google: "No one can guarantee a #1 ranking on Google."
- Submission to hundreds of directories. "Low-quality directory or bookmark site links" is enumerated link spam, and Google says you "usually don't need to do anything except publish your site on the web."
- The meta keywords tag. Google: "not used by Google Search, and it has no effect on indexing and ranking at all." Dead since 2009.
- Keyword density targets. Google specifies no density, and in the one controlled generative-engine experiment stuffing scored below making no change at all. Debunked for ordinary search and AI search, by separate evidence.
- Monthly blog volume, or a word-count minimum. Google's scaled content abuse policy targets pages "generated for the primary purpose of manipulating search rankings," and says "the length of the content alone doesn't matter." Using AI is not the violation. Volume without value is.
- "Helpful content update recovery." Google folded that system into core ranking in March 2024 and says "there's no longer one signal or system used to do this." It is sold as recovery from a system that no longer exists.
- llms.txt as a visibility service. Google: "Google Search itself doesn't use them." This site publishes one because it costs nothing. It is not charged for, and nothing is claimed for it.
- An audit that is really a tool export. Google: "Be wary of tools that claim to be 'acceptable' or 'approved' by Google Search." A third-party score grades you against a vendor's heuristics, not Google's systems.
Sources, in roughly the order they get ignored: Google spam policies, Do you need an SEO?, SEO starter guide, Meta tags Google supports, March 2024 core update and spam policies.
The work
So this is what I actually do.
Only the things above that survived. Nothing from the unknown list is billed as settled, and nothing from the list that does not work appears at all.
What is included
Search and AI visibility
Built so the page can be read, ranked and quoted, with every decision traceable to something public.
- Hand-written HTML with the answer in the source, before any script runs
- Google's three technical requirements, checked rather than assumed
- Real links, real titles, canonical discipline, and a mobile page carrying everything
- Structured data for rich-result eligibility, stating facts that match the page
- Writing built to be quoted: specific, checkable statements instead of adjectives
- Crawler policy set on purpose, training and retrieval decided per company
- Tracking in your own accounts, so the numbers are yours and you can audit me
What you will not get: a ranking promise, a citation promise, a monthly score with no error bars, or an invoice for a file that Google says it ignores.
Questions
The things people ask about search.
What is answer engine optimization?
Answer engine optimization, or AEO, is the work of making a page quotable by AI assistants rather than only rankable by a search engine. In practice that means a short plain-text answer near the top of each page, real text in the HTML rather than text drawn by a script, and headings that match how people actually ask.
Do you guarantee I will rank, or that AI assistants will mention me?
No, and nobody honestly can. Google's own guidance on hiring an SEO says no one can guarantee a number one ranking, and that you should find someone else if they promise it. What I control is the work: a fast site, clean structure, real text in the HTML, and tracking in your own accounts so you can check the results yourself.
Should I block AI crawlers from my site?
That is two decisions, not one. Each company runs a separate crawler for training and for search. OpenAI documents GPTBot for training and OAI-SearchBot for its search results, and states that the settings are independent of each other. Blocking the search crawler removes you from the answers. Blocking the training crawler does not.
Does structured data get me quoted by AI assistants?
No. Google's own documentation says structured data is not required for generative AI search, and that there is no special markup to add for it. Schema earns its place by making a page eligible for rich results in ordinary search, and by stating facts that agree with the visible page. I add it for that.
Is llms.txt worth paying for?
No, and I do not charge for it. It is a community proposal rather than a standard, and Google states plainly that Google Search does not use these files. No major AI company has committed to reading one for citation. I publish one on this site because it costs nothing, and I claim nothing for it.
How do you measure AI visibility?
Carefully, and with the spread shown. Asking an assistant the same question twice can return different sources, so a single check proves very little. I run each question repeatedly and report the range rather than one number. A tool reporting a tidy monthly score from one run per question is reporting noise.
Start a project
Ask me to show you the source.
For anything on this page, or anything in a proposal someone else has sent you.
A written price within one business day, and a straight answer about what is known and what is not.