How to get cited by Claude

By Aram Alekian, Founder of Citematic. Tracks brand citations across ChatGPT, Claude, Perplexity and Google AI Overviews. Published August 22, 2026.

Claude reaches the web through three separate crawlers, and blocking the wrong one quietly removes you from its answers. Anthropic documents what each bot does and what happens when you disable it. Beyond that access layer, Claude behaves like a selective reader: low volume, long-form pages, specific URLs rather than domains.

Which crawlers does Claude use?

Three, each with its own robots.txt token: ClaudeBot for model training, Claude-User for user-initiated fetches, and Claude-SearchBot for indexing that feeds search answers.

The distinction matters more here than on most engines, because the three do genuinely different jobs and Anthropic ties a different consequence to each. A site that blocks the training bot and keeps the other two open has opted out of the training corpus while staying fully eligible for citation. A site that reaches for a blanket AI block loses the citations along with the training. Both are legitimate choices. What is not legitimate is making the second one by accident, which is what happens when a robots.txt lists one Claude token and assumes it covers the rest.

What happens if you block one of them?

Anthropic spells out each consequence separately. Blocking ClaudeBot only excludes future content from training. Blocking either of the other two costs you visibility.

Two operational details sit underneath that table and are easy to miss. First, Anthropic supports the non-standard Crawl-delay directive, so a site with crawl-rate concerns has an option short of blocking. Second, Anthropic states plainly that blocking by IP address may not work correctly or guarantee a persistent opt-out, because it also impedes the ability to read your robots.txt in the first place. The company publishes its crawler IP ranges as a JSON file for verification rather than for blocking; the version checked for this guide listed twenty-three IPv4 prefixes and carried a creation timestamp of August 13, 2026, which is a useful reminder that the list rotates.

The failure mode worth checking today is the one nobody configures deliberately. A WAF rule or bot-management preset can challenge or return 403 to these agents while robots.txt says Allow. Anthropic's bots respect anti-circumvention technologies and will not attempt to bypass CAPTCHAs, so an interstitial is a hard stop, not a speed bump. The same principle applies across engines, which is why crawler access sits at step one of the cross-engine checklist rather than being treated as a Claude-specific chore.

How does Claude decide which pages to cite?

Anthropic publishes no selection or ranking criteria. The documentation covers access only, so any specific claim about Claude's citation logic is inference, not fact.

This is an unsatisfying answer, and it is the correct one. Google publishes an optimization guide for its AI surfaces; OpenAI publishes a crawler reference; Anthropic publishes a crawler reference too, and stops there. There is no documented equivalent of "these are the signals that make a page citable in Claude." Guides that supply one are reverse-engineering from observed outputs, which is a reasonable thing to do and a bad thing to present as documentation.

What can be said honestly is that the observable behavior is consistent across independent datasets, and that behavior is what the rest of this guide optimizes for. Where a mechanism is genuinely documented, we cite it. Where only behavior is measured, we say so - the same distinction we draw in the Google AI Overviews guide, where the documentation is unusually explicit and the folklore is unusually loud.

How much traffic does Claude actually send?

It depends entirely on whose panel you read. Two credible 2026 studies put Claude's share of AI referrals an order of magnitude apart.

Both numbers are honestly reported and they are not really in conflict; they are answering different questions. Previsible measures a broad mix of consumer-facing and business-facing sites and finds Claude small but growing fast. Goodie measures business-facing brands and finds Claude second. Goodie flags the obvious caveat itself, noting that its panel may overindex on audiences that use Claude, and Previsible's methodology warns that absolute figures should not be compared across its own reports without accounting for sample composition. Two disciplined studies, two panels, one order of magnitude.

The practical reading: your Claude share is a function of who your audience is, not of a market-wide constant. If you sell to developers, technical buyers, or professional services, the Goodie picture is the one to plan against. If you publish for a general audience, the Previsible picture is closer. Anyone quoting a single Claude percentage at you without naming the panel is quoting a number that does not exist. The same warning applies to every engine, which is why we attribute sourced statistics rather than presenting them as constants.

One more figure belongs here for scale, with its age attached. In July 2025 Cloudflare put the crawl-to-visit ratio in stark terms, estimating that it had become roughly 30,000 times harder to get a visit from Anthropic than from the Google of a decade earlier, against 750 times for OpenAI. That was a snapshot taken before Claude's 2026 referral growth, and both 2026 datasets show the trend moving the other way. Read together, they describe a surface that reads a great deal and sends comparatively little, from a very low base that is now rising quickly.

What kind of content does Claude favor?

Long-form explanatory pages. Referral data describes Claude as a content-selection engine that picks specific URLs and sends readers to guides and research-oriented material.

Previsible groups the engines into two behavioral families, and the split is more actionable than any share number. Search-pattern models such as ChatGPT and Gemini trust a domain broadly but hesitate at the page level; ChatGPT sends 28.8% of its traffic to internal search pages, effectively naming the brand and letting the user find the page. Content-selection models such as Perplexity and Claude do the opposite: they pick the page. Claude skews furthest toward educational material - guides, courses, research-oriented pages - and its referrals engage with longer-form content at higher rates than any other platform's.

That has a direct consequence for how you write. On a search-pattern engine, a thin page on a trusted domain can still earn the session, because the engine is betting on the domain. On Claude, the page is the unit being chosen, so the page has to be complete on its own: the full explanation, the caveats, the sources, in one place. Splitting a subject across five shallow posts to chase five phrasings is counterproductive here, and on Google's surfaces the same tactic is explicitly named as spam. Depth on one URL is the strategy that survives on every engine at once.

The answer-capsule technique still applies, with a caveat about why. On ChatGPT and Perplexity, a short self-contained answer under a question heading is what gets lifted into the response. On Claude the evidence for a direct lift is weaker, but the answer-first pattern makes a long page navigable, which matters more when the engine is evaluating whether one page answers a whole question. Same technique, different justification.

How do you qualify for Claude citations?

Allow all three bots, serve server-rendered HTML, and give Claude what it prefers to send people to: a complete, sourced, long-form page.

1. Name all three tokens in robots.txt. ClaudeBot, Claude-User and Claude-SearchBot. Listing one and assuming it covers the others is the most common way sites lose Claude citations without noticing.
2. Decide the training question deliberately. Allowing ClaudeBot puts your content in the training corpus; disallowing it does not affect citation eligibility. Either is defensible, but make it a decision rather than a default.
3. Check the layer robots.txt does not control. CDN rules, WAF presets and bot-management defaults can return 403 or an interstitial regardless of what robots.txt permits. Test the actual response for each user agent.
4. Verify by user agent and IP, never block by IP. Anthropic publishes its crawler ranges for verification and warns that IP blocking is unreliable as an opt-out because it also blocks robots.txt retrieval.
5. Serve content in the initial HTML. Retrieval agents read what the server returns. A client-rendered single-page app is effectively empty to them, and static or server-side rendering is the safe baseline.
6. Write one complete page per subject. Claude selects pages, not domains. Consolidate rather than fragmenting, and make sure the page can stand alone without the rest of the cluster.
7. Attribute every number. Name the source, the date and the sample in the sentence. On a research-skewed surface this is the difference between a page that can be quoted safely and one that cannot.
8. Use Crawl-delay if crawl rate is the real concern. Anthropic supports it. Rate limiting is the proportionate response; blocking is not.

How do you measure Claude visibility?

Server logs and a fixed prompt set. Claude referrals are small enough that analytics alone will show zero long after citations have started.

Referrer data understates every AI engine and understates Claude worst, because the volumes are smallest and the loss mechanisms are the same everywhere: links copied out of a chat lose the referrer, native desktop and mobile apps strip it, and privacy-focused browsers strip it too. Goodie's report treats this as a structural undercount rather than noise. At Claude's volumes the practical consequence is stark - a site can be cited regularly and still show a flat zero in a referral report for months.

Two measurements do work. Server logs show whether Claude-User and Claude-SearchBot are arriving at all and which URLs they take, which is the earliest available signal and the only one that is not sampled. And a fixed prompt set, run on a schedule and recorded, shows whether you are named and cited in the answers your buyers would actually ask for. Neither is a vanity metric and neither requires a vendor's internal access, which no third party has. How the two fit together across engines is covered in getting cited by ChatGPT and in how Perplexity selects sources, where the same method runs against surfaces that report differently.

Automating Claude visibility

Claude rewards the least glamorous version of this work: one complete page per subject, every claim attributed, access verified at the edge as well as in robots.txt, and a prompt set re-run often enough to notice a change. None of that is difficult and all of it decays quietly. Citematic can automate sourced-claim briefs, keep the access layer verified across every engine's user agents, and report Share of Model from a consistent prompt set so a small surface is still measurable. For the full technique set, see which techniques the evidence supports.

Key takeaways

• Claude uses three independently controllable crawlers. Only Claude-User and Claude-SearchBot affect citation; ClaudeBot governs training, and blocking it costs nothing in visibility.
• Anthropic documents access, not selection. Any confident account of how Claude ranks sources is inference from observed behavior, and should be labelled as such.
• Claude's measured share of AI referrals ranges from about 1% to 18.5% depending on the panel, and the deciding variable is how business-facing your audience is.
• Claude selects specific pages rather than domains and skews toward long-form educational content, so depth on one URL beats coverage across many.

Frequently asked questions

Which crawler does Claude use?

Three, not one. Anthropic documents ClaudeBot for collecting web content that may contribute to model training, Claude-User for retrieving pages when someone asks Claude a question, and Claude-SearchBot for indexing content to improve search result quality. Each has its own robots.txt user-agent token and can be allowed or disallowed independently.

Does blocking ClaudeBot remove me from Claude's answers?

No. Anthropic states that restricting ClaudeBot signals that a site's future materials should be excluded from its AI model training datasets. Citation eligibility runs through the other two agents: Claude-User for user-directed retrieval and Claude-SearchBot for search indexing. You can opt out of training and stay eligible to be cited.

How does Claude decide which pages to cite?

Anthropic does not publish selection or ranking criteria. Its crawler documentation covers access and opt-out only. Any specific account of how Claude ranks or picks sources is third-party inference rather than documented behavior, so treat confident claims about the mechanism with caution and optimize for access and clarity instead.

What content length does Claude prefer?

Referral data points to long-form. Previsible's 2026 study describes Claude as skewing toward educational content such as guides, courses, and research-oriented pages, and reports that Claude users engage with longer-form content at higher rates than referrals from any other platform. That is measured behavior, not a stated ranking rule.

Is getting cited by Claude the same as getting cited by ChatGPT?

No. The access layers differ: OpenAI and Anthropic use separate crawler tokens with separate consequences, so allowing one says nothing about the other. The behavior differs too. ChatGPT frequently sends traffic to internal search pages, while Claude tends to select a specific URL and send the reader there.

Sources

• Anthropic Help Center, Does Anthropic crawl data from the web, and how can site owners block the crawler?. Article dated April 7, 2026. Checked August 22, 2026.
• Anthropic, published crawler IP ranges. 23 IPv4 prefixes, creation timestamp August 13, 2026. Checked August 22, 2026.
• Previsible, 2026 State of AI Discovery Report. 6.77M sessions across 166 GA4 properties, November 2024 to May 2026. Published July 6, 2026. Checked August 22, 2026.
• Goodie, 2026 AI Search Traffic Report, Wave 2. Brand-averaged B2B panel, March to April 2026. Published May 21, 2026. Checked August 22, 2026.
• Cloudflare, Content Independence Day. Crawl-to-visit ratio estimates, July 1, 2025. Checked August 22, 2026.

Back to AI citation strategies | Get cited by ChatGPT | Get cited in Google AI Overviews