Technical access audit
Can AI crawlers read your site? A blocked site cannot benefit from any content work.
Up to eleven engines · 23 markets · refreshed daily
Robots rules, rendering, sitemaps and the crawler log in one screen: whether the engines can read your pages before you spend a month improving them.
Can AI crawlers read your site? A blocked site cannot benefit from any content work.
✓ Start audit| Status | Check | Finding | Open task |
|---|---|---|---|
| blocking | llms.txt | No llms.txt. Not required, but a cheap way to describe the site’s structure to AI crawlers. | Open task |
| fix this | Sitemap | The sitemap has not been updated for 47 days. Freshness matters especially to Perplexity. | Open task |
| fix this | Response speed (TTFB) | Time to first byte 840 ms. Slow responses burn crawl budget. | Open task |
| passing | robots.txt — AI crawlers | Every AI crawler can read the site. | — |
| passing | Content without JavaScript | The content is present in the raw HTML. | — |
| passing | Structured data (JSON-LD) | Schemas found: Organization, Product, FAQPage. | — |
| passing | Bot protection (CDN/WAF) | Bot protection is not blocking AI crawlers. | — |
The scan runs when you start it, with “Start audit” or from a workflow; fixes are re-verified on the next scan you start. Tasks you open go to you or your developer as instructions.
A page that cannot be fetched cannot be quoted, and a surprising number of sites refuse the crawlers behind AI answers without ever deciding to. The audit reads your robots rules crawler by crawler, fetches pages the way a model fetches them to see what survives without JavaScript, checks the sitemap and IndexNow readiness for Bing — the index ChatGPT and Copilot answer from — and puts the crawler log next to all of it, so you can see which crawler actually arrived and when.
An answer engine cannot cite a page it could not fetch. Before any of the interesting questions — is the writing good, is the claim clear — there is a dull one: did the crawler get a 200, and did the text arrive in the HTML or only after JavaScript ran? A surprising share of invisibility is this and nothing more.
Answer crawlers are not search crawlers and should not be read as one line in a log. GPTBot, ClaudeBot, PerplexityBot and Google-Extended arrive on their own schedules, obey their own rules and are blocked by different lines in robots.txt — and a CDN rule written years ago against scrapers is the most common reason one of them never comes back.
Structured data is the cheapest fix on this screen and the one most often left half-done. A product without a price, an FAQ without an answer, an article without a date — each is a field a model wanted and did not find, and each is usually a template change rather than a project. The screen lists them by how much they cost you, not by how hard they look.
Can AI crawlers read your site? A blocked site cannot benefit from any content work.
Are AI crawlers actually reading your site, and which pages? With the WordPress plugin installed this screen fills on its own; without it, upload your server log. Either way no visitor records are kept — only the crawler lines.
Builds the index the assistants answer from. Block it and you vanish from search answers at once.
A person is waiting on this request right now. If it fails, an answer breaks while they watch.
Feeds a future model. Blocking it costs nothing today and your presence in the model tomorrow.
A full crawl of your own site: on-page issues, broken links, duplicate titles and Core Web Vitals. No external cost.
Crawl the siteFrom Google's PageSpeed Insights. Field data where enough Chrome users visited, lab data otherwise.
| URL | Performance | LCP | CLS | INP | Core Web Vitals |
|---|---|---|---|---|---|
| / | 91 | 1.9s | 0.03 | 140ms | |
| /pricing | 84 | 2.4s | 0.06 | 180ms | |
| /integrations/gmail | 88 | 2.1s | 0.02 | 160ms | |
| /blog/crm-for-real-estate-teams | 72 | 3.1s | 0.12 | 230ms |
/blog/crm-migration-checklist · /docs/api-v1 · /integrations/outlook …
/features/forecasting-beta · /partners/old
/blog/pipeline-stages · /blog/sales-forecasting · /templates …
/ · /integrations · /customers …
/blog/best-crm-for-small-business · /compare/hubspot · /blog/gmail-crm …
/templates/sales · /templates/support · /templates/hr …
/pricing · /security · /about
/changelog/2026-08 · /legal/dpa
/lp/webinar-2025 · /lp/black-friday · /beta
Each of these is a step the product runs, not a line on a slide — the name is the one it carries in the workflow builder.
Queues a technical audit for the selected pages.
Pages with an answer-format score below the threshold.
Adds the selected pages to the action list as refreshes.
Writes the run summary to the Inbox.
| Metric | What it counts | How to read it |
|---|---|---|
| Search index | Builds the index the assistants answer from. Block it and you vanish from search answers at once. | A person is waiting on this request right now. If it fails, an answer breaks while they watch. |
| Technical access audit | The audit tells you in two minutes whether AI bots can read your site. | The scan runs when you start it, with “Start audit” or from a workflow; fixes are re-verified on the next scan you start. Tasks you open go to you or your developer as instructions. |
| AI access | Can AI crawlers read your site? A blocked site cannot benefit from any content work. | Nothing is blocking the crawlers — AI engines can read your site. |
Measured gaps become briefs, briefs become drafts; nothing is published until you approve it.
Visits to your site from AI assistants. This is where visibility converts to traffic.
Order steps from the catalogue: measure → check → act → notify. Runs manually, daily or weekly; what each step found shows in the run log.
The four screens below are the whole path from a measurement to a task.
Visits to your site from AI assistants. This is where visibility converts to traffic.
Open page →Everything AEO, GEO and SEO found, as one ranked list — split by who does the work.
Open page →What the system will do on your behalf. Approve or cancel what is waiting.
Open page →Visits to your site from AI assistants. This is where visibility converts to traffic.
This measures movement together, not cause. Nothing in this data can separate it from a good month for the whole category.
| Assistant | AI share | Session | Conversion |
|---|---|---|---|
| ChatGPT | 49% | 1,468 | 41 |
| Perplexity | 24.9% | 745 | 39 |
| Gemini | 15.5% | 465 | 35 |
| Microsoft Copilot | 10.6% | 319 | 39 |
Everything AEO, GEO and SEO found, as one ranked list — split by who does the work.
HubSpot beats you on 104 questions. "X vs Y" and "X alternative" pages are what assistants quote directly on the comparison questions buyers ask most.
We write and publish the page · Generate a comparison brief for this rival from the Competitors page; the rival’s side is written only from collected, real sources.
Pipedrive beats you on 15 questions. "X vs Y" and "X alternative" pages are what assistants quote directly on the comparison questions buyers ask most.
We write and publish the page · Generate a comparison brief for this rival from the Competitors page; the rival’s side is written only from collected, real sources.
Zoho CRM beats you on 13 questions. "X vs Y" and "X alternative" pages are what assistants quote directly on the comparison questions buyers ask most.
We write and publish the page · Generate a comparison brief for this rival from the Competitors page; the rival’s side is written only from collected, real sources.
monday CRM beats you on 4 questions. "X vs Y" and "X alternative" pages are what assistants quote directly on the comparison questions buyers ask most.
We write and publish the page · Generate a comparison brief for this rival from the Competitors page; the rival’s side is written only from collected, real sources.
Salesforce beats you on 58 questions. "X vs Y" and "X alternative" pages are what assistants quote directly on the comparison questions buyers ask most.
We write and publish the page · Generate a comparison brief for this rival from the Competitors page; the rival’s side is written only from collected, real sources.
On gemini-grounding, Salesforce appears 100% of the time and you appear 0%.
We write and publish the page · Write a page that answers this question directly: a clear answer in the first paragraph, a table, numbers, and the date it was updated.
On perplexity-sonar, Salesforce appears 100% of the time and you appear 0%.
We write and publish the page · Write a page that answers this question directly: a clear answer in the first paragraph, a table, numbers, and the date it was updated.
What the system will do on your behalf. Approve or cancel what is waiting.
Autopilot settings| Autopilot | Status | Risk tier | Runs: | |
|---|---|---|---|---|
Update the Business Profile description to the 2026 positioning | Failed | Your own accounts | 12 days ago | Retry |
Send the TechRadar review-seat pitch | Awaiting approval | Your own accounts | — | ApproveCancel |
Publish “The best CRM for nonprofits in 2026” | Awaiting approval | Your own site | — | ApproveCancel |
Publish the pricing FAQ block | Failed | Your own site | 19 days ago | Retry |
Send the crm.org pricing correction | Cancelled | Your own accounts | — | |
Publish “FlowCRM for real estate teams” | Done | Your own site | 22 days ago | |
Send the G2 review campaign invitation | Done | Your own accounts | last month |
Brands and agencies track their AI visibility with Rixe
Rixe turns your crawled pages into clean Markdown and keeps llms.txt and llms-full.txt up to date. Serve them to AI assistants with a ready Cloudflare Worker or a WordPress snippet, or download a zip. The content is the same your visitors see.
In the app: AI access & pages › AI-ready pages
AI agents now open sites to book or buy for a person. The audit checks whether robots.txt lets ChatGPT-User, Claude-User, Perplexity-User and Google-Agent in, whether your forms can be filled in by an agent, and whether buttons and links can be found. WebMCP, an emerging proposal, is reported for information only.
In the app: AI access & pages › Audit
AI agent readiness now asks whether a shopping agent can find how to pay on your site without a person typing a card number. The audit looks for UCP, ACP (ChatGPT Instant Checkout), x402, an MCP server card and shopping links in llms.txt. A site that sells online with none of them gets a warning and a minimal UCP profile to start from; a site without online checkout passes for information.
In the app: AI Access & Pages › Audit
Give an AI agent a task on your own site, such as finding the price of a plan, and watch it work in a real browser, step by step. It never pays, signs in or sends a form: it stops at that step and marks it for your approval. Whatever got in its way becomes a finding with a fix. It only visits domains you have verified as yours. From the Growth plan.
In the app: AI Access & Pages › Agent simulator
See which AI bots read your pages and what they got back, from Cloudflare, the WordPress plugin, a log drain or an uploaded server log. Visits are grouped by purpose (live user fetch, search index, training crawl), checked against the address ranges each company publishes, and listed per page, with a badge when AI answers cited it. Only real visits are shown, never estimates.
In the app: Sources & AI Traffic › Crawlers
A robots rule copied years ago, a bot filter at the CDN, and content that only exists after JavaScript runs. All three are visible here, named per crawler rather than as one pass-or-fail verdict.
ChatGPT and Copilot answer from the Bing index. A site can be well indexed by Google and nearly absent from Bing, which quietly removes it from two of the biggest answer surfaces.
Whether a model's crawler can fetch your page, read it without running JavaScript, and find the answer in the markup rather than behind an interaction.
From your own server or CDN log, connected once. It shows GPTBot, ClaudeBot, PerplexityBot, Google-Extended and the rest by name, with status codes.
More common than teams expect, usually through a managed rule nobody set deliberately. It is also the fastest thing on this page to fix.
It removes you from the answer. It does not stop the content being described from third-party sources, so the usual result is losing the mention without gaining privacy.
Yes, more than for search. Several AI crawlers do not execute JavaScript at all, so a client-rendered page is an empty page to them.
Any non-200 to a named AI crawler, and especially a 403 that only they receive — that is usually a bot rule catching them by accident.
It helps a model find the fact, it does not make the model cite you. Clear prose in readable markup does more than a schema block on a thin page.
Monthly is enough if nothing changed; after any CDN, WAF or CMS change it is worth re-reading the same week, because that is when access quietly breaks.
Three numbers, one screen. Enough to know which way the week went.
What's the best project management tool for small teams?
What are the alternatives to Asana?
The CRMs that come up most often for small teams are HubSpot, Pipedrive and FlowCRM. FlowCRM is the one people pick when the team already lives in Gmail.
Can you recommend a CRM with multilingual support?
The CRMs that come up most often for small teams are HubSpot, Pipedrive and FlowCRM. FlowCRM is the one people pick when the team already lives in Gmail.
Which reporting tool do agencies use?
What's the best project management tool for small teams?
What are the alternatives to Asana?
The CRMs that come up most often for small teams are HubSpot, Pipedrive and FlowCRM. FlowCRM is the one people pick when the team already lives in Gmail.
Can you recommend a CRM with multilingual support?
The CRMs that come up most often for small teams are HubSpot, Pipedrive and FlowCRM. FlowCRM is the one people pick when the team already lives in Gmail.
Which reporting tool do agencies use?
Which tool has the most generous free plan?
Jira or Linear — which one is easier?
Which software is the fastest to set up?
Best analytics tool for e-commerce?
Which tool has the most generous free plan?
Jira or Linear — which one is easier?
Which software is the fastest to set up?
Best analytics tool for e-commerce?
Which tool is GDPR compliant?
What should a new company start with?
What is the best CRM for a small team?
Which tool is GDPR compliant?
What should a new company start with?
What is the best CRM for a small team?
“Being well-ranked on Google is no longer the same as being recommended by ChatGPT. It's good to have tools that try to measure this difference.”See how
AI engines on the menu, each measured on its own surface.
markets, each asked in its own language.
measurement, with a confidence range on every score.
Search Console, Analytics, Shopify, Slack, Notion, Linear — and anything else your team and your agents run on.
Rixe asks the AI engines your buyers use the questions they actually ask, every day, and shows which names come back and why.
Stay as involved as you like: the plan says what to fix, and on managed plans Rixe carries it out.