Technical SEO audit
A full crawl plus Search Console, analytics and server log data where available. You get a prioritized list of issues, each with impact, effort and an owner.
We find the problems that stop Google, Bing and AI crawlers from reading your pages properly, fix them, and keep checking as your site changes.
Technical SEO decides whether search engines and AI systems can crawl your pages, render them, index them and work out what each one is about.
When it breaks, good content and strong links underperform, and the cause rarely shows up in a marketing dashboard.
Can Googlebot, Bingbot and AI crawlers reach every page that matters, and are they wasting time on pages that don't?
Does your content exist in the HTML a crawler receives, or only after JavaScript runs in a browser?
Are the right pages in Google's index under the right canonical URL, with duplicates and thin pages kept out?
Do your structure, internal links and schema make it obvious what each page covers and how it relates to the rest of the site?
Each engagement starts with a full audit. The work after that follows the audit's priorities, so the fixes that affect revenue get done first.
A full crawl plus Search Console, analytics and server log data where available. You get a prioritized list of issues, each with impact, effort and an owner.
robots.txt, XML sitemaps, canonical tags, noindex rules, redirects, parameter handling and index coverage. Crawl budget on large sites.
URL structure, navigation, hub pages, orphaned pages and click depth, so authority reaches the pages that sell.
Largest Contentful Paint, Interaction to Next Paint and Cumulative Layout Shift, measured on real-user data, with fixes for images, scripts, fonts and hosting.
We compare the raw HTML with the rendered page and flag content, links and metadata that only appear after scripts run.
Organization, product, software, article, breadcrumb and FAQ markup that matches what's on the page, validated and monitored.
We check robots.txt, CDN and firewall rules for OAI-SearchBot, PerplexityBot, Claude-SearchBot, Googlebot and Bingbot, and set your policy for training crawlers separately.
hreflang, country and language structure, and duplicate handling for sites that serve more than one market.
Redirect maps, staging checks, launch-day monitoring and a recovery plan for redesigns, replatforms and domain changes.
Search crawlers decide whether you can appear in AI answers. Training crawlers decide whether your content can be used to train models. Those are separate decisions, and plenty of sites block the wrong ones through a CDN or firewall setting nobody remembers switching on.
| Crawler | Company | What it does | Our default |
|---|---|---|---|
| OAI-SearchBot | OpenAI | Finds pages to show in ChatGPT search answers | Allow |
| ChatGPT-User | OpenAI | Fetches a page when a ChatGPT user's request needs it | Allow |
| GPTBot | OpenAI | Collects content that may be used to train OpenAI models | Your call |
| PerplexityBot | Perplexity | Surfaces and links sites in Perplexity results | Allow |
| Claude-SearchBot | Anthropic | Indexes content to improve Claude's search results | Allow |
| Claude-User | Anthropic | Fetches a page when a Claude user asks a question | Allow |
| ClaudeBot | Anthropic | Collects content that may be used to train Claude | Your call |
| Google-Extended | Controls use of your content for Gemini training and grounding. Google says it does not affect inclusion or ranking in Google Search | Your call | |
| Googlebot, Bingbot | Google, Microsoft | Search indexing. Google's AI features and Microsoft Copilot depend on these indexes | Allow |
Sources: OpenAI crawler documentation · Anthropic crawler documentation · Perplexity crawler documentation · Google crawler documentation
A full crawl, a data review and a prioritized issue list. Urgent problems get flagged the day we find them.
We rank fixes by revenue impact and effort, and agree the order with you and your developers.
We make changes in your CMS where we have access. For code changes, we write tickets with acceptance criteria and check the work after release.
Recurring crawls, Search Console checks and alerts for new errors, so problems get caught before rankings drop.
We write everything for the person who has to act on it: a prioritized list for marketing, and tickets with acceptance criteria for engineering.
The program put technical foundations and high-intent category pages first during an aggressive growth window.
Two to three weeks for most sites, longer for sites with hundreds of thousands of URLs. We send urgent findings as we go, so a serious indexing problem doesn't sit waiting for the final report.
Both. We make changes directly in your CMS when we have access, including Webflow, WordPress, Shopify, HubSpot and most headless setups. When a fix needs code, we write tickets your developers can pick up and verify the result after release.
Yes. We map every URL, write the redirect plan, test the staging site, and monitor launch day and the weeks after, with a rollback list ready. Bring us in before the design is signed off: problems caught before launch cost far less to fix.
When Vercel analyzed AI crawler traffic in December 2024, none of the major AI crawlers executed JavaScript. If your important content only appears after scripts run, those crawlers may see an empty page. We check what each crawler receives and close the gaps with server-side rendering or pre-rendering. (Vercel research)
Yes. AI assistants can only cite pages they can reach and read. Google says a page must be indexed and eligible to show a snippet before it can appear as a link in AI Overviews or AI Mode, so a page with indexing problems can't be cited there either. (Google Search Central)
Technical SEO is part of every SEO retainer, and retainers start at $2,000 a month. Standalone audits and migrations are fixed-fee projects. See pricing for what changes the fee.
Tell us about your site, your platform and any recent changes. We reply within one business day to book a call, then send a proposal for an audit or a retainer.