AI readiness & governance
PitchAI helps companies and webshops turn AI policy into operational controls, authoritative web content, structured data, governed automation, and a practical implementation roadmap.
Why this matters
Search crawlers, user-requested fetchers, and model-training crawlers serve different purposes. A serious policy records which uses the organisation permits, protects non-public systems, and assigns an owner who can review the decision as platforms change.
Technical access is only the front door. Answer engines still need clear service and product information, stable URLs, accurate data, useful internal links, and structured data that agrees with the visible page. If those foundations are weak, allowing a crawler does not make the organisation understandable.
What we do
The work is scoped to the organisation's real goals and risk boundaries. We do not promise placement in an AI answer; we establish the conditions for accurate discovery and controlled use.
01
Define approved uses, accountable owners, data boundaries, human review, escalation, supplier decisions, and a review cycle that people can actually operate.
02
Audit robots.txt, page-level directives, canonicals, sitemaps, redirects, response codes, and CDN or firewall behaviour for OpenAI, Claude, Perplexity, Google, and Bing discovery.
03
Make important services, products, policies, locations, and company facts available as concise, crawlable text with semantic HTML and accurate structured data.
04
Turn the readiness decision into a bounded delivery plan: integrations, workflow automation, evaluation, permissions, monitoring, and a controlled path to production.
Crawler policy
One blanket allow or deny rule hides material differences. We document the purpose of each control and keep private or operational paths out of scope.
OAI-SearchBot · Claude-SearchBot · PerplexityBot · Googlebot · Bingbot
These crawlers support search indexes and answer-engine visibility. Access makes a page eligible for discovery; it does not guarantee indexing, ranking, or citation.
ChatGPT-User · Claude-User · Perplexity-User
These fetchers may retrieve a page when a user asks a product to visit it. Their behaviour and robots support differ, so access must also be checked at the CDN or firewall edge.
GPTBot · ClaudeBot · Google-Extended
These controls are distinct from search inclusion. The organisation should make and record this policy choice separately from its answer-engine visibility decision.
Typical output
A concise record of permitted uses, crawler choices, owners, review dates, and protected boundaries.
Evidence for robots, metadata, sitemap, canonical, response, rendering, and edge-access behaviour.
A prioritised map of missing service, product, policy, company, and structured information.
Bounded implementation steps with acceptance criteria, dependencies, owners, and production checks.
Field note
Our field note explains the policy choices, technical controls, content foundations, and verification sequence in plain language.
Read the practical guideDiscuss AI readiness
We will review the context and tell you whether a focused audit, governance track, or implementation engagement is the useful next step.
info@pitchai.net