AI readiness for the agent-driven web

Prepare your site for AI crawlers, agents, and discoverability tooling. AI readiness findings account for 20% of the overall score.

Files checked

CF Ready looks for public assets in public/, project root, or static/ (first match wins):

File	Search paths	Severity if missing
`robots.txt`	`public/robots.txt`, `robots.txt`, `static/robots.txt`	Medium
`sitemap.xml`	`public/sitemap.xml`, `sitemap.xml`, `static/sitemap.xml`	Medium
`llms.txt`	`public/llms.txt`, `llms.txt`	Medium
`llms-full.txt`	`public/llms-full.txt`, `llms-full.txt`	Medium
`openapi.json` / `openapi.yaml`	Project root or `public/openapi.json`	Medium (when API routes exist)
`mcp-server-card.json`	Project root	Low (when API routes exist)
`auth.md`	Project root	Low (when auth patterns detected)

Auth patterns are detected by scanning for next-auth, Clerk, Auth0, Lucia, JWT signing, session handling, and similar libraries in up to 200 source files.

fix --ai-readiness output

cf-ready fix --ai-readiness generates draft assets. Existing files are skipped unless you pass --force.

Output file	When generated	Contents
`public/robots.txt`	Always	Crawler rules, sitemap link, GPTBot/ClaudeBot/PerplexityBot hints
`public/llms.txt`	Always	Project summary, key pages, API section, usage policy
`public/llms-full.txt`	Always	Extended llms.txt with framework and package manager context
`public/sitemap.xml`	Always	URL set from detected routes + `criticalRoutes` (up to 20)
`openapi.json`	When API routes detected	OpenAPI 3.0.3 draft with one GET per route
`mcp-server-card.json`	When API routes or auth patterns exist	Agent discovery card with endpoints list
`auth.md`	When auth patterns detected	Agent-friendly auth documentation template

Review before publishing. Generated files are starting points. Update productionUrl in config, verify crawler policy, and flesh out OpenAPI schemas before go-live.

aiPolicy setting

Set in cf-ready.config.json (default allow-assistive-agents):

Value	Meaning
`allow-assistive-agents`	Default — robots.txt allows common AI crawlers; llms.txt encourages agent use
`restrict`	Limit agent access — tighten robots.txt rules after generation
`block`	Disallow AI crawlers — update generated robots.txt accordingly

The aiPolicy value is recorded in ai-readiness-report.md. Adjust generated robots.txt crawler sections to match your policy.

Commands

cf-ready ai-ready              # Run AI readiness checks only
cf-ready fix --ai-readiness    # Generate draft AI assets
cf-ready fix --ai-readiness --force   # Overwrite existing files
cf-ready ai-optimize           # LLM-assisted content optimization (requires Workers AI config)

Fix commands only run when explicitly invoked — a scan never writes files. See Configuration for productionUrl and criticalRoutes.