manta.com

7 of 8 AI crawlers may read the site, 1 disallowed

• GET /robots.txt -> HTTP 200, 9.4 KB, 6 user-agent groups
• Allowed: GPTBot (3 path rules), ClaudeBot (3 path rules), OAI-SearchBot (via *) (55 path rules), PerplexityBot (via *) (55 path rules), Google-Extended (via *) (55 path rules), Amazonbot (3 path rules), Bytespider (via *) (55 path rules)
• Disallowed: CCBot
• Sitemap declared: https://www.manta.com/site_map_claimed_world_index_1.xml, https://www.manta.com/site_map_manta_blog.xml, https://www.manta.com/site_map_marketing_pages.xml (+59)

Fix: Add a group for CCBot with "Allow: /" (or drop the blanket "Disallow: /") if you want answer engines to cite you.

None of the 7 machine-readable surfaces is published (a sitemap is declared in robots.txt)

• /llms.txt: HTTP 403
• /sitemap.xml: HTTP 403
• /.well-known/x402: HTTP 403
• /.well-known/agent-card.json: HTTP 403
• /.well-known/mcp-registry-auth: HTTP 403
• /openapi.json: HTTP 403
• /ai.txt: HTTP 403
• Sitemap declared in robots.txt instead: https://www.manta.com/site_map_claimed_world_index_1.xml

Fix: Publish /llms.txt (a short Markdown guide to your key pages); one agent surface: /openapi.json, /.well-known/agent-card.json, or /.well-known/x402 if you sell to agents.

Home page is blocked by a bot challenge (HTTP 403)

• GET https://manta.com/ -> HTTP 403 after 2 redirects (final: https://www.manta.com/), text/html, 5.5 KB
• <title>: "Just a moment..."

Fix: Your edge serves a challenge or error to non-browser fetchers; allowlist the AI crawlers you want (at least for the home page) or serve them a static version.

Home page returned HTTP 403 (challenge); structured data not measurable

• GET https://manta.com/ -> HTTP 403, text/html

Fix: Let non-browser fetchers reach the home page first.

Unavailable

• takeitdown.agentexchange.work/check.json: refused takeitdown.agentexchange.work: hosts on agentexchange.work cannot be graded from inside the same Cloudflare zone; every check would - unavailable

Fix: Re-run in a few minutes; the notice check could not be completed.

Exposes /mcp but is not in the census

• census unavailable (refused index.agentexchange.work: hosts on agentexchange.work cannot be graded from inside the same Cloudflare zone; every check would time )
• GET /mcp -> HTTP 403 (text/html): an endpoint answers
• Index MCP probe record: refused index.agentexchange.work: hosts on agentexchange.work cannot be graded from inside the same Cloudflare zone; every check would time

Fix: Publish /.well-known/x402 with your paid routes so payment indexers can list you.