facebook.com

5 of 8 AI crawlers may read the site, 3 disallowed

• GET /robots.txt -> HTTP 200, 34 KB, 36 user-agent groups
• Allowed: GPTBot (41 path rules), ClaudeBot (41 path rules), PerplexityBot (41 path rules), Google-Extended (41 path rules), Amazonbot (41 path rules)
• Disallowed: OAI-SearchBot (via *), CCBot (via *), Bytespider (via *)
• "User-agent: *" disallows the whole site; only crawlers named explicitly may read it

Fix: Add a group for OAI-SearchBot, CCBot, Bytespider with "Allow: /" (or drop the blanket "Disallow: /") if you want answer engines to cite you.

None of the 7 machine-readable surfaces is published

• /llms.txt: HTTP 200 but an HTML page (catch-all route, not a real file)
• /sitemap.xml: HTTP 200 but an HTML page (catch-all route, not a real file)
• /.well-known/x402: HTTP 404
• /.well-known/agent-card.json: HTTP 404
• /.well-known/mcp-registry-auth: HTTP 404
• /openapi.json: HTTP 200 but an HTML page (catch-all route, not a real file)
• /ai.txt: HTTP 200 but an HTML page (catch-all route, not a real file)

Fix: Publish /llms.txt (a short Markdown guide to your key pages); /sitemap.xml; one agent surface: /openapi.json, /.well-known/agent-card.json, or /.well-known/x402 if you sell to agents.

Home page is readable

• GET https://facebook.com/ -> HTTP 200 after 1 redirect (final: https://www.facebook.com/), text/html, 61 KB
• 560 characters of visible text with scripts, styles and tags removed; <title>: "Facebook"
• 1 external script, 17 inline; <noscript> present
• first words: "Facebook Email or phone Password Forgotten account? Sign Up Not Logged In Not Logged In Please log in to see this page. Return home English ..."

Fix: Nothing to fix here.

No JSON-LD, Open Graph complete

• 0 JSON-LD blocks
• Open Graph: og:title "Log in or sign up to view"; og:description present
• meta description: absent

Fix: add a JSON-LD block (Organization or WebSite, plus Product, FAQPage or Article where relevant) so answer engines describe you accurately.

Unavailable

• takeitdown.agentexchange.work/check.json: refused takeitdown.agentexchange.work: hosts on agentexchange.work cannot be graded from inside the same Cloudflare zone; every check would - unavailable

Fix: Re-run in a few minutes; the notice check could not be completed.

Exposes /mcp but is not in the census

• census unavailable (refused index.agentexchange.work: hosts on agentexchange.work cannot be graded from inside the same Cloudflare zone; every check would time )
• GET /mcp -> HTTP 500 (text/html): an endpoint answers
• Index MCP probe record: refused index.agentexchange.work: hosts on agentexchange.work cannot be graded from inside the same Cloudflare zone; every check would time

Fix: Publish /.well-known/x402 with your paid routes so payment indexers can list you.