============================================================================= AGENTINDEXR v2.0.2 OFFICIAL DOCUMENTATION ============================================================================= Welcome to AgentIndexr! This documentation provides a comprehensive guide to installing, configuring, and maximizing the potential of the AgentIndexr WordPress plugin. AgentIndexr transforms your WordPress site into a modern, discoverable, and highly readable hub for AI agents, LLMs, and autonomous crawlers. --- TABLE OF CONTENTS: 1. Overview & Architecture 2. Installation & Updates 3. The Dashboard (Control Center) 4. AI Prompt Intelligence Layer 5. AI Visibility & Keywords (Google Search Console) 6. Settings: Basic Modules 7. Settings: Advanced Discovery Protocols 8. Settings: Robots & Traffic 9. Troubleshooting & Verification ============================================================================= 1. OVERVIEW & ARCHITECTURE ============================================================================= AgentIndexr consists of two primary architectural components: A. The Client Plugin (What you install on your WordPress site) B. The Server Proxy (Hosted at agentindexr.com) The Client plugin generates standards-compliant metadata, `robots.txt` rules, and endpoints (e.g., `/.well-known/*`) so that AI crawlers (like GPTBot, ClaudeBot, PerplexityBot) can ingest your site intelligently. The Server proxy acts as a secure intermediary for features like Google Search Console OAuth authentication. This means you do not need to create complex Google Cloud Platform API credentials yourself—the 1-click connection securely handles it for you. ============================================================================= 2. INSTALLATION & UPDATES ============================================================================= INSTALLATION: 1. Upload the `agentindexr` folder to the `/wp-content/plugins/` directory. 2. Activate the plugin through the 'Plugins' menu in WordPress. 3. Upon activation, the plugin automatically creates necessary database tables (e.g., `aco_prompt_index`, `aco_bot_analytics`) securely. UPDATES: AgentIndexr includes a secure, manual-download update engine. 1. Every 24 hours, your site securely pings `agentindexr.com` to check for updates. 2. If an update is available, you will see a notice on your WordPress Plugins page and inside the AgentIndexr Dashboard. 3. Click "Download Update" to retrieve the latest ZIP from the AgentIndexr portal, then manually upload it via the WordPress plugin installer. No settings or data will be lost during updates. ============================================================================= 3. THE DASHBOARD (CONTROL CENTER) ============================================================================= The Control Center is your primary hub, featuring a modern SaaS-style layout. A. SCORE CARDS - Overall AI Visibility Score: A weighted aggregate (60% Infrastructure, 40% Optimization) indicating your site's readiness for AI agents. - Infrastructure Score: Measures core endpoint availability (robots.txt, sitemaps, MCP, etc.). - Optimization Score: Measures proactive configurations (llms.txt, Schema, Analytics, DNS-AID). B. DISCOVERY HEALTH CENTER An automated Verification Engine actively queries your live site endpoints (e.g., checking if `/.well-known/mcp.json` returns HTTP 200). - Passed (Green): Endpoint is healthy. - Warning (Yellow): Endpoint is active but missing key directives. - Failed (Red): Endpoint is returning a 404/Error or is misconfigured. Click "Fix Issue" to navigate to the correct settings page. C. AI CRAWLER TRAFFIC Displays recent visits from known AI bots (GPTBot, ClaudeBot, etc.) intercepted natively by the plugin before they hit discovery endpoints. ============================================================================= 4. AI PROMPT INTELLIGENCE LAYER ============================================================================= This layer helps you discover and track natural language prompts that AI assistants might use to find your content. A. AUTO PROMPT GENERATOR - Navigate to any Post or Page editor in WordPress. - Look for the "AgentIndexr: AI Prompt Opportunities" meta box on the sidebar. - Click "Generate AI Prompts". The plugin will dynamically generate 10-30 intent-based prompts based on your content. - You can copy these to your clipboard or add them directly to the Prompt Index. B. AI PROMPT INDEX (AgentIndexr -> Prompt Index) - A dedicated dashboard to track your target AI prompts. - Manually add prompts, assign a target URL, intent, and priority. - The plugin computes a "Status" (Weak, Improving, Strong) based on keyword overlap between your prompt and the target page's title/content. ============================================================================= 5. AI VISIBILITY & KEYWORDS (GOOGLE SEARCH CONSOLE) ============================================================================= AgentIndexr integrates real Google Search Console (GSC) data to show you exactly how your content performs in organic search. 1. Navigate to Settings -> Integrations. 2. Click "Connect Account" under Google Search Console Integration. 3. Authenticate via Google. The secure proxy will return an access token to your WordPress database. 4. Navigate to "Keywords & Ranking". You will now see: - Performance Data Table: Impressions, Clicks, CTR, and Position. - Quick Wins: High impressions, ranking positions 4-10. - Page 2 Opportunities: Ranking positions 8-20. - Low CTR Alerts: High impressions, but CTR < 2%. ============================================================================= 6. SETTINGS: BASIC MODULES ============================================================================= Navigate to Settings -> Basic to toggle core infrastructure features. - Robots.txt Manager: Generates dynamic AI crawler rules and a `Content-Signal` header. (Note: Ensure you delete any physical `robots.txt` file in your root directory, as it overrides WordPress's virtual robots file). - XML Sitemap Manager: Exposes an AI-optimized sitemap at `/ai-sitemap.xml`. - llms.txt Generation: Creates a standard Markdown file at `/llms.txt` summarizing your site's context for LLMs. You can edit the exact content here. - Agent Skills & skill.md: Generates `/agentindexr/skill.md` and the `/.well-known/agent-skills/index.json` schema directory. - Structured Data / JSON-LD: Automatically injects semantic Schema.org metadata into your site's ``. ============================================================================= 7. SETTINGS: ADVANCED DISCOVERY PROTOCOLS ============================================================================= Navigate to Settings -> Advanced for developer-oriented protocols. - RFC 8288 Link Headers: Injects `Link:` headers into HTTP responses (e.g., `rel="mcp"`). - AI Markdown Negotiation: Intercepts requests looking for Markdown (e.g., `Accept: text/markdown` or appending `.md` to a URL) and returns clean text instead of raw HTML. - MCP Endpoints: Exposes Model Context Protocol at `/.well-known/mcp.json`. - WebMCP Support: Injects `navigator.modelContext` javascript to expose specific tools (like WP Search or WooCommerce Search) directly to browser-based agents. - API Catalog (RFC 9727): Publishes `/.well-known/api-catalog` linkset mapping. - OAuth Discovery: Publishes OpenID configuration and authorization server metadata so agents understand your authentication requirements. - DNS-AID Recommendations: While the plugin cannot change your DNS, this module provides the exact TXT records you need to copy into your domain registrar to enable DNS-level agent discovery. ============================================================================= 8. SETTINGS: ROBOTS & TRAFFIC ============================================================================= Navigate to Settings -> Robots & Traffic to fine-tune crawler behavior. - Content Signals Control: Adjust the `ai-train`, `search`, and `ai-input` preferences appended to your `robots.txt` file. By default, `ai-train=no` is set. - Per-bot Overrides: Provide a JSON string to assign specific Content-Signal rules to specific bots (e.g., `{"GPTBot": {"ai-train": "yes"}}`). - AI Bots Control: Explicitly "Allow" or "Disallow" major crawlers like GPTBot, ClaudeBot, and PerplexityBot. ============================================================================= 9. TROUBLESHOOTING & VERIFICATION ============================================================================= ISSUE: "Endpoint Live Validation Failed" in Dashboard FIX: AgentIndexr intercepts requests dynamically via `parse_request`. If an endpoint returns a 404, verify that you have "Pretty Permalinks" enabled in WordPress (Settings -> Permalinks). ISSUE: "Physical robots.txt overrides plugin" FIX: Connect via FTP or File Manager and delete the hardcoded `robots.txt` file in your public WordPress root. WordPress will natively fall back to our dynamically generated file. ISSUE: "OAuth service temporarily unavailable" FIX: The proxy server at `agentindexr.com` may be experiencing high load. Try authenticating again in a few minutes. ISSUE: "WebMCP not detected" FIX: WebMCP relies on javascript executed on window load. Ensure your caching or minification plugins are not blocking or stripping inline `