=== md4AI ===
Contributors: codekraft
Tags: GEO, AI, SEO, markdown, llms.txt
Requires at least: 6.2
Tested up to: 6.9
Requires PHP: 7.4
Stable tag: 1.3.0
License: GPLv2 or later
License URI: https://www.gnu.org/licenses/gpl-2.0.html

Optimise content for generative engines (GEO) by serving custom Markdown and a site-wide llms.txt.

== Description ==

The **md4AI** plugin provides a powerful, multi-layered approach to Generative Engine Optimization (GEO).

### 1. Automatic AI Bot Detection (Zero-Config)

md4AI automatically detects requests from a comprehensive list of known AI agents (like GPTBot, ClaudeBot, Google-Extended, etc.) and serves them a lightweight, on-the-fly Markdown version of your content instead of the standard HTML theme. This ensures a clean, parsable data feed for LLMs right out of the box.

### 2. Per-Post Custom Markdown (Metabox)

The plugin adds a new metabox to your post and page editor, giving you full control over the AI-optimized version of your content.
Serving an md version of the website is particular useful if your website has editors like Elementor / Beaver Builder / Divi Builder / other div bloated editors because bots usually skips complex and too much nested content.

* **Generate Custom Markdown:** Convert your post's content into Markdown with one click.
* **Manually Edit:** Modify and enhance the Markdown to be exactly as you want AIs to see it.
* **(Optional) Enhance with AI:** If you also use the **[AI Services](https://wordpress.org/plugins/ai-services/)** plugin, you can use the "Generate with AI" button to automatically add FAQs, discussion questions, or key takeaways extracted from your post.
* **Serve Enhanced Content:** When an AI bot visits this *specific post*, it will be served your new, custom-tailored Markdown version, giving it far richer data.
* **Automatic Fallback:** If you don't create a custom version, the plugin falls back to the automatic on-the-fly conversion.

### 3. Site-Wide AI Instructions (llms.txt)

The plugin adds a new main page under 'Utilities' with dedicated tabs for configuration and insights. From here, you can create and manage a `llms.txt` file for your entire site.

* **What is `llms.txt`?** This is a file, similar to `robots.txt`, that provides general instructions, context, and useful links to AI crawlers. It's a new standard to help guide LLMs in understanding your site's content and purpose.
* **Manage Content:** Use the settings page to write your `llms.txt` content, which will then be served automatically to bots that look for it.
* **Learn More:** You can find more details on this new standard at [https://llmstxt.org/](https://llmstxt.org/).

This combination gives you full control, from automatic optimization to granular, AI-enhanced, per-post content and site-wide AI directives.

### 4. GEO/AIO Insights & Validation

The dedicated GEO/AIO Insights tab provides a unique validation mechanism for your site's AI presence, offering measurable scores (0-100) on how well your brand is understood by Generative Engines.

* **The Validation Process**: We ask a Large Language Model (LLM) to complete a "knowledge card" about your site (e.g., brand name, main topics, product categories).
* **Ground Truth Check**: The AI response is immediately compared with real data extracted from your WordPress database (e.g., site title, author names, WooCommerce categories)
* **Final Score**: Based on this cross-check, the insight provides a score for brand identity and technical relevance, highlighting exactly where the AI provided incorrect or insufficient data (hallucinations or missing data).

== Installation ==

1.  Upload the `md4ai` folder to the `/wp-content/plugins/` directory.
2.  Activate the plugin through the 'Plugins' menu in WordPress.
3.  **(Optional but Recommended):** Install and configure the **[AI Services](https://wordpress.org/plugins/ai-services/)** plugin to enable the "Generate with AI" features in the post metabox.
4.  The automatic bot detection is now live.
5.  Go to the new **"md4AI" options page** in your dashboard (usually under 'Settings') to configure your site-wide `llms.txt` file.
6.  Go to any post or page edit screen to find the **"md4AI Custom Markdown"** metabox to customize content for that specific page.

== Frequently Asked Questions ==

= What is md4AI? =

md4AI is a plugin that optimizes your website for Generative Engines (GEO) by serving custom Markdown and a site-wide llms.txt file.

= What is GEO? =

GEO is a new standard for Generative Engine Optimization (GEO) that helps guide LLMs in understanding your site's content and purpose.

= How does it detect an AI bot? =

The plugin checks the incoming HTTP `User-Agent` string against an internal, pre-defined list of known AI and large language model (LLM) crawlers (e.g., gptbot, claudebot, google-extended, perplexitybot, etc.).

= How does the AI Services integration work? =

If you have the `ai-services` plugin installed, md4AI adds a "Generate with AI" button to its **post metabox**. This takes the generated Markdown for that post and sends it, along with a prompt, to your chosen AI provider. The AI-enhanced text (e.g., your post *plus* a new FAQ section) then replaces the content in the metabox, ready to be served to bots.

= What's the difference between the Custom Markdown and the llms.txt file? =

They serve two different, important purposes:

* **Custom Markdown (Per Post/Page):** This is controlled via a **metabox** on each post/page. It allows you to create a specific, enhanced Markdown *version of that page's content*. This is what an AI bot will receive *instead* of the HTML when it crawls that specific URL.
* **llms.txt (Site-Wide):** This is controlled from the plugin's main **options page**. It's a *single file* for your entire site, similar to `robots.txt`. It doesn't contain post content, but rather gives *general instructions* and context to AI crawlers (e.g., site purpose, useful links, etc.). For more details, see [https://llmstxt.org/](https://llmstxt.org/).

= What's the difference between 'Automatic' and 'Manual' Markdown? =

This applies to your posts and pages:

* **Automatic:** If you do nothing, the plugin automatically converts your post's HTML to Markdown on-the-fly when a bot visits.
* **Manual:** If you use the metabox to "Generate" or "Generate with AI," you create a *static* version that is saved to your database. This *saved* version will be served to bots instead. This allows you to add enhanced content (like FAQs) that only the AI sees.

= Does this affect regular search engine optimization (SEO)? =

No. Regular users and traditional search engine crawlers (like standard Googlebot) will continue to receive your standard HTML page. This functionality is specifically targeted at AI agents.

= Are there any performance concerns? =

No. For automatic detection, the plugin runs early and immediately stops execution (`exit;`) after serving the Markdown content. For manual generation, the content is pre-generated and saved, so serving it is extremely fast.

== Changelog ==

= 1.3.0 =
* The user can now select the preferred model for the post description generation
* Prevent admin bar links from leaking when extracting header and footer links. (thanks to @alh0319 for reporting this issue)
* Fix: llms.txt file 404s. (thanks to @alh0319 for reporting this issue)

= 1.2.0 =
* UI enhancements

= 1.1.1 =
- Store and Display access logs (source and search terms only)
- Updated dependencies
- Updated dashboard UI

= 1.1.0 =
- Enhanced admin UI
- GEO insights page
- Added crawler stats
- Added integration with the AI Services plugin.
- Added "Generate with AI" functionality to enhance content using prompts (e.g., add FAQs, summaries).
- Implemented REST API endpoints for generating base .md and .txt content.

= 1.0.0 =
- Added metaboxes to the post editor to manually generate and store custom Markdown (.md) file.
- The plugin now serves the custom, manually-generated content to bots if it exists, falling back to the automatic conversion if not.
- Initial release of the md4AI plugin.
- Includes a list of 14 known AI and search bots.
- Implements basic HTML to Markdown conversion for optimal AI consumption.
