Serve Markdown to agents instead of HTML

INSTRUCTIONS.md

Rewrite Markdown URLs For Agents

Prerequisites

• Serve Markdown files for .md extensions (see #generating-markdown)
  • See how we do it at Rivet by copying .mdx to the public folder in Next.js

Steps

1. Log in to the Cloudflare Dashboard
   Go to https://dash.cloudflare.com and select your site.

2. Navigate to Snippets

   • In the sidebar, click Rules > Snippets
   • Click Create Snippet.

3. Add Your Snippet Code

   • Name your snippet (e.g., docs_url_rewriter).
   • Paste in your code (see below)

4. Deploy the Snippet

5. Test Your Snippet

   • Visit your site at https://example.com/docs/.... Confirm this works as expected
   • Then call curl "https://example.com/docs/..." to verify that you get served the markdown version

snippet.js

export default {
  async fetch(request, env, ctx) {
    const url = new URL(request.url);

    // Check if the path starts with /docs/ and Accept header is NOT text/html
    const acceptHeader = request.headers.get("accept") || "";
    if (
      url.pathname.startsWith("/docs/") &&
      !acceptHeader.includes("text/html")
    ) {
      // Rewrite the URL to add ".md" at the end if it doesn't already have it
      if (!url.pathname.endsWith(".md")) {
        url.pathname = `${url.pathname}.md`;
      }

      // Create a new request with the rewritten URL
      request = new Request(url.toString(), request);
    }

    return await fetch(request);
  },
};