Driver

ApexYard's marketing site (site/{index,architecture,skills}.html) has no robots.txt and no sitemap.xml. Result: search engines have no allowlist (default-permissive is OK but missed nudges to the right pages), no Sitemap: directive to point Google Search Console at, and new pages are slow to index. Verified by SEO audit at origin/dev 9170ed2 — findings S6 + S7. Both are high-severity because they directly degrade discoverability of every page on the site.

These are 5-minute fixes that compound — every day they're missing is a day Google's crawler is doing extra work to find your pages.

Scope

Add two files at site/ root:

1. site/robots.txt

User-agent: *
Allow: /

Sitemap: https://<marketing-site>/sitemap.xml

Conservative: allow all human and well-behaved bot crawlers, point them at the sitemap. AI-crawler directives (per the upcoming /generative-engine-audit recommendations from #311) can be added later — keep this first iteration simple.

2. site/sitemap.xml

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://<marketing-site>/</loc>
    <lastmod>2026-05-20</lastmod>
    <priority>1.0</priority>
  </url>
  <url>
    <loc>https://<marketing-site>/architecture.html</loc>
    <lastmod>2026-05-20</lastmod>
    <priority>0.8</priority>
  </url>
  <url>
    <loc>https://<marketing-site>/skills.html</loc>
    <lastmod>2026-05-20</lastmod>
    <priority>0.8</priority>
  </url>
</urlset>

Three URLs, lastmod set to the date of each PR landing (or refreshed via a tiny script run at release-cut time).

3. Submit to Google Search Console + Bing Webmaster Tools

After deploy, paste https://<marketing-site>/sitemap.xml into both consoles. One-time setup, ongoing benefit.

Acceptance Criteria

• site/robots.txt exists with the content above (or improved variant)
• site/sitemap.xml exists with all 3 current pages listed
• Netlify deploy includes both files at the site root (verify by curl https://<marketing-site>/robots.txt after deploy)
• Sitemap submitted to Google Search Console
• (Optional) Sitemap submitted to Bing Webmaster Tools
• Add to the release-cut process: refresh lastmod dates in sitemap.xml when site/ changes (or write a tiny shell-loop that updates them based on git log -1 --format=%cs site/<page>.html)

Out of scope

• AI-crawler directives in robots.txt (deferred to /generative-engine-audit recommendations once [Feature] /generative-engine-audit — LLM/agent SEO sibling to /seo-audit #311 lands)
• News-sitemap, image-sitemap, video-sitemap extensions — none applicable to apexyard's marketing site
• Dynamic sitemap generation — 3 pages doesn't justify a generator

Refs SEO-audit 2026-05-20T07-00-36Z findings S6 + S7 (persisted at projects/apexyard/audits/seo-audit/)