fix(seo): valid robots.txt syntax and trailing-slash gallery canonical

Drops the non-standard LLM-Content: directive that Lighthouse flagged as
a parse error (the /llms.txt discovery path stays in place via a comment).
Folds the previously orphaned Allow: lines into the User-agent: * stanza
where they are semantically effective.

Aligns the gallery canonical URL with Astro's trailing-slash static output
to resolve 27 "Alternative page with correct canonical tag" exclusions
reported in Google Search Console.

Author: Bissbert

public/robots.txt

@@ -3,14 +3,18 @@
 
 User-agent: *
 Allow: /
+Allow: /crystals/
+Allow: /og-images/
+Allow: /favicon.svg
 Disallow: /api/
 Disallow: /admin/
 
 # Sitemap location
 Sitemap: https://gemmology.dev/sitemap-index.xml
 
-# LLM crawler index (https://llmstxt.org/)
-LLM-Content: https://gemmology.dev/llms.txt
+# LLM crawler index (https://llmstxt.org/) — discoverable at /llms.txt directly.
+# The LLM-Content directive is non-standard for robots.txt parsers, so we
+# advertise the location via a comment plus the well-known /llms.txt path.
 
 # Named AI-crawler stanzas.
 # Several AI agents (notably PerplexityBot) require an explicit Allow per
@@ -65,7 +69,3 @@ Allow: /
 Disallow: /api/
 Disallow: /admin/
 
-# Static assets are explicitly allowed for all crawlers above.
-Allow: /crystals/
-Allow: /og-images/
-Allow: /favicon.svg

src/pages/gallery/index.astro

@@ -16,7 +16,7 @@ const initialSearch = url.searchParams.get('search') || '';
 const initialOrigin = url.searchParams.get('origin') || '';
 
 // Always point to base gallery URL as canonical to prevent duplicate content from filters
-const canonicalURL = 'https://gemmology.dev/gallery';
+const canonicalURL = 'https://gemmology.dev/gallery/';
 
 const { familyCount, expressionCount } = await getSiteStats();