Add llms.txt and llms-full.txt#436
Open
fntn wants to merge 8 commits into
Open
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
There was a problem hiding this comment.
Pull request overview
Adds two static text endpoints — /llms.txt (compact nav-tree index) and /llms-full.txt (concatenated plain-markdown dump) — following the llms.txt convention so LLMs can consume the Cloudsmith docs. Pages are filtered by src/content/menu.json, MDX-specific syntax (imports, JSX components, <br/>) is stripped, and both routes use force-static so generation happens at next build time.
Changes:
- New
src/lib/llms.tswithbuildIndexandbuildFullDocs, including a basic MDX-stripping pass. - New App Router route handlers serving
text/plainresponses for/llms.txtand/llms-full.txt. - Reuses existing
loadMdxInfofromsrc/lib/markdown/utilto enumerate MDX files per section.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/lib/llms.ts | Builds the nav index and full-content dump; collects menu paths and strips MDX syntax. |
| src/app/llms.txt/route.ts | Static GET handler returning the navigation index. |
| src/app/llms-full.txt/route.ts | Static GET handler returning the full docs dump. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…th-docs into docs-llms-dot-txt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds two plain-text endpoints for LLM consumption, following the llms.txt convention.
Heavily inspired by https://vercel.com/llms.txt
What it does
/llms.txt— a compact navigation index: a markdown link tree mirroring the site's nav structure, with a pointer to the full dump./llms-full.txt— the full content dump: every page's MDX content stripped to plain markdown and concatenated, organised by section.What's included / excluded
.mdxfiles on disk but aren't in the nav (drafts, hidden pages, etc.) are excluded.mobileNavbar) are automatically skipped.menu.json— adding a new nav section does not require changes to this code, as long as it maps to a known content directory (documentation,guides,api).Noteheadlines become bold labels,Cardtitle/description/href are rendered as linked prose, andBlockImagealt text is preserved.When it's built
Both endpoints use Next.js App Router route handlers with
force-static, so the files are generated atnext buildtime and served as static assets — no per-request computation.Discovery
A
<link rel="alternate" type="text/plain">tag is added to every page's<head>via the root layout metadata.