# Token Efficiency
URL: /docs/token-efficiency
LLM index: /llms.txt
Description: How @farming-labs/docs keeps your project lean for AI-assisted development — without sacrificing flexibility
Related: /docs/guides/agent-friendly-docs, /docs/customization/agent-primitive, /docs/customization/llms-txt, /docs/customization/mcp
# Token Efficiency
Use this page when the user asks about this topic: How @farming-labs/docs keeps your project lean for AI-assisted development - without sacrificing flexibility.
Keep answers grounded in the exact options, routes, commands, and examples documented here.
If the request moves beyond this page, point to the closest related docs instead of inventing config.
AI tools like Cursor, Copilot, and Claude read your project files to help you write code. Every file they read costs **tokens** — the units that determine response speed, API cost, and how much of your project fits in a single conversation.
`@farming-labs/docs` is built to keep the framework footprint small so more of your token budget goes to your actual documentation content — while still giving you the full flexibility of a modern docs framework.
## One Config, Full Control
The core idea: everything about your docs site — theme, colors, typography, sidebar, AI chat, metadata — lives in a single `docs.config.ts` file.
```ts title="docs.config.ts"
import { defineDocs } from "@farming-labs/docs";
import { pixelBorder } from "@farming-labs/theme/pixel-border";
export default defineDocs({
entry: "docs",
theme: pixelBorder(),
nav: { title: "My Docs", url: "/docs" },
ai: { enabled: true, model: "gpt-4o-mini" },
metadata: {
titleTemplate: "%s – Docs",
description: "My documentation site",
},
});
```
An AI agent reads this one file and immediately understands your entire docs setup — what theme you're using, where content lives, how AI chat is configured, how pages are titled. There's a provider wrapper (`RootProvider`) and a docs layout file, but they're minimal one-liners that the CLI generates for you. The real configuration surface is this single file.
This matters because every additional config file, layout wrapper, or routing file is something an AI has to read, reason about, and avoid accidentally breaking. Fewer framework files means more room in the context window for what actually matters: your documentation content.
## The CLI Does the Heavy Lifting
You don't have to set any of this up manually. The CLI scaffolds everything in seconds — for both new and existing projects.
### Starting from scratch
Use `--template` to bootstrap a complete project with your framework and theme of choice:
```bash title="terminal"
npx @farming-labs/docs init --template next --name my-docs
```
```bash title="terminal"
pnpm dlx @farming-labs/docs init --template next --name my-docs
```
```bash title="terminal"
yarn dlx @farming-labs/docs init --template next --name my-docs
```
```bash title="terminal"
bunx @farming-labs/docs init --template next --name my-docs
```
This creates a new `my-docs/` folder with a fully working docs site — config, routes, CSS, sample pages, dependencies installed, dev server running. Pick from `next`, `tanstack-start`, `nuxt`, `sveltekit`, or `astro`.
Want a specific theme? Add `--theme`:
```bash title="terminal"
npx @farming-labs/docs init --template next --name my-docs --theme pixel-border
```
```bash title="terminal"
pnpm dlx @farming-labs/docs init --template next --name my-docs --theme pixel-border
```
```bash title="terminal"
yarn dlx @farming-labs/docs init --template next --name my-docs --theme pixel-border
```
```bash title="terminal"
bunx @farming-labs/docs init --template next --name my-docs --theme pixel-border
```
That's it — a beautiful themed docs site in one command.
### Adding to an existing project
Already have a Next.js, TanStack Start, SvelteKit, Astro, or Nuxt project? Just run `init` inside it:
```bash title="terminal"
npx @farming-labs/docs init
```
```bash title="terminal"
pnpm dlx @farming-labs/docs init
```
```bash title="terminal"
yarn dlx @farming-labs/docs init
```
```bash title="terminal"
bunx @farming-labs/docs init
```
The CLI auto-detects your framework from `package.json`, asks you to pick a theme, generates the config and minimal routing files, installs dependencies, and starts the dev server. Your existing code is untouched — it only adds what's needed for docs.
See the [CLI reference](/docs/cli) for all flags and options.
## Why This Matters for AI
### More context for your content
AI models have limited context windows. Framework boilerplate eats into that budget. With `@farming-labs/docs`, the framework surface is roughly **~15 lines of config** instead of hundreds of lines spread across many files. That leaves more room for your actual documentation pages when an AI is answering questions or making changes.
### Fewer files to reason about
When an AI agent needs to modify your docs setup — change a theme, enable AI chat, adjust the sidebar — it reads your `docs.config.ts` and makes the change in one place. No hunting through layout files, provider trees, slug handlers, and CSS imports to figure out how everything connects.
### Less risk of breaking things
A declarative config file is hard to break. An AI can add `ai: { enabled: true }` or change `theme: darksharp()` without worrying about import paths, component hierarchies, or routing logic. The framework handles all of that internally.
## Compact Page-Level Agent Docs
The framework already gives agents clean page-level markdown through `.md` routes, canonical URLs
with `Signature-Agent` in Next.js, hidden `...` blocks, and sibling `agent.md` files.
When you want that machine-readable layer to be even tighter, use `docs agent compact`.
The command resolves the same page contract the runtime already uses:
- existing `agent.md`
- embedded `Agent` blocks
- normal page markdown
Then it sends that resolved document to the Token Company compression API and writes a sibling
`agent.md` file for the page.
The visible docs UI still renders `page.mdx` or `page.md`. The generated `agent.md` becomes the
page-level source for `.md` routes, `Signature-Agent` markdown reads in Next.js,
`GET /api/docs?format=markdown&path=`, and MCP `read_page()`.
```ts title="docs.config.ts"
export default defineDocs({
entry: "docs",
agent: {
compact: {
apiKeyEnv: "TOKEN_COMPANY_API_KEY",
model: "bear-1.2",
aggressiveness: 0.3,
protectJson: true,
},
},
});
```
```bash title="terminal"
pnpm exec docs agent compact installation configuration
pnpm exec docs agent compact --all
```
The CLI also loads `.env` and `.env.local`, so `TOKEN_COMPANY_API_KEY` can stay out of checked-in
config. Use `--dry-run` when you want to preview the compression call without writing files. For the
full command surface, see the [CLI page](/docs/cli). For the `agent.compact` config shape, see
[Configuration](/docs/configuration).
## Built-in AI Features
Beyond being token-efficient to work with, `@farming-labs/docs` includes features designed for AI consumption:
### llms.txt
Your docs are automatically served in LLM-optimized format — no extra routes needed:
```text
/llms.txt → llms.txt (index of all pages)
/llms-full.txt → llms-full.txt (full content)
```
See the [llms.txt docs](/docs/customization/llms-txt) for details.
### Sitemaps
Enable `sitemap` when agents and crawlers need a compact inventory before reading pages:
```text
/sitemap.xml → canonical URLs with lastmod dates
/sitemap.md → semantic docs map with descriptions and markdown URLs
/docs/sitemap.md → docs-scoped alias for agents that look under the docs section
JSON-LD → per-page title, canonical URL, freshness, and breadcrumbs
```
JSON-LD structured data is injected automatically on docs pages. See the
[Sitemaps docs](/docs/customization/sitemaps) for static export and manifest details.
### Robots.txt
Generate `robots.txt` when crawlers and AI agents need an explicit public access policy for docs and
machine-readable routes:
```text
/robots.txt → allows docs, .md routes, llms.txt, sitemaps, AGENTS.md, skill.md, MCP, and agent discovery
```
Use `docs robots generate`, or `docs robots generate --append` when the project already owns a
custom `robots.txt`. The agent doctor checks the resolved file path and warns when the policy blocks
agent-readable routes or common AI crawlers.
### AI Chat with RAG
Built-in AI chat with retrieval-augmented generation. A simple setup is just two lines:
```ts title="ask-ai.ts"
ai: {
enabled: true,
model: "gpt-4o-mini",
}
```
Need multiple models from different providers? Use the `providers` map and a `model` object with a selectable dropdown:
```ts title="ask-ai.ts"
ai: {
enabled: true,
providers: {
openai: {
baseUrl: "https://api.openai.com/v1",
apiKey: process.env.OPENAI_API_KEY,
},
groq: {
baseUrl: "https://api.groq.com/openai/v1",
apiKey: process.env.GROQ_API_KEY,
},
},
model: {
models: [
{ id: "gpt-4o-mini", label: "GPT-4o mini (fast)", provider: "openai" },
{ id: "gpt-4o", label: "GPT-4o (quality)", provider: "openai" },
{ id: "llama-3.3-70b-versatile", label: "Llama 3.3 70B", provider: "groq" },
],
defaultModel: "gpt-4o-mini",
},
}
```
Users get a model selector dropdown in the chat interface. The backend automatically routes each request to the correct provider with the right credentials. No separate API routes, no vector databases, no embedding pipelines — search indexing, context retrieval, and streaming responses are all handled internally.
Works with any OpenAI-compatible provider: OpenAI, Groq, Together, Fireworks, OpenRouter, Ollama, or any vLLM deployment.
See the [AI Chat docs](/docs/customization/ai-chat) for the full configuration reference.
## Summary
`@farming-labs/docs` keeps the framework lean — one config file, minimal routing, CLI-generated scaffolding — so you and your AI tools spend less time on plumbing and more time on content. And it does this without sacrificing flexibility: themes, multi-provider AI chat, search, custom components, and multi-framework support are all still there, just configured from one place.