-
Notifications
You must be signed in to change notification settings - Fork 471
Agent Docs - Iterate #6288
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Agent Docs - Iterate #6288
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -6,49 +6,52 @@ title: 'Model parameters' | |
| description: 'Configure model selection and generation parameters for ClickHouse Agents' | ||
| keywords: ['AI', 'ClickHouse Cloud', 'agents', 'model parameters', 'temperature', 'top-p', 'top-k', 'thinking', 'prompt caching'] | ||
| doc_type: 'reference' | ||
| draft: true | ||
| --- | ||
|
|
||
| import BetaBadge from '@theme/badges/BetaBadge'; | ||
|
|
||
| <BetaBadge/> | ||
|
|
||
| Model parameters control which model an agent uses and how that model generates responses. Configure them in the Agent Builder's **Model Parameters** panel. | ||
| Model parameters control which model an agent uses and how that model generates responses. | ||
| From the **Agent Builder** tab, click **Select a model** under the **Model** heading. | ||
| This will open the **Model Parameters** window. | ||
|
|
||
| ## Provider and model {#provider-and-model} | ||
|
|
||
| - **Provider** — the upstream LLM provider. | ||
| - **Model** — the specific model from that provider. Different models have different strengths: large reasoning models for planning-heavy tasks, faster small models for routine queries. | ||
| - **Provider** - select the upstream LLM provider. | ||
| - **Model** - select the specific model from that provider. Different models have different strengths: large reasoning models for planning-heavy tasks, faster small models for routine queries. | ||
|
|
||
| Provider and model are required. The rest of the panel adapts based on what the selected model supports. | ||
| Both **Provider** and **Model** are required. The rest of the panel adapts based on what the selected model supports. | ||
|
|
||
| ## Context and output limits {#context-and-output-limits} | ||
|
|
||
| - **Max Context Tokens** — caps total context the agent sends to the model. Leave as **System** to use the model's default. Lower it to reduce cost; raise it for agents that need to reason over large inputs. | ||
| - **Max Output Tokens** — caps the size of the agent's response. **System** uses the model's default. Set lower if responses are too long, higher if they're being cut off. | ||
| - **File Token Limit** — caps how many tokens a single uploaded file contributes to context. Useful when users attach large files and you don't want them to crowd out the rest of the conversation. | ||
| - **Max Context Tokens** - caps total context the agent sends to the model. Leave as **System** to use the model's default. Lower it to reduce cost; raise it for agents that need to reason over large inputs. | ||
| - **Max Output Tokens** - caps the size of the agent's response. **System** uses the model's default. Set lower if responses are too long, higher if they're being cut off. | ||
| - **File Token Limit** - caps how many tokens a single uploaded file contributes to context. Useful when users attach large files and you don't want them to crowd out the rest of the conversation. | ||
|
|
||
| ## Sampling {#sampling} | ||
|
|
||
| - **Temperature** — randomness. Higher values (0.7–1.0) = more random, while lower values (0.0–0.3) = more focused and deterministic. We recommend altering this or Top P but not both. | ||
| - **Top P** — nucleus sampling. Changes how the model selects tokens for output. | ||
| - **Top K** — restricts sampling to the top K most likely tokens at each step. Supported by some providers; controls determinism along a different axis than temperature. | ||
| - **Temperature** - randomness. Higher values (0.7–1.0) = more random, while lower values (0.0–0.3) = more focused and deterministic. We recommend altering this or Top P but not both. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same here |
||
| - **Top P** - nucleus sampling. Changes how the model selects tokens for output. | ||
| - **Top K** - restricts sampling to the top K most likely tokens at each step. Supported by some providers; controls determinism along a different axis than temperature. | ||
|
|
||
| If you're not tuning for a specific behavior, leave the sliders near their defaults — small changes here rarely move the needle and large ones can degrade output quality. | ||
| If you're not tuning for a specific behavior, leave the sliders near their defaults as small changes rarely move the needle and large ones can degrade output quality. | ||
|
|
||
| ## Reasoning controls {#reasoning-controls} | ||
|
|
||
| Available on models that expose extended reasoning. The exact set varies by provider. | ||
|
|
||
| - **Thinking** — toggles the model's extended reasoning mode. When on, the model produces internal thinking tokens before its final answer; this usually improves accuracy on hard tasks at the cost of latency and tokens. | ||
| - **Thinking Budget** — token budget for the thinking phase. The model stops thinking and answers once it has spent this many tokens. | ||
| - **Effort** — high-level reasoning effort dial (**Auto**, low, medium, high). Used by reasoning models that don't expose a thinking-token budget directly. | ||
| - **Thought Visibility** — controls whether the model's thinking is shown to the user inline, hidden behind a collapsed view, or omitted entirely. | ||
| - **Thinking** - this toggles the model's extended reasoning mode. When on, the model produces internal thinking tokens before its final answer; this usually improves accuracy on hard tasks at the cost of latency and tokens. | ||
| - **Thinking Budget** - sets a token budget for the thinking phase. The model stops thinking and answers once it has spent this many tokens. | ||
| - **Effort** - high-level reasoning effort dial (**Auto**, low, medium, high). Used by reasoning models that don't expose a thinking-token budget directly. | ||
| - **Thought Visibility** - this controls whether the model's thinking is shown to the user inline, hidden behind a collapsed view, or omitted entirely. | ||
|
|
||
| ## Conversation behavior {#conversation-behavior} | ||
|
|
||
| - **Resend Files** — when on, files attached in earlier turns are re-sent on every subsequent turn so the model doesn't lose track of them. Turn off to save tokens if the conversation is short or the model is summarizing files as it goes. | ||
| - **Use Prompt Caching** — when supported by the provider, caches reusable parts of the prompt to reduce cost and latency on conversations where instructions and tool descriptions repeat across turns. | ||
| - **Web Search** — toggles provider-native web search on supported models. This is distinct from the [Web search tool](/cloud/features/ai-ml/agents/builder/web-search), which runs as one of the agent's tools rather than as a provider capability. | ||
| - **Resend Files** - when on, files attached in earlier turns are re-sent on every subsequent turn so the model doesn't lose track of them. Turn off to save tokens if the conversation is short or the model is summarizing files as it goes. | ||
| - **Use Prompt Caching** - when supported by the provider, caches reusable parts of the prompt to reduce cost and latency on conversations where instructions and tool descriptions repeat across turns. | ||
| - **Web Search** - toggles provider-native web search on supported models. This is distinct from the [Web search tool](/cloud/features/ai-ml/agents/builder/web-search), which runs as one of the agent's tools rather than as a provider capability. | ||
|
|
||
| ## Reset {#reset} | ||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't see this when I try the model builder, is this just not available in the Open House demo?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, this is only available for certain models, none of which are enabled in production yet. I'm ok to remove this and add back when those are available.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've added
draft=trueto the frontmatter of this doc so that it doesn't show in production. When these features are available in production we'll just need to remove that attribute and add it to the overview page again.