> ## Documentation Index
> Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Controlling Model Parameters
> Explains how to use temperature, topP, and maxTokens to control model output randomness, diversity, and length.
In this lesson you'll learn how to control model parameters to influence how a model generates responses. These controls help you steer output length, creativity, and determinism without changing the model’s pre-trained knowledge or reasoning capabilities.
How can you steer a model’s responses? In practice, model outputs may be too long, too random, or too inconsistent for your application. Some use cases require highly repeatable, deterministic outputs (e.g., code generation, templates); others benefit from diverse, creative outputs (e.g., marketing copy, storytelling). To meet those needs, you supply parameter controls with each inference request to influence token selection during generation.
## Why parameters matter
When generating text, the model predicts the next token from a set of candidate tokens and associated probabilities. Parameters let you influence which candidates are considered and how strongly high-probability options are favored. They tune generation behavior—creativity, randomness, and length—without changing the model’s underlying knowledge or training.
The key parameters covered here:
* Temperature — controls how strongly the model favors high-probability tokens (affects randomness).
* Top P (nucleus sampling) — defines the probability mass threshold for the candidate pool (affects diversity).
* Max tokens — caps the number of tokens the model may generate (controls length and cost).
## Temperature
Temperature adjusts how strongly the model favors the highest-probability next token:
* Low temperature (close to 0): outputs are predictable and deterministic — ideal for structured tasks like code or factual answers.
* High temperature (closer to 1): outputs are more varied and creative — useful for brainstorming, slogans, or storytelling.
Example Python (Amazon Bedrock Runtime SDK) showing `temperature` and `maxTokens` in `inferenceConfig`:
```python theme={null}
import boto3
client = boto3.client("bedrock-runtime", region_name="us-east-1")
response = client.converse(
modelId="us.meta.llama3-2-3b-instruct-v1:0",
messages=[
{
"role": "user",
"content": [{"text": "Write a short slogan for a cybersecurity company."}]
}
],
inferenceConfig={
"temperature": 0.7,
"maxTokens": 100
}
)
print(response["output"]["message"]["content"][0]["text"])
```
A temperature of `0.7` is moderately high and tends to produce more creative outputs than lower values.
## Top P (nucleus sampling)
Top P controls diversity by specifying a cumulative probability threshold for the candidate token pool. The model samples only from the smallest set of tokens whose total probability mass is at least `topP`.
* Lower `topP` (e.g., `0.6`): limits the pool to the most probable tokens (less diverse).
* Higher `topP` (e.g., `0.95` or `1.0`): expands the pool to include less probable tokens (more creative).
Top P determines the candidate set; temperature governs how adventurous the sampling is within that set. Adjust one parameter at a time when experimenting.
Example snippet showing `topP`:
```python theme={null}
# Example showing topP in inferenceConfig
inferenceConfig = {
"topP": 0.9,
"maxTokens": 100
}
```
## Max tokens
`maxTokens` sets an explicit upper bound on the number of tokens the model may generate in a single response. Use it to prevent runaway output and control cost.
* Use `maxTokens` to enforce concise answers for UI or API constraints.
* Lowering `maxTokens` forces brevity and reduces request cost.
Example snippet (integrate into the same `client.converse` call as above):
```python theme={null}
inferenceConfig = {
"temperature": 0.3,
"maxTokens": 50
}
```
## Quick reference table
| Parameter | Purpose | Typical range | When to use |
| - | - | -: | - |
| Temperature | Controls randomness / determinism | `0.0` — `1.0` (common: `0.0`–`1.0`) | Low for code/factual output; higher for creative text |
| Top P | Limits candidate token pool (nucleus sampling) | `0.0` — `1.0` (common: `0.6`–`0.95`) | Lower for focused outputs; higher for diversity |
| Max tokens | Caps the generated output length | Integer (tokens) | Enforce UI length or control cost |
## Practical guidance
* Change only one parameter at a time to isolate its effect. Record settings and outputs for reproducibility.
* For deterministic or structured tasks (code, templates): use low `temperature` (e.g., `0.0`–`0.3`) and optionally a low `topP`.
* For creative tasks (slogans, storytelling): increase `temperature` and/or `topP`.
* Always set a reasonable `maxTokens` to avoid excessive output and cost.
When testing, record the exact settings (`temperature`, `topP`, `maxTokens`) and sample outputs. Run controlled experiments by changing one parameter at a time to evaluate its effect.
## Analogy: menu and adventure
Think of `topP` as how many desserts are listed on the menu (the pool of choices) and `temperature` as how adventurous you feel. A wide menu (`topP` high) plus high adventurousness (`temperature` high) increases the chance you’ll pick an unusual dessert. The two controls together determine the final choice.
## What parameters do not do
Model parameters:
* Do not make the model smarter.
* Do not add knowledge or change training data.
* Do not fundamentally improve the model’s reasoning capability.
They only influence next-token selection during generation.
## Expected results
When you tune these parameters appropriately, you should see:
* Improved consistency when limiting randomness.
* Greater flexibility and creativity when allowing more diversity.
* Fewer prompt-engineering iterations by adjusting generation behavior directly.
## Key takeaway
Model parameters such as `temperature`, `topP`, and `maxTokens` let you influence how text is generated. Use them to tailor output length and creative variability to your application’s requirements while remembering they do not change the model’s underlying knowledge.
Should you use all parameters together? It depends on your use case. Understanding each control gives you the flexibility to produce either consistent or creative outputs as needed.
That concludes this lesson on controlling model parameters. Try the hands-on lab to practice tuning these settings and observe how each parameter affects generation.
Links and references
* Amazon Bedrock documentation: [https://docs.aws.amazon.com/bedrock](https://docs.aws.amazon.com/bedrock)
* Sampling and decoding strategies (overview): [https://distill.pub/2016/softmax/](https://distill.pub/2016/softmax/)
* Practical sampling discussion (temperature, top-p): [https://www.eleuther.ai/posts/top-p-sampling/](https://www.eleuther.ai/posts/top-p-sampling/)
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.