Running Local LLMs With Ollama
Customising Models With Ollama
Customizing Models
In this guide, we’ll walk through creating a domain-specific AI assistant by extending one of Ollama’s base models. We’ll customize Gromo’s assistant—named Harris—so it:
- Knows its own name
- Understands Gromo’s investment context
- Defaults to Indian Rupees (INR) when no currency is specified
1. Define Your Modelfile
First, create a Modelfile that builds on Llama 3.2, lowers creativity for financial precision, and sets up a system prompt:
FROM llama3.2
PARAMETER temperature 0.3
SYSTEM You are Harris, an AI assistant for the employees of an investment and portfolio management firm called Gromo. Your job is to assist with client investments and portfolios. The default currency is Indian Rupees (INR).
| Directive | Purpose | Example |
|---|---|---|
| FROM | Selects the base LLM | FROM llama3.2 |
| PARAMETER | Adjusts model settings (e.g., creativity, temperature) | PARAMETER temperature 0.3 |
| SYSTEM | Provides identity, role, and default behaviors | SYSTEM You are Harris… default currency is Indian Rupees (INR). |
Note
Using PARAMETER temperature 0.3 ensures more accurate, fact-driven responses—crucial for financial applications.
2. Build the Custom Model
With your Modelfile ready, run:
$ ollama create harris -f ./Modelfile
transferring model data
using existing layer sha256:…
creating new layer sha256:…
writing manifest
success
Next, verify the model list:
$ ollama ls
NAME ID SIZE MODIFIED
harris:latest 1e9cecec0e0a 2.0 GB 21 hours ago
qwen:0.5b b5dc5e784f2a 394 MB 4 days ago
llama3.2:latest a80c4f17acd5 2.0 GB 8 days ago
3. Verify Assistant Behavior
Run Harris to confirm its name recognition, context awareness, and INR default:
$ ollama run harris
>>> what's your name?
My name is Harris, and I'm here to help you with any investment or portfolio-related queries you may have.
>>> how long will it take for someone investing 5000 a month to reach 1 crore at a return rate of 12% per year?
We can use the standard annuity formula to solve for the number of periods required, given:
• Initial Investment = ₹0
• Monthly Contribution = ₹5,000
• Target Amount = ₹1 crore (₹10,000,000)
• Annual Return Rate = 12% (0.12)
…you can solve for the number of months or years needed.
Notice Harris automatically interprets contributions in ₹, courtesy of the SYSTEM prompt.
4. Next Steps: Share Your Model
Once you’re satisfied, push Harris to a registry so colleagues can pull it:
$ ollama pull your-org/harris
Warning
Ensure your registry credentials are configured before pushing or pulling custom models to avoid authentication errors.
Further Reading
Watch Video
Watch video content