> ## Documentation Index
> Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Demo Creating an App Using Ollama OpenAI Python Client
> Guide to building a Flask AI Poem Generator using the OpenAI Python client and local Ollama backend, covering setup, example code, environment configuration, and production considerations
This guide demonstrates how to structure a small web application that uses the OpenAI Python client to talk to a local Ollama backend (or switch later to OpenAI's hosted API with minimal changes). We’ll build a simple Flask-based AI Poem Generator that sends user prompts to a chat completion endpoint and renders the model's output.
Using a client library (instead of curl) helps keep your app code clean and portable across local development and hosted providers.
## What you’ll need
* Python 3.8+ installed
* Ollama running locally for local development (optional if you target OpenAI hosted APIs)
* Familiarity with virtual environments and Flask
Useful references:
* [OpenAI Python client docs](https://platform.openai.com/docs/api-reference)
* [Ollama docs](https://docs.ollama.ai/) (for running a local model server)
* [Flask documentation](https://flask.palletsprojects.com/)
## Project setup
1. Create a project folder, for example:
```bash theme={null}
mkdir -p ~/code/ollama-app
cd ~/code/ollama-app
```
2. Create and activate a virtual environment:
```bash theme={null}
python -m venv ollama-app
source ollama-app/bin/activate
```
3. Install dependencies:
```bash theme={null}
pip install openai flask python-dotenv
```
4. Open your editor and create `server.py` at the project root.
## server.py — full implementation
This example exposes a single route ("/"): GET renders a small form, POST sends the prompt to the chat completions endpoint using the OpenAI Python client and displays the returned poem.
```python theme={null}
# server.py
import os
from flask import Flask, request, render_template_string
from openai import OpenAI
from dotenv import load_dotenv
load_dotenv() # take environment variables from .env
app = Flask(__name__)
client = OpenAI(
api_key=os.environ.get("OPENAI_API_KEY"),
base_url=os.environ.get("LLM_ENDPOINT")
)
# Simple HTML template for the app UI
HTML_TEMPLATE = """
AI Poem Generator
AI Poem Generator
{% if poem %}
Your AI-Generated Poem
{{ poem }}
{% endif %}
"""
@app.route("/", methods=["GET", "POST"])
def index():
poem = None
if request.method == "POST":
try:
input_message = request.form["input"]
response = client.chat.completions.create(
model=os.environ.get("MODEL"),
messages=[
{"role": "system", "content": "You are an AI assistant specialized in writing poems."},
{"role": "user", "content": input_message}
],
)
# Extract the generated text from the response
# Structure: response.choices[0].message.content
poem = response.choices[0].message.content
except Exception as e:
# log the exception, and show a friendly error to the user
print("Error:", str(e))
poem = "An error occurred when trying to fetch your poem."
return render_template_string(HTML_TEMPLATE, poem=poem)
if __name__ == "__main__":
port = int(os.getenv("PORT", 3000))
# Development server — do not use this directly in production
app.run(host="0.0.0.0", port=port)
```
## Configuration — environment variables
Store runtime configuration in a `.env` file at the project root. This keeps credentials and endpoints out of your code and makes switching between local Ollama and OpenAI hosted APIs straightforward.
| Variable | Purpose | Example |
| ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------ |
| OPENAI\_API\_KEY | API key used by the OpenAI client. For local Ollama testing this can be any value. Replace with a real key when using OpenAI hosted APIs. | kodekloud |
| LLM\_ENDPOINT | Base URL for the LLM endpoint. For Ollama local server use [http://localhost:11434/v1](http://localhost:11434/v1) | [http://localhost:11434/v1](http://localhost:11434/v1) |
| MODEL | Model identifier to call (configurable without code changes). | llama3.2 |
Example `.env` contents:
```bash theme={null}
OPENAI_API_KEY=kodekloud
LLM_ENDPOINT=http://localhost:11434/v1
MODEL=llama3.2
```
## How the app works (high-level)
* The HTML form posts the user's prompt to "/".
* The Flask route builds a `messages` array: a system message to define role plus the user's message.
* The OpenAI Python client sends that to the chat completions endpoint (`client.chat.completions.create(...)`) and returns a response object.
* We extract the model output at `response.choices[0].message.content` and render it inside the page.
## Start Ollama's REST API (local)
Ensure the Ollama local service is running so the OpenAI client can reach it. Typical local start command:
```bash theme={null}
ollama serve
```
By default Ollama exposes endpoints such as POST /v1/chat/completions and listens on `localhost:11434`.
Representative Ollama server log when started:
```log output theme={null}
time=2025-01-24T13:05:44.483+05:30 level=INFO msg="Listening on 127.0.0.1:11434 (version 0.5.7)"
time=2025-01-24T13:05:44.530+05:30 level=INFO msg="inference compute" id=0 library=metal total="10.7 GiB" available="10.7 GiB"
```
## Run the Flask app
With your virtualenv active and `.env` in place:
```bash theme={null}
python server.py
```
You should see the Flask development server start:
```text theme={null}
* Serving Flask app 'server'
* Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
* Running on all addresses (0.0.0.0)
* Running on http://127.0.0.1:3000
Press CTRL+C to quit
```
Open [http://127.0.0.1:3000](http://127.0.0.1:3000), enter a prompt such as "Write a short poem about cats" and click "Generate Poem." The app will POST the prompt to the model and display the returned poem.
## Logs you will see
* Flask logs GET and POST requests to `/`.
* Ollama logs incoming requests to `/v1/chat/completions`.
Example Flask access log:
```text theme={null}
127.0.0.1 - - [24/Jan/2025 13:07:03] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [24/Jan/2025 13:07:43] "POST / HTTP/1.1" 200 -
```
## Next steps and production considerations
* This demo illustrates a minimal integration pattern. To prepare for production:
* Use a production WSGI server (gunicorn/uvicorn) instead of Flask’s dev server.
* Store secrets securely (e.g., a secret manager or environment variable service).
* Add robust error handling, rate limiting, and input validation/sanitization.
* Cache or paginate long-running requests and handle model timeouts gracefully.
* When switching to OpenAI hosted APIs, change the `LLM_ENDPOINT` and set a valid `OPENAI_API_KEY`.
This is a simple demo intended for local development. For production, use a production WSGI server (gunicorn/uvicorn), secure environment secrets properly, and follow best practices for rate-limiting, error handling, and user input sanitization.