Demo Running a Local Model

% ollama list
NAME              ID                SIZE    MODIFIED
gemma4:31b        6316f0629137      19 GB   10 minutes ago
gpt-oss:20b       17052f91a42e      13 GB   43 minutes ago

% ollama run gpt-oss:20b
>>> in two sentences, what is a VPN?
Thinking...
...done thinking.

A VPN, or virtual private network, creates a secure, encrypted tunnel between your device and a remote server, masking your IP address and encrypting all traffic between them. This protects your privacy, helps bypass geographic restrictions, and secures connections on untrusted networks like public Wi‑Fi.

>>> /bye
%

# run_local.py
import os
import sys
import pathlib
import argparse
import urllib.request
import json

DEFAULT_MODEL = "gpt-oss:20b"
OLLAMA = os.environ.get("OLLAMA_HOST", "http://localhost:11434").rstrip("/")
if not OLLAMA.startswith("http"):
    OLLAMA = "http://" + OLLAMA

here = pathlib.Path(__file__).parent
SEARCH_DIRS = [here, here.parent / "Token Economics"]

def find(name):
    for d in SEARCH_DIRS:
        p = d / name
        if p.exists():
            return p
    sys.exit(f"Could not find {name} next to this script or in ../Token Economics")

def preflight(model):
    # placeholder: check model availability, memory, etc.
    pass

def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--model", default=DEFAULT_MODEL)
    ap.add_argument("--show-thinking", action="store_true", help="print the model's reasoning")
    ap.add_argument("--think", choices=["off", "low", "medium", "high"], default="low", help="thinking level")
    ap.add_argument("--ctx", type=int, default=16384)
    ap.add_argument("--max-tokens", type=int, default=1024)
    args = ap.parse_args()

    system_text = find("system-prompt.txt").read_text()
    user_text = find("ticket-and-logs.txt").read_text()

    preflight(args.model)

    body = {
        "model": args.model,
        "messages": [
            {"role": "system", "content": system_text},
            {"role": "user", "content": user_text},
        ],
        "stream": True,
        "options": {"num_ctx": args.ctx, "num_predict": args.max_tokens},
    }

    # Code to POST to Ollama and stream responses omitted for brevity.
    # After run completes, parse stats and print usage like below.

% python3 run_local.py
Summary: Dana’s VPN was rejected because the Endpoint Agent on her MacBook is too old – the policy requires version 5.4.0, but the log shows agent 5.2.1.

Root cause:
2026-09-21 08:02:22 vpn-gw-01 sslvpn[2213]: sess=c47d19 user=dwhitfield event=POSTURE_CHECK policy=Corp-Remote-Std rev=14 result=FAIL reason=agent_version_below_minimum agent=bwl-ea/5.2.1 required=5.4.0
2026-09-21 08:02:22 vpn-gw-01 sslvpn[2213]: sess=c47d19 user=dwhitfield event=SESSION_REJECT code=0x4C msg="Connection rejected by policy"

Fix (Runbook R3 – agent_version_below_minimum):
1. Ask Dana to open "Self Service" from the Dock.
2. In Self Service, locate "BWL Endpoint Agent" and click "Install" to upgrade to 5.4.0.
3. Instruct her to reboot the MacBook.
4. Have her retry the VPN connection.

If the Self Service install is missing or the update fails, escalate to Endpoint Engineering.

Escalate to: None (unless update cannot be installed).

Confidence: High – the log unambiguously shows a version mismatch causing the policy reject.

--- usage (what a local run gives you instead of a bill) ---
model                  gpt-oss:20b
input tokens           5,996
output tokens          662    (thinking tokens included)
load time              6.0 s
prompt processing      656 tokens/s
generation             38.1 tokens/s
total time             32.6 s
cost in tokens         $0.00   (you paid in hardware and time)

(thinking hidden: 1,354 characters. Rerun with --show-thinking to watch it.)
%

% ollama ps
NAME            ID              SIZE    PROCESSOR    CONTEXT    UNTIL
gpt-oss:20b     17052f91a42e    12 GB   100% GPU     16384      3 minutes from now
%

% ollama stop gpt-oss:20b
% ollama run gemma4:31b
# After running the same request against the larger model:
--- usage (what a local run gives you instead of a bill) ---
model               gemma4:31b
input tokens         7,369
output tokens        1,200    (thinking tokens included)
load time            11.6 s
prompt processing    85 tokens/s
generation           9.1 tokens/s
total time           230.0 s
cost in tokens       $0.00   (you paid in hardware and time)

(thinking hidden: 2,513 characters. Rerun with --show-thinking to watch it.)
(stopped at --max-tokens before the answer finished; raise it or use --think low)
%