Skip to main content
So you open ChatGPT, type a question, hit Enter, and a moment later an answer appears. The name “ChatGPT” actually combines two pieces: the Chat part — the application (the website or client) — and the GPT part — the model running behind the scenes that reads your question and computes an answer.
A minimalist infographic titled "What is ChatGPT?" showing two hand-drawn boxes: one labeled "Chat" (the app) with a chat window, and one labeled "GPT" (the model) with a smiling robot face. A small circular photo of a person speaking is in the bottom-right and a KodeKloud logo sits in the lower-left.
What sits under the application is a model. Before getting technical, imagine the simplest possible model: a tiny machine where you drop a number in and a different number comes out. Drop in 3 and you get 6. Drop in 10 and you get 20. The machine doubles whatever you give it. That little machine is a model: it takes an input, performs math, and returns an output.
A slide titled "A Model Is a Formula With Weights" showing inputs 3 and 10 entering a box labeled "×2" and producing outputs 6 and 20. A small circular inset photo of a man appears in the bottom-right corner.
Look closely: a model has two parts.
  • The formula (the structure the code implements — e.g., input → multiply → output). This structure is fixed by the architecture.
  • The weights (the numbers inside the formula, e.g., 2 in the ×2 example). These are learned from data.
Change the weight from 2 to 3 and the same structure triples the input instead. For real tasks one weight isn’t enough: predicting a house price uses features like size, bedrooms, and age. Each feature gets multiplied by a learned weight, and the results are summed — the model learns those weights by looking at many examples. That learning process is called training.
An illustrated diagram titled "A Bigger Formula, More Weights" showing weights (300, 50,000, −1,000, 25,000) multiplied by features like size (sqft), bedrooms, and age to compute price. A small circular photo of a man appears in the bottom-right.
For language tasks, these formulas become vastly larger. Inputs and weights expand into high-dimensional vectors; computations are arranged into many stacked layers. If you grow the network in the right pattern you get the transformer — the architecture behind modern large language models. The transformer is the code; what differentiates implementations like GPT, Claude, and Llama are the weights they contain.
A stylized diagram of a fully connected neural network with multiple layers and many interconnections, captioned "More Weights, and More, and More." A small circular photo of a man appears in the bottom-right and a KodeKloud logo is in the bottom-left.
The surprising part: a trained transformer is essentially a large file of numbers. Real models have billions of learned numbers stored on disk. Typical model file sizes:
  • Small: ~2 GB
  • Midsize: ~16 GB
  • 70B parameters: ~140 GB
  • Very large models: several hundred gigabytes
To run a model you load its weights from disk into memory, give it some text, run the transformer’s math over those numbers, and get text back. In short: load the file into memory, feed text in, get text out.
An illustrated infographic titled "A Model Is Just a Big File" showing four hand-drawn circles with dot clusters and labels for 2 GB, 16 GB, 140 GB, and 700 GB to compare model sizes. A small circular video thumbnail of a person is overlaid in the lower-right corner.
If a model is just a file and running it is loading it into memory and doing math, why not run one on your laptop? You can for small models, but larger models need specialized hardware. Understanding that requires a quick look at CPU vs. GPU and system memory. Every computer has two parts relevant here: the CPU (processor) that performs calculations and the RAM (system memory) that stores data and instructions. These are separate and connected by a channel (bus). For each operation the CPU reads operands from memory, computes, and writes results back — a round trip for each operation.
A hand-drawn infographic explaining CPU vs RAM using a calculator example (6 × 7 = 42), with arrows showing the CPU reading numbers from and writing the answer to memory. A small circular video overlay of a man appears in the bottom-right.
Running a transformer is mostly billions of small, independent multiplications (matrix math). Those operations are parallelizable. CPUs are optimized for a small number of powerful cores handling complex sequential workloads. Feeding billions of independent operations to a CPU results in doing them slowly, a few at a time. That’s why we use GPUs. GPUs are built for parallelism: thousands of simpler cores that do many math operations at once. Data-center GPUs deliver orders-of-magnitude higher parallel math throughput (TFLOPS) than CPUs, making them far better for the matrix-heavy workloads models require.
A simple diagram comparing a CPU (left) with a few large cores and ≈10 TFLOPS to a GPU (right) with many small cores and ≈1,000 TFLOPS, under the title "So we use a GPU." A small circular video thumbnail of a person appears in the bottom-right corner.
A GPU fixes compute parallelism but introduces another bottleneck: feeding data fast enough to the cores. Weights must be delivered from memory to GPU cores. If weights sit in system RAM, they travel over PCIe or another interconnect whose effective bandwidth is often tens to a few hundred GB/s — much lower than the GPU’s internal bandwidth. Cores would be starved waiting for data. The solution is local, fast memory on the GPU — VRAM — that sits next to the compute units. VRAM provides very high bandwidth but is capacity‑limited. For example, an NVIDIA T4 may have only 16 GB VRAM while some models require hundreds of GB to fit entirely in memory. So three numbers summarize a GPU’s suitability for a model:
  • Compute: how much math it can do (e.g., TFLOPS)
  • Capacity: how much fits in VRAM (e.g., GB)
  • Bandwidth: how fast the GPU can read VRAM (e.g., TB/s)
Each GPU generation improves these three metrics.
A comparison table listing compute (TFLOPS), bandwidth (TB/s) and memory (GB) for desktops, rack servers, and NVIDIA data‑center GPUs (A100, H100, H200) with numeric values. A small circular webcam photo of a man is overlaid in the bottom-right.
Example comparisons (approximate):
  • Desktop GPUs: strong compute but limited VRAM (single- to low-tens of GB), best for smaller models and experimentation.
  • Rack servers: more CPU cores and larger system RAM for dataset handling and orchestration, but heavy matrix math still runs on GPUs.
  • Data-center GPUs: cards like NVIDIA A100 come with 40 GB or 80 GB VRAM; H100 and newer generations increase throughput and memory. New accelerators continue to push TFLOPS and VRAM to support larger models.
Be mindful of VRAM limits. If a model’s weights don’t fit in a GPU’s VRAM, performance drops drastically due to transfers over PCIe or the need for complex memory streaming. For production deployments, choose hardware where model capacity, compute, and bandwidth align with your model size and latency requirements.
A compact comparison table: Hardware is inert without software. PyTorch is a common framework that loads models onto GPUs and orchestrates computation. Using the Hugging Face transformers library (which uses PyTorch under the hood), loading and running a small model can be done in a few lines:
This minimal script runs a single invocation on one machine. Production services must handle many users concurrently and run continuously — listening for requests, batching them, managing GPU memory, and scaling across multiple GPUs and machines. That’s the role of a model server. A model server wraps the model and runtime into a long‑lived service with a standard web API. Popular model servers like vLLM provide production-ready features:
  • Multi-user request handling
  • Efficient batching and scheduling
  • Memory and GPU management
  • Multi-GPU and multi-node setups
If you’re experimenting locally, use the Hugging Face pipelines and a single GPU. For production inference at scale, evaluate model servers (e.g., vLLM) and hardware that match your model’s capacity and latency needs.
Links and references

Watch Video