Skip to main content
Up to now we’ve focused on the inner training loop: the model takes inputs, makes predictions, computes a loss, and updates its parameters using gradients. This inner loop is where the model actually learns from a fixed dataset and a fixed configuration. Outside of that, there is an outer loop that covers the broader experimental process: choosing features, selecting model architectures, tuning hyperparameters, cleaning and augmenting data, and evaluating on validation sets. The outer loop is where you change the environment in which the inner loop runs — for example, adjusting the learning rate schedule, trying different regularization strategies, adding or removing features, or switching models. The interplay between inner and outer loops is how you build a practical machine learning system that improves over many experiments rather than a single training run. Supervised learning uses labeled examples. The next question is: what kind of label are we trying to predict? This distinction defines two common supervised tasks: regression and classification.
  • Regression predicts continuous values.
    Example: predicting a house sale price. Given features such as square footage, number of bedrooms, and neighborhood, a regression model predicts a numeric value like 700000, 850000, or 1_200_000. Other regression examples include predicting rent, delivery time, temperature, revenue, or demand. The output is a real number (or vector of real numbers), and it can take many possible values.
  • Classification predicts discrete categories.
    Example: a spam filter predicts whether an email belongs to the “spam” or “not spam” class. MNIST digit recognition is also classification: the model looks at an image of a handwritten digit and chooses one of 10 discrete classes (0–9). Classification outputs are often probabilities across classes (for example, spam: 92%, not-spam: 8%), and the predicted label is usually the class with the highest probability.
A common way to visualize the difference:
  • For classification, the model learns decision boundaries that separate classes in feature space. Points on one side of a boundary belong to one class, points on the other side belong to another.
  • For regression, the model fits a function (a line or curve) through the data to predict a numerical value from inputs (for example, square footage on the x-axis and price on the y-axis).
How these differences affect training and evaluation:
  • Regression training often uses losses that measure distance between predicted and true values (mean squared error, mean absolute error). Evaluation metrics include RMSE, MAE, and R².
  • Classification training commonly uses probabilistic losses such as cross-entropy, and evaluation metrics include accuracy, precision, recall, F1 score, and calibration measures.
For more on supervised learning and common metrics, see:
Summary: Regression predicts numbers (use MSE/MAE during training and RMSE/MAE/R² for evaluation), while classification predicts categories (use cross-entropy during training and accuracy/precision/recall/F1 for evaluation). Regardless of task, the model learns a mapping from inputs to outputs — simpler problems can be solved with simpler models; harder problems require more expressive models, more data, better features, and robust validation.
To summarize, regression is about predicting numerical values, and classification is about predicting categories. Whether we are predicting price, digits, or whether an email is spam, the model still has to learn a mapping from inputs to outputs. Simple models can work for simple patterns, but complex problems often require more expressive models, more data, stronger regularization, and careful validation.
A hand-drawn machine learning diagram showing a "House Prices" dataset table (sqft, beds, area, price) on the left and model/processing arrows to a neural network on the right. The right side also illustrates classification vs. regression with scatter plots, decision boundaries, and example labels.

Watch Video