Classification Fundamentals

Source: Concepts adapted from Ace the Data Science Interview — Chapter 7 (“Machine Learning”) by Kevin Huo & Nick Singh. Rewritten in original wording for this course.

Classification assigns each example to one of $K$ classes (churn / not, click / not, fraud / legit) instead of predicting a continuous number. Interviews love this topic because the business framing — especially false positives vs false negatives — is half the answer.

This lesson covers the shared framework and how to evaluate classifiers — confusion matrix, precision/recall/F1, ROC — with interactive sims and slider explorers for threshold trade-offs. Model families (logistic regression, Naive Bayes, SVMs, trees) are in Classification models.

Generative vs discriminative

Two broad approaches both end up maximizing a posterior over classes, but they get there differently:

Style	Idea	Decision rule (sketch)
Generative	Model the joint $P(X,Y)$ (often via $P(X\mid Y)P(Y)$)	Pick class with largest posterior from Bayes
Discriminative	Model $P(Y\mid X)$ (or a decision boundary) directly	$\hat y = \arg\max_k P(Y=k\mid X)$

Binary ($K=2$) is the interview default; multi-class adapts the same ideas (one-vs-rest, softmax, etc.).

Quiz

Classification framework

1 / 4

Classification differs from regression mainly because it predicts…

Pick an answer to see if you are right.

Evaluating classifiers (and the accuracy paradox)

If a disease hits 1 in 10,000 people, always predicting “no disease” is 99.99% accurate — and useless. That is the accuracy paradox: with imbalance, accuracy alone can hide a model that never finds the rare positive class.

You need metrics that account for false positives (FP) and false negatives (FN).

Confusion matrix

For binary classification:

	Pred +	Pred −
Actual +	TP	FN (type II)
Actual −	FP (type I)	TN

• False positive: predict disease when healthy (alarm / costly treatment).
• False negative: miss a real case (false security).

Quiz

Confusion matrix

1 / 4

A false negative means…

Pick an answer to see if you are right.

Practice: confusion counts → rates

Precision, recall, and F1

• Precision = $\mathrm{TP}/(\mathrm{TP}+\mathrm{FP})$ — of predicted positives, how many were right?
• Recall (sensitivity) = $\mathrm{TP}/(\mathrm{TP}+\mathrm{FN})$ — of actual positives, how many did you catch?
• Specificity = $\mathrm{TN}/(\mathrm{TN}+\mathrm{FP})$ — of actual negatives, how many correctly cleared?
• F1 = $2\cdot\frac{\mathrm{precision}\cdot\mathrm{recall}}{\mathrm{precision}+\mathrm{recall}}$ — harmonic mean when both matter

Trade-off: higher recall (catch more cancer) often means more false alarms; higher precision (trust positives) often means more misses. Interview answers must tie FP/FN costs to the product.

The threshold explorer above already shows precision / recall / F1 moving together — re-slide $\tau$ while you read this section.

Quiz

Precision, recall, F1

1 / 4

Precision answers…

Pick an answer to see if you are right.

Compute

Compute: P / R / F1 / specificity

1 / 5

Story problem — figure out which numbers are the formula inputs, then compute.

TP=20, FP=10. Precision = TP/(TP+FP). Round to 2 decimals.

Enter a number (at most 2 decimal places).

Practice: F1

ROC curves and AUC

Sweep the classification threshold and plot true positive rate (recall) against false positive rate ($\mathrm{FP}/(\mathrm{FP}+\mathrm{TN})$). The AUC (area under the ROC) summarizes ranking quality: closer to 1 is better separation; ~0.5 is chance.

A strong curve “hugs” the top-left (high TPR, low FPR).

Quiz

ROC & AUC

1 / 4

ROC plots…

Pick an answer to see if you are right.

Practice: TPR, FPR, AUC

Quick synthesis

Idea	Sound bite
Goal	Assign classes (or class probabilities), not continuous $y$
Generative vs discriminative	Joint vs $P(Y\mid X)$ — both → posterior decisions
Accuracy paradox	Imbalance makes accuracy misleading
Confusion matrix	TP / FP / FN / TN organize errors
Precision / recall / F1	Trust + vs catch +; harmonic mean when both matter
ROC / AUC	Threshold sweep; ranking quality

Next: Classification models — logistic regression, Naive Bayes, SVMs, and decision trees.