Probability

Source: Concepts adapted from Ace the Data Science Interview — Chapter 5 (“Probability”) by Kevin Huo & Nick Singh. Rewritten in original wording for this course.

Probability is one of the skills interviewers probe hardest — especially conditional probability, Bayes’ rule, and matching the right distribution to a story. Finance loops lean on expectation and betting-style decisions; stats rounds dig deeper on top of these foundations.

This lesson covers the interview core: conditionals and total probability, counting, random variables, common distributions, and Markov chains. Each section ends with a concept quiz plus a compute quiz — randomized story problems (map the context to $n$, $k$, $p$, $\lambda$, …; answers to at most 2 decimals). Use New story for another scenario.

Conditional probability and Bayes’ rule

You often want $P(A \mid B)$ — the chance of $A$ once you know $B$ happened. Bayes’ rule rewrites that as:

$$P(A \mid B) = \frac{P(B \mid A)\, P(A)}{P(B)}$$

• $P(A)$ — prior
• $P(B \mid A)$ — likelihood
• $P(A \mid B)$ — posterior

If $P(A \mid B) = P(A)$, then $A$ and $B$ are independent: learning $B$ does not change $A$. They can also be conditionally independent given $C$:

$$P(A \cap B \mid C) = P(A \mid C)\, P(B \mid C)$$

Interview habit: any “given that…” probability is a Bayes hint. Rare-disease testing is the classic trap — a “highly accurate” test still needs the base rate. In ML, many models are about choosing a good conditional distribution of the target given the data.

Quiz

Conditional probability & Bayes

1 / 6

In Bayes’ rule, P(A|B) is called the…

Pick an answer to see if you are right.

Compute

Compute: Bayes’ rule

1 / 3

Story problem — figure out which numbers are the formula inputs, then compute.

A medical clinic screens for a rare condition that affects about 1.0% of people in this population. Their lab test correctly flags 99% of true cases, but also falsely flags 10% of healthy people. Someone just tested positive. What is the probability they actually have the condition? Round to 2 decimals.

Enter a number (at most 2 decimal places).

Law of total probability

If $\{B_i\}$ partitions the sample space (disjoint, cover everything), then:

$$P(A) = \sum_i P(A \mid B_i)\, P(B_i)$$

Think of a tree of outcomes: weight each branch by its probability, then sum. Common DS framing: purchase probability averaged over customer segments.

Quiz

Law of total probability

1 / 2

The law of total probability expresses P(A) as…

Pick an answer to see if you are right.

Compute

Compute: total probability

1 / 3

Story problem — figure out which numbers are the formula inputs, then compute.

50% of shoppers today are first-time visitors; each of them buys with probability 0.2. Returning shoppers buy with probability 0.1. What’s the overall purchase probability for a random shopper? Round to 2 decimals.

Enter a number (at most 2 decimal places).

Counting: permutations vs combinations

Counting shows up as “how many ways…?” or as the denominator/numerator of a probability (“four cards of the same suit”).

• Order matters → permutations:

  $$P(n,k) = \frac{n!}{(n-k)!}$$

  Example: password character sequences.

• Order does not matter → combinations:

  $$\binom{n}{k} = \frac{n!}{k!(n-k)!}$$

  Example: choosing a set of restaurants from a map.

First decide whether order matters, then apply the right formula. The same ideas appear in combinatorial and graph problems.

Quiz

Counting

1 / 5

Order the usual interview checklist for a counting problem (top = first).

OrderDrag the handle to rearrange — top should be first.

1. ⋮⋮1Decide whether order / labels matter
2. ⋮⋮2Apply P(n,k) or C(n,k) (or multiply choices)
3. ⋮⋮3Read the story (password, committee, hands, …)

Drag items into the correct order (or use ↑↓), then check.

Compute

Compute: permutations & combinations

1 / 3

Story problem — figure out which numbers are the formula inputs, then compute.

A map shows 12 nearby restaurants. You want to pick 3 of them for a weekend crawl (order of the set doesn’t matter). How many choices? (Integer)

Enter a number (at most 0 decimal places).

Random variables, PMF, PDF, and CDF

A random variable is a numeric outcome with a distribution. It is discrete (countable values) or continuous (uncountable range).

Type	Distribution function	Notation idea
Discrete	Probability mass function (PMF)	$f_X(x) = P(X = x)$
Continuous	Probability density function (PDF)	densities on intervals; $P(X = x) = 0$

Both must be non-negative and normalize:

• Discrete: $\sum_x f_X(x) = 1$
• Continuous: $\int f_X(x)\, dx = 1$

The CDF is $F_X(x) = P(X \le x)$ — a sum of the PMF or an integral of the PDF. It is non-negative and non-decreasing. In interviews, name the right PMF/PDF/CDF before deriving properties.

Quiz

Random variables

1 / 3

A continuous PDF at a single point x gives…

Pick an answer to see if you are right.

Compute

Compute: PMF / CDF

1 / 3

Story problem — figure out which numbers are the formula inputs, then compute.

A fair coin is flipped 7 times. What’s the probability of getting exactly 6 heads? Round to 2 decimals.

Enter a number (at most 2 decimal places).

Joint, marginal, and conditional distributions

When two variables move together, use a joint PMF/PDF. For continuous $(X,Y)$:

$$\iint f_{X,Y}(x,y)\, dx\, dy = 1$$

Marginal of $X$: integrate (or sum) out $Y$:

$$f_X(x) = \int f_{X,Y}(x,y)\, dy$$

Joint CDF: $F_{X,Y}(x,y) = P(X \le x,\, Y \le y)$.

Conditional densities extend Bayes’ idea, e.g. relating $f_X(x)$ to $f_Y(y)$ and $f_{X \mid Y}(x \mid y)$. Deep technical rounds may ask for these; whenever multiple RVs appear, start from the joint.

Quiz

Joint / marginal / conditional

1 / 2

The marginal PDF of X from a joint PDF f_{X,Y} is found by…

Pick an answer to see if you are right.

Compute

Compute: joint → marginal

1 / 3

Story problem — figure out which numbers are the formula inputs, then compute.

In an A/B log, each session is (clicked?, purchased?): P(no click, no buy)=0.35, P(no click, buy)=0.1, P(click, no buy)=0.25, P(click, buy)=0.3. What fraction of sessions had a click? Round to 2 decimals.

Enter a number (at most 2 decimal places).

Discrete distributions: Binomial and Poisson

Interviewers care that you recognize the story and can use the PMF — not that you memorize every identity.

Binomial

$k$ successes in $n$ independent trials, each with success probability $p$:

$$P(X = k) = \binom{n}{k} p^k (1-p)^{n-k}, \quad \mathbb{E}[X] = np, \quad \mathrm{Var}(X) = np(1-p)$$

Stories: coin flips, signups, any binary success/failure count with fixed $n$.

Poisson

Count of events in a fixed interval at constant rate $\lambda$:

$$P(X = k) = e^{-\lambda} \frac{\lambda^k}{k!}, \quad \mathbb{E}[X] = \lambda, \quad \mathrm{Var}(X) = \lambda$$

Stories: visits in an hour, defects per square foot — a rate process, not a fixed number of Bernoulli trials.

Quiz

Binomial & Poisson

1 / 5

Number of heads in 20 fair coin flips is best modeled as…

Pick an answer to see if you are right.

Compute

Compute: Binomial & Poisson

1 / 4

Story problem — figure out which numbers are the formula inputs, then compute.

13 users each independently convert with probability 0.5. What’s the probability that exactly 3 of them convert? Round to 2 decimals.

Enter a number (at most 2 decimal places).

Continuous distributions: Uniform, Exponential, Normal

Uniform$(a,b)$

Constant density on $[a,b]$:

$$f(x) = \frac{1}{b-a}, \quad \mathbb{E}[X] = \frac{a+b}{2}, \quad \mathrm{Var}(X) = \frac{(b-a)^2}{12}$$

Stories: random number generation, some sampling / null setups in testing.

Exponential$(\lambda)$

Waiting time between events of a Poisson process with rate $\lambda$:

$$f(x) = \lambda e^{-\lambda x} \quad (x \ge 0), \quad \mathbb{E}[X] = \frac{1}{\lambda}, \quad \mathrm{Var}(X) = \frac{1}{\lambda^2}$$

Stories: time until purchase, time until default. Famous property: memorylessness — remaining wait does not depend on how long you have already waited.

Normal$(\mu, \sigma^2)$

Bell curve with PDF

$$f(x) = \frac{1}{\sqrt{2\pi}\,\sigma} \exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right)$$

Everywhere in interviews because many phenomena look roughly Normal, and because of the Central Limit Theorem.

Quiz

Uniform, Exponential, Normal

1 / 4

Time until the next customer purchase in a Poisson-arrivals story is typically…

Pick an answer to see if you are right.

Compute

Compute: Uniform, Exponential, Normal

1 / 3

Story problem — figure out which numbers are the formula inputs, then compute.

Support tickets are answered with Exponential waiting times at rate λ=0.5 per hour. What’s P(you wait more than 3 hours)? Round to 2 decimals.

Enter a number (at most 2 decimal places).

Markov chains

A Markov chain has a finite set of states; the next state depends only on the current state (Markov property: past and future are conditionally independent given the present).

Transitions live in a matrix $P$ with $P_{ij} =$ probability of going $i \to j$.

Useful vocabulary:

Term	Meaning
Recurrent state	Once entered, you return with probability 1
Transient state	Positive chance you leave and never return
Stationary distribution $\pi$	$\pi = \pi P$ — long-run fraction of time in each state

Product example: user states new / active / churned, ask about long-run mix. Reach for Markov chains when you have discrete states, transitions, and long-run behavior.

Quiz

Markov chains

1 / 6

The Markov property says that, given the current state…

Pick an answer to see if you are right.

Compute

Compute: Markov chain

1 / 3

Story problem — figure out which numbers are the formula inputs, then compute.

Users are inactive (0) or active (1). Each week, 20% of inactive users become active, and 15% of active users go inactive. In the long run, what fraction are active? Round to 2 decimals.

Enter a number (at most 2 decimal places).

Interview checklist

• Hear “given that” → conditionals / Bayes / base rates
• Hear a tree of scenarios → total probability
• Hear how many ways → permutations vs combinations
• Hear a data-generating story → name Binomial, Poisson, Uniform, Exponential, or Normal
• Hear states and long-run mix → Markov chain + $\pi = \pi P$