Statistics

Source: Concepts adapted from Ace the Data Science Interview — Chapter 6 (“Statistics”) by Kevin Huo & Nick Singh. Rewritten in original wording for this course.

Statistics is the language of A/B tests, uncertainty, and “is this lift real?” Interviewers start with LLN/CLT, then p-values, confidence intervals, Type I/II errors, and sometimes MLE/MAP. Each section below has an explorer, a concept quiz, and a compute story quiz (≤2 decimals; use New story).

Expectation, variance, and standard deviation

For a random variable $X$ with density/mass $f$:

$$\mathbb{E}[X] = \int x\, f(x)\, dx \quad\text{(or }\sum x f(x)\text{ if discrete)}$$ $$\mathrm{Var}(X) = \mathbb{E}[(X-\mathbb{E}[X])^2] = \mathbb{E}[X^2] - (\mathbb{E}[X])^2 \ge 0$$

Standard deviation $\sigma = \sqrt{\mathrm{Var}(X)}$. Conditional expectation $\mathbb{E}[X \mid Y=y]$ uses the conditional density. Covariance and correlation:

$$\mathrm{Cov}(X,Y)=\mathbb{E}[XY]-\mathbb{E}[X]\mathbb{E}[Y], \qquad \rho = \frac{\mathrm{Cov}(X,Y)}{\sigma_X \sigma_Y} \in [-1,1]$$

Quiz

Moments & correlation

1 / 4

Var(X) is always…

Pick an answer to see if you are right.

Compute

Compute: expectation & variance

1 / 3

Story problem — figure out which numbers are the formula inputs, then compute.

A user converts (X=1) with probability 0.2, else X=0. What is Var(X)? Round to 2 decimals.

Enter a number (at most 2 decimal places).

Compute

Compute: correlation

1 / 2

Story problem — figure out which numbers are the formula inputs, then compute.

Cov(X,Y)=1, SD(X)=4, SD(Y)=4. What is the correlation ρ? Round to 2 decimals.

Enter a number (at most 2 decimal places).

Uniform moments (interview derivation target)

If $X \sim \mathrm{Uniform}[a,b]$:

$$\mathbb{E}[X]=\frac{a+b}{2}, \qquad \mathrm{Var}(X)=\frac{(b-a)^2}{12}$$

You should be able to re-derive these from the PDF $1/(b-a)$ when asked.

Quiz

Uniform moments

1 / 2

For Uniform(0, 6), E[X] equals…

Pick an answer to see if you are right.

Compute

Compute: Uniform E / Var

1 / 3

Story problem — figure out which numbers are the formula inputs, then compute.

X ~ Uniform(2, 11). What is Var(X)? Round to 2 decimals.

Enter a number (at most 2 decimal places).

Law of Large Numbers

Independently sample $X_1,\ldots,X_n$ from the same distribution. The sample mean $\bar X_n$ converges to $\mathbb{E}[X]$ as $n\to\infty$. Short streaks (five heads) do not contradict LLN — long-run averages do.

Quiz

Law of Large Numbers

1 / 3

LLN says that as n grows, the sample mean…

Pick an answer to see if you are right.

Central Limit Theorem

For large $n$, $\bar X_n$ is approximately Normal with mean $\mu$ and variance $\sigma^2/n$, even if $X$ itself is not Normal:

$$\frac{\bar X_n - \mu}{\sigma/\sqrt{n}} \approx N(0,1)$$

CLT underwrites most z-tests and A/B proportion tests.

Quiz

Central Limit Theorem

1 / 4

CLT is about the distribution of…

Pick an answer to see if you are right.

Compute

Compute: SE and z

1 / 3

Story problem — figure out which numbers are the formula inputs, then compute.

Scores have σ=5. You average n=64 independent scores. What is the standard error of the sample mean (σ/√n)? Round to 2 decimals.

Enter a number (at most 2 decimal places).

Hypothesis testing

1. State $H_0$ (baseline) and $H_1$ (what you hope to support).
2. Compute a test statistic under $H_0$ → p-value.
3. Compare to significance level $\alpha$. Reject $H_0$ if $p < \alpha$. Failing to reject is not proof $H_0$ is true.

One-sided: $H_0:\mu=\mu_0$ vs $H_1:\mu>\mu_0$ (or $<$).
Two-sided: $H_1:\mu \ne \mu_0$.

A/B testing is hypothesis testing on product metrics (e.g. equal conversion rates vs uplift).

Test statistics

Test	Use when
Z	Large $n$ (CLT) or known $\sigma$
t	Small $n$, $\sigma$ unknown (use $s$, df $=n-1$)
χ²	Goodness of fit / independence in contingency tables

$$z = \frac{\bar x - \mu_0}{\sigma/\sqrt{n}}, \qquad t = \frac{\bar x - \mu_0}{s/\sqrt{n}}$$

Proportions and A/B

Members are Bernoulli; $\hat p$ is a mean. Under $H_0:p=p_0$:

$$z = \frac{\hat p - p_0}{\sqrt{p_0(1-p_0)/n}}$$

Two-sample A/B uses a pooled SE for $\hat p_B - \hat p_A$. Always name the statistic and that CLT justifies Normality for large $n$.

Quiz

Hypothesis tests

1 / 5

Order the usual hypothesis-testing workflow (top = first).

OrderDrag the handle to rearrange — top should be first.

1. ⋮⋮1Compare p to α; reject or fail to reject H₀
2. ⋮⋮2State H₀ and H₁
3. ⋮⋮3Compute the test statistic and p-value under H₀

Drag items into the correct order (or use ↑↓), then check.

Compute

Compute: χ²

1 / 2

Story problem — figure out which numbers are the formula inputs, then compute.

Under H₀ three categories are equally likely. Observed counts are 25, 40, 22. What is χ² = Σ(O−E)²/E? Round to 2 decimals.

Enter a number (at most 2 decimal places).

Compute

Compute: A/B z

1 / 3

Story problem — figure out which numbers are the formula inputs, then compute.

A/B test: control 150/1000 conversions, treatment 190/1000. Using the pooled two-proportion z-statistic, what is z? Round to 2 decimals.

Enter a number (at most 2 decimal places).

p-values and confidence intervals

A p-value is the probability, under $H_0$, of a test statistic at least as extreme as observed. Small $p$ → data are surprising under $H_0$.

A $(1-\alpha)$ confidence interval is a random interval that covers the true parameter in $(1-\alpha)$ of repeated samples. For a mean with known $\sigma$:

$$\bar x \pm z_{\alpha/2}\cdot \frac{\sigma}{\sqrt{n}}$$

For a proportion: $\hat p \pm z_{\alpha/2}\sqrt{\hat p(1-\hat p)/n}$. If the null value (e.g. 0 lift) lies inside the CI, you typically do not reject $H_0$.

Quiz

p-values & CIs

1 / 3

A p-value is computed assuming…

Pick an answer to see if you are right.

Compute

Compute: CI endpoint

1 / 3

Story problem — figure out which numbers are the formula inputs, then compute.

x̄=10, σ=4, n=49. What is the lower endpoint of a 95% CI for μ (use z=1.96)? Round to 2 decimals.

Enter a number (at most 2 decimal places).

Type I / II errors, power, Bonferroni

	Reject $H_0$	Fail to reject
$H_0$ true	Type I (false positive), rate $\alpha$	Correct
$H_0$ false	Correct (power $1-\beta$)	Type II (false negative), rate $\beta$

Larger $n$ generally raises power. For $m$ tests at family-wise $\alpha$, Bonferroni uses per-test $\alpha/m$ so you do not expect $m\alpha$ false positives by chance.

Quiz

Errors & multiple testing

1 / 5

Type I error means…

Pick an answer to see if you are right.

Compute

Compute: Bonferroni & power

1 / 3

Story problem — figure out which numbers are the formula inputs, then compute.

A test has Type II error rate β=0.3. What is its power (1−β)? Round to 2 decimals.

Enter a number (at most 2 decimal places).

MLE and MAP

MLE: $\hat\theta = \arg\max_\theta L(\theta)$ with $L(\theta)=\prod_i f(x_i\mid\theta)$. Work with $\log L = \sum \log f(x_i\mid\theta)$.

MAP: maximize prior × likelihood: $\arg\max_\theta g(\theta)\, f(\mathbf{x}\mid\theta)$.

Classic drill: Bernoulli MLE is $\hat p = k/n$; Normal mean MLE is the sample mean.

Quiz

MLE & MAP

1 / 2

We maximize log-likelihood because…

Pick an answer to see if you are right.

Compute

Compute: Bernoulli MLE

1 / 3

Story problem — figure out which numbers are the formula inputs, then compute.

You observe 13 successes in 20 i.i.d. Bernoulli trials. What is the MLE of p? Round to 2 decimals.

Enter a number (at most 2 decimal places).

Interview checklist

• Moments: $\mathbb{E}$, Var, $\sigma$, Cov, $\rho$
• Uniform (and friends): re-derive $\mathbb{E}$ / Var when asked
• LLN → long-run averages; CLT → $\bar X$ ≈ Normal, SE $=\sigma/\sqrt{n}$
• Hypothesis test steps; Z vs t vs χ²; A/B = proportion z + CLT
• Explain p-value and CI in plain language
• Type I/II, power, Bonferroni for many metrics
• MLE (log-likelihood) vs MAP (prior)