Reinforcement Learning

Source: Concepts adapted from Ace the Data Science Interview — Chapter 7 (“Machine Learning”) by Kevin Huo & Nick Singh. Rewritten in original wording for this course.

Reinforcement learning (RL) sits outside supervised and unsupervised learning. An agent takes actions in an environment, visits states, and receives rewards. The goal is to learn a behavior that maximizes long-term return — not to imitate labeled “correct” outputs.

Famous use cases: games (AlphaGo, chess, StarCraft) and robotics. Prefer RL when the problem is decisions over time and you do not already know the right action labels (supervised assumes you do). Unless RL is on your résumé, most DS interviews will not drill it deeply — still know the vocabulary.

Core components

Piece	Role
Reward function	Scores good vs bad outcomes in a state (defines the goal)
Policy	Maps states → actions (how the agent chooses)
Model	Predicts next state / reward given (state, action) — optional; model-free methods skip an explicit dynamics model
Value function	Predicts expected discounted future return from a state (or state–action); re-estimated as the agent learns

Learning loop: act → observe reward & next state → update policy / values → repeat.

Quiz

RL basics

1 / 4

RL differs from supervised learning mainly because…

Pick an answer to see if you are right.

Discounting, exploration, Q-learning

Return with discount $\gamma\in[0,1]$:

$$G_t = r_t + \gamma r_{t+1} + \gamma^2 r_{t+2} + \cdots$$

One-step Bellman sketch: $V(s) \approx r + \gamma V(s')$.

Q-learning (model-free) updates action-values:

$$Q(s,a) \leftarrow Q(s,a) + \alpha\big(r + \gamma \max_{a'} Q(s',a') - Q(s,a)\big)$$

ε-greedy: with probability $1-\varepsilon$ pick $\arg\max_a Q(s,a)$; otherwise explore randomly — because good actions are not labeled up front.

Quiz

Discounting & Q-learning

1 / 3

γ close to 0 makes the agent…

Pick an answer to see if you are right.

Compute

Compute: returns, Bellman, Q-update

1 / 5

Story problem — figure out which numbers are the formula inputs, then compute.

Discounted return G=Σ γ^k r_k for rewards [1, 1, 1] and γ=0.5. Compute G (process from the end). Round to 2 decimals.

Enter a number (at most 2 decimal places).

Practice: RL arithmetic

Interview note

When RL shows up	What to say
Gaming / robotics / sequential control	Agent, reward, policy, value, explore/exploit
Typical product DS	Prefer supervised / unsupervised unless RL is in scope
Contrast	Supervised = labeled y; RL = learn actions from reward

Next: End-to-end ML workflow — how interviews expect you to walk a business problem from clarification to deployment.