End-to-End ML Workflow
Source: Concepts adapted from Ace the Data Science Interview — Chapter 7 (“Machine Learning”) by Kevin Huo & Nick Singh. Rewritten in original wording for this course.
End-to-end ML interview questions test whether you can apply theory to a business problem — not jump straight to “random forest.” Interviewers want clarifying questions, metrics tied to impact, data strategy, cleaning / features, model choice, evaluation, deployment, and iteration. Pair this lesson with case-style product sense practice later.
Step 1 — Clarify the problem & constraints
Half the work is framing. Cover business context, assumptions, and stakeholder concerns before picking an algorithm.
Useful prompts:
- What is the dependent variable / success definition? (e.g. how is churn defined?)
- What is the baseline today, and what beat-the-baseline bar makes the project a win?
- Is ML needed, or do heuristics / rules / humans-in-the-loop suffice?
- Legal / ethical limits? (e.g. lending rules on demographic features)
- How do users consume the output? Standalone vs integrated?
- Clear business value? Who else is affected?
- Cost of wrong predictions? (spam in inbox ≪ approving a bad mortgage)
- One monolith vs decoupled subsystems? (perception / planning / control)
Technical anchors (systems design):
- Latency — autocomplete is useless if slower than typing; maybe inference is real-time but training is batch
- Throughput — predictions per minute
- Deploy target — on-device size / cost (GPU in a car ≠ GPU on a tiny drone)
Ask enough to scope — not 57 toddler questions. Time-boxed interviews reward concise, relevant clarification.
Clarify the problem
1 / 3A strong opening on an open-ended ML question is to…
Pick an answer to see if you are right.
Step 2 — Establish metrics
Pick simple, observable, attributable metrics aligned to the business KPI. Translate model scores into impact (“90% ticket routing accuracy → 10% faster resolution”).
Interview pattern: lead with one primary metric (easier to rank models / align teams), then hedge with secondary metrics or satisficing (e.g. maximize precision subject to recall ≥ 0.95). Blended OEC (overall evaluation criterion) weights trade-offs (FP vs FN costs).
Define success vs baseline — 100% accuracy is rarely the bar; beating today’s process often is.
Metrics
1 / 2Satisficing means…
Pick an answer to see if you are right.
Steps 3–5 — Data sources, EDA, cleaning
Garbage in, garbage out. Beyond internal tables: crowdsourcing, onboarding prompts, purchased data, ethical scraping, labeling vendors. Ask freshness, dictionaries, SME context, collection bias.
Boost hard cases with augmentation / synthesis / simulation (self-driving rare events).
EDA: profile columns (variance, noise, missingness), summary stats, visualize distributions / categories, correlation matrix vs target.
Clean: drop junk / dupes, fix schema mismatches, handle missingness (mean/median, model impute, drop last), treat outliers (remove / truncate / keep — know the cause). Watch multivariate outliers (4-year-old who is 5′ tall).
Data & cleaning
1 / 2Before modeling, missing values should…
Pick an answer to see if you are right.
Step 6 — Feature engineering
Selection (domain-driven subset) + preprocessing (present structure cleanly so models need less heroics).
Quantitative: transforms (log, caps), standardize (z-score) / min-max, binning, PCA-style reduction.
Categorical: one-hot, hashing for high cardinality.
Text (if on résumé): stemming vs lemmatization, stopword filter, bag-of-words / n-grams, embeddings (word2vec / GloVe).
Compute: scaling & splits
1 / 5Min-max scale to [0,1]: (x−min)/(max−min). x=5, min=0, max=10. Round to 2 decimals.
Enter a number (at most 2 decimal places).
Practice: feature transforms
Steps 7–8 — Model selection, training & evaluation
Choose models using constraints: speed, budget, data volume / dimension, categorical vs numeric, explainability.
Cheat-sheet intuition (interview shorthand):
| Need | Lean toward |
|---|---|
| Fast / interpretable baseline | Linear / logistic, Naive Bayes, shallow trees |
| Tabular accuracy | Forests / boosting |
| Huge / unstructured | Neural nets (with cost/latency awareness) |
| Clusters / structure | k-means, hierarchical, DBSCAN, GMM, PCA |
Training talk track: train/val/test, CV, hyperparameter search, class imbalance (undersample / oversample / SMOTE), bias, feature importance, regularization, learning curves. If data is huge, discuss sampling carefully (random vs stratified).
Steps 9–10 — Deployment & iterate
MLOps concepts > specific tools (Airflow / MLflow come and go).
| Mode | When | Watch-outs |
|---|---|---|
| Online | Low latency inference | Cost, caching features, monitoring, redundancy |
| Batch | High throughput / not real-time (many recommenders) | Stale predictions until next run |
| Hybrid | Mix both | Complexity |
Training–serving skew / drift: winter coat recommender in July. Mention retrain cadence, refresh triggers, mix of new vs historical data, logging to catch degradation.
Iterate: error analysis — bucket failure modes, prioritize data/features/model work (more rainy traffic-light images, etc.). Deployment is not the end.
Deploy & iterate
1 / 3Training–serving skew often comes from…
Pick an answer to see if you are right.
Quick synthesis
| Step | Sound bite |
|---|---|
| 1 Clarify | Goals, baseline, constraints, failure costs |
| 2 Metrics | One primary + hedges / satisficing; tie to KPI |
| 3–5 Data | Source creatively; EDA; clean missing/outliers |
| 6 Features | Select + transform (scale, encode, text prep) |
| 7–8 Model | Fit constraints; CV / tune / imbalance |
| 9–10 Ship | Online/batch; drift; retrain; error analysis |
You now have Ace Ch.7’s closing pair: RL vocabulary plus the interview workflow that glues classical ML, deep learning, and business sense together. Next: ML problems — Easy.