Meta interview prep

Leakage occurs when training uses information unavailable at prediction time. Examples include scaling with statistics from the full dataset, using future transactions, or placing near-duplicate users in both splits. Split first, fit preprocessing only on training data, and use temporal or grouped validation when the deployment setting requires it.

What a strong answer covers: A strong answer connects the split to the actual prediction moment and explains why a random split can be misleading.

Follow-up: How would you validate a model predicting a customer's next purchase?

Last reviewed 2026-09-28

Compare training and validation performance using learning curves. Poor performance on both can indicate underfitting, weak features, or optimization problems. Strong training results and weak validation results suggest overfitting or distribution mismatch. Try regularization, simpler models, more representative data, and cross-validation while keeping a final test set untouched.

What a strong answer covers: Mention that label noise, leakage, and a mismatched split can also explain these patterns.

Follow-up: What would you try if both errors remain high as data grows?

Last reviewed 2026-09-28

Read batches or stream observations, retaining a running count and mean. For a new value x at count n, update mean = mean + (x - mean) / n. Use a generator so loading is lazy. Define how missing and invalid values are handled; use a stable algorithm such as Welford's if variance is also needed.

What a strong answer covers: This uses constant auxiliary memory. Discuss floating-point limits and deterministic aggregation for parallel processing.

Follow-up: How would you merge the counts and means of two independent workers?

Last reviewed 2026-09-28

Use a real example: describe the user need, measured quality difference, and operational constraints. Compare latency, interpretability, cost, and maintenance. Explain the decision you owned, the outcome, and what evidence would make you revisit it. If you lack work experience, use a project and be explicit about its scope.

What a strong answer covers: The goal is a defensible decision supported by measurements, not a claim that simple models always win.

Follow-up: Which production metric would trigger a reevaluation?

Last reviewed 2026-09-28

Recent observations may be incorrectly labeled negative because conversions have not happened yet. Define a label window, train on sufficiently mature examples, and evaluate cohorts at comparable maturity. If freshness is essential, consider delay-aware modeling. Monitor the delay distribution and distinguish label delay from actual changes in user behavior.

What a strong answer covers: A training dataset can be temporally fresh but systematically mislabeled. Explain the trade-off between recency and completeness.

Follow-up: How would you compare campaigns whose conversion delays differ?

Last reviewed 2026-09-28

Retrieve candidates from relevant sources, rank with multiple predictions, and apply constraints for quality, diversity, and eligibility. Define user-value metrics with longer-term outcomes, not just clicks. Use offline checks and staged online experiments, logging exposures and monitoring groups for regressions.

What a strong answer covers: Different objectives can conflict. State the trade-offs and how weights or constraints would be selected.

Follow-up: How would you measure whether diverse recommendations improve the experience?

Last reviewed 2026-09-28

Confirm the randomization unit, exposure logging, sample ratio, and preselected metric. Check power, confidence intervals, novelty effects, multiple testing, and guardrails. Account for interference between users when applicable. Avoid repeatedly stopping at the first significant result unless the test design supports sequential analysis.

What a strong answer covers: Statistical significance does not establish practical value or rule out instrumentation errors.

Follow-up: When would you randomize by household or geographic area rather than user?

Last reviewed 2026-09-28

Predicting the majority class can achieve high accuracy while missing almost every positive. Inspect precision, recall, the precision-recall curve, calibration, and errors at the operating threshold. Choose the threshold based on the cost of misses and false alarms. Keep validation prevalence representative even when training uses resampling.

What a strong answer covers: A good answer connects the metric to a concrete decision and avoids using a universal threshold.

Follow-up: What happens to precision if the positive rate drops after deployment?

Last reviewed 2026-09-28

Track availability, latency, resource cost, input quality, feature distributions, and output distributions. When labels arrive, monitor outcome quality and calibration by segment. Define alert thresholds, ownership, and rollback paths. Investigate drift before retraining because a distribution change does not always imply degraded performance.

What a strong answer covers: Monitoring must connect to an action. Labels may be delayed, so proxy signals need explicit limitations.

Follow-up: How would you distinguish a broken data pipeline from genuine behavior change?

Last reviewed 2026-09-28

Attention computes a weighted sum of values, where weights come from the dot product of queries and keys passed through softmax. Dividing by the square root of the key dimension prevents the dot products from growing large when the dimension is high, which would push softmax into regions with very small gradients. Multi-head attention runs this in parallel across learned projection subspaces so the model can attend to different aspects of the input simultaneously.

What a strong answer covers: A strong answer explains the gradient issue numerically, not just verbally, and connects multi-head design to representational diversity.

Follow-up: What changes when you replace full self-attention with a sparse or linear approximation?

Last reviewed 2026-10-07

Human annotators rank model outputs; a reward model is trained to predict those preferences. A policy is then fine-tuned with reinforcement learning to maximize predicted reward while a KL penalty keeps it close to the original model. The reward model can overfit to annotator biases, be gamed through reward hacking, or fail to generalise to novel outputs. Distribution shift between training and deployment makes reward model reliability difficult to guarantee.

What a strong answer covers: Cover both the training loop and the failure modes. Reward hacking and annotation disagreement are the two most cited weaknesses interviewers probe.

Follow-up: What would you do if annotators systematically preferred verbose but less accurate answers?

Last reviewed 2026-10-07

Adam adapts learning rates per parameter using first and second moment estimates, making it a good default for most networks. SGD with momentum can generalise better on some vision tasks and is preferred when fine-tuning large pretrained models. AdaGrad accumulates all past squared gradients and effectively reduces learning rates to zero for dense features, so it is more suited to sparse settings. Monitor the training loss curve, gradient norms, and validation performance rather than choosing blindly.

What a strong answer covers: Distinguish the update rule from the generalization property. A strong answer mentions the flat-minimum hypothesis linking SGD to better generalization on some tasks.

Follow-up: How does weight decay interact differently with Adam versus SGD?

Last reviewed 2026-10-07

Separate an offline store for batch-computed features used in training from an online store for low-latency serving. Maintain a registry of feature definitions, versioning, and ownership. Compute features once and reuse them to avoid training-serving skew. Include point-in-time correct joins for training to prevent leakage. Define freshness SLAs per feature, monitor staleness, and plan for backfills when a feature definition changes.

What a strong answer covers: Training-serving skew is the most critical concern. A strong answer also mentions cost: materialising all features is expensive and some are better computed on the fly.

Follow-up: How would you handle a feature that must be computed from raw events arriving out of order?

Last reviewed 2026-10-07

Consider quasi-experimental methods: difference-in-differences if parallel trends hold before the intervention, regression discontinuity if assignment depends on a threshold, instrumental variables if a valid instrument exists, or synthetic control for aggregate outcomes. Each method requires explicit assumptions; state them and describe how you would test them. Compare estimates across methods as a robustness check and be explicit about the effect you can actually identify versus the one the business wants.

What a strong answer covers: The answer should name at least two methods and articulate the identifying assumption for each. Claiming that observational data can recover causal effects without assumptions is a red flag.

Follow-up: How would you test the parallel trends assumption for a difference-in-differences analysis?

Last reviewed 2026-10-07

Ensembles reduce variance by averaging independent errors, so they help most when individual models overfit or are unstable. Bagging trains models on bootstrap samples; random forests add feature subsampling to decorrelate trees. Boosting reduces bias by sequentially fitting residuals. Stacking learns to combine diverse base models. Gains diminish when base learners are highly correlated or when the dominant error is bias rather than variance.

What a strong answer covers: Distinguish when to reach for an ensemble versus regularising a single model. Mention that diversity among base models is essential and that ensembles add latency and maintenance cost.

Follow-up: How would you measure whether a new model adds value to an existing ensemble?

Last reviewed 2026-10-07

Stream interaction events through a reliable pipeline, apply deduplication and delay handling, and compute features with point-in-time correctness. Update the model on mini-batches, applying a learning rate schedule that prevents catastrophic forgetting of older patterns. Shadow-deploy the updated model, compare offline and online metrics, and gate full rollout on guardrails. Keep a stable fallback version and log both model versions for retrospective analysis.

What a strong answer covers: The hardest parts are label delay and catastrophic forgetting, not the update mechanism itself. A strong answer addresses both and defines what "fresh" means for the specific use case.

Follow-up: How would you detect when online updates degrade performance on an older cohort of users?

Last reviewed 2026-10-07

L2 adds the sum of squared weights to the loss, shrinking all weights toward zero proportionally. L1 adds the sum of absolute weights and can set some weights exactly to zero, producing sparse models. Use L2 when you believe most features contribute and want smooth shrinkage. Use L1 when you expect many irrelevant features and want automatic feature selection. Elastic net combines both. Regularization strength is a hyperparameter tuned by cross-validation.

What a strong answer covers: The key distinction is sparsity: L1 produces it, L2 does not. Explain why in terms of the geometry of the constraint region.

Follow-up: Why might L1 regularization still leave many small but non-zero weights in practice?

Last reviewed 2026-10-07

A list comprehension evaluates immediately and stores all values in memory. A generator expression or generator function uses yield to produce values one at a time and holds only the current state in memory. Use a generator when the sequence is large, infinite, or expensive to compute, and you only need to iterate once. Use a list when you need random access, repeated iteration, or the full collection for operations such as sorting or length checks.

What a strong answer covers: The core concept is lazy evaluation. Mention that generators are exhausted after one pass, which is a common source of bugs when the same generator is iterated twice.

Follow-up: How would you implement a generator that yields batches of rows from a large CSV file?

Last reviewed 2026-10-07

Use the STAR format: describe the context and the decision you disagreed with, the specific concern you raised and how you raised it (data, an experiment, or a clear argument), the outcome, and what you would do differently. Show that you can advocate for your position while remaining constructive, and that you can commit to a decision once it has been made even when you disagree.

What a strong answer covers: The goal is to show technical judgment and professional communication, not to relitigate old grievances. Avoid framing the story as you versus them. A strong answer includes evidence you used to make your case.

Follow-up: What would you do if the team made the same decision again and the outcome was worse than you predicted?

Last reviewed 2026-10-07

Use STAR: explain the project goal, what you built, what happened, and what you did next. Own your share of the failure rather than attributing it entirely to data, compute, or other constraints. Name a specific technical mistake — a leaky split, a flawed evaluation metric, a mislabeled dataset — not a vague sense that the model was wrong. Describe what you learned and how you applied it on a subsequent project.

What a strong answer covers: A strong answer names a specific technical root cause, not bad luck or insufficient time. The retrospective action matters as much as the failure itself.

Follow-up: What would be the first thing you would investigate if you had three days to debug the same project now?

Last reviewed 2026-10-07

Use STAR: describe the decision, what information was missing, what assumptions you made and why, how you documented those assumptions, and what you did to limit downside risk. Show that you can move forward without perfect information while keeping the team informed about your uncertainty and the conditions that would change your decision.

What a strong answer covers: This tests judgment and risk management. A strong answer distinguishes between uncertainty that is worth waiting to resolve and uncertainty you can safely assume away for now.

Follow-up: How did you decide what additional information was worth waiting for versus what you could assume away?

Last reviewed 2026-10-07

Use STAR: set the scene, describe the failure and how you found it, and explain how you communicated it. Lead with the business impact first, then explain the cause at a level the audience can act on. Show that you gave them what they needed to make a decision rather than a full technical post-mortem. Include what you proposed to fix and what the outcome was.

What a strong answer covers: The best answers show empathy for what the stakeholder needs to know, not just what you know. Avoid jargon without explanation and avoid false reassurance about a fix that is not yet validated.

Follow-up: How would you set up a review process to catch similar issues before they reach stakeholders?

Last reviewed 2026-10-07

Use STAR: describe the problem you noticed, why it was not clearly owned, what you chose to do, how you involved or informed the right people, and the outcome. Show judgment about when to escalate versus act, and demonstrate that you understood the constraints and worked within or around them thoughtfully rather than bypassing important processes.

What a strong answer covers: This tests initiative and judgment. Interviewers probe whether you involved the right people at the right time. A strong answer shows you didn't silently take over someone else's work.

Follow-up: How did you decide this was your problem to solve rather than someone else's?

Last reviewed 2026-10-07

Compute the dot product of the two vectors divided by the product of their L2 norms. Handle the zero-norm edge case by returning 0.0 to avoid division by zero. Raise a ValueError if the vectors have different lengths.

def cosine_similarity(a: list[float], b: list[float]) -> float:
    if len(a) != len(b):
        raise ValueError("Vectors must have equal length")
    dot = sum(x * y for x, y in zip(a, b))
    norm_a = sum(x ** 2 for x in a) ** 0.5
    norm_b = sum(x ** 2 for x in b) ** 0.5
    if norm_a == 0.0 or norm_b == 0.0:
        return 0.0
    return dot / (norm_a * norm_b)

What a strong answer covers: The implementation should handle equal-length validation and the zero-norm case without importing math. Mention that floating-point precision can push results slightly outside [-1, 1] for normalised vectors.

Follow-up: How would you adapt this to compute all pairwise similarities for a matrix of N embeddings efficiently?

Last reviewed 2026-10-07

Use Welford's online algorithm, which updates the mean and the sum of squared deviations incrementally without storing all values. The variance property returns the population variance; divide by n-1 for sample variance.

class RunningStats:
    def __init__(self) -> None:
        self._n = 0
        self._mean = 0.0
        self._M2 = 0.0   # sum of squared deviations from current mean

    def update(self, x: float) -> None:
        self._n += 1
        delta = x - self._mean
        self._mean += delta / self._n
        self._M2 += delta * (x - self._mean)  # uses updated mean

    @property
    def mean(self) -> float:
        return self._mean

    @property
    def variance(self) -> float:          # population variance
        return self._M2 / self._n if self._n > 0 else 0.0

    @property
    def count(self) -> int:
        return self._n

What a strong answer covers: Welford's algorithm is numerically stable and O(1) in memory. The key is using the updated mean in the M2 accumulation step. Mention that two RunningStats instances can be merged by combining their n, mean, and M2 fields.

Follow-up: How would you merge the running statistics from two independent workers processing different data shards?

Last reviewed 2026-10-07

Find each user's first-seen date, group users into weekly cohorts, then count how many returned within 30 days. Divide retained users by cohort size to get the retention rate.

-- Table: events(user_id BIGINT, event_date DATE)
WITH first_seen AS (
    SELECT user_id, MIN(event_date) AS cohort_date
    FROM events
    GROUP BY user_id
),
cohort_size AS (
    SELECT
        DATE_TRUNC('week', cohort_date) AS cohort_week,
        COUNT(DISTINCT user_id)          AS total_users
    FROM first_seen
    GROUP BY 1
),
retained AS (
    SELECT
        DATE_TRUNC('week', f.cohort_date) AS cohort_week,
        COUNT(DISTINCT e.user_id)          AS retained_users
    FROM first_seen f
    JOIN events e
      ON  e.user_id    = f.user_id
      AND e.event_date >  f.cohort_date
      AND e.event_date <= f.cohort_date + INTERVAL '30 days'
    GROUP BY 1
)
SELECT
    c.cohort_week,
    c.total_users,
    COALESCE(r.retained_users, 0)                                    AS retained_users,
    ROUND(COALESCE(r.retained_users, 0)::numeric / c.total_users, 3) AS retention_rate
FROM cohort_size  c
LEFT JOIN retained r USING (cohort_week)
ORDER BY c.cohort_week;

What a strong answer covers: COALESCE handles cohorts with zero retention. The strict inequality on event_date excludes the sign-up day itself from the retained count. Clarify with the interviewer whether returning on the same day should count.

Follow-up: How would you extend this to compute 7-day, 30-day, and 90-day retention in a single query?

Last reviewed 2026-10-07

Separate data validation, training, evaluation, and deployment into distinct reproducible stages. Gate each stage: fail if data quality checks fail, if evaluation metrics fall below a threshold, or if the model regresses on a golden dataset. Version models, datasets, and configurations together. Use a model registry to promote from staging to production. Run integration tests against a staging environment before promoting, and support rollback by keeping the previous version active alongside the new one.

What a strong answer covers: An ML CI/CD pipeline is more complex than software CI/CD because training is non-deterministic and evaluation requires agreement on what "better" means. Mention the need to test code changes and data changes as separate triggers.

Follow-up: How would you handle a training run that is too expensive to run on every commit?

Last reviewed 2026-10-07

Keep the previous model version live and route a small traffic slice to it even after promoting the new version. Define alert thresholds on business metrics, error rates, and output distributions that trigger rollback or human review. Store model artifacts, preprocessing code, and feature definitions together so a rollback is fully reproducible. After rolling back, diagnose the regression on offline data before redeploying.

What a strong answer covers: A rollback that restores the model but not the feature store or preprocessing logic often reproduces the same problem. Everything the model depends on must be versioned together.

Follow-up: What single metric would you monitor to decide within 30 minutes whether a new model needs to be rolled back?

Last reviewed 2026-10-07

Shadow deployment routes a copy of live traffic to the new model but only the existing model's predictions reach users. Canary deployment serves a small percentage of users with the new model's actual predictions. Use shadow deployment to validate predictions and latency with zero user risk. Use canary deployment once shadow results are acceptable and you need to measure business metrics that require real user responses. Both require careful traffic allocation to avoid biasing the evaluation.

What a strong answer covers: Shadow deployment cannot measure user behaviour effects because it never changes the response. Mention that shadow deployment adds latency overhead on the mirrored path, and that canary deployment requires guarding against novelty effects.

Follow-up: How would you decide when to ramp a canary from 1% to 10% of production traffic?

Last reviewed 2026-10-07

Define a proxy for long-term value: subscription retention, weekly active days, content satisfaction ratings, or a composite. Train a model to predict this metric rather than next-click engagement. Use offline simulation to check that optimising the proxy actually changes the long-term outcome, and run online experiments with a measurement window long enough to observe retention effects. Instrument the system to log counterfactuals so you can estimate long-term effects from historical data.

What a strong answer covers: Short-term engagement optimises for what is clickable, which can surface addictive or low-quality content. The hardest part is defining and measuring long-term value before you can optimise for it.

Follow-up: How would you measure whether a recommendation change improved long-term satisfaction in an A/B test that must conclude in two weeks?

Last reviewed 2026-10-07