Google interview prep

Leakage occurs when training uses information unavailable at prediction time. Examples include scaling with statistics from the full dataset, using future transactions, or placing near-duplicate users in both splits. Split first, fit preprocessing only on training data, and use temporal or grouped validation when the deployment setting requires it.

What a strong answer covers: A strong answer connects the split to the actual prediction moment and explains why a random split can be misleading.

Follow-up: How would you validate a model predicting a customer's next purchase?

Last reviewed 2026-09-28

Compare training and validation performance using learning curves. Poor performance on both can indicate underfitting, weak features, or optimization problems. Strong training results and weak validation results suggest overfitting or distribution mismatch. Try regularization, simpler models, more representative data, and cross-validation while keeping a final test set untouched.

What a strong answer covers: Mention that label noise, leakage, and a mismatched split can also explain these patterns.

Follow-up: What would you try if both errors remain high as data grows?

Last reviewed 2026-09-28

Read batches or stream observations, retaining a running count and mean. For a new value x at count n, update mean = mean + (x - mean) / n. Use a generator so loading is lazy. Define how missing and invalid values are handled; use a stable algorithm such as Welford's if variance is also needed.

What a strong answer covers: This uses constant auxiliary memory. Discuss floating-point limits and deterministic aggregation for parallel processing.

Follow-up: How would you merge the counts and means of two independent workers?

Last reviewed 2026-09-28

Use a real example: describe the user need, measured quality difference, and operational constraints. Compare latency, interpretability, cost, and maintenance. Explain the decision you owned, the outcome, and what evidence would make you revisit it. If you lack work experience, use a project and be explicit about its scope.

What a strong answer covers: The goal is a defensible decision supported by measurements, not a claim that simple models always win.

Follow-up: Which production metric would trigger a reevaluation?

Last reviewed 2026-09-28

Clarify relevance, latency, freshness, and language requirements. Use a fast retrieval stage to select candidates, then a richer ranker to score them. Build training examples from relevance judgments and carefully debiased interaction data. Evaluate retrieval recall and ranking quality separately, then run an online experiment with guardrails for latency and user satisfaction.

What a strong answer covers: Candidate retrieval limits what the ranker can recover. Include cold-start queries and monitoring for index freshness.

Follow-up: How would you handle position bias in click data?

Last reviewed 2026-09-28

Lexical search can retrieve exact identifiers and rare terms; vector search captures semantic similarity. Run both retrieval methods, combine candidates with a method such as reciprocal rank fusion, then optionally rerank. Evaluate representative query types and inspect failures before adding complexity.

What a strong answer covers: Score scales are not automatically comparable. Rank fusion can avoid relying on raw score equivalence.

Follow-up: How would you test whether hybrid search improves exact product-code queries?

Last reviewed 2026-09-28

Separate document ingestion and indexing from model retraining. Monitor indexing delays, failed ingestion, query drift, and relevance by content age. Refresh embeddings when necessary, version indexes alongside models, and support rollback. Include fresh-document queries in evaluation and use a deployment strategy that limits inconsistent indexes.

What a strong answer covers: Fresh content and a new ranking model are different update problems, with different schedules and failure modes.

Follow-up: What happens when the embedding model's dimension changes?

Last reviewed 2026-09-28

Use a labeled set with expected evidence. Check whether retrieval includes that evidence, then test generation with known-good context. Measure retrieval recall separately from answer correctness and citation support. Inspect chunk boundaries, ambiguous queries, and unanswerable questions, and compare changes on a held-out set.

What a strong answer covers: End-to-end answer accuracy alone cannot locate the failure. Controlled context experiments isolate the components.

Follow-up: What metric would you use when several documents can correctly support an answer?

Last reviewed 2026-09-28

Confirm the randomization unit, exposure logging, sample ratio, and preselected metric. Check power, confidence intervals, novelty effects, multiple testing, and guardrails. Account for interference between users when applicable. Avoid repeatedly stopping at the first significant result unless the test design supports sequential analysis.

What a strong answer covers: Statistical significance does not establish practical value or rule out instrumentation errors.

Follow-up: When would you randomize by household or geographic area rather than user?

Last reviewed 2026-09-28

Track availability, latency, resource cost, input quality, feature distributions, and output distributions. When labels arrive, monitor outcome quality and calibration by segment. Define alert thresholds, ownership, and rollback paths. Investigate drift before retraining because a distribution change does not always imply degraded performance.

What a strong answer covers: Monitoring must connect to an action. Labels may be delayed, so proxy signals need explicit limitations.

Follow-up: How would you distinguish a broken data pipeline from genuine behavior change?

Last reviewed 2026-09-28

Attention computes a weighted sum of values, where weights come from the dot product of queries and keys passed through softmax. Dividing by the square root of the key dimension prevents the dot products from growing large when the dimension is high, which would push softmax into regions with very small gradients. Multi-head attention runs this in parallel across learned projection subspaces so the model can attend to different aspects of the input simultaneously.

What a strong answer covers: A strong answer explains the gradient issue numerically, not just verbally, and connects multi-head design to representational diversity.

Follow-up: What changes when you replace full self-attention with a sparse or linear approximation?

Last reviewed 2026-10-07

Fine-tune when prompt engineering cannot reliably elicit the required format, domain vocabulary, or task behavior, or when latency and cost make large in-context examples impractical. Parameter-efficient methods such as LoRA adapt the model with far fewer trainable parameters. Use a representative evaluation set to confirm improvement over a well-crafted prompt before committing to the training overhead. Consider catastrophic forgetting, data quality, and whether the base model license permits fine-tuning.

What a strong answer covers: Mention that fine-tuning is a cost with diminishing returns; a good answer weighs it against the quality ceiling of prompting for the specific task.

Follow-up: How would you decide how much of the model to unfreeze for a domain-adaptation task?

Last reviewed 2026-10-07

Human annotators rank model outputs; a reward model is trained to predict those preferences. A policy is then fine-tuned with reinforcement learning to maximize predicted reward while a KL penalty keeps it close to the original model. The reward model can overfit to annotator biases, be gamed through reward hacking, or fail to generalise to novel outputs. Distribution shift between training and deployment makes reward model reliability difficult to guarantee.

What a strong answer covers: Cover both the training loop and the failure modes. Reward hacking and annotation disagreement are the two most cited weaknesses interviewers probe.

Follow-up: What would you do if annotators systematically preferred verbose but less accurate answers?

Last reviewed 2026-10-07

Adam adapts learning rates per parameter using first and second moment estimates, making it a good default for most networks. SGD with momentum can generalise better on some vision tasks and is preferred when fine-tuning large pretrained models. AdaGrad accumulates all past squared gradients and effectively reduces learning rates to zero for dense features, so it is more suited to sparse settings. Monitor the training loss curve, gradient norms, and validation performance rather than choosing blindly.

What a strong answer covers: Distinguish the update rule from the generalization property. A strong answer mentions the flat-minimum hypothesis linking SGD to better generalization on some tasks.

Follow-up: How does weight decay interact differently with Adam versus SGD?

Last reviewed 2026-10-07

Separate an offline store for batch-computed features used in training from an online store for low-latency serving. Maintain a registry of feature definitions, versioning, and ownership. Compute features once and reuse them to avoid training-serving skew. Include point-in-time correct joins for training to prevent leakage. Define freshness SLAs per feature, monitor staleness, and plan for backfills when a feature definition changes.

What a strong answer covers: Training-serving skew is the most critical concern. A strong answer also mentions cost: materialising all features is expensive and some are better computed on the fly.

Follow-up: How would you handle a feature that must be computed from raw events arriving out of order?

Last reviewed 2026-10-07

Data parallelism replicates the model across devices and splits the batch; gradients are aggregated at each step. It is simpler but requires the full model to fit on each device and incurs gradient synchronisation cost. Model parallelism splits layers or tensors across devices and is necessary when the model exceeds a single device memory, but introduces pipeline bubbles or recomputation overhead. Tensor parallelism can split individual weight matrices. Choose based on model size, batch size, communication bandwidth, and hardware topology.

What a strong answer covers: Cover both the memory constraint that forces model parallelism and the communication overhead that limits scaling. Mention pipeline parallelism as a hybrid approach for very deep models.

Follow-up: How would gradient checkpointing change your analysis of memory versus compute trade-offs?

Last reviewed 2026-10-07

Fixed-size overlapping chunks are simple and predictable but can split sentences. Sentence or paragraph boundaries respect natural structure. Semantic chunking groups text by embedding similarity. Recursive splitting tries larger units first and falls back to smaller ones. The right size depends on the embedding model context window, the typical query length, and whether answers usually span a single passage or multiple sections. Evaluate retrieval recall on representative queries at different chunk sizes before deciding.

What a strong answer covers: There is no universally optimal chunk size. A strong answer connects the strategy to retrieval quality metrics, not just implementation convenience.

Follow-up: How would you handle a document where tables and prose require different chunking logic?

Last reviewed 2026-10-07

Ensembles reduce variance by averaging independent errors, so they help most when individual models overfit or are unstable. Bagging trains models on bootstrap samples; random forests add feature subsampling to decorrelate trees. Boosting reduces bias by sequentially fitting residuals. Stacking learns to combine diverse base models. Gains diminish when base learners are highly correlated or when the dominant error is bias rather than variance.

What a strong answer covers: Distinguish when to reach for an ensemble versus regularising a single model. Mention that diversity among base models is essential and that ensembles add latency and maintenance cost.

Follow-up: How would you measure whether a new model adds value to an existing ensemble?

Last reviewed 2026-10-07

Stream interaction events through a reliable pipeline, apply deduplication and delay handling, and compute features with point-in-time correctness. Update the model on mini-batches, applying a learning rate schedule that prevents catastrophic forgetting of older patterns. Shadow-deploy the updated model, compare offline and online metrics, and gate full rollout on guardrails. Keep a stable fallback version and log both model versions for retrospective analysis.

What a strong answer covers: The hardest parts are label delay and catastrophic forgetting, not the update mechanism itself. A strong answer addresses both and defines what "fresh" means for the specific use case.

Follow-up: How would you detect when online updates degrade performance on an older cohort of users?

Last reviewed 2026-10-07

L2 adds the sum of squared weights to the loss, shrinking all weights toward zero proportionally. L1 adds the sum of absolute weights and can set some weights exactly to zero, producing sparse models. Use L2 when you believe most features contribute and want smooth shrinkage. Use L1 when you expect many irrelevant features and want automatic feature selection. Elastic net combines both. Regularization strength is a hyperparameter tuned by cross-validation.

What a strong answer covers: The key distinction is sparsity: L1 produces it, L2 does not. Explain why in terms of the geometry of the constraint region.

Follow-up: Why might L1 regularization still leave many small but non-zero weights in practice?

Last reviewed 2026-10-07

A list comprehension evaluates immediately and stores all values in memory. A generator expression or generator function uses yield to produce values one at a time and holds only the current state in memory. Use a generator when the sequence is large, infinite, or expensive to compute, and you only need to iterate once. Use a list when you need random access, repeated iteration, or the full collection for operations such as sorting or length checks.

What a strong answer covers: The core concept is lazy evaluation. Mention that generators are exhausted after one pass, which is a common source of bugs when the same generator is iterated twice.

Follow-up: How would you implement a generator that yields batches of rows from a large CSV file?

Last reviewed 2026-10-07

Filter methods rank features by a statistical criterion such as mutual information, correlation, or variance and are fast but ignore feature interactions. Wrapper methods such as recursive feature elimination use model performance to select subsets and are more powerful but expensive. Embedded methods such as L1 regularization and tree-based importance select during training. Start with a fast filter to remove clearly irrelevant features, then use embedded importance for the remaining set. Validate any selection on a held-out set to avoid optimistic estimates.

What a strong answer covers: Mention that importance from a tree trained on correlated features distributes arbitrarily across them, and that permutation importance or SHAP can be more reliable.

Follow-up: How would you handle two highly correlated features where domain knowledge says both matter?

Last reviewed 2026-10-07

Use the STAR format: describe the context and the decision you disagreed with, the specific concern you raised and how you raised it (data, an experiment, or a clear argument), the outcome, and what you would do differently. Show that you can advocate for your position while remaining constructive, and that you can commit to a decision once it has been made even when you disagree.

What a strong answer covers: The goal is to show technical judgment and professional communication, not to relitigate old grievances. Avoid framing the story as you versus them. A strong answer includes evidence you used to make your case.

Follow-up: What would you do if the team made the same decision again and the outcome was worse than you predicted?

Last reviewed 2026-10-07

Use STAR: explain the project goal, what you built, what happened, and what you did next. Own your share of the failure rather than attributing it entirely to data, compute, or other constraints. Name a specific technical mistake — a leaky split, a flawed evaluation metric, a mislabeled dataset — not a vague sense that the model was wrong. Describe what you learned and how you applied it on a subsequent project.

What a strong answer covers: A strong answer names a specific technical root cause, not bad luck or insufficient time. The retrospective action matters as much as the failure itself.

Follow-up: What would be the first thing you would investigate if you had three days to debug the same project now?

Last reviewed 2026-10-07

Use STAR: describe the decision, what information was missing, what assumptions you made and why, how you documented those assumptions, and what you did to limit downside risk. Show that you can move forward without perfect information while keeping the team informed about your uncertainty and the conditions that would change your decision.

What a strong answer covers: This tests judgment and risk management. A strong answer distinguishes between uncertainty that is worth waiting to resolve and uncertainty you can safely assume away for now.

Follow-up: How did you decide what additional information was worth waiting for versus what you could assume away?

Last reviewed 2026-10-07

Use STAR: set the scene, describe the failure and how you found it, and explain how you communicated it. Lead with the business impact first, then explain the cause at a level the audience can act on. Show that you gave them what they needed to make a decision rather than a full technical post-mortem. Include what you proposed to fix and what the outcome was.

What a strong answer covers: The best answers show empathy for what the stakeholder needs to know, not just what you know. Avoid jargon without explanation and avoid false reassurance about a fix that is not yet validated.

Follow-up: How would you set up a review process to catch similar issues before they reach stakeholders?

Last reviewed 2026-10-07

Use STAR: describe the problem you noticed, why it was not clearly owned, what you chose to do, how you involved or informed the right people, and the outcome. Show judgment about when to escalate versus act, and demonstrate that you understood the constraints and worked within or around them thoughtfully rather than bypassing important processes.

What a strong answer covers: This tests initiative and judgment. Interviewers probe whether you involved the right people at the right time. A strong answer shows you didn't silently take over someone else's work.

Follow-up: How did you decide this was your problem to solve rather than someone else's?

Last reviewed 2026-10-07

Compute the dot product of the two vectors divided by the product of their L2 norms. Handle the zero-norm edge case by returning 0.0 to avoid division by zero. Raise a ValueError if the vectors have different lengths.

def cosine_similarity(a: list[float], b: list[float]) -> float:
    if len(a) != len(b):
        raise ValueError("Vectors must have equal length")
    dot = sum(x * y for x, y in zip(a, b))
    norm_a = sum(x ** 2 for x in a) ** 0.5
    norm_b = sum(x ** 2 for x in b) ** 0.5
    if norm_a == 0.0 or norm_b == 0.0:
        return 0.0
    return dot / (norm_a * norm_b)

What a strong answer covers: The implementation should handle equal-length validation and the zero-norm case without importing math. Mention that floating-point precision can push results slightly outside [-1, 1] for normalised vectors.

Follow-up: How would you adapt this to compute all pairwise similarities for a matrix of N embeddings efficiently?

Last reviewed 2026-10-07

Use Welford's online algorithm, which updates the mean and the sum of squared deviations incrementally without storing all values. The variance property returns the population variance; divide by n-1 for sample variance.

class RunningStats:
    def __init__(self) -> None:
        self._n = 0
        self._mean = 0.0
        self._M2 = 0.0   # sum of squared deviations from current mean

    def update(self, x: float) -> None:
        self._n += 1
        delta = x - self._mean
        self._mean += delta / self._n
        self._M2 += delta * (x - self._mean)  # uses updated mean

    @property
    def mean(self) -> float:
        return self._mean

    @property
    def variance(self) -> float:          # population variance
        return self._M2 / self._n if self._n > 0 else 0.0

    @property
    def count(self) -> int:
        return self._n

What a strong answer covers: Welford's algorithm is numerically stable and O(1) in memory. The key is using the updated mean in the M2 accumulation step. Mention that two RunningStats instances can be merged by combining their n, mean, and M2 fields.

Follow-up: How would you merge the running statistics from two independent workers processing different data shards?

Last reviewed 2026-10-07

Give the model a clear task description, a typed tool schema, and instructions to emit structured calls. Validate tool arguments before executing them and return structured results including errors. Limit the number of steps and total tokens consumed. Separate the loop controller from the model to enable retries, logging, and policy enforcement. Keep tools idempotent where possible and require explicit confirmation before irreversible actions.

What a strong answer covers: Reliability comes from the system around the model, not the model itself. Non-determinism makes it hard to guarantee a specific execution path, so monitoring and graceful failure matter more than forcing a fixed sequence.

Follow-up: How would you prevent an agent from entering an infinite loop when a tool keeps failing?

Last reviewed 2026-10-07

Define tasks with known correct outcomes or verifiable end states. Measure task completion rate, number of steps, cost, and latency. Where ground truth is hard to define, use a judge model or a rubric to rate intermediate steps and final outputs. Curate a diverse evaluation set including edge cases and adversarial examples. Run evaluations deterministically using fixed seeds and mocked external calls, and track regressions across versions.

What a strong answer covers: End-to-end task completion is the primary signal but can be noisy. Per-step quality metrics help locate where a version regressed. Mention that judge models have their own biases and should be calibrated against human ratings.

Follow-up: How would you build an evaluation set for a task where correct answers are inherently open-ended?

Last reviewed 2026-10-07

Decide which content is essential for the current query. For conversations, keep the system prompt, the most recent turns, and a compressed summary of older turns. For documents, retrieve relevant chunks rather than passing the full text. Always inform the model which information has been omitted. Monitor how often truncation occurs and whether query quality degrades — that signals the retrieval or summarisation step needs improvement.

What a strong answer covers: Different strategies suit different use cases: retrieval for document QA, summarisation for long conversations. Truncating silently is usually worse than explicit omission with a note to the model.

Follow-up: How would you decide how much conversation history to summarise versus pass verbatim?

Last reviewed 2026-10-07

Separate data validation, training, evaluation, and deployment into distinct reproducible stages. Gate each stage: fail if data quality checks fail, if evaluation metrics fall below a threshold, or if the model regresses on a golden dataset. Version models, datasets, and configurations together. Use a model registry to promote from staging to production. Run integration tests against a staging environment before promoting, and support rollback by keeping the previous version active alongside the new one.

What a strong answer covers: An ML CI/CD pipeline is more complex than software CI/CD because training is non-deterministic and evaluation requires agreement on what "better" means. Mention the need to test code changes and data changes as separate triggers.

Follow-up: How would you handle a training run that is too expensive to run on every commit?

Last reviewed 2026-10-07

Keep the previous model version live and route a small traffic slice to it even after promoting the new version. Define alert thresholds on business metrics, error rates, and output distributions that trigger rollback or human review. Store model artifacts, preprocessing code, and feature definitions together so a rollback is fully reproducible. After rolling back, diagnose the regression on offline data before redeploying.

What a strong answer covers: A rollback that restores the model but not the feature store or preprocessing logic often reproduces the same problem. Everything the model depends on must be versioned together.

Follow-up: What single metric would you monitor to decide within 30 minutes whether a new model needs to be rolled back?

Last reviewed 2026-10-07

Shadow deployment routes a copy of live traffic to the new model but only the existing model's predictions reach users. Canary deployment serves a small percentage of users with the new model's actual predictions. Use shadow deployment to validate predictions and latency with zero user risk. Use canary deployment once shadow results are acceptable and you need to measure business metrics that require real user responses. Both require careful traffic allocation to avoid biasing the evaluation.

What a strong answer covers: Shadow deployment cannot measure user behaviour effects because it never changes the response. Mention that shadow deployment adds latency overhead on the mirrored path, and that canary deployment requires guarding against novelty effects.

Follow-up: How would you decide when to ramp a canary from 1% to 10% of production traffic?

Last reviewed 2026-10-07

A model card describes: the intended use cases and explicitly out-of-scope uses, training data sources and known biases, evaluation metrics broken down by relevant subgroups, known limitations and failure modes, and recommendations for monitoring. It matters because downstream users make deployment decisions based on it — an incomplete card leads to misuse in contexts the model was not designed for. Include both what the model does well and where it is known to struggle.

What a strong answer covers: Subgroup-disaggregated evaluation is often the most critical section because aggregate accuracy can hide serious performance gaps on important demographics. A strong answer explains why omitting this is not just an oversight but a risk.

Follow-up: How would you keep a model card accurate and current as you retrain the model on new data?

Last reviewed 2026-10-07

Full fine-tuning updates all model weights: highest quality ceiling, highest compute and memory cost. LoRA freezes the base model and adds small trainable rank-decomposition matrices to attention layers, reducing trainable parameters by 10–100× while recovering most of the quality gain. Prompt tuning adds learnable input tokens and trains only those, but generally underperforms LoRA on complex tasks. Choose full fine-tuning when quality is critical and compute is available. Choose LoRA when you need multiple adapters on the same base model or must train efficiently at scale. Choose prompt tuning when even LoRA is too expensive.

What a strong answer covers: The three approaches form a spectrum of parameter efficiency versus quality. A significant operational advantage of LoRA is that adapters are small, composable, and easy to swap at inference time, enabling personalisation at scale.

Follow-up: How would you choose the rank hyperparameter r in LoRA for a new task with no prior experiments?

Last reviewed 2026-10-07

Common issues: benchmark contamination (training data contains benchmark examples), saturation (models already score near ceiling, leaving no signal), narrow coverage (a benchmark captures only a slice of real-world use), and metric mismatch (BLEU or accuracy does not reflect the quality users care about). Use task-specific evaluations on held-out data not in training, human preference studies for open-ended outputs, and red-teaming for safety properties. Track metrics that correspond to deployment objectives.

What a strong answer covers: Benchmark contamination is the most frequently underestimated problem: even accidental inclusion of benchmark data in pretraining can inflate scores significantly and makes fair comparison between models trained on different data very difficult.

Follow-up: How would you design an evaluation suite for a customer-support LLM that could not be gamed by training on public benchmarks?

Last reviewed 2026-10-07