Microsoft interview prep

Leakage occurs when training uses information unavailable at prediction time. Examples include scaling with statistics from the full dataset, using future transactions, or placing near-duplicate users in both splits. Split first, fit preprocessing only on training data, and use temporal or grouped validation when the deployment setting requires it.

What a strong answer covers: A strong answer connects the split to the actual prediction moment and explains why a random split can be misleading.

Follow-up: How would you validate a model predicting a customer's next purchase?

Last reviewed 2026-09-28

Compare training and validation performance using learning curves. Poor performance on both can indicate underfitting, weak features, or optimization problems. Strong training results and weak validation results suggest overfitting or distribution mismatch. Try regularization, simpler models, more representative data, and cross-validation while keeping a final test set untouched.

What a strong answer covers: Mention that label noise, leakage, and a mismatched split can also explain these patterns.

Follow-up: What would you try if both errors remain high as data grows?

Last reviewed 2026-09-28

Read batches or stream observations, retaining a running count and mean. For a new value x at count n, update mean = mean + (x - mean) / n. Use a generator so loading is lazy. Define how missing and invalid values are handled; use a stable algorithm such as Welford's if variance is also needed.

What a strong answer covers: This uses constant auxiliary memory. Discuss floating-point limits and deterministic aggregation for parallel processing.

Follow-up: How would you merge the counts and means of two independent workers?

Last reviewed 2026-09-28

Use a real example: describe the user need, measured quality difference, and operational constraints. Compare latency, interpretability, cost, and maintenance. Explain the decision you owned, the outcome, and what evidence would make you revisit it. If you lack work experience, use a project and be explicit about its scope.

What a strong answer covers: The goal is a defensible decision supported by measurements, not a claim that simple models always win.

Follow-up: Which production metric would trigger a reevaluation?

Last reviewed 2026-09-28

Lexical search can retrieve exact identifiers and rare terms; vector search captures semantic similarity. Run both retrieval methods, combine candidates with a method such as reciprocal rank fusion, then optionally rerank. Evaluate representative query types and inspect failures before adding complexity.

What a strong answer covers: Score scales are not automatically comparable. Rank fusion can avoid relying on raw score equivalence.

Follow-up: How would you test whether hybrid search improves exact product-code queries?

Last reviewed 2026-09-28

Ingest and version documents, retain access-control metadata, and chunk content with source references. Enforce user permissions during retrieval, rerank relevant results, and ask the model to answer with citations. Support abstention when evidence is missing. Evaluate retrieval, groundedness, access isolation, latency, and cost using realistic questions.

What a strong answer covers: Authorization must be enforced by the application. A prompt telling the model to protect data is insufficient.

Follow-up: How would you remove a deleted document from indexes, caches, and existing sessions?

Last reviewed 2026-09-28

Use a labeled set with expected evidence. Check whether retrieval includes that evidence, then test generation with known-good context. Measure retrieval recall separately from answer correctness and citation support. Inspect chunk boundaries, ambiguous queries, and unanswerable questions, and compare changes on a held-out set.

What a strong answer covers: End-to-end answer accuracy alone cannot locate the failure. Controlled context experiments isolate the components.

Follow-up: What metric would you use when several documents can correctly support an answer?

Last reviewed 2026-09-28

Treat retrieved content as untrusted data. Keep tool permissions narrow, enforce authorization outside the model, validate tool arguments, and require confirmation for consequential actions. Separate instructions from content, restrict data exposure, and test adversarial documents. Record failures and evaluate mitigations without assuming that prompt text alone provides a security boundary.

What a strong answer covers: The answer should cover what the model can actually do, not just whether it follows a misleading instruction.

Follow-up: How would you prevent a retrieved URL from exfiltrating private context?

Last reviewed 2026-09-28

Track availability, latency, resource cost, input quality, feature distributions, and output distributions. When labels arrive, monitor outcome quality and calibration by segment. Define alert thresholds, ownership, and rollback paths. Investigate drift before retraining because a distribution change does not always imply degraded performance.

What a strong answer covers: Monitoring must connect to an action. Labels may be delayed, so proxy signals need explicit limitations.

Follow-up: How would you distinguish a broken data pipeline from genuine behavior change?

Last reviewed 2026-09-28

Attention computes a weighted sum of values, where weights come from the dot product of queries and keys passed through softmax. Dividing by the square root of the key dimension prevents the dot products from growing large when the dimension is high, which would push softmax into regions with very small gradients. Multi-head attention runs this in parallel across learned projection subspaces so the model can attend to different aspects of the input simultaneously.

What a strong answer covers: A strong answer explains the gradient issue numerically, not just verbally, and connects multi-head design to representational diversity.

Follow-up: What changes when you replace full self-attention with a sparse or linear approximation?

Last reviewed 2026-10-07

Fine-tune when prompt engineering cannot reliably elicit the required format, domain vocabulary, or task behavior, or when latency and cost make large in-context examples impractical. Parameter-efficient methods such as LoRA adapt the model with far fewer trainable parameters. Use a representative evaluation set to confirm improvement over a well-crafted prompt before committing to the training overhead. Consider catastrophic forgetting, data quality, and whether the base model license permits fine-tuning.

What a strong answer covers: Mention that fine-tuning is a cost with diminishing returns; a good answer weighs it against the quality ceiling of prompting for the specific task.

Follow-up: How would you decide how much of the model to unfreeze for a domain-adaptation task?

Last reviewed 2026-10-07

Human annotators rank model outputs; a reward model is trained to predict those preferences. A policy is then fine-tuned with reinforcement learning to maximize predicted reward while a KL penalty keeps it close to the original model. The reward model can overfit to annotator biases, be gamed through reward hacking, or fail to generalise to novel outputs. Distribution shift between training and deployment makes reward model reliability difficult to guarantee.

What a strong answer covers: Cover both the training loop and the failure modes. Reward hacking and annotation disagreement are the two most cited weaknesses interviewers probe.

Follow-up: What would you do if annotators systematically preferred verbose but less accurate answers?

Last reviewed 2026-10-07

Choose explanation methods that match the audience and question. SHAP values attribute predictions to features globally and locally and are consistent across model types. LIME approximates the model locally for a single prediction. Partial dependence plots show the marginal effect of a feature. For high-stakes decisions, pair local explanations with error analysis and communicate confidence. Avoid overstating what a local explanation tells you about the model globally.

What a strong answer covers: Distinguish global explanations for model auditing from local explanations for individual decisions. Mention that explanations of black-box models are approximations, not ground truth.

Follow-up: How would you validate that a SHAP explanation actually reflects model behavior?

Last reviewed 2026-10-07

Fixed-size overlapping chunks are simple and predictable but can split sentences. Sentence or paragraph boundaries respect natural structure. Semantic chunking groups text by embedding similarity. Recursive splitting tries larger units first and falls back to smaller ones. The right size depends on the embedding model context window, the typical query length, and whether answers usually span a single passage or multiple sections. Evaluate retrieval recall on representative queries at different chunk sizes before deciding.

What a strong answer covers: There is no universally optimal chunk size. A strong answer connects the strategy to retrieval quality metrics, not just implementation convenience.

Follow-up: How would you handle a document where tables and prose require different chunking logic?

Last reviewed 2026-10-07

L2 adds the sum of squared weights to the loss, shrinking all weights toward zero proportionally. L1 adds the sum of absolute weights and can set some weights exactly to zero, producing sparse models. Use L2 when you believe most features contribute and want smooth shrinkage. Use L1 when you expect many irrelevant features and want automatic feature selection. Elastic net combines both. Regularization strength is a hyperparameter tuned by cross-validation.

What a strong answer covers: The key distinction is sparsity: L1 produces it, L2 does not. Explain why in terms of the geometry of the constraint region.

Follow-up: Why might L1 regularization still leave many small but non-zero weights in practice?

Last reviewed 2026-10-07

A list comprehension evaluates immediately and stores all values in memory. A generator expression or generator function uses yield to produce values one at a time and holds only the current state in memory. Use a generator when the sequence is large, infinite, or expensive to compute, and you only need to iterate once. Use a list when you need random access, repeated iteration, or the full collection for operations such as sorting or length checks.

What a strong answer covers: The core concept is lazy evaluation. Mention that generators are exhausted after one pass, which is a common source of bugs when the same generator is iterated twice.

Follow-up: How would you implement a generator that yields batches of rows from a large CSV file?

Last reviewed 2026-10-07

Use the STAR format: describe the context and the decision you disagreed with, the specific concern you raised and how you raised it (data, an experiment, or a clear argument), the outcome, and what you would do differently. Show that you can advocate for your position while remaining constructive, and that you can commit to a decision once it has been made even when you disagree.

What a strong answer covers: The goal is to show technical judgment and professional communication, not to relitigate old grievances. Avoid framing the story as you versus them. A strong answer includes evidence you used to make your case.

Follow-up: What would you do if the team made the same decision again and the outcome was worse than you predicted?

Last reviewed 2026-10-07

Use STAR: explain the project goal, what you built, what happened, and what you did next. Own your share of the failure rather than attributing it entirely to data, compute, or other constraints. Name a specific technical mistake — a leaky split, a flawed evaluation metric, a mislabeled dataset — not a vague sense that the model was wrong. Describe what you learned and how you applied it on a subsequent project.

What a strong answer covers: A strong answer names a specific technical root cause, not bad luck or insufficient time. The retrospective action matters as much as the failure itself.

Follow-up: What would be the first thing you would investigate if you had three days to debug the same project now?

Last reviewed 2026-10-07

Use STAR: describe the decision, what information was missing, what assumptions you made and why, how you documented those assumptions, and what you did to limit downside risk. Show that you can move forward without perfect information while keeping the team informed about your uncertainty and the conditions that would change your decision.

What a strong answer covers: This tests judgment and risk management. A strong answer distinguishes between uncertainty that is worth waiting to resolve and uncertainty you can safely assume away for now.

Follow-up: How did you decide what additional information was worth waiting for versus what you could assume away?

Last reviewed 2026-10-07

Use STAR: set the scene, describe the failure and how you found it, and explain how you communicated it. Lead with the business impact first, then explain the cause at a level the audience can act on. Show that you gave them what they needed to make a decision rather than a full technical post-mortem. Include what you proposed to fix and what the outcome was.

What a strong answer covers: The best answers show empathy for what the stakeholder needs to know, not just what you know. Avoid jargon without explanation and avoid false reassurance about a fix that is not yet validated.

Follow-up: How would you set up a review process to catch similar issues before they reach stakeholders?

Last reviewed 2026-10-07

Use STAR: describe the problem you noticed, why it was not clearly owned, what you chose to do, how you involved or informed the right people, and the outcome. Show judgment about when to escalate versus act, and demonstrate that you understood the constraints and worked within or around them thoughtfully rather than bypassing important processes.

What a strong answer covers: This tests initiative and judgment. Interviewers probe whether you involved the right people at the right time. A strong answer shows you didn't silently take over someone else's work.

Follow-up: How did you decide this was your problem to solve rather than someone else's?

Last reviewed 2026-10-07

Compute the dot product of the two vectors divided by the product of their L2 norms. Handle the zero-norm edge case by returning 0.0 to avoid division by zero. Raise a ValueError if the vectors have different lengths.

def cosine_similarity(a: list[float], b: list[float]) -> float:
    if len(a) != len(b):
        raise ValueError("Vectors must have equal length")
    dot = sum(x * y for x, y in zip(a, b))
    norm_a = sum(x ** 2 for x in a) ** 0.5
    norm_b = sum(x ** 2 for x in b) ** 0.5
    if norm_a == 0.0 or norm_b == 0.0:
        return 0.0
    return dot / (norm_a * norm_b)

What a strong answer covers: The implementation should handle equal-length validation and the zero-norm case without importing math. Mention that floating-point precision can push results slightly outside [-1, 1] for normalised vectors.

Follow-up: How would you adapt this to compute all pairwise similarities for a matrix of N embeddings efficiently?

Last reviewed 2026-10-07

Give the model a clear task description, a typed tool schema, and instructions to emit structured calls. Validate tool arguments before executing them and return structured results including errors. Limit the number of steps and total tokens consumed. Separate the loop controller from the model to enable retries, logging, and policy enforcement. Keep tools idempotent where possible and require explicit confirmation before irreversible actions.

What a strong answer covers: Reliability comes from the system around the model, not the model itself. Non-determinism makes it hard to guarantee a specific execution path, so monitoring and graceful failure matter more than forcing a fixed sequence.

Follow-up: How would you prevent an agent from entering an infinite loop when a tool keeps failing?

Last reviewed 2026-10-07

Define tasks with known correct outcomes or verifiable end states. Measure task completion rate, number of steps, cost, and latency. Where ground truth is hard to define, use a judge model or a rubric to rate intermediate steps and final outputs. Curate a diverse evaluation set including edge cases and adversarial examples. Run evaluations deterministically using fixed seeds and mocked external calls, and track regressions across versions.

What a strong answer covers: End-to-end task completion is the primary signal but can be noisy. Per-step quality metrics help locate where a version regressed. Mention that judge models have their own biases and should be calibrated against human ratings.

Follow-up: How would you build an evaluation set for a task where correct answers are inherently open-ended?

Last reviewed 2026-10-07

Decide which content is essential for the current query. For conversations, keep the system prompt, the most recent turns, and a compressed summary of older turns. For documents, retrieve relevant chunks rather than passing the full text. Always inform the model which information has been omitted. Monitor how often truncation occurs and whether query quality degrades — that signals the retrieval or summarisation step needs improvement.

What a strong answer covers: Different strategies suit different use cases: retrieval for document QA, summarisation for long conversations. Truncating silently is usually worse than explicit omission with a note to the model.

Follow-up: How would you decide how much conversation history to summarise versus pass verbatim?

Last reviewed 2026-10-07

Enforce tool permissions outside the model: the model requests an action, the controller decides whether to execute it. Apply the principle of least privilege to each tool. Validate and sanitise all arguments before execution. Require human confirmation for high-impact or irreversible actions. Log every tool call with its arguments and result. Rate-limit calls and apply a token budget. Treat retrieved content as untrusted data and test for prompt injection in tool outputs.

What a strong answer covers: The model cannot be trusted to self-enforce safety boundaries. The system architecture must assume the model can be manipulated and enforce constraints in code, not prompts.

Follow-up: How would you prevent a retrieved web page from hijacking the agent with an injected instruction?

Last reviewed 2026-10-07

Separate data validation, training, evaluation, and deployment into distinct reproducible stages. Gate each stage: fail if data quality checks fail, if evaluation metrics fall below a threshold, or if the model regresses on a golden dataset. Version models, datasets, and configurations together. Use a model registry to promote from staging to production. Run integration tests against a staging environment before promoting, and support rollback by keeping the previous version active alongside the new one.

What a strong answer covers: An ML CI/CD pipeline is more complex than software CI/CD because training is non-deterministic and evaluation requires agreement on what "better" means. Mention the need to test code changes and data changes as separate triggers.

Follow-up: How would you handle a training run that is too expensive to run on every commit?

Last reviewed 2026-10-07

Keep the previous model version live and route a small traffic slice to it even after promoting the new version. Define alert thresholds on business metrics, error rates, and output distributions that trigger rollback or human review. Store model artifacts, preprocessing code, and feature definitions together so a rollback is fully reproducible. After rolling back, diagnose the regression on offline data before redeploying.

What a strong answer covers: A rollback that restores the model but not the feature store or preprocessing logic often reproduces the same problem. Everything the model depends on must be versioned together.

Follow-up: What single metric would you monitor to decide within 30 minutes whether a new model needs to be rolled back?

Last reviewed 2026-10-07

A model card describes: the intended use cases and explicitly out-of-scope uses, training data sources and known biases, evaluation metrics broken down by relevant subgroups, known limitations and failure modes, and recommendations for monitoring. It matters because downstream users make deployment decisions based on it — an incomplete card leads to misuse in contexts the model was not designed for. Include both what the model does well and where it is known to struggle.

What a strong answer covers: Subgroup-disaggregated evaluation is often the most critical section because aggregate accuracy can hide serious performance gaps on important demographics. A strong answer explains why omitting this is not just an oversight but a risk.

Follow-up: How would you keep a model card accurate and current as you retrain the model on new data?

Last reviewed 2026-10-07

Full fine-tuning updates all model weights: highest quality ceiling, highest compute and memory cost. LoRA freezes the base model and adds small trainable rank-decomposition matrices to attention layers, reducing trainable parameters by 10–100× while recovering most of the quality gain. Prompt tuning adds learnable input tokens and trains only those, but generally underperforms LoRA on complex tasks. Choose full fine-tuning when quality is critical and compute is available. Choose LoRA when you need multiple adapters on the same base model or must train efficiently at scale. Choose prompt tuning when even LoRA is too expensive.

What a strong answer covers: The three approaches form a spectrum of parameter efficiency versus quality. A significant operational advantage of LoRA is that adapters are small, composable, and easy to swap at inference time, enabling personalisation at scale.

Follow-up: How would you choose the rank hyperparameter r in LoRA for a new task with no prior experiments?

Last reviewed 2026-10-07