Back to interview questions
By
Team Sherlock.sh
Generative AI interviews reveal whether a candidate can connect model behavior to useful, reliable systems. Strong candidates understand how models learn and generate content, but also know how to evaluate evidence, manage uncertainty, protect data, and constrain automated actions. Recruiters should assess those practical distinctions rather than relying on fluent terminology or a polished demo.
This guide to 100+ Generative AI Interview Questions and Answers contains 120 questions across beginner, intermediate, and experienced levels. It covers generative model families, transformers, language and multimodal models, retrieval-augmented generation, adaptation, evaluation, serving, and responsible deployment. The questions distinguish foundational knowledge from implementation ability and production judgment without assuming that one model or provider is best for every task.
How can recruiters identify potential AI cheating, deepfakes, impersonation, and unauthorized assistance during generative AI interviews?
Ask candidates to diagnose a retrieval failure, explain a sampling calculation, or defend a tool-authorization boundary. Because working with generative AI may legitimately involve model-assisted tools, define the permitted assistance before the interview and assess whether candidates can explain and verify their own work.
Sherlock AI helps recruiters detect potential AI-assisted cheating, deepfakes, candidate impersonation, and unauthorized real-time assistance during generative AI interviews.
Combine interview-integrity checks with a focused technical exercise and follow-up questions about assumptions, evidence, and failure modes. Review suspicious signals in context rather than treating a single signal as proof of misconduct.
Generative AI Interview Questions
Choose questions that match the role: model research, application engineering, evaluation, infrastructure, or governance. State the task, available data, deployment constraints, and permitted tools before evaluating an answer. Ask candidates to distinguish measured results from assumptions and illustrative designs.
Generative AI Interview Questions for Beginners
Establish a foundation in generative model families, language modeling, tokens, embeddings, transformers, training, sampling, prompting, and model limitations. Look for accurate distinctions and simple examples rather than memorized definitions.
What is generative AI, and what does it actually learn?
How does generative AI differ from discriminative AI?
What is a conditional generative model?
Which major families of generative models should a beginner know?
When is generative AI useful, and when is a simpler method preferable?
How does autoregressive generation work?
What is a generative adversarial network?
What is a variational autoencoder?
How does a diffusion model generate an image?
How would you compare autoregressive, GAN, VAE, and diffusion models?
What is a large language model?
What is a token, and why is it not the same as a word?
What problem does subword tokenization solve?
What is an embedding?
What is a transformer, and how do encoder and decoder variants differ?
What does self-attention do?
Why do transformers need positional information?
What is a context window?
What are model parameters, and do more parameters guarantee a better model?
What does self-supervised learning mean?
What are cross-entropy loss and perplexity?
How does pretraining differ from fine-tuning?
What is instruction tuning?
What do alignment, RLHF, and preference optimization mean?
Does a language model automatically know current facts or search the internet?
What is greedy decoding?
How does temperature affect generation?
What is top-p sampling, and how is it different from top-k?
What makes a prompt clear and useful?
What are zero-shot and few-shot prompting?
How do system, user, and assistant messages relate to the prompt?
What is a hallucination in generative AI?
What is retrieval-augmented generation at a high level?
How do prompting, RAG, and fine-tuning differ?
What does multimodal mean, and is every multimodal model generative?
How should a beginner evaluate a generative AI answer?
Why can the same prompt produce different outputs?
What basic privacy precautions matter when using generative AI?
What does responsible use of generative AI mean in a beginner project?
How would you choose a model for a simple generative AI project?
Generative AI Interview Questions for Intermediate
Explore retrieval pipelines, grounded generation, structured outputs, evaluation design, fine-tuning, tools, memory, and multimodal workflows. Ask candidates to explain how they would test each component and identify the source of a failure.
How would you implement and debug a basic RAG pipeline?
How do you choose chunk sizes and overlap for retrieval?
Why must document and query embedding configurations be compatible?
When do cosine similarity, dot product, and Euclidean distance give the same ranking?
How would you evaluate an approximate nearest-neighbor index?
How do metadata filters interact with semantic retrieval?
How would you combine lexical and dense retrieval with reciprocal rank fusion?
What does a reranker add after initial retrieval?
When is query rewriting useful, and how would you test it?
How do you assemble retrieved passages into an effective generation context?
How would you encourage grounded answers and sensible abstention?
How do you verify that citations actually support generated claims?
Why is schema-constrained generation not the same as a correct answer?
How would you validate a model-generated extraction before using it?
How would you design a fair prompt experiment?
How do you select few-shot examples without overfitting the prompt?
How do you split generative-AI datasets to avoid leakage?
What belongs in a useful evaluation dataset for a RAG assistant?
How do precision@k, recall@k, and MRR differ for retrieval?
When would you use NDCG instead of a binary retrieval metric?
How would you choose metrics for generated answers?
What is the difference between groundedness, correctness, and relevance?
How would you calibrate an LLM-as-a-judge?
How would you compare two prompts when their scores are close?
What does a good supervised fine-tuning example look like?
How would you decide between RAG and fine-tuning for a task?
How does LoRA reduce trainable parameter count?
What is the difference between LoRA and QLoRA?
How do SFT, DPO, and reward-model-based RLHF use different supervision?
How do you know whether a fine-tuning run actually improved the model?
What happens during a model tool-calling cycle?
What should a well-designed tool interface return to a language model?
How would you keep a simple agent loop from running indefinitely?
How would you manage conversation memory without keeping every message?
How would you calculate an input and output token budget?
How is a KV cache different from a prompt or prefix cache?
How would you test temperature and top-p without confusing their effects?
How do guidance scale and inference steps affect diffusion generation?
How do image-to-image strength and inpainting masks change an editing task?
How would you evaluate a multimodal extraction or question-answering system?
Generative AI Interview Questions for Experienced
Use these scenarios to assess system architecture, capacity planning, security boundaries, evaluation governance, incident handling, and launch decisions. Strong candidates reason about trade-offs using workload-specific evidence.
How would you decide between a hosted model and self-hosting for a production application?
How do tensor, pipeline, and data parallelism affect an inference deployment?
How would you design model routing without making quality unpredictable?
How do you design evaluation slices that expose production risk rather than improve a headline score?
How would you compare models statistically when requests from the same customer are correlated?
Why can a training-compute-optimal model be the wrong choice for serving?
What would a defensible canary rollout look like for a generative-AI feature?
How would you distinguish data drift, model regression, and evaluator drift?
How would you diagnose high time to first token versus slow token streaming?
How would you tune continuous batching while protecting interactive latency?
How do you estimate KV-cache memory before admitting long-context requests?
What security and correctness issues arise when sharing prefix caches across tenants?
How would you decide whether quantization is acceptable in a production model?
When does speculative decoding help, and when can it make serving worse?
How would you assess the real cost of increasing the context window?
How would you estimate GPU capacity for an inference service?
How would you calculate cost per successful task rather than cost per token?
How would you enforce tenant and document permissions in a RAG system?
How would you handle stale documents, permission revocation, and deletion in RAG?
How would you threat-model prompt injection in an enterprise assistant?
How should tool authorization differ from tool-call validation?
How would you prevent an assistant from exfiltrating confidential data through tools?
What should a sandbox for model-generated code actually restrict?
How would you design privacy controls across the generative-AI data lifecycle?
How would you govern data provenance and intellectual-property risk in model development?
What makes human review a meaningful safety control rather than a checkbox?
How would you make a long-running agent reliable enough for production?
How would you prevent duplicate external actions when an agent retries after a timeout?
How would you combine retries, deadlines, and circuit breakers in an agent workflow?
When would you reject a multi-agent design in favor of a single workflow?
What observability is necessary to debug a generative-AI incident?
How much reproducibility can you promise for generative-AI experiments?
Why is alignment training insufficient as the main security boundary?
How would you choose between DPO-style preference optimization and an online reward-based approach?
What additional deployment tests does a multimodal assistant need beyond text evaluation?
How would you evaluate a diffusion image-generation system for a real product?
How would you use synthetic training data without creating a self-reinforcing feedback loop?
How would you respond to an incident involving an incorrect or unauthorized agent action?
How would you design fallbacks for a model-provider outage?
What evidence would you require before approving a generative-AI system for launch?
Generative AI Interview Questions and Answers
These answers follow the question bank in the same order and begin with direct responses candidates can adapt in an interview. Small calculations and Python examples illustrate concepts; pseudocode describes application boundaries rather than a complete production implementation. Provider-specific APIs, model behavior, and operational limits must be verified in the target environment.
Generative AI Interview Questions and Answers for Beginners
The beginner answers explain what generative models learn and how their outputs should be interpreted. They connect mathematical concepts to practical behavior while separating fluency from factual reliability.
1. What is generative AI, and what does it actually learn?
Generative AI refers to models that learn patterns in data and use those patterns to produce outputs such as text, images, audio, or code. A model learns a statistical representation of possible outputs, often conditioned on an input. It does not need to retrieve an existing document to generate a sentence, although an application can separately connect it to retrieval.
For example, a support assistant can draft a reply from a customer's question and a supplied policy. The draft is generated content, not automatically a verified statement of company policy. Its usefulness depends on whether it follows the request, preserves the evidence, and avoids unsupported details.
I distinguish generation from originality and correctness. A new sequence of words can still contain memorized material, reproduce stereotypes, or make a factual error. The practical goal is therefore not simply to produce plausible content: it is to produce content that meets a defined task, with checks appropriate to the consequences of being wrong.
2. How does generative AI differ from discriminative AI?
A discriminative model commonly learns a decision boundary or a conditional distribution such as P(label | input). A generative model represents how data can be produced, for example P(text) or P(image | caption). The distinction concerns the modeling objective, not whether the system uses a neural network.
Consider product reviews. A sentiment classifier maps a review to positive or negative. A text generator can create a review, summarize one, or produce the word โpositiveโ when prompted to classify it. The same user-facing classification task can therefore be implemented with different modeling approaches.
I would not say that discriminative models only understand while generative models create: โunderstandingโ is too vague, and generative models can perform classification. I would choose a focused classifier when its accuracy, latency, cost, and predictable output fit the problem. Generation is particularly useful when the output is open-ended, but extra flexibility creates additional validation work rather than automatically making it the better solution.
3. What is a conditional generative model?
A conditional generative model produces an output using additional information, written conceptually as P(output | condition). The condition might be a question, a class label, a source document, an image, or several of these together. An unconditional model instead generates without that particular external condition, although its architecture and training still shape its outputs.
A text-to-image system conditioned on โa red bicycle beside a treeโ should produce an image consistent with that description. An assistant conditioned on an invoice and an extraction instruction should return information from that invoice. These are different tasks, but both illustrate conditioning.
The condition guides a learned distribution; it is not a hard guarantee that every requested constraint will hold. A bicycle may have incorrect geometry, and an extracted invoice field may be invented. I would test the actual constraints separately. Conditioning also does not necessarily update model weights: the same trained model can handle different requests by receiving different inputs at inference time.
4. Which major families of generative models should a beginner know?
Four useful families are autoregressive models, variational autoencoders, generative adversarial networks, and diffusion models. Autoregressive models generate a sequence using previously generated elements. VAEs learn probabilistic latent representations and a decoder. GANs train a generator against a discriminator. Diffusion models learn to reverse a corruption process, commonly by progressively denoising.
These are organizing ideas, not mutually exclusive product categories. A system may encode an image into a latent representation, use a diffusion process there, and decode the result into pixels. A transformer is an architecture that can participate in different modeling approaches; it is not a fifth training objective equivalent to those four families.
In an interview, I would explain the generation mechanism before making quality claims. For example, sequential generation and iterative denoising imply different computational patterns, but neither description alone determines latency or output quality. Dataset quality, model size, training design, sampling method, and implementation all matter. There is no universally best family for every modality and task.
5. When is generative AI useful, and when is a simpler method preferable?
Generative AI is useful when a task benefits from flexible language or media generation: drafting, summarization, translation, brainstorming, or transforming information into a requested format. It can also provide a natural-language interface to other capabilities. I would first identify the user's outcome rather than assume that every workflow needs a chatbot.
For example, drafting a polite explanation of a failed payment can benefit from generation. Calculating the amount owed should still use validated arithmetic and the actual billing records. The generator can explain a result without being the authority that determines it.
A template, search interface, rules engine, or conventional classifier may be preferable for a small, stable set of outputs. I would compare solutions using representative cases, error consequences, maintenance, and operating cost. A fluent demonstration is not enough. If the application cannot tolerate unsupported content, I would constrain the output, validate critical fields, or keep a human approval step rather than rely on persuasive wording.
6. How does autoregressive generation work?
An autoregressive model represents a sequence as a product of conditional probabilities: P(x1, โฆ, xn) = product of P(xt | x1, โฆ, x[tโ1]). During text generation, it predicts a distribution for the next token, selects a token with a decoding method, appends it to the sequence, and repeats until a stopping condition is met.
If the prompt ends with โThe meeting starts atโ, the model might assign probability to several possible continuations. Selecting โnineโ changes the context for the next prediction. One decision therefore influences later decisions; the model is not independently filling every output position at once.
Training and inference have different computational patterns. With causal masking and known training text, predictions for many positions can be computed in parallel. Ordinary inference must wait for earlier generated tokens. The factorization also does not guarantee factuality: it assigns probabilities to sequences. A likely continuation may sound natural while supplying the wrong meeting time if the relevant evidence is missing.
7. What is a generative adversarial network?
A generative adversarial network, or GAN, trains two components with competing objectives. The generator transforms an input such as random noise into a synthetic sample. The discriminator learns to distinguish generated samples from examples in the training data. Feedback from the discriminator helps improve the generator.
For an image example, the generator might produce synthetic product photographs while the discriminator assesses whether each image resembles the training distribution. At generation time, the generator is normally the component needed to create a new sample; the discriminator is a training mechanism, not a universal authenticity checker.
An important limitation is that realistic samples do not prove broad coverage of the data distribution. A generator might repeatedly produce a narrow range of convincing images, a problem commonly called mode collapse. I would evaluate both quality and diversity. I would also avoid describing GAN training as guaranteed to converge: the two components change together, and their balance can make optimization difficult in practice.
8. What is a variational autoencoder?
A variational autoencoder, or VAE, is a probabilistic latent-variable model with an encoder and decoder. The encoder estimates a distribution over latent variables for an input; the decoder models how an input can be reconstructed or generated from those variables. Unlike a plain deterministic autoencoder, it explicitly works with a latent distribution.
Training balances reconstruction with regularization of the inferred latent distribution toward a chosen prior. The usual objective is an evidence lower bound, or ELBO. The reparameterization trick expresses a sample using a differentiable transformation of noise, allowing gradient-based training of the encoder.
For a simple image example, a latent vector might capture combinations of visual properties, and sampling from the prior allows the decoder to generate an image. I would not promise that each latent coordinate corresponds to a clean human concept such as color or rotation. Meaningful structure can emerge, but disentanglement is not automatic. Reconstruction quality, generative quality, and useful representations are related but separate evaluation goals.
9. How does a diffusion model generate an image?
A diffusion model learns to reverse a process that gradually corrupts data with noise. In a common formulation, training samples an image, a noise level, and noise, then teaches a network to predict a quantity that supports denoising. At generation time, a sampler starts from noise and repeatedly applies learned updates to obtain a structured sample.
A text-conditioned image system uses the prompt to influence those updates. Some systems operate in a compressed latent space rather than directly on full-resolution pixels, so the final result requires decoding back into an image. โDiffusionโ therefore describes a modeling approach, not one fixed network or image size.
The number and design of sampling steps affect the speedโquality trade-off, but more steps do not universally guarantee a better image. I would assess prompt adherence, visual coherence, diversity, and artifacts separately. A convincing image is not photographic evidence that an event occurred, and an attractive output can still fail a precise requirement such as the number of objects.
10. How would you compare autoregressive, GAN, VAE, and diffusion models?
I would compare their mechanisms and the requirements of the task rather than rank the names. Autoregressive models factor generation into ordered predictions. GANs learn through adversarial feedback. VAEs combine latent-variable inference with a probabilistic decoder. Diffusion models use a learned reverse corruption process. These differences shape training objectives and sampling behavior.
For example, if I need a latent representation for exploring variations of a small dataset, a VAE is a useful conceptual starting point. If I need sequential text completion, autoregressive modeling naturally fits the output structure. Neither example establishes that one implementation will beat all alternatives.
My evaluation checklist would include sample quality, diversity, conditioning accuracy, inference cost, training stability, and whether likelihood or an encoder is needed. I would also check the available data and evaluation budget. The categories can overlap inside a larger system, and many details vary by implementation. A defensible comparison states the workload and measurements instead of repeating claims such as โGANs are fastโ or โdiffusion is always best.โ
11. What is a large language model?
A large language model, or LLM, is a language model trained with substantial data and computational capacity to represent patterns in token sequences. Many generative LLMs predict the next token, although language modeling also includes other objectives. โLargeโ does not have one universal parameter threshold, and a language model is not necessarily a chat assistant.
A base model may continue a passage in the style of its training data. Further training and application instructions can make it more useful for following requests, answering questions, or interacting through a chat interface. Search, calculators, and databases are separate capabilities when connected by an application; they are not automatically present because the model is large.
I would describe its output as a learned response conditioned on the input, not as a database lookup or a guarantee of reasoning correctness. Strong performance on language tasks can coexist with arithmetic mistakes, fabricated details, and uneven language coverage. For a real use case, I would evaluate the specific behavior needed rather than infer it from parameter count alone.
12. What is a token, and why is it not the same as a word?
A token is a unit represented by an identifier in a model's vocabulary. Depending on the tokenizer, a token can correspond to a word, part of a word, punctuation, whitespace-related text, or a byte-level fragment. Special tokens can also mark structural information such as message boundaries or the end of a sequence.
For example, an unfamiliar technical term may split into several pieces while a frequent word remains a single token. The exact split depends on the tokenizer, so I would not present a guessed tokenization as a verified result. Two models can tokenize the same sentence differently.
This matters because context limits and many usage measurements are expressed in tokens, not words. A fixed word count is not a reliable capacity estimate across languages, code, and ordinary prose. I would measure with the tokenizer used by the actual model and include formatting or message overhead where applicable. Tokenization is an encoding step; an individual token does not need to have a complete standalone meaning.
13. What problem does subword tokenization solve?
Subword tokenization balances two inconvenient extremes. A word-level vocabulary can become very large and struggle with unseen words. A character-level representation needs a small vocabulary but often produces much longer sequences. Subword methods reuse recurring pieces so that many uncommon words can be expressed without dedicating one vocabulary entry to each word.
Byte pair encoding learns merges of frequent units, while WordPiece and unigram approaches use different vocabulary-learning procedures. The important beginner distinction is between learning the tokenizer vocabulary and using that fixed tokenizer to encode new text. Tokenization is not simply splitting a string at spaces.
For example, a new product name can be represented by several known pieces even if it never appeared as a whole word during tokenizer training. That does not mean the language model understands the product. I would preserve the modelโtokenizer pairing: swapping vocabularies changes token identifiers and can invalidate the learned embedding lookup. I would also test multilingual and code inputs because segmentation efficiency varies with the data.
14. What is an embedding?
An embedding is a numerical vector used to represent an item such as a token, sentence, image, or document. Learned embeddings can place items with useful similarities near one another according to a chosen comparison. A token embedding inside a language model and a sentence embedding used for search are related ideas, but they are not interchangeable objects.
Cosine similarity compares vector direction. This toy calculation uses invented vectors, not outputs from an embedding model:
Python example
A similarity score is not a probability that two statements mean the same thing, nor does it establish truth. For instance, a statement and its negation may share substantial vocabulary and subject matter. I would evaluate embeddings on the intended comparison task and handle zero-length vectors if implementing a general cosine function.
15. What is a transformer, and how do encoder and decoder variants differ?
A transformer is a neural-network architecture built around attention and position-wise transformations, with mechanisms such as residual connections and normalization. It processes representations of tokens rather than operating directly on human words. The original architecture combined an encoder and a decoder, while later models use different subsets and variants.
An encoder typically builds representations using input context on both sides of a token, which is useful for understanding or classification tasks. A causal decoder predicts outputs using preceding context. An encoderโdecoder model reads a source sequence and generates a target sequence, a natural structure for translation or summarization.
These are common patterns, not rules that determine everything a model can do. Training objectives, attention masks, data, and adaptation matter. For example, calling a model a transformer does not tell me whether it is instruction-tuned, supports images, or can search the web. In an interview, I would separate architecture from training objective and from the capabilities supplied by the surrounding application.
16. What does self-attention do?
Self-attention lets each token representation incorporate information from other positions in the same sequence. In scaled dot-product attention, queries and keys determine relevance scores; a softmax converts scores into weights; those weights combine value vectors. A compact expression is Attention(Q,K,V) = softmax(QKแต / sqrt(dk))V, with masks applied where appropriate.
For example, interpreting โitโ in a sentence may require information from an earlier noun. Attention provides a trainable mechanism for connecting the positions, though an attention weight alone is not a reliable explanation of the model's full decision. Multiple heads and layers can learn different patterns.
In a causal language model, masking prevents a position from reading future tokens when making its prediction. That is essential during training: otherwise the target could leak into the input used to predict it. I would also distinguish self-attention from cross-attention, where queries come from one representation and keys and values from another, such as a decoder attending to an encoded source.
17. Why do transformers need positional information?
Attention needs a way to account for sequence order. Without position-dependent information or an equivalent architectural mechanism, a collection of token representations does not by itself distinguish all ordered arrangements. Positional methods help the model represent relationships such as which token came first and how far apart two tokens are.
Consider โthe dog chased the catโ and โthe cat chased the dog.โ The words overlap, but their order changes who performed the action. A language model must be sensitive to that difference rather than treat the sentence as an unordered bag of words.
Implementations can use learned position embeddings, sinusoidal encodings, relative-position methods, or rotary approaches. These are not identical, and their behavior at longer sequence lengths depends on design and training. I would not assume that changing a configuration value makes a model reliable at arbitrary lengths. Positional information supplies an order signal; it does not automatically guarantee that the model correctly tracks every relationship across a very long document.
18. What is a context window?
A context window is the amount of tokenized information a model can process within a particular invocation or generation setup. It can include instructions, conversation history, supplied documents, and generated tokens, depending on the model and interface. An application may separately impose output limits, so I would check both rather than quote one number as every limit.
If a user submits a long report, the relevant issue is not only whether it fits. The model must also locate and use the right evidence. More available context does not guarantee equal attention to every detail, and placing a fact in the prompt does not ensure that the answer will preserve it.
Conversation memory is also distinct from context. An application can store earlier messages or summaries, but the model only receives what is included in its current input. I would ask what is retained, truncated, summarized, or retrieved. For a precise question about a report, a focused evidence passage may be easier to verify than an unnecessarily large, noisy input.
19. What are model parameters, and do more parameters guarantee a better model?
Parameters are learned numerical values, such as weights in neural-network layers and entries in embedding matrices. Training adjusts them to improve an objective. They are different from settings chosen around training or inference, such as learning rate, batch size, temperature, or maximum output length, which are commonly called hyperparameters or configuration settings.
A larger parameter count can provide greater representational capacity, but it does not independently establish quality. Training data, optimization, architecture, compute allocation, and post-training all affect the result. For mixture-of-experts models, total parameters and parameters activated for a token are also different quantities.
For example, a smaller model adapted to a constrained extraction task may be more useful than a larger general model if it meets the accuracy requirement with lower latency. I would compare measured performance on representative inputs rather than choose solely by size. Parameters also are not database rows: they encode distributed patterns, so extracting or updating a single fact is not equivalent to editing a record.
20. What does self-supervised learning mean?
Self-supervised learning derives training targets from the data itself rather than requiring a person to label every example for a particular task. In next-token prediction, the sequence supplies both the preceding context and the next token to predict. In masked language modeling, parts of the input are hidden and the model learns to recover them.
For example, from โThe package arrived yesterday,โ a causal training example can use โThe package arrivedโ as context and โyesterdayโ as a target, subject to the actual tokenization. The target exists in the text; a human does not need to annotate its sentiment or topic for this objective.
Self-supervised does not mean that data needs no curation, that there is no supervision signal, or that the model is immune to bias. Decisions about sourcing, filtering, deduplication, and representation strongly affect learning. It also does not describe every later training stage: a pretrained model may subsequently learn from demonstrations, preference labels, or other feedback created for specific behaviors.
21. What are cross-entropy loss and perplexity?
For next-token prediction, cross-entropy measures how much probability the model assigns to the observed target tokens. A confident correct prediction receives lower loss than a prediction that assigns little probability to the target. With natural logarithms, perplexity is the exponential of the average negative log-likelihood.
This toy example uses invented target probabilities:
Python example
Lower perplexity means better prediction of the evaluated token sequence under that setup. It is not a direct score for helpfulness, factuality, or safe behavior. Comparisons require compatible tokenization, evaluation data, and context-handling methods. A model can predict common text well yet fail a specialized instruction. I would therefore use language-model loss as one diagnostic and evaluate the actual application task separately. Training loss alone is insufficient: held-out evaluation helps reveal whether the model generalizes beyond examples used to update its weights.
22. How does pretraining differ from fine-tuning?
Pretraining develops broad representations and predictive behavior from a large training corpus. Fine-tuning starts from an existing model and updates some or all of its parameters using additional data for a more focused purpose. That purpose might be following instructions, working in a domain, or performing a specific task.
For example, a pretrained text model may have encountered many kinds of documents. A later training set of requests and well-formed answers can help shape its response behavior. This is different from simply placing examples into a prompt, which normally changes the current input rather than the stored weights.
The boundary is partly about the training pipeline, not a magical change in the model. Continued domain pretraining and supervised task fine-tuning are different adaptation choices. I would ask what objective is used and what behavior should improve. Fine-tuning also carries risks such as overfitting or degrading existing abilities. Evaluation should include the target task and important behaviors that need to remain intact after adaptation.
23. What is instruction tuning?
Instruction tuning trains a model on tasks expressed as instructions, usually paired with desired responses. Its aim is to improve how the model follows requests across tasks rather than merely continue arbitrary text. It is a form of model adaptation and should be distinguished from writing instructions in a prompt at inference time.
For example, training examples might ask for a summary, a translation, or a classification and show an appropriate answer. A useful dataset makes the requested format and behavior consistent while still covering meaningful variation. If all examples reward long answers, the model may become verbose even when a short answer is better.
Instruction tuning does not automatically add reliable current knowledge or remove every hallucination. It teaches behavior through the selected examples and objective. I would test whether the model follows unseen instructions, respects format constraints, and handles requests that lack enough information. Good performance on training-style examples is not sufficient evidence that it generalizes to every user phrasing or every subject area.
24. What do alignment, RLHF, and preference optimization mean?
Alignment is the broad goal of making model behavior better match intended human requirements, including helpfulness and safety. It is not one algorithm and does not imply that a model is perfectly reliable. Reinforcement learning from human feedback, or RLHF, is one approach: human judgments can train a reward model that guides further optimization of the language model.
Preference optimization also includes methods such as direct preference optimization, or DPO, which learns from preferred and rejected responses without the same explicit reward-model-and-RL training loop. The key beginner idea is that feedback concerns which behavior is preferred, not only which next token appeared in a corpus.
For example, reviewers might prefer an answer that acknowledges missing evidence over an invented confident answer. But preferences can be inconsistent, biased, or overly reward style. I would therefore evaluate factual accuracy and other requirements independently of preference scores. Alignment work can reduce specific failure patterns while leaving important limitations; a model's pleasant tone is not proof of correctness.
25. Does a language model automatically know current facts or search the internet?
No. A model's parameters reflect information and patterns learned during training, not a continuously synchronized view of the world. A product may connect the model to search, documents, or other tools, but those capabilities depend on the surrounding system and whether they are actually used for a particular answer.
For example, asking about today's delivery status requires access to the relevant tracking record. A plausible delivery date generated from language patterns is not a lookup result. Even when the answer happens to be correct, I would distinguish verified evidence from an unsupported guess.
A stated training cutoff is also not a guarantee that every earlier fact is known or accurately retained. Training data coverage is uneven, and a model can mix up dates or entities. For changing information, I would use an appropriate current source, preserve when it was checked, and state uncertainty if the source is unavailable. Supplying external evidence can improve the answer, but the model still needs to interpret that evidence correctly.
26. What is greedy decoding?
Greedy decoding selects the highest-scoring next token at each generation step. It makes a locally preferred decision, appends that token, and repeats. It does not search every possible complete answer, so a sequence of locally likely choices is not necessarily the globally most probable sequence or the most useful response.
Here is an isolated decision using invented probabilities:
Python example
Greedy decoding avoids random sampling in that selection step. However, I would not promise identical results across every hardware configuration, model revision, or serving implementation. Changes to numerical computation or input formatting can still affect outputs.
A deterministic-looking answer is also not necessarily a factual answer. If the model strongly prefers an incorrect continuation, greedy decoding will choose it consistently. I would use it when predictable output is useful, but test task quality independently rather than treat low randomness as a substitute for validation.
27. How does temperature affect generation?
Temperature changes how sharply a model's token scores are converted into sampling probabilities. For positive temperature T, the distribution is proportional to exp(logit/T). Lower values concentrate probability on higher-scoring tokens; higher values flatten the distribution. It affects selection variability, not the model's factual knowledge or intelligence.
This pure-Python illustration uses invented logits:
Python example
The function assumes a positive temperature; setting it to zero would divide by zero. Interfaces that accept zero typically implement a special decoding behavior rather than this formula literally. I would choose settings using task examples, not label high temperature โcreative modeโ as if it guaranteed useful novelty. Lower temperature can still repeat the same unsupported claim, while higher temperature can make formatting and consistency harder to maintain.
28. What is top-p sampling, and how is it different from top-k?
Top-p, or nucleus, sampling limits candidates to a high-probability set whose cumulative probability reaches a chosen threshold. The remaining probabilities are renormalized before sampling. Top-k instead keeps a fixed number of highest-scoring candidates. Top-p therefore adapts the candidate count to the shape of the current distribution.
For an illustrative distribution of 0.50, 0.25, 0.15, and 0.10, a top-p threshold of 0.80 retains the first three candidates under the usual smallest-prefix rule: the first two total only 0.75, while three total 0.90. It does not mean every retained token has probability at least 0.80.
These settings alter which continuations can be sampled; they do not verify those continuations. Temperature can also reshape the distribution before filtering, and exact processing order depends on the implementation. I would avoid changing several decoding controls at once when diagnosing behavior. Compare outputs on a fixed task set, recording the model, prompt, and settings so observed changes can be interpreted meaningfully.
29. What makes a prompt clear and useful?
A useful prompt states the task, provides necessary context, identifies important constraints, and describes the desired output. It should make success observable. Adding more words is not automatically helpful; conflicting requirements or irrelevant background can make the task harder to interpret.
For example, instead of โAnalyze this,โ I might write: โSummarize the supplied incident report in three bullets: impact, established cause, and next action. Use only the report. If the cause is not established, say so.โ That prompt gives a scope and an explicit way to handle missing information. It still needs an accurate report and output checking.
I would separate instructions from quoted material and avoid asking the model to infer unstated business rules. If a field must use a fixed vocabulary, I would list the allowed values. Prompting is an iterative design activity: test representative and difficult examples, inspect failures, and revise the instruction that caused ambiguity. A polished prompt is useful, but it is not a security boundary or proof that all outputs will comply.
30. What are zero-shot and few-shot prompting?
Zero-shot prompting gives a task without examples of inputโoutput pairs for that task. Few-shot prompting includes a small set of demonstrations in the input. In both cases, ordinary inference leaves the model weights unchanged. The examples condition the current response; they are not the same as supervised fine-tuning.
For instance, a sentiment prompt could show โWorks perfectly โ positiveโ and โArrived broken โ negativeโ before asking for a label on a new review. The demonstrations communicate the task and format. They can also accidentally bias the output if they cover only obvious cases or repeatedly favor one label.
I would select examples that illustrate meaningful distinctions, including an ambiguous case when that matters. More examples are not always better: they consume context and can introduce contradictions. I would also keep evaluation cases separate from demonstrations so the test remains informative. Few-shot success is evidence of useful in-context behavior, not evidence that the model permanently learned a new rule for every future conversation.
31. How do system, user, and assistant messages relate to the prompt?
Chat interfaces represent a conversation using messages with roles, commonly including system, user, and assistant. A system message often supplies application-level guidance, a user message carries a request, and an assistant message represents a previous or generated reply. Exact roles and instruction-priority semantics depend on the model and interface.
Before inference, a chat template usually converts that structure into the sequence and control tokens expected by the model. This is why passing raw text in an arbitrary format can behave differently from using the correct template. The model ultimately processes a structured encoding, not abstract role labels floating outside its input.
For example, an application might set a consistent concise style while the user asks for a summary. The template should preserve which text is instruction and which is conversation content. I would not treat a system message as a place to store secrets or as an access-control mechanism. Role formatting helps communicate intended behavior, but sensitive actions and permissions require enforcement outside generated text.
32. What is a hallucination in generative AI?
A hallucination is an output that presents unsupported, fabricated, or incorrect content as though it were valid in the task. In a grounded task, an answer can also be unfaithful to the supplied source even if a statement happens to be true elsewhere. It is useful to distinguish factual correctness from faithfulness to evidence.
For example, an assistant summarizing a report might invent a launch date or cite a nonexistent section. The sentence can be grammatical and confident while failing the actual requirement. Asking the model whether it is certain is not an independent verification procedure.
I would reduce the risk by supplying relevant evidence, narrowing the task, allowing explicit uncertainty, and checking consequential claims against trusted sources. None of these alone eliminates the problem. Lower temperature does not turn the model into a fact database, and adding a citation-shaped string does not prove support. Evaluation should inspect whether the evidence exists and actually entails the claim, rather than merely checking that the answer sounds plausible.
33. What is retrieval-augmented generation at a high level?
Retrieval-augmented generation, or RAG, combines access to external information with generated answers. At a high level, the system retrieves potentially relevant material and makes it available to the generator. The aim is to ground the response in information beyond what is stored in the model parameters.
For example, an employee asks about an expense policy. The application retrieves the relevant policy section, and the model explains it using that text. Updating the underlying document can make new information available without retraining the generator, provided retrieval and indexing are also kept up to date.
RAG is not synonymous with a vector database, and it is not a guarantee against hallucinations. The retriever might find the wrong passage, the source might be outdated, or the generator might misread it. I would assess both whether the right evidence was available and whether the answer used it faithfully. At beginner level, the essential distinction is between obtaining evidence and composing a response: success in one stage does not prove success in the other.
34. How do prompting, RAG, and fine-tuning differ?
Prompting changes the instructions and context given to a model. RAG adds a way to obtain relevant external evidence for that context. Fine-tuning changes learned parameters through training. These approaches solve overlapping but different problems, and a useful application can combine them rather than choose exactly one.
For example, an internal policy assistant might use prompting for the desired answer format, retrieval for the current policy text, and fine-tuning for a consistent specialized response style. This is an illustrative design, not a requirement to use all three. A simpler prompt and good evidence may already meet the task.
I would diagnose the failure before picking a technique. Missing or changing facts suggest better access to information; inconsistent behavior may suggest clearer prompts or carefully evaluated adaptation. Fine-tuning is not a dependable substitute for a frequently updated authoritative database. Likewise, retrieval cannot automatically teach a model every procedural skill. I would start with the least complex approach that meets measured requirements and justify added training or infrastructure using observed gaps.
35. What does multimodal mean, and is every multimodal model generative?
A multimodal model processes or connects more than one kind of information, such as text, images, audio, or video. Its supported input and output modalities are separate properties. A model that accepts images and text but returns only text is multimodal without necessarily generating images.
Not every multimodal model is generative. For example, contrastive imageโtext models can learn aligned representations for matching images with descriptions. Their useful output may be a similarity score or embedding rather than a newly created picture or paragraph.
For an application that answers questions about a chart, I would check whether it can read small labels, interpret units, and distinguish visual estimates from exact values. Supplying a picture does not guarantee accurate spatial reasoning or data extraction. I would also consider missing modalities: an audio transcript may omit tone, and a video summary based on selected frames may miss a brief event. โMultimodalโ names a broad capability category, so evaluation must specify the actual inputs and outputs involved.
36. How should a beginner evaluate a generative AI answer?
I would start with the task's acceptance criteria, not a single universal score. For a factual answer, I would check correctness, evidence support, relevance, and whether uncertainty is handled appropriately. For a summary, I would also check coverage and whether it introduces information absent from the source. For code, I would use appropriate tests rather than judge by appearance alone.
A small evaluation set should include typical cases and difficult cases such as missing information, ambiguous instructions, and conflicting evidence. If several answers can be valid, a reference response is a guide rather than an exact string that every output must match.
I would record failures by category and compare candidate changes against the same cases. Human review needs a clear rubric, while automated checks should target properties they can genuinely measure, such as valid structure or required fields. Fluency is easy to notice but can conceal serious errors. A useful evaluation explains what passed, what failed, and which important properties remain untested.
37. Why can the same prompt produce different outputs?
Sampling is one reason: the model may choose among several plausible next tokens rather than always taking the highest-scoring one. Each choice changes the subsequent context, so small early differences can lead to noticeably different answers. But sampling is not the only source of variation.
The actual input may include hidden application formatting, retrieved material, conversation history, or changed instructions. The model revision and execution environment can also change. Even with a fixed seed, numerical behavior and nondeterministic operations can prevent identical results across every software and hardware setup.
For a meaningful comparison, I would record the complete relevant input, model identifier or revision when available, decoding settings, and retrieved evidence. A seed can help reproduce a controlled experiment but is not a universal portability guarantee. I would also distinguish exact repeatability from stable quality: a system can produce different valid paraphrases while remaining reliable, or repeatedly produce the same wrong answer. The evaluation target should match the application rather than require identical wording without a reason.
38. What basic privacy precautions matter when using generative AI?
I would first minimize the sensitive information sent to a model and confirm that the chosen service and configuration are approved for the intended data. Privacy depends on the entire workflow: prompts, uploaded documents, logs, stored outputs, retrieval sources, and any connected tools. It cannot be inferred just from the model's name or whether the interface looks private.
For example, a debugging question rarely needs a real customer's full name, payment details, or an active API secret. A synthetic example or carefully redacted record may preserve the technical issue with less exposure. Redaction itself must be checked so identifiers do not remain in surrounding text or attachments.
Training-data memorization is another concern, distinct from service retention policies: generative models can sometimes reproduce material from training. I would not treat model output as guaranteed anonymous. The practical response is data minimization, access control, verified handling policies, and a clear escalation path for accidental disclosureโnot a promise that a prompt instruction such as โkeep this confidentialโ enforces privacy.
39. What does responsible use of generative AI mean in a beginner project?
Responsible use means defining who can be affected, what can go wrong, and which safeguards are proportionate to the task. I would consider unsupported claims, biased outputs, privacy exposure, misleading synthetic content, and overreliance. These concerns remain relevant even when a prototype performs well on a few examples.
For a rรฉsumรฉ-writing assistant, helpful safeguards include preserving the user's actual experience and flagging invented credentials. For an image-generation tool, users should not be encouraged to present a synthetic scene as documentary evidence. The right controls depend on the workflow, not simply on adding a generic warning below every answer.
I would keep consequential decisions reviewable, provide a way to correct mistakes, and test examples that expose unequal or inappropriate behavior. I would also clearly distinguish model-generated suggestions from verified records. Human review is useful only when the reviewer has enough evidence, time, and authority to challenge the output. Calling something โhuman in the loopโ does not by itself establish that the process is safe or fair.
40. How would you choose a model for a simple generative AI project?
I would define the task and minimum acceptable behavior before comparing models. Relevant dimensions include supported modalities and languages, accuracy on representative examples, context and output requirements, latency, operating cost, deployment constraints, and data-handling requirements. A model that excels on an unrelated benchmark may still be unsuitable for the project.
For example, a short product-description generator has different needs from a policy assistant that must cite internal evidence. I would test both ordinary and difficult inputs, inspect concrete errors, and compare against a simpler baseline such as a template. I would measure the whole workflow, including any retrieval or validation, rather than only the raw model response.
I would also check licensing and operational requirements before committing to a deployment, without assuming that downloadable weights mean unrestricted use. The best choice is the least complicated option that meets the actual requirements with acceptable risk. If no candidate meets an important condition, the honest outcome is to narrow the task or add a review stepโnot declare a winner from an impressive demonstration.
Generative AI Interview Questions and Answers for Intermediate
The intermediate answers focus on implementable contracts, useful measurements, and bounded workflows. Examples make assumptions explicit and distinguish a syntactically valid output from a correct or authorized result.
41. How would you implement and debug a basic RAG pipeline?
I would separate ingestion from answering. Ingestion extracts text, preserves document identities and headings, divides content into retrievable units, and stores embeddings with provenance. Answering prepares the query, retrieves candidates, optionally reranks them, assembles a bounded evidence context, and asks the generator to answer from that evidence.
I would make each stage observable rather than judging only the final sentence. For a question about an expense limit, I want to inspect the extracted policy text, the retrieved chunk IDs, their order, the exact prompt context, and the resulting claims. If the correct paragraph never enters the candidate set, changing the answer prompt is unlikely to fix the root cause.
My first baseline would be deliberately simple: one corpus snapshot, one embedding configuration, fixed top-k retrieval, and a small labeled question set. I would then change one component at a time. Retrieval adds accessible evidence; it does not guarantee that the generator uses it correctly or that the source itself is current.
42. How do you choose chunk sizes and overlap for retrieval?
I would choose chunk boundaries around the units people ask about, not start with a universal token count. A short policy clause may fit in one chunk, while a table needs its headers and a procedure may need several adjacent steps. I would preserve section titles, document IDs, and offsets so a small retrieval hit can be expanded into useful context.
Overlap reduces the chance that an answer falls across a boundary, but it increases index size and can flood results with near-duplicates. For example, 400-token chunks with 80-token overlap advance by 320 tokens; that is an experimental setting, not a recommended default for every corpus.
I would compare several settings using retrieval recall, answer quality, index size, and context tokens on the same questions. I would also inspect failed examples manually. If a retrieved chunk contains a number but loses its unit or exception clause, a larger or structure-aware chunk may help more than switching embedding models. Deduplication is important when assembling overlapping hits.
43. Why must document and query embedding configurations be compatible?
An index is meaningful only when its document vectors and query vectors occupy a compatible representation space. Matching dimensions is necessary but not sufficient: two unrelated models can both emit 768 numbers without assigning the same meaning to any direction. I would record the model revision, preprocessing, truncation, pooling, normalization, and any required query-versus-document prompts with the index.
Some retrieval models intentionally use different instructions or encoders for queries and passages. Compatibility means following that trained pairing, not forcing identical input strings through an arbitrary encoder. A query such as โrefund deadlineโ and a full policy paragraph also have different lengths and roles, which the retrieval setup should reflect.
When changing the embedding model, I would build a separate index and re-embed the corpus rather than silently mixing vector versions. Before a full rebuild, I would run a representative evaluation subset and inspect score distributions. Similarity thresholds calibrated for the old model should not automatically carry over, even if both systems use cosine similarity.
44. When do cosine similarity, dot product, and Euclidean distance give the same ranking?
Cosine similarity compares vector directions, whereas an unnormalized dot product also depends on vector magnitudes. Euclidean distance measures geometric separation. For unit-normalized vectors, dot product equals cosine similarity, and squared Euclidean distance is 2 - 2 * cosine; maximizing similarity then gives the same ranking as minimizing distance.
Executable Python, using only the standard library:
Python example
I would use the metric expected by the embedding model and configure the index accordingly. Normalization is not automatically an improvement: a model trained for raw inner-product retrieval may encode useful information in vector norms.
45. How would you evaluate an approximate nearest-neighbor index?
I would distinguish embedding quality from index approximation. First, exact search over the same vectors gives a reference nearest-neighbor result. Then I compare the approximate index against that reference using top-k neighbor overlap, latency, memory, and build cost. This measures how faithfully the index approximates the chosen metric, not whether the neighbors actually answer usersโ questions.
For example, recovering nine of the exact top ten neighbors gives 0.9 overlap recall for that query. Those nine could still be semantically unhelpful, so I would separately measure retrieval relevance against human-labeled evidence. An index parameter that improves ANN recall might not improve answer accuracy once enough useful evidence is already present.
I would test across query lengths and metadata-filter selectivities, not just average traffic. A tiny corpus may be better served by exact search because it is simpler and already fast enough. I would tune the search-versus-latency tradeoff on validation queries, then report the selected configuration on a held-out set.
46. How do metadata filters interact with semantic retrieval?
Metadata filters restrict which documents are eligible, while semantic similarity ranks eligible content. I would use explicit metadata for constraints such as product version, language, document type, or an effective date instead of hoping the embedding understands every constraint. A query about version 3 should not be answered from a more similar version-2 paragraph.
The order of filtering and candidate selection matters. If a system retrieves ten global neighbors and only then removes nine, it may return one result even though many eligible documents exist farther down the ranking. A filtered search implementation or larger candidate pool may address that, but the right behavior depends on the index.
I would test filter selectivity and missing metadata explicitly. For example, โeffective before 2025-01-01โ needs a defined date field and boundary semantics, not just a text comparison. I would log both filter expressions and surviving evidence IDs when debugging. If no eligible evidence remains, the answer should indicate the gap rather than silently relaxing a userโs constraint.
47. How would you combine lexical and dense retrieval with reciprocal rank fusion?
I would use lexical retrieval to preserve exact identifiers and rare terms, and dense retrieval to capture semantic matches that use different wording. Their raw scores usually have different scales, so adding them without calibration can make one system dominate accidentally. Reciprocal rank fusion, or RRF, combines ranks instead: a document receives a contribution of 1 / (c + rank) from each list where it appears.
Executable Python for two illustrative rankings:
Python example
The constant smooths the influence of very high ranks; 60 here is an example, not a universal optimum. I would deduplicate by stable document or chunk ID and evaluate candidate depths separately. Fusion cannot recover evidence absent from every input list, and overlapping chunks can still crowd out diverse evidence.
48. What does a reranker add after initial retrieval?
A first-stage retriever efficiently narrows a large corpus to a candidate set. A cross-encoder reranker then scores each queryโpassage pair jointly, allowing richer interactions than comparing two independently computed embeddings. I would use it when relevant evidence is present in the candidate set but poorly ordered.
For example, I might retrieve 50 candidates and rerank them to select six context passages. Those numbers would be tuned against latency and answer quality, not treated as architectural requirements. A reranker cannot rescue a correct passage that never reached those 50 candidates, so I would check candidate recall before investing in ranking changes.
I would also inspect truncation: a reranker with a short input limit may never see the relevant sentence at the end of a long passage. Evaluation should compare the original and reranked ordering on identical candidates, then separately measure end-to-end answers. Finally, I would avoid interpreting a reranker score as a calibrated probability of truth; it is usually a relevance signal within its modelโs scoring setup.
49. When is query rewriting useful, and how would you test it?
Query rewriting is useful when the userโs wording is incomplete or poorly aligned with the corpus. In a conversation, โDoes that cover contractors?โ may need to become โDoes the remote-work equipment policy cover contractors?โ before retrieval. I would resolve only what the conversation actually establishes and preserve the original query alongside the rewrite.
The main failure mode is intent drift. A rewrite can invent a product, remove a negation, or turn uncertainty into an assumption. I would compare original-only retrieval, rewritten retrieval, and a union of both on labeled questions. Helpful rewrites should increase relevant evidence recall without causing a disproportionate rise in irrelevant context.
For ambiguous requests, producing two interpretations or asking a clarification can be better than one confident rewrite. I would inspect exact identifiers, dates, and exclusions with deterministic checks where possible. I would also avoid evaluating only the fluency of the rewritten question: the meaningful outcome is whether the downstream answer addresses the original request with valid evidence.
50. How do you assemble retrieved passages into an effective generation context?
I would assemble context as an evidence set, not simply concatenate the first k search results. Each passage should carry a stable source ID, relevant heading, and enough surrounding text to preserve units, exceptions, and references. I would remove near-duplicates and make sure the set covers the different parts of the question.
For a question comparing two plans, six passages about one plan are less useful than balanced evidence for both. I would reserve tokens for instructions, the question, and the answer before filling the remaining budget with evidence. A compact extract can help, but any compression must preserve qualifications and provenance.
I would test passage ordering rather than assume that everything inside the context window is equally usable. Long-context research shows that evidence position can affect performance under tested conditions, but the exact behavior needs measurement on my model and task. My evaluation would include distracting passages and answer-bearing text at different positions. More context is useful only when the generator can identify and correctly use the relevant parts.
51. How would you encourage grounded answers and sensible abstention?
I would define grounding as support for the answerโs claims in the supplied evidence, then make that expectation explicit. The prompt should ask the model to distinguish supported facts, unresolved conflicts, and missing information. An abstention should explain the specific evidence gap rather than produce a generic refusal.
For example, a policy may establish that employees qualify for reimbursement but say nothing about contractors. The grounded answer is that contractor eligibility is not established by the available material, not that contractors are definitely excluded. That distinction prevents the model from turning absence of evidence into a factual prohibition.
I would evaluate answerable and unanswerable questions together. Otherwise, a system that refuses everything can look perfectly grounded while being useless. I would track unsupported claims, correct answers, unnecessary abstentions, and answers given despite insufficient evidence. A retrieval similarity threshold can be one signal, but it is not a universal answerability detector. I would calibrate any threshold on the actual corpus and include cases with conflicting or incomplete passages.
52. How do you verify that citations actually support generated claims?
I would check citation validity and citation support separately. Validity asks whether the cited identifier maps to a passage the system actually retrieved. Support asks whether that passage establishes the adjacent claim, including its number, unit, scope, and qualifiers. A real document attached to an unrelated sentence is still a bad citation.
I would provide the generator a closed set of source IDs and validate every emitted ID against that set. Then I would split substantive statements into checkable claims and inspect the cited text. For example, a source saying โup to 30 days for domestic ordersโ does not support โall orders arrive within 30 days.โ
Citation coverage matters too: one well-supported sentence does not justify an uncited paragraph of extra claims. For a small evaluation set, I would use human annotations of claimโevidence support; automated entailment or model judging can help triage but needs calibration. I would preserve offsets or quoted spans so reviewers can find the evidence quickly and detect extraction or document-version mistakes.
53. Why is schema-constrained generation not the same as a correct answer?
A schema constrains the shape of a response: required keys, data types, allowed values, and supported structural rules. It does not establish that the values are true or appropriate. A valid object containing an invoice total of 900 can still be wrong if the document says 90.
I would use structured generation to reduce parsing ambiguity, then apply semantic validation separately. For an extracted booking, that could mean verifying that departure precedes arrival, the currency is supported, and the cited text contains the claimed amount. Cross-field or business rules may not be expressible in the providerโs supported schema subset, so application validation remains necessary.
I would also handle refusal, truncation, and transport errors as separate outcomes rather than assuming every request produces a complete matching object. The exact API behavior should be checked for the chosen model and endpoint. In an interview, I would describe schema adherence as an interface guarantee within documented conditions, not a guarantee of factuality, authorization, or task success.
54. How would you validate a model-generated extraction before using it?
I would validate in layers: parse the response, verify the schema, check domain constraints, and compare critical fields with the source. Missing data should have an explicit representation such as null; I would not force the model to invent a value simply because a downstream system prefers complete records.
Executable Python illustrating a cross-field check after parsing and type validation:
Python example
This arithmetic does not prove that the quantity or price was extracted correctly. I would still compare those values with their evidence spans. Real invoices may include discounts, taxes, or rounding rules, so the check must match the documentโs meaning rather than impose a simplistic formula.
On failure, I would return a precise validation error or request a bounded correction with the original evidence. Repeatedly retrying until something passes can conceal source ambiguity. I would retain the original extraction and validation result for diagnosing systematic mistakes.
55. How would you design a fair prompt experiment?
I would begin with a specific hypothesis, such as whether explicit evidence-selection instructions reduce unsupported claims. I would hold the model configuration, retrieval results, evaluation cases, and scoring rubric constant while changing the prompt. If I change all of them at once, I cannot attribute any improvement to the wording.
I would develop prompts on a development set and reserve a separate test set for the final comparison. The cases should include normal requests, edge cases, and examples where the current prompt fails. For stochastic generation, I would run repeated samples or otherwise quantify variability instead of trusting one favorable output.
I would report task quality alongside latency, output length, and failure rates. A prompt that writes twice as much may appear more helpful to a judge without becoming more correct. I would version the full messages, few-shot examples, and parser rules, not just the final instruction sentence. After selecting a candidate, I would inspect changed outcomes: which errors disappeared, which new ones appeared, and whether the gain is relevant to users.
56. How do you select few-shot examples without overfitting the prompt?
I would select examples that demonstrate the behavior the model needs to learn from context: the output format, decision boundaries, handling of missing evidence, and important exceptions. Several nearly identical easy examples are less useful than a small, diverse set showing meaningful contrasts.
For a classification task, I would cover each label and include a borderline case with the intended rationale. I would keep formatting consistent and make sure the demonstrations do not contradict the written instructions. If examples are selected dynamically, I would evaluate the selector as part of the system because it changes the effective prompt for each query.
I would not include held-out evaluation answers or close paraphrases of them. That can turn a generalization test into a memory test. I would also try alternate example orders and selections on the development set because in-context behavior can be sensitive to demonstrations. Finally, I would compare against a zero-shot baseline: examples consume context, and they should earn their place through measurable gains rather than intuition alone.
57. How do you split generative-AI datasets to avoid leakage?
I would split at the unit that must generalize, not automatically at the individual row. If many questions come from one customer conversation or document, those related items should usually remain in the same partition. Otherwise, a training example can reveal almost the entire answer to a nominally unseen test question.
I would deduplicate exact matches and inspect near-duplicates across partitions. For a future-facing task, a time-based split can be more realistic than random splitting, provided timestamps reflect when information became available. For document QA, I would state whether the test measures new questions about known documents or entirely unseen documents; both are valid, but they test different capabilities.
Preprocessing that learns from data should be fitted only where the protocol permits. The same principle applies to prompt-example selection, synthetic example generation, and threshold tuning. I would freeze the final test set and keep it out of iterative prompt development. If the test set influences decisions repeatedly, I would treat it as development data and create a fresh final evaluation.
58. What belongs in a useful evaluation dataset for a RAG assistant?
I would build cases around the assistantโs actual jobs rather than collect only easy factual questions. Each case should include the question, relevant conversation context, an expected answer or scoring rubric, supporting evidence where available, and tags for important categories. I would include single-passage answers, multi-document synthesis, ambiguous questions, outdated documents, conflicting evidence, and genuinely unanswerable requests.
I would separate evidence labels from answer labels. A question can have several valid supporting passages and several acceptable phrasings, so one exact reference string may be too restrictive. For numerical answers, I would record units and tolerances explicitly. For abstentions, I would specify why the available corpus cannot answer.
A practical starting set might be small enough for careful human review, then expanded with sampled real failures. Synthetic questions can improve coverage, but I would audit them for unnatural phrasing and accidental answer leakage. I would report results by category as well as overall, because a strong average can hide failures on a small but important class of questions.
59. How do precision@k, recall@k, and MRR differ for retrieval?
Precision@k measures how many retrieved items in the first k are relevant; recall@k measures how many of all labeled relevant items were recovered. Reciprocal rank focuses on how early the first relevant result appears, and MRR averages that quantity across queries. These answer different questions, so I would choose them according to the task.
Executable Python for one query with complete relevance labels:
Python example
MRR would be 0.5 only if this were the sole query. I would deduplicate retrieved IDs and define how queries with no relevant documents are handled. Incomplete judgments can undercount useful evidence, and high retrieval scores alone do not prove that the final answer is grounded or correct.
60. When would you use NDCG instead of a binary retrieval metric?
I would use normalized discounted cumulative gain when relevance has grades and order matters. A passage that directly answers the question should count more than one that provides weak background, and useful material near the top should receive more credit than the same material far down the list.
One common convention is DCG@k = sum((2**rel_i - 1) / log2(i + 1)) for one-based ranks. NDCG divides that value by the DCG of the ideal ordering under the same convention. Other implementations use linear gains, so I would state the formula rather than assume every library computes identical values.
For example, an ordering with relevance grades [3, 0, 2] should score below [3, 2, 0]: both retrieve the same documents, but the second exposes useful evidence sooner. I would define the zero-ideal-gain case and tie handling. NDCG is useful for reranking comparisons, but its quality depends on meaningful relevance judgments and a candidate pool that includes the evidence users need.
61. How would you choose metrics for generated answers?
I would choose metrics from the taskโs failure modes. Exact match is useful for canonical labels or identifiers after carefully defined normalization. Numeric tasks need unit-aware comparisons and tolerances. Open-ended explanations need a rubric for factual correctness, completeness, relevance, and unsupported claims rather than one string comparison.
Token-overlap metrics can help track changes, but they can penalize correct paraphrases and reward answers that share words while getting a key fact wrong. Embedding similarity is more tolerant of paraphrasing, yet two statements with opposite conclusions can still be semantically close. I would not use either as a standalone truth detector.
For a support assistant, I might separately score whether the answer resolves the question, cites valid evidence, and avoids inventing policy. I would inspect disagreement between automated metrics and human reviewers to find weaknesses in the evaluation. Finally, I would report failure categories and uncertainty, not just a single aggregate score. A small average gain is less compelling if it comes with more confidently wrong answers on important cases.
62. What is the difference between groundedness, correctness, and relevance?
Groundedness asks whether the answer is supported by the supplied evidence. Correctness asks whether it is factually right under the taskโs reference standard. Relevance asks whether it addresses the userโs question. These properties overlap, but none implies all the others.
Suppose the user asks for the current return window. An answer can faithfully repeat an outdated policy and therefore be grounded in the retrieved text but incorrect for the current question. Another answer can give the correct current window from model memory while citing no supporting evidence, making it correct but ungrounded in that retrieval context. A paragraph accurately describing the storeโs opening hours can be both grounded and true while failing relevance.
I would score these dimensions independently and connect failures to the appropriate component. Stale evidence points toward corpus or filtering problems; unsupported additions point toward generation behavior; answering the wrong question may indicate query interpretation or prompt issues. I would also distinguish factual completeness from groundedness: a short answer can contain no unsupported claims while omitting the central information the user requested.
63. How would you calibrate an LLM-as-a-judge?
I would treat a model judge as an imperfect measurement instrument. First, I would define a rubric with concrete pass and fail examples, then assemble a human-labeled calibration set covering ordinary answers and known failure modes. I would measure agreement and inspect disagreements rather than assume fluent judgments are reliable.
For pairwise comparisons, I would randomize answer order and repeat a subset with the order reversed to detect position bias. I would avoid showing model identities and control for irrelevant verbosity where possible. If the task involves evidence, the judge should see the same evidence and evaluate individual claims against it, not rely only on its own memory.
I would keep the judge version, prompt, and settings fixed during an experiment. Changing the judge changes the measuring scale. Borderline or high-impact cases should receive human review. Finally, I would report judge-derived scores as estimates under that rubric, not objective ground truth; agreement on one dataset does not establish reliability for every domain or language.
64. How would you compare two prompts when their scores are close?
I would evaluate both prompts on the same cases and analyze paired differences. This removes some variation caused by one prompt accidentally receiving easier questions. I would count wins, losses, ties, and critical regressions, then inspect whether the observed gain is concentrated in one category.
If each case has a numerical score, I can bootstrap paired case-level differences to estimate an uncertainty interval. Repeated outputs from one question are not fully independent examples, so I would preserve that grouping rather than inflate the sample size. With stochastic generation, I would also distinguish variation across questions from variation across repeated runs of the same question.
I would define a practically meaningful improvement before choosing the winner. A tiny score increase may not justify longer outputs or more latency. Conversely, an unchanged average can hide a useful reduction in severe errors. If the evidence remains inconclusive, I would report that and collect more representative cases rather than announce a winner based on a rounded decimal or one impressive example.
65. What does a good supervised fine-tuning example look like?
A good SFT example pairs the input the model will actually see with the response we want it to produce. For a chat task, that includes the correct roles, relevant conversation turns, and the intended assistant answer. I would use the modelโs supported chat template and verify tokenization before training.
The training objective is usually next-token prediction on selected target tokens. Whether loss applies to the full sequence, only the completion, or only assistant messages depends on the configuration and data format. I would inspect a tokenized example with its loss mask rather than assume the trainer inferred the intended boundary.
Quality matters more than merely increasing row count. I would remove contradictory labels, fix malformed records, and include examples of uncertainty and missing information when those behaviors are desired. I would also examine length distributions: truncation can remove the answer or leave an incomplete target. For tool-using tasks, examples should preserve the relationship between calls and tool results instead of flattening the exchange into misleading plain text.
66. How would you decide between RAG and fine-tuning for a task?
I would ask whether the main problem is access to information or consistent behavior. RAG is a natural fit when the answer depends on a changing document collection, requires source attribution, or needs details too specific to expect from model memory. Fine-tuning is more relevant when examples can teach a stable output style, classification boundary, or task procedure that prompting does not handle reliably.
For example, an assistant answering questions about a frequently updated handbook needs retrieval. If its responses must follow a specialized triage format, supervised examples may help teach that behavior. The two methods can be combined: retrieval supplies evidence, and a tuned model learns how to use it.
I would first establish a prompt-and-retrieval baseline, then classify its failures. Fine-tuning should not be used as a substitute for fixing missing or incorrect evidence. Likewise, adding more documents will not necessarily teach a precise response protocol. I would compare solutions using held-out task performance, maintenance effort, and the ability to diagnose errors, rather than choosing based on model size alone.
67. How does LoRA reduce trainable parameter count?
LoRA freezes a base weight matrix and learns a low-rank update. For a weight matrix with output dimension d_out and input dimension d_in, the update can be written as B @ A, where A has shape (r, d_in) and B has shape (d_out, r). A scaling factor is commonly applied to that update.
Executable Python for the parameter-count comparison:
Python example
This is a 256-fold reduction for the update parameters of that one matrix, not a promise of 256-fold lower total training memory. Activations, frozen base weights, optimizer states, and which layers receive adapters all matter. I would tune rank and target modules on validation performance. A higher rank increases capacity but can also increase cost or overfitting, so it is not automatically better.
68. What is the difference between LoRA and QLoRA?
LoRA is a parameter-efficient adaptation method: the base weights are frozen and low-rank adapter weights are trained. QLoRA combines adapter training with a quantized frozen base model, reducing the memory needed to hold those base weights. The original QLoRA work uses four-bit quantization and introduces techniques including NormalFloat quantization, double quantization, and paged optimizers.
The important distinction is between storage precision and computation. A four-bit stored base does not mean every arithmetic operation or trainable adapter uses four-bit precision. Implementations dequantize as needed for computation, and gradients update the adapters rather than the frozen quantized base weights.
I would describe memory savings in terms of the complete training configuration, not simply divide all memory by four. Activations, sequence length, optimizer state, and temporary buffers still contribute. I would compare validation quality with an appropriate non-quantized adapter baseline and inspect numerical stability. Finally, I would save the adapter together with the exact base-model and quantization configuration so the resulting setup can be reconstructed and evaluated consistently.
69. How do SFT, DPO, and reward-model-based RLHF use different supervision?
SFT learns from demonstrations: given an input, increase the likelihood of the desired response. Preference training uses comparisons, such as a chosen answer and a rejected answer for the same prompt. That signal says which response is preferred, not necessarily that either response is perfect.
In a common reward-model-based RLHF pipeline, preference pairs train a reward model, and a policy is then optimized using that reward, often with a constraint against drifting too far from a reference policy. DPO instead uses a direct preference objective involving policy and reference response likelihoods, avoiding a separate learned reward model and an online reinforcement-learning sampling loop in its basic formulation.
I would choose based on the available supervision and the behavior to improve. Demonstrations are useful for establishing a task format; comparisons can refine qualities that are easier to rank than to write from scratch. I would audit preference data for shortcuts such as always preferring longer answers. Better preference fit is not proof of factual correctness, so evaluation must still check the actual task.
70. How do you know whether a fine-tuning run actually improved the model?
I would compare the tuned model with the untuned baseline on a held-out task set using the same prompts and inference settings. Lower training loss means the model fits its training targets better; it does not establish that users receive better answers. Even validation loss can miss changes in instruction following, answer format, or factual reliability.
I would track task metrics, important subgroups, and regression cases. For a structured extraction model, that means field accuracy and semantic validation failures, not only valid JSON. I would also test representative capabilities outside the narrow tuning objective to detect unwanted degradation.
Before blaming the learning rate, I would inspect data quality, chat templates, loss masks, and truncation. A small overfit test on a tiny dataset can expose a broken training pipeline, but it is not evidence of generalization. I would select checkpoints using the development protocol and reserve the test set for the final comparison. If gains vanish under a small prompt variation, I would investigate brittleness before treating the tuning as successful.
71. What happens during a model tool-calling cycle?
The model proposes a tool name and arguments; the application decides whether and how to execute that request. The application validates the arguments, calls the tool, records the result, and returns it to the model in the required conversation format. The model can then answer or propose another call. Generating a tool call is not the same as executing the action.
I would preserve the call identifier so each result is associated with the correct request, especially when multiple calls are produced. I would also distinguish a successful empty result from a timeout or validation error. Those outcomes should lead to different next steps.
For example, a weather lookup returning โno matching locationโ should trigger clarification rather than a fabricated forecast. The toolโs schema helps constrain the request shape, but application checks still establish whether the location and operation are meaningful. I would test the lifecycle with local stub tools before connecting external systems, including malformed arguments, missing results, and multiple calls, so orchestration bugs are isolated from model quality.
72. What should a well-designed tool interface return to a language model?
I would return a compact, explicit result with enough structure for the next decision. A useful contract distinguishes success, empty results, and failure, and includes stable identifiers and relevant evidence rather than an unbounded dump. The input schema should describe meaningful argument constraints, but the implementation must still validate them.
For a document lookup, I might return a status, the normalized query, and a list of passages with IDs, titles, and text. For an error, I would provide a machine-readable category and a concise explanation of whether retrying could help. I would not present a timeout as an empty list because the model might incorrectly conclude that no documents exist.
I would define size limits and truncation indicators so the model knows when it has only partial data. If numeric values are returned, units and timestamps should travel with them. During testing, I would feed the same result shapes into the agent repeatedly, including edge cases, and verify that it interprets the contract consistently instead of guessing from prose.
73. How would you keep a simple agent loop from running indefinitely?
I would make termination an explicit part of the loop rather than rely on the model eventually stopping. Useful boundaries include a maximum number of tool steps, a total token budget, a time limit, and detection of repeated calls that produce no new information. The final outcome should distinguish success from a budget-limited or failed attempt.
Illustrative pseudocode, not executable Python:
Plain-text example
I would define progress in task terms: finding a new relevant document may count, while repeating the same search does not. Retries should be limited to failures that might be transient, not applied blindly to invalid arguments. The agent should preserve useful partial findings and explain the unresolved step instead of pretending a forced stop completed the task.
74. How would you manage conversation memory without keeping every message?
I would separate recent conversational context from durable task state and retrievable history. Recent turns preserve local references; structured state tracks facts such as the current goal, selected options, and unresolved questions; retrieval can recover older relevant details when needed. A summary is useful, but it is a lossy representation rather than a perfect replacement for the original conversation.
I would attach provenance and update semantics to important facts. If the user changes a delivery preference, the current preference should supersede the old one rather than coexist as an unexplained contradiction. I would avoid converting a modelโs speculation into a stored user fact.
For example, โconsidering Tuesdayโ and โconfirmed Tuesdayโ should remain different states. I would test memory with corrections, long gaps, and multiple similar tasks to detect cross-task confusion. When a detail is consequential or ambiguous, I would retrieve the original message or ask for clarification. The success measure is whether memory improves task continuity without introducing stale assumptions, not how much conversation text the system can compress.
75. How would you calculate an input and output token budget?
I would treat the context window as a shared budget for all content the model must process, including instructions, history, retrieved passages, tool schemas, and generated tokens under the modelโs accounting rules. I would reserve output capacity before adding optional context, then leave a margin for formatting and estimation error.
Executable Python using illustrative token counts:
Python example
Those numbers are not a tokenizer. In practice I would use the correct model tokenizer or documented counting method, include tool definitions and message overhead, and account for any model-specific reasoning-token rules. Character counts can mislead, especially across languages or for code.
If the budget is exceeded, I would remove redundant evidence, summarize older context with provenance, or split the task. I would not silently truncate the latest user request or the answer-bearing passage. I would also detect output truncation so an incomplete structured response is not treated as successful.
76. How is a KV cache different from a prompt or prefix cache?
A KV cache stores attention keys and values for tokens already processed during autoregressive generation. It avoids recomputing those representations at every decoding step, although the model still computes the new token and attends to the retained context. The cache consumes memory that generally grows with sequence length and concurrent sequences.
Prefix caching reuses computation for a shared prompt prefix, potentially across requests, when the serving system supports it. It is not the same as returning a previously generated answer. A result cache stores an output; a prefix cache helps compute a new output more efficiently.
I would distinguish these when explaining performance. A long static instruction prefix may benefit from prefix reuse, while a long unique conversation still needs substantial prefill work. Exact cache matching, model revision, and serving configuration matter; not every superficially similar prompt is reusable. I would measure time to first token and subsequent decoding separately. Caching should be an optimization whose outputs remain consistent with the intended model computation, not a reason to assume requests have equivalent meaning.
77. How would you test temperature and top-p without confusing their effects?
I would hold the prompt, model, and evaluation cases fixed, then vary one sampling parameter at a time. Temperature rescales logits before sampling; top-p restricts sampling to a probability-mass nucleus. Their interaction matters, so after understanding individual effects I would test a small combined grid rather than assume two settings with similar-looking numbers behave alike.
For factual extraction, I would prioritize field accuracy, consistency, and validation failure rate. For brainstorming, I might also measure diversity and usefulness across several outputs. A lower temperature can reduce sampling variation but does not make the underlying distribution factually correct; a confidently preferred wrong answer can remain wrong.
I would use repeated samples and record seeds when supported, while acknowledging that a seed is not a universal cross-hardware or cross-version reproducibility guarantee. I would also keep output limits constant because longer responses change the opportunity for both useful detail and errors. The selected setting should match the taskโs tolerance for variation, not a blanket rule that creativity always requires high temperature.
78. How do guidance scale and inference steps affect diffusion generation?
For diffusion pipelines that use classifier-free guidance, guidance scale controls how strongly the conditional prediction is emphasized relative to an unconditional prediction. Increasing it can improve prompt adherence, but excessive guidance may reduce diversity or introduce visual artifacts. I would not assume the same numerical setting has the same effect across model families and implementations.
Inference steps control the sampling procedureโs sequence of updates. More steps can improve a result within a suitable setup, but returns depend on the model, scheduler, and whether the model was trained or distilled for a particular step regime. Doubling steps is not a guarantee of better images.
I would run a small controlled grid with fixed prompts and seeds, changing guidance and steps separately. Evaluation would include prompt adherence, visual defects, and generation time rather than aesthetic preference alone. I would save the scheduler and model configuration with the results. For comparisons, I would inspect multiple seeds, since one unusually good image can make an inferior configuration appear stronger than it is.
79. How do image-to-image strength and inpainting masks change an editing task?
In image-to-image diffusion, strength typically controls how much noise is added to the starting image and therefore how far the result can depart from it. Higher strength generally allows larger changes; lower strength tends to preserve more of the source structure. The exact mapping to denoising steps depends on the pipeline and scheduler.
An inpainting mask identifies the region intended for editing. I would verify the mask convention for the chosen implementation rather than assume white and black mean the same thing everywhere. Mask resolution, edge softness, and alignment with the input image can strongly affect seams and unintended changes.
For a product-photo edit, I would evaluate both the requested change and preservation outside the edited region. A visually attractive output is still wrong if it changes the product logo or shape. I would compare several strengths with fixed seeds and inspect boundary artifacts. I would also keep the original image unchanged and retain the mask and settings, so a reviewer can distinguish model behavior from a mistaken preprocessing or coordinate transform.
80. How would you evaluate a multimodal extraction or question-answering system?
I would separate perception errors from reasoning errors. For a chart question, the system first needs to read labels, associate marks with values, and identify the correct series; only then can it calculate or explain the result. A wrong answer caused by a misread axis needs a different fix from correct extraction followed by bad arithmetic.
I would build cases with small text, rotated images, multiple panels, occlusion, and visually similar categories, while preserving realistic input quality. The labels should specify both the expected answer and the visual evidence, such as a page region or table cell. Numeric answers need units and tolerances, and text extraction needs a defined normalization policy.
I would test image resizing and cropping explicitly because preprocessing can remove crucial details. If the task includes generated images, I would separately score instruction adherence, object counts, text accuracy, and preservation of requested content. A text-only judge cannot establish all visual properties reliably. I would use deterministic checks where possible and human review for ambiguous visual judgments rather than collapse everything into one similarity score.
Generative AI Interview Questions and Answers for Experienced
The experienced answers examine how to operate generative AI systems under real constraints. They connect serving efficiency, data controls, human oversight, and failure recovery without promising that a single safeguard or benchmark establishes reliability.
81. How would you decide between a hosted model and self-hosting for a production application?
I would compare complete operating models, not API token prices against GPU rental prices. First I would establish acceptance criteria: task success on representative traffic, tail latency, regional processing constraints, availability, retention requirements, and the permissions the application needs. A candidate that fails a mandatory constraint is not rescued by a lower unit price.
For hosted inference, I would evaluate contractual data handling, rate limits, version changes, outage behavior, and whether exported traces support debugging. For self-hosting, I would include accelerator utilization, spare capacity, deployment engineering, security patches, and on-call ownership. A lightly used private deployment can be expensive even when each forward pass is cheap.
I would prototype both paths against the same workload distribution and report cost per accepted task, including retries and human review. My decision document would specify the traffic or compliance change that triggers reconsideration. I would preserve a narrow provider interface, while avoiding a lowest-common-denominator abstraction that hides meaningful differences in tools, token accounting, or failure semantics.
82. How do tensor, pipeline, and data parallelism affect an inference deployment?
I would choose parallelism from memory requirements and interconnect measurements. Tensor parallelism divides operations within layers across accelerators, so frequent communication can dominate when devices have slow links. Pipeline parallelism divides layers into stages; it can accommodate a model that does not fit on one device, but stage imbalance and pipeline bubbles can hurt latency. Data parallelism adds independent replicas and is usually attractive once a single replica fits comfortably.
For a model spanning two servers, I would not assume that doubling GPUs doubles throughput. I would benchmark realistic prompt lengths, output lengths, concurrency, and network contention. A configuration that improves aggregate tokens per second may worsen interactive time to first token.
I would also inspect failure domains. If one replica requires eight accelerators, losing one accelerator may remove that whole replica, not one eighth of its capacity. Deployment planning therefore includes spare replica capacity, startup time, communication topology, and whether smaller independently replicated models satisfy the quality target more economically.
83. How would you design model routing without making quality unpredictable?
I would route by observable task requirements and measured risk, rather than asking a model whether it feels confident. Useful routing features include requested operation, input size, language, retrieval availability, and whether the output will trigger an external action. Each route needs its own quality, latency, and cost evidence.
For example, a smaller model might classify an incoming support request, while a stronger model drafts an answer when several policies conflict. Escalation should be triggered by explicit checks such as missing evidence, invalid structure, or a high-impact request. A second model's agreement is not proof of correctness because both models may share failure modes.
I would evaluate the router and downstream models as one system, including false easy classifications and the cost of repeated attempts. I would reserve randomized audit traffic for each route so routing does not hide failures from evaluation. Finally, I would version routing rules, cap escalation loops, and maintain a safe manual or non-generative path when no route meets the acceptance criteria.
84. How do you design evaluation slices that expose production risk rather than improve a headline score?
I would define slices before looking at candidate results. Product slices might include language, customer tier, input length, rare document types, and tasks involving irreversible actions. Risk slices would include ambiguous authorization, contradictory evidence, sensitive content, and unsupported requests. These categories answer different questions and should not be collapsed into a single leaderboard.
I would report both traffic-weighted performance and critical-slice results. Suppose ninety-five percent of traffic is routine summarization: a small improvement there can conceal a serious regression in a low-volume approval workflow. That workflow should have an independent release gate, even if its contribution to the average is tiny.
Sample size and dependence matter. Ten paraphrases of one scenario are not ten independent production situations. I would preserve source-case identifiers, distinguish exploratory from confirmatory slices, and report uncertainty rather than ranking tiny differences. New incident-derived cases would enter a development set first, with a separate held-out set retained for release decisions and protection against evaluation overfitting.
85. How would you compare models statistically when requests from the same customer are correlated?
I would define the unit of inference before selecting a statistical test. If many requests come from the same customer or conversation, treating every request as independent produces overly narrow intervals. For a paired offline comparison, I would run both candidates on the same cases and preserve the customer or conversation grouping.
One practical approach is a cluster bootstrap: sample customers with replacement, retain their paired results, and recompute the chosen metric. I would specify whether the target is average customer experience or average request experience; equal customer weighting and traffic weighting estimate different quantities. A large customer should not dominate accidentally.
I would report the estimated difference, uncertainty interval, and a practical improvement threshold. Safety regressions require their own decision rule rather than being traded against stylistic wins. For an online experiment, assignment should usually remain stable within a user or account to reduce contamination. I would also predefine stopping rules: repeatedly checking for a favorable result without a valid sequential design undermines the claimed error control.
86. Why can a training-compute-optimal model be the wrong choice for serving?
Training efficiency and lifetime product efficiency optimize different objectives. Chinchilla studied how to allocate a fixed training budget between model size and training tokens. A deployed product must additionally pay for every subsequent inference, and may serve vastly more tokens than were considered in the training decision.
I would compare total cost over an explicit horizon: training or adaptation cost plus expected inference volume multiplied by measured serving cost, with operational overhead added. A smaller model trained longer can be attractive if its extra training cost is repaid by cheaper repeated inference, provided it satisfies the application's quality requirements. That is an economic hypothesis to test, not a universal scaling law.
For mixture-of-experts models, I would distinguish total parameters from active parameters per token. Reduced active computation does not eliminate weight storage, routing overhead, or inter-device communication. I would benchmark the actual serving topology and traffic mix rather than infer latency from parameter count. The best choice depends on utilization, context lengths, acceptable quality loss, and how quickly requirements or model versions change.
Python example
87. What would a defensible canary rollout look like for a generative-AI feature?
I would promote an immutable release bundle containing the model identifier, prompts, retrieval configuration, tool schemas, policies, and evaluator versions. Changing several of these without recording the bundle makes a regression difficult to attribute or reverse.
The first canary would receive limited, representative traffic with stable assignment and clear rollback conditions. I would monitor service metrics immediately, but recognize that answer correctness and business outcomes may arrive later. A clean latency dashboard is not evidence that the content is safe. High-impact actions would remain behind review until enough relevant outcome evidence exists.
I would compare treatment with a contemporaneous control, inspect critical slices, and account for user-level correlation. Before expansion, I would verify that the rollback path actually restores compatible prompts, indexes, and tools, not just a model name. Shadow traffic can help, but it must not duplicate side effects or send data to an unauthorized provider. Progression should depend on predefined evidence, not simply the passage of a quiet hour.
88. How would you distinguish data drift, model regression, and evaluator drift?
I would maintain separate views of the input distribution, deployed system behavior, and measurement process. Input drift includes changes in languages, document types, request lengths, or task frequency. A model regression is a degradation on a comparable workload after a system change. Evaluator drift occurs when the judge, rubric, or annotation process changes what counts as success.
To separate them, I would keep a frozen sentinel set and replay it across release bundles and evaluator versions. I would also sample fresh production cases, since a frozen set cannot reveal a genuinely new use case. If production scores fall but sentinel scores remain stable, I would investigate traffic and retrieval changes before blaming model weights.
I would audit human and automated scoring disagreement, including by slice. Replacing the judge without rescoring a common bridge set creates a break in the time series. My response would depend on the cause: repair a pipeline, adjust coverage, revise policy, or roll back a release. Automatic retraining is not a universal remedy and may amplify mislabeled feedback.
89. How would you diagnose high time to first token versus slow token streaming?
I would decompose end-to-end latency into queueing, retrieval and other dependencies, prompt processing, first-token generation, and subsequent decoding. A user-visible delay may come from a slow search service rather than the model, so I would trace the complete request before tuning kernels.
Long prompts primarily increase prefill work; long outputs extend decode occupancy. Under heavy load, queued requests can dominate both. I would compare latency by input length, output length, concurrency, and cache-hit status, using percentiles rather than only averages. Measurements should include rejected and timed-out requests, not just successful responses.
If prefill is the problem, I would test shorter evidence selection, legitimate prefix reuse, or prefill scheduling changes. If decoding is slow, I would investigate memory bandwidth, batching, model size, and output budgets. Separating prefill and decode workers may help some workloads but adds transfer and operational costs. I would judge a change against the complete service objective: accelerating initial tokens while making long streams unusable is not an improvement.
90. How would you tune continuous batching while protecting interactive latency?
I would optimize throughput subject to explicit latency and fairness constraints, not maximize batch size in isolation. Continuous batching admits and removes sequences as generation progresses, reducing wasted capacity compared with waiting for every request in a fixed batch to finish. However, requests still compete for compute and KV-cache memory.
I would replay a realistic mixture of short chats, long prompts, and long generations. Then I would vary token budgets and concurrency limits while measuring queue time, time to first token, inter-token delay, memory pressure, and cancellation behavior. Large prefill requests can disrupt active decoding unless the scheduler manages their work carefully.
Admission control is part of tuning. I would bound queued work by estimated tokens as well as request count, separate offline jobs from interactive traffic when necessary, and enforce per-tenant limits. Canceled requests should promptly release resources. The production setting would retain headroom for bursts and accelerator failures; a benchmark configuration that saturates every resource can collapse when normal traffic variability appears.
91. How do you estimate KV-cache memory before admitting long-context requests?
For a conventional decoder with grouped-query attention, I would start with the number of layers, KV heads, head dimension, stored tokens, bytes per element, and concurrent sequences. The factor of two accounts for keys and values. Query-head count is not interchangeable with KV-head count.
Python example
Sixteen such sequences require roughly sixteen GiB for this cache alone, before model weights, activations, allocator overhead, and serving workspaces. This is an illustrative estimate, not a sizing guarantee; alternative attention architectures and cache layouts change it.
I would reserve space for tokens still to be generated rather than admit requests using only current prompt length. I would validate estimates against measured peak allocations and fragmentation. When memory is exhausted, uncontrolled preemption can sharply increase latency, so admission limits, output budgets, and workload isolation should be deliberate parts of the service contract.
92. What security and correctness issues arise when sharing prefix caches across tenants?
I would treat prefix-cache reuse as an optimization within a trust boundary, not as a reason to merge tenant boundaries. Identical token prefixes may permit computation reuse, but cache timing, metadata, and implementation bugs can still create information exposure. A shared global cache needs a documented threat model rather than an assumption that cached tensors are harmless.
I would include tenant or security-domain identity in the cache namespace where isolation is required, along with all configuration that affects computation: model revision, adapter, tokenizer behavior, and relevant execution settings. I would distinguish model KV caches from application answer caches; the latter additionally require authorization, freshness, and output-policy checks.
For example, revoking access to a document must invalidate any application response that could still reveal it, even if the underlying KV cache is only an internal optimization. I would test cross-tenant collisions, stale adapters, eviction behavior, and timing-sensitive abuse. I would also measure whether isolation removes enough reuse to change the business case; security requirements take precedence over the benchmark hit rate.
93. How would you decide whether quantization is acceptable in a production model?
I would treat quantization as a new model-and-runtime configuration requiring qualification. Lower precision can reduce weight memory and bandwidth demands, but the realized speedup depends on hardware kernels, batch size, and whether weights, activations, or the KV cache are quantized. Smaller files do not automatically mean lower latency.
I would compare the quantized candidate with the reference on representative tasks and stress slices: numerical extraction, long contexts, rare tokens, structured outputs, and tool arguments. An average score may miss a small but operationally serious increase in malformed identifiers or incorrect amounts. Calibration data should resemble deployment inputs without contaminating the held-out test set.
Operationally, I would measure peak memory, throughput at the target latency objective, cold-start behavior, and compatibility with adapters or speculative decoding. I would keep a rollback artifact and record the quantization recipe, runtime, and hardware. If the quality loss is concentrated in one workflow, routing that workflow to a reference model may be preferable to abandoning the optimization or applying it universally.
94. When does speculative decoding help, and when can it make serving worse?
Speculative decoding is attractive when producing draft tokens is cheap and the target model accepts enough of them to reduce serial target-model steps. Exact speculative sampling uses a verification and correction procedure designed to preserve the target distribution; simply accepting a draft because it looks plausible does not provide that property.
I would measure accepted tokens per verification step, draft-model overhead, target utilization, memory use, and end-to-end latency. A draft model that works well for ordinary prose may have poor acceptance on code or unfamiliar languages. At high concurrency, the target GPU may already be efficiently utilized, leaving little benefit from extra speculative work.
My experiment would compare identical traffic distributions and output settings, including short outputs where setup costs dominate. I would test compatibility with constrained decoding and other runtime features rather than assume every combination preserves semantics. The deployment decision would be workload-specific, with a straightforward disable switch. I would not promise the speedup reported by a research paper on a different model, accelerator, or serving regime.
95. How would you assess the real cost of increasing the context window?
I would separate maximum accepted context from useful context. A model may accept a large input without reliably connecting every relevant fact, and adding irrelevant material can increase both cost and error opportunities. Long-context qualification should test evidence position, distractors, conflicting versions, and questions requiring several distant passages.
I would compare alternatives using the same task set: full-document prompting, retrieval, hierarchical summaries, and targeted follow-up retrieval. For each, I would measure accuracy, omitted evidence, first-token latency, memory consumption, and cost per accepted answer. Summaries need special scrutiny because an early omission can become invisible downstream.
The infrastructure effect is also important. Longer sequences enlarge KV-cache requirements and reduce the concurrency that fits on a device. Attention costs depend on architecture and implementation, so I would avoid projecting a universal quadratic wall-clock curve. My preferred policy would allocate context by task value, reserve output space, and reject or transform oversize inputs explicitly rather than silently truncate the material most likely to determine the answer.
96. How would you estimate GPU capacity for an inference service?
I would begin with measured throughput at the required latency objective, not a vendor's peak FLOPS. The load model needs arrival rate, prompt and output distributions, burstiness, and the mix of cached versus uncached requests. Prefill and decoding capacity should both be checked because either can become the bottleneck.
Python example
This simplified example only budgets output-token throughput. It does not prove that four replicas satisfy prefill load, memory limits, or tail latency. If each replica needs multiple GPUs, accelerator count is larger than replica count.
I would replay bursts, long-tail outputs, and a replica failure, then add capacity so the remaining fleet still meets the chosen degradation policy. Autoscaling must account for model-loading and warm-up delays. Admission control and bounded queues are necessary because provisioning new GPUs may be much slower than a traffic spike.
97. How would you calculate cost per successful task rather than cost per token?
I would include every attempt and every supporting service in the numerator, then divide by tasks that meet the acceptance criterion. A cheap model that frequently retries, invokes expensive tools, or requires human correction may have a higher effective cost than a more capable model.
Python example
The rates are illustrative, not a provider quote. Other cost could include retrieval and allocated review expense; production accounting should also allocate idle infrastructure and operations where appropriate.
I would report quality and cost together, with rejected or abandoned tasks visible. I would also examine expensive slices separately: the aggregate can hide an unsustainable long-document workflow. Optimizations should target redundant work, unnecessarily long outputs, and avoidable retries before sacrificing the correctness threshold.
98. How would you enforce tenant and document permissions in a RAG system?
I would enforce authorization before protected content enters model context. The application would derive identity and tenant membership from authenticated server-side state, not from a user-supplied prompt or a model-generated filter. Retrieval must intersect relevance with current access rights; a generation-time instruction to ignore unauthorized documents is too late.
I would propagate document identity, permission version, and tenant through ingestion, chunking, search, reranking, citations, and caches. A reranker or external model is another processing boundary, so sending it unauthorized candidate text is itself a leak even if the final answer omits that text. Missing or stale permission metadata should fail closed for protected material.
My tests would include users with similar names, group removal, shared documents, cross-tenant identifiers, and revoked access during an active conversation. I would verify that citations and downloadable originals enforce the same policy. Index-level tenant isolation can simplify reasoning, while shared indexes may be more economical; neither eliminates the need to test the complete authorization path and its synchronization behavior.
99. How would you handle stale documents, permission revocation, and deletion in RAG?
I would design deletion as a lineage problem, not just a vector-store delete call. One document may produce chunks, embeddings, summaries, cached answers, conversation attachments, evaluation fixtures, and backups. Each derivative needs an owner, retention rule, and a way to locate it from the source identifier.
For urgent revocation, I would use an authoritative access check or tombstone layer that blocks use immediately, while asynchronous cleanup removes derived copies. Otherwise, an indexing delay becomes an unintended permission grace period. Updates should carry versions so an older ingestion job cannot overwrite a newer deletion or resurrect obsolete content.
I would expose freshness requirements per workflow. A historical research assistant and a live policy adviser need different guarantees. Retrieval should return version or timestamp evidence, and conflicting versions should be resolved deliberately rather than blended by the model. Finally, I would test deletion end to end, including answer caches and previously generated summaries, and distinguish prompt-time retrieval deletion from attempts to remove information already incorporated into model weights.
100. How would you threat-model prompt injection in an enterprise assistant?
I would identify what the attacker can control, what the assistant can read, and what actions or network destinations it can reach. The main risk is not merely an offensive answer: it is untrusted content influencing a privileged operation. Retrieved documents, web pages, email, images, and tool responses are potential instruction carriers.
I would draw explicit boundaries between user intent, application policy, external evidence, and executable actions. Delimiters and model instructions help communicate those boundaries, but they are not an authorization mechanism. The model should be allowed to propose actions only within capabilities independently checked by application code.
My abuse tests would follow realistic chains: a malicious document asks the assistant to export a private attachment, alter a recipient, or hide a failed operation. I would measure whether the external action is blocked, not just whether the model verbally recognizes an attack. Controls would be layered, with least privilege, constrained data flows, sandboxing, and narrowly scoped approvals reducing impact even when the model follows an injected instruction.
101. How should tool authorization differ from tool-call validation?
Schema validation establishes that a request has the expected shape; authorization establishes that this actor may perform this specific action now. A syntactically valid transfer, file share, or deletion can still be unauthorized. I would therefore keep authorization outside the language model and evaluate it against authenticated identity, resource ownership, action scope, and current policy.
An approval should bind to the exact proposed operation: resource, recipient, amount, relevant version, and expiry. If those fields change after approval, the application should request a new decision rather than reuse the old approval. This prevents an innocent preview from authorizing a different operation at execution time.
I would also separate read capabilities from write capabilities and avoid giving a general-purpose credential to the model. Tool execution would use narrow server-side credentials, record the decision, and verify preconditions again at commit time. The model's explanation of why an action is allowed is useful context for a human, but it is never a substitute for the policy check or the user's actual authority.
102. How would you prevent an assistant from exfiltrating confidential data through tools?
I would control destinations and data movement, not merely scan the final answer. An assistant can leak information through an email body, URL query parameter, uploaded file, image request, or tool argument. Blocking suspicious prose in chat does not cover these channels.
I would establish which data classes may flow to which destinations, then enforce that policy at tool boundaries. An external upload would require an approved destination and appropriate authorization for the specific data. Internal service credentials would never be inserted into model context. Network egress would be restricted so an injected URL cannot become an arbitrary outbound request.
For example, a research tool allowed to read a public site should not automatically be able to submit private document contents to that site's form. I would test redirects, encoded payloads, attachments, and indirect references, while avoiding claims that any detector catches all sensitive information. Where classification is uncertain, high-impact transfers should require review or be blocked. Audit records should reveal what left the boundary without themselves becoming a second unprotected copy of the secret.
103. What should a sandbox for model-generated code actually restrict?
I would assume generated code can be wrong or hostile, regardless of whether the user request appears benign. The sandbox should restrict filesystem access, network egress, available credentials, process creation, resource consumption, and execution duration. A container label alone does not establish an adequate security boundary.
For a document-analysis task, I would mount only the required input files read-only, provide a disposable output directory, and omit cloud credentials and host sockets. I would set CPU, memory, disk, and process limits, then terminate and clean up the environment after the task. Package installation and network access would be separate capabilities rather than ambient privileges.
I would validate output artifacts before exposing them downstream because a generated file can contain active content or misleading links. The application would retain enough metadata to diagnose failures without logging unnecessary private inputs. Finally, I would test escape attempts and denial-of-service cases with the security team. If the environment cannot provide the required isolation, I would narrow the feature instead of relying on a prompt that asks the code to behave safely.
104. How would you design privacy controls across the generative-AI data lifecycle?
I would inventory data flows before choosing a model endpoint. Inputs may travel through retrieval, inference, telemetry, human review, feedback datasets, and backups. A privacy promise about one model call is incomplete if the same content is retained indefinitely in tracing or reused for adaptation.
I would minimize collection, separate identity from task data where possible, and assign retention and access rules to each storage location. Redaction should occur before unnecessary exposure, but I would test whether it removes information required for the task. Pseudonymization is useful, yet it does not automatically make data anonymous when other fields permit re-identification.
For a sensitive support assistant, I might store structured error categories and document identifiers by default, while restricting full-content traces to approved investigations with short retention. I would verify provider terms and technical settings for the actual deployment rather than assume all endpoints behave identically. Deletion requests, export requests, and incident handling should be exercised end to end. Legal or regulatory conclusions would be reviewed by the responsible specialists for the applicable jurisdiction.
105. How would you govern data provenance and intellectual-property risk in model development?
I would maintain a dataset and artifact inventory that records origin, collection method, permitted uses, consent where relevant, transformations, and downstream model versions. A dataset being publicly downloadable does not establish that every intended use is permitted. I would involve legal and procurement specialists rather than infer rights from technical accessibility.
Provenance needs to survive preprocessing. If records are deduplicated, translated, or converted into synthetic examples, the lineage should still connect them to the original source and the transformation. I would also record external model and dataset terms at acquisition time because later reproduction may depend on those conditions.
For generated outputs, I would use risk-based review: public brand assets, code distributed to customers, and internal brainstorming have different exposure. Similarity checks can flag possible copying but do not resolve ownership or infringement questions. My release evidence would state known gaps, excluded sources, and approved uses. A clean provenance record supports investigation and remediation; it is not a blanket guarantee that all possible outputs are legally safe.
106. What makes human review a meaningful safety control rather than a checkbox?
I would place review where a person can still prevent the consequential action. Asking someone to inspect a summary after a payment or publication has already occurred is monitoring, not approval. The review interface should display the proposed action, exact affected resources, source evidence, uncertainty, and material changes since any previous approval.
I would assign reviewers with appropriate expertise and authority, then measure whether they catch representative errors under realistic workload. A high approval rate alone is ambiguous: it may indicate good suggestions or fatigued rubber-stamping. Deliberately seeded evaluation cases, disagreement analysis, and review-time distributions can reveal weaknesses without exposing users to real harmful actions.
The system should make rejection and escalation easy, preserve the review decision, and revalidate important preconditions at execution. For high-volume low-risk tasks, sampling may be appropriate; irreversible or high-impact operations may require individual approval. I would explicitly budget reviewer capacity, since an overloaded queue often turns a nominal safeguard into either an unsafe shortcut or an unusable product.
107. How would you make a long-running agent reliable enough for production?
I would implement the surrounding process as an explicit state machine with durable checkpoints. The model can suggest the next step, but the application should own execution state, budgets, permissions, and terminal conditions. Otherwise a crash or repeated prompt can leave the agent unsure which effects already occurred.
I would define observable success for each stage: a file exists and validates, an external operation has a confirmed identifier, or a reviewer has approved a specific proposal. A persuasive final message is not completion evidence. Tool failures, missing evidence, and ambiguous external states should lead to typed recovery paths rather than another unconstrained reasoning loop.
For a ten-step workflow with independent ninety-eight-percent step success, naive all-step success is approximately 0.98 to the tenth power, or 81.7 percent. Real failures may be correlated, making this only an illustration. I would reduce unnecessary steps, add verification at high-risk boundaries, and test interruption and resume behavior. Reliability comes from controlled recovery and bounded authority, not simply a longer instruction prompt.
Python example
108. How would you prevent duplicate external actions when an agent retries after a timeout?
A timeout means the caller did not receive a timely response; it does not prove the operation failed. I would use an application-generated idempotency key tied to a specific user intent and preserve it across retries. Generating a new key on every retry defeats deduplication.
The server should durably associate that key with the normalized request and result, rejecting reuse with incompatible parameters. For an operation involving multiple systems, I would not promise exactly-once effects unless their contracts support it. An outbox, reconciliation process, or compensating action may be required.
For example, after a timed-out invoice creation, the agent should query by the original operation identifier or retry under the same supported idempotency contract. It should not invent a second invoice and later hope to detect the duplicate. I would test crashes before commit, after commit but before response, and during recovery. Retention windows for idempotency records must cover plausible delayed retries, and the user-facing status should remain pending when the external outcome is genuinely unknown.
109. How would you combine retries, deadlines, and circuit breakers in an agent workflow?
I would classify failures before retrying. A transient transport error may be retryable; an authorization denial, invalid argument, or policy rejection usually requires a different response. Side-effecting operations also need an idempotency or reconciliation strategy. Blind retries can multiply cost and duplicate actions.
Each workflow would have an overall deadline and resource budget, with smaller per-call limits. Backoff with jitter helps avoid synchronized retry storms, but retry ownership should be clear so nested SDK and application retries do not multiply unexpectedly. A circuit breaker can stop repeated calls to an unhealthy dependency, with controlled probes to establish recovery.
I would distinguish dependency failure from a low-quality model answer. Repeating an identical prompt indefinitely is not a meaningful recovery policy. A bounded alternative might request missing evidence, choose an authorized fallback, or escalate to a person. Metrics would include attempts per task, time spent waiting, and abandoned work. The desired behavior during an outage is predictable degradation, not a swarm of agents independently consuming their full budgets against the same failing service.
110. When would you reject a multi-agent design in favor of a single workflow?
I would reject it when the additional agents do not create a measurable advantage on the actual task. Multiple agents add coordination overhead, context transfer, duplicated work, and more opportunities for inconsistent state. Their agreement is not independent evidence when they use similar models and sources.
A multi-agent design is more plausible when subtasks are genuinely separable, can run concurrently, and have explicit output contracts. For example, independent analysis of different document collections may parallelize well, while several agents editing the same external record create coordination hazards. I would assign one owner for final decisions and side effects.
My comparison would hold the total token and latency budgets visible and evaluate a strong single-workflow baseline. I would test missing workers, contradictory outputs, stale context, and malicious content passed between agents. Shared memory needs access controls and provenance; an agent's message should not acquire authority merely because another agent produced it. I would keep the simpler architecture unless the measured quality or completion-time improvement justifies the larger operational and security surface.
111. What observability is necessary to debug a generative-AI incident?
I would connect user-visible outcomes to a structured execution trace: release bundle, authenticated tenant, retrieval identifiers and versions, model calls, tool proposals, authorization decisions, external operation identifiers, and timing. The goal is to reconstruct what happened, not to collect every possible token indefinitely.
I would separate service health from task quality. Latency, errors, queue depth, and resource use explain availability; evidence coverage, invalid outputs, failed verifications, and user corrections explain usefulness. An apparently successful HTTP response may contain a completely failed task. Dashboards should expose failures by workflow and critical slice, not just overall request success.
Sensitive content needs restricted access and retention. Hashes, identifiers, and structured categories may support routine diagnosis, while content-level traces require stronger controls. I would preserve enough release and dependency information to replay a sanitized case, but mark when exact reproduction is impossible. During an incident, the most valuable signal is often the first violated invariant or unauthorized boundary crossing, rather than a long model-generated explanation of its own behavior.
112. How much reproducibility can you promise for generative-AI experiments?
I would promise a reproducible experimental procedure, not automatically identical text. A seed and temperature setting are only part of the execution state. Model revisions, runtime kernels, batching, hardware, retrieval indexes, tool responses, and sampling implementations can change results. Some hosted services do not expose enough internals for bit-for-bit replay.
I would version prompts, datasets, preprocessing, model artifacts or identifiers, runtime configuration, and scoring code. For retrieval, I would preserve document versions and the actual retrieved evidence, since rerunning a query against today's index is a different experiment. For side-effecting tools, replay would use recorded or simulated responses rather than repeating the real action.
My report would separate deterministic checks, such as schema validity on a saved output, from stochastic performance estimates across repeated runs. I would use multiple seeds or repeated trials where variation affects the decision. If a regression appears only under production batching, a serial notebook reproduction is insufficient; I would investigate the serving configuration while keeping the original trace and release bundle available.
113. Why is alignment training insufficient as the main security boundary?
Alignment training can make desirable behavior more likely, but it does not turn a probabilistic model into an access-control system. Training examples cannot enumerate every future input, tool combination, or deployment context. A model that usually refuses harmful instructions may still misinterpret a legitimate request or follow an indirect instruction embedded in external data.
I would therefore evaluate alignment as one layer in a larger system. The application still needs independently enforced permissions, safe tool interfaces, bounded execution, monitoring, and recovery. I would test both harmful compliance and excessive refusal, because a system that blocks legitimate work indiscriminately is not well aligned with its intended purpose.
I would also distinguish preference from truth. Users may prefer fluent, agreeable answers even when the evidence is weak, and feedback can encode conflicting values or annotator bias. My acceptance criteria would include factual and operational checks that cannot be satisfied by persuasive wording alone. Alignment claims should be tied to a defined evaluation and deployment scope, with documented residual risks rather than a general guarantee of safe behavior.
114. How would you choose between DPO-style preference optimization and an online reward-based approach?
I would begin with the available feedback and the behavior I need to improve. DPO-style training can use fixed preferred-versus-rejected response pairs without a separate online reinforcement-learning loop. That simplifies parts of the pipeline, but the dataset still needs relevant prompts, trustworthy comparisons, and adequate coverage of failure modes.
An online reward-based approach can explore responses produced by the evolving policy, potentially addressing weaknesses absent from a static dataset. It also introduces reward-model error, sampling cost, optimization instability, and opportunities to exploit the reward signal rather than improve the underlying task. Neither method makes evaluator quality optional.
I would run a controlled comparison from the same starting model, track held-out task success and safety slices, and examine how much the policy moves from its reference behavior. Preference wins should be checked against verbosity, correctness, and refusal changes. I would avoid choosing the method solely because its training loss improves. The useful outcome is a deployable behavioral improvement that survives independent evaluation, not a higher score from the signal used to train it.
115. What additional deployment tests does a multimodal assistant need beyond text evaluation?
I would test the complete media-processing path, not just the model on clean benchmark images. Uploaded files can fail during decoding, resizing, page selection, OCR, or audio segmentation before the model reasons about anything. The application should detect unsupported formats, corrupted inputs, and partial processing rather than silently answer from incomplete evidence.
For a document assistant, I would vary scan resolution, rotation, handwriting, language, table density, and page order. I would check whether a cited value comes from the intended cell and whether missing pages trigger an explicit limitation. For audio or video, I would test timestamp alignment, overlapping speakers, and errors introduced by sampling frames or truncating recordings.
Security tests should include instructions embedded in visible or extracted media content, oversized files, and sensitive metadata. Operational metrics need upload size, preprocessing latency, and modality-specific failure reasons. A model's strong score on a broad multimodal benchmark is useful context, but the release decision depends on these end-to-end tests and the consequences of mistakes in the intended workflow.
116. How would you evaluate a diffusion image-generation system for a real product?
I would separate distributional image quality from the user's requested outcome. FID compares generated and reference feature distributions, but a favorable score does not establish that an individual image follows a prompt, preserves a specific product, or avoids prohibited content. I would not use it as a single release criterion.
For a product-photo workflow, I would measure object identity, geometry, brand constraints, background instructions, text legibility, and consistency across edits. Human reviewers would compare outputs blindly using a task-specific rubric. I would include difficult combinations and record rejection rates, since selecting only attractive examples hides the cost of regeneration.
I would control resolution, preprocessing, sample count, prompt set, and sampling configuration when comparing candidates. Seed sweeps help expose instability and diversity loss; one favorable seed proves little. Operational tests would include generation time, memory, interrupted jobs, and artifact provenance. The final decision would weigh usable outputs per unit cost, not merely average aesthetic preference, with separate gates for privacy, likeness, and content-policy concerns relevant to the product.
117. How would you use synthetic training data without creating a self-reinforcing feedback loop?
I would treat synthetic data as a controlled intervention with lineage, not as free ground truth. Each example should record its generator, prompt or transformation, validation method, and any human review. Generated labels can repeat the teacher's errors, while filtering only for confident or fluent outputs can reduce diversity.
I would preserve independently collected human or real-world evaluation data and prevent it from entering the generation pipeline. A useful experiment would compare a baseline, a synthetic-data mixture, and a matched-volume alternative under the same training budget. I would inspect rare cases and distribution tails rather than rely only on average accuracy.
For verifiable tasks, executable checks or authoritative references can make synthetic examples more trustworthy, although the checks themselves need coverage. I would monitor whether successive rounds increasingly resemble the model's own preferences instead of the target population. Research on recursive generated-data training identifies collapse risks under studied conditions; it does not mean every synthetic example is harmful. The practical question is whether this particular data mixture adds validated information and preserves needed diversity.
118. How would you respond to an incident involving an incorrect or unauthorized agent action?
I would first contain the harmful capability while preserving evidence. Depending on the incident, that could mean disabling one tool, revoking a credential, pausing a release, or blocking an affected tenant's workflow. A blanket shutdown may be necessary, but a targeted control can preserve safe functionality when the scope is known.
I would reconstruct the chain from user intent through retrieved content, model proposal, authorization decision, execution, and confirmation. The key question is which control failed, not simply whether the model produced a bad sentence. I would identify affected external records and reconcile ambiguous operations before issuing retries that could worsen the incident.
Recovery would include correcting authorized reversible effects, notifying the responsible people through the established process, and validating the repaired control with incident-derived tests. I would also inspect neighboring failure modes rather than patch only the exact prompt. The postmortem should assign owners and evidence-based completion criteria. Restoring traffic requires proof that the relevant boundary now holds and that rollback, monitoring, and customer remediation are ready.
119. How would you design fallbacks for a model-provider outage?
I would define acceptable degradation before the outage. A fallback might be a second authorized provider, a smaller local model, a retrieval-only response, a queued task, or a human handoff. These are not interchangeable: each has different quality, latency, privacy, and tool-capability implications.
I would qualify the fallback on the workflows it may actually receive and verify its data-processing authorization. Automatically sending sensitive traffic to an unapproved provider is not resilience. Prompts, structured-output formats, tokenizer limits, and tool semantics may differ, so an adapter alone does not prove behavioral compatibility.
During failover, I would avoid replaying completed side effects and preserve operation identifiers. Capacity planning matters because a backup sized for normal trickle traffic may fail immediately when the primary disappears. I would exercise failover and recovery under realistic load, including partially streamed responses and long-running jobs. Where no safe fallback exists, an honest temporary limitation is preferable to silently lowering correctness or authorization requirements while presenting the result as equivalent.
120. What evidence would you require before approving a generative-AI system for launch?
I would require a clearly bounded use case, an accountable owner, and explicit definitions of success and unacceptable failure. The launch packet should connect requirements to measured evidence: representative task evaluations, critical risk slices, load tests, security tests, privacy review, and operational recovery exercises.
I would look for demonstrated boundaries, not only favorable averages. Can unauthorized documents enter context? Can a model-generated tool call bypass approval? Are incomplete tasks reported honestly? Does a rollback restore a compatible release bundle? Can the team identify and correct an external action after an ambiguous timeout? These questions expose gaps that a benchmark score cannot resolve.
I would also require an observability plan, an incident owner, user-facing limitations, and a staged rollout with stopping conditions. Known weaknesses need explicit acceptance by the appropriate decision-maker or a restriction that prevents exposure. Launch is not proof of permanent safety: new traffic, model revisions, and dependency changes require continued evaluation. My recommendation would state what is approved, what remains excluded, and which evidence or incident triggers reconsideration.
Generative AI Documentation and Further Reading
This guide emphasizes durable concepts and explicitly scoped examples rather than a latest-model ranking. Model capabilities, library APIs, provider terms, hardware behavior, and security guidance evolve; verify the chosen versions and deployment requirements. Local arithmetic and example checks do not establish model quality, training convergence, production security, or operational performance.
Use these primary research papers and official resources to explore model foundations, adaptation, retrieval, evaluation, serving, and risk management. This is a curated reading list rather than every source consulted. Historical paper results describe their experimental settings, not universal or current model rankings.
Similar Posts