Generative AI
3 interview questions in this topic. Expand any question to read the full answer.
“Fine-tuning vs prompting: when should you use each?”
Answer
Think of it as a ladder of adaptation, from cheapest to most involved — you climb only as high as you need to.
Prompting (including few-shot) steers a general model with a well-crafted instruction and a few examples. Nothing about the model changes. It's zero training cost, instant to iterate, and works with the latest base models the moment they ship — but every call pays for the long prompt in tokens, and there's a ceiling on how consistent or specialized you can make it.
RAG layers on top of prompting: it retrieves relevant documents and injects them as context. It's the answer to "the model doesn't know this," not "the model doesn't behave how I want."
Fine-tuning continues training the model on your own labeled examples, so the behavior is baked into the weights. It buys strong, consistent adherence to a style or output format, deep specialization, and shorter prompts at inference — and a small fine-tuned model can match a large general one on a narrow task, cutting cost. The price: you need a quality labeled dataset, it costs time and compute, and it's a snapshot that can go stale.
The mental model that keeps it straight: prompting and RAG change what the model knows and sees; fine-tuning changes how it behaves. So if you have both a knowledge gap and a behavior gap, do both. In practice, start with prompting, add RAG for facts, and reach for fine-tuning — usually parameter-efficient (LoRA) — only when prompting hits a wall.
💡 Note Prompting is giving a talented new hire clear instructions. RAG is handing them the reference binder to look things up. Fine-tuning is sending them through weeks of training so the company's way of doing things becomes second nature. You instruct first, provide references next, and invest in training only when the role demands deep, repeatable expertise.
Related Questions
“How do you reduce hallucinations in an LLM?”
Answer
Be clear about why it happens, because that drives every fix. An LLM is a next-token predictor trained to continue text plausibly. It has no built-in notion of truth, so when a question lands in a gap in its knowledge it doesn't stop — it produces a fluent, confident fabrication. Fluency is the trap: the output reads authoritative whether or not it's correct.
There's no single switch, so you stack defenses, roughly by leverage:
- Ground the model with retrieval (RAG). The highest-leverage fix — the model answers from documents you retrieve instead of from memory, turning "recall a fact" into "summarize this passage."
- Prompt for humility. Instruct it to answer only from the provided context and to say "I don't know" when the context is silent, which removes the pressure to invent.
- Lower the temperature for factual tasks, so it picks the safe, high-probability token instead of a random guess.
- Add a verification step — cite the source sentence for each claim, or run a second pass that checks the answer against the context. Self-consistency (sample several answers, keep what agrees) catches unstable fabrications.
The honest part: you can't drive hallucinations to zero. So a trustworthy design also measures factuality with a benchmark and surfaces uncertainty — show citations and let users verify, rather than hiding the risk.
💡 Note An LLM is a supremely confident student who never learned to say "I'm not sure." Ask a question outside what they studied and they'll answer smoothly anyway. RAG is handing them the reference book; the prompt instruction is telling them it's okay to answer "it's not in here." Both are needed.
Related Questions
“What is retrieval-augmented generation (RAG)?”
Answer
An LLM on its own is a closed book: it only knows what was in its training data, which has a cutoff date and nothing about your private documents. Ask it about last week's release or your internal policy and it will refuse or confidently make something up. Retrieval-augmented generation turns the closed book into an open-book exam — it gives the model the facts it needs at the moment it answers.
The pipeline has two phases:
- Indexing (offline, once per document): split each document into chunks, run each chunk through an embedding model to get a vector, and store the vectors (plus the text) in a vector database.
- Retrieval + generation (per question): embed the user's question, find the top-k chunks whose vectors are nearest to it (semantic search, not keyword match), insert those chunks into the prompt as context, and ask the model to answer grounded in that text — ideally citing it.
The key mechanism is semantic retrieval: because chunks and questions live in the same embedding space, "How do I reset my password?" retrieves a passage titled "Account recovery steps" even with no shared keywords. The wins are concrete — fewer hallucinations, fresh and private knowledge without retraining, and citations you can verify. But the whole system lives or dies on retrieval quality: if the right chunk isn't retrieved, the model can't use it, which is why real systems invest in chunking strategy, hybrid search, and rerankers.
💡 Note Fine-tuning is studying for a closed-book exam — you cram everything into your head and hope you remember it. RAG is an open-book exam — you walk in with the textbook and look up the right page when the question is asked. For facts that change or are too many to memorize, open-book wins.