Gen AI

Citations and Attribution

Users don't trust AI. To build trust, a RAG system must provide exact, verifiable citations pointing back to the specific source documents it used to generate its answer.

The LLM doesn't just generate text; it outputs structured markers [Doc1] that the UI maps directly back to the original source text.

Why Does This Exist?

The entire purpose of Retrieval-Augmented Generation (RAG) is to ground the LLM's answers in factual data to prevent hallucination. But from the user's perspective, how do they know the system didn't hallucinate?

If a corporate chatbot says, "You can expense up to $500 for a home office desk," the employee isn't going to just blindly buy a desk. They need to see the actual HR policy. If the chatbot doesn't provide a link to the policy, the employee has to go search the intranet manually, defeating the entire purpose of the chatbot.

Citations and Attribution are the UI and architectural mechanisms that prove the AI's work. By forcing the LLM to explicitly cite its sources inline, you transform a "magic black box" into a verifiable research assistant.

Think of It Like This

The Wikipedia Standard

Imagine reading a Wikipedia article about a controversial historical event. If there are no little bracketed numbers [1] at the end of the sentences, you immediately suspect the author made it up.

When you see the [1], you can click it, scroll to the bottom, and see that the fact came from a specific page in a published history book. You now trust the fact. A production-grade RAG system must operate on the exact same "Wikipedia Standard."

How It Actually Works

Implementing accurate citations is surprisingly difficult because LLMs naturally want to weave information together seamlessly. There are three common approaches to forcing attribution:

1. The Naive Approach (Appended Sources)

The easiest way to do attribution is to just list the titles of the documents the vector database retrieved at the bottom of the answer. Answer: "You get 20 days of PTO." Sources: [HR_Policy.pdf, IT_Hardware.pdf]

Why it's bad: The LLM might have ignored HR_Policy.pdf and hallucinated the answer entirely. Just because the database retrieved a document doesn't mean the LLM actually used it.

2. Prompt Engineering (Inline Citations)

You instruct the LLM in the system prompt: "You will be provided with documents containing a source ID. You must cite your claims using the [SourceID] format."

You format the retrieved context like this:

1[DOC_1]: Employees receive 20 days of PTO.2[DOC_2]: Laptops must be returned upon termination.

The LLM generates: "Employees are eligible for 20 days of paid time off [DOC_1]." The UI parses the [DOC_1] string, turns it into a clickable hyperlink, and displays a tooltip containing the text of [DOC_1].

Why it's better: It forces the LLM to explicitly attribute specific sentences. However, the LLM can still hallucinate a fake citation (e.g., placing [DOC_2] after a claim about PTO).

3. Post-Hoc Verification (The Gold Standard)

Advanced systems use a second, smaller LLM (or an algorithm like Natural Language Inference) to verify the citations after the main LLM generates the answer. The verifier looks at the sentence: "Employees are eligible for 20 days of paid time off [DOC_1]." It reads [DOC_1]. If [DOC_1] does not actually contain that fact, the verifier strips the citation or flags the sentence as a hallucination in the UI (e.g., highlighting it in red).

Show Me the Code

This example demonstrates how to format the context and the prompt to force inline citations.

1import openai2import re3
4def generate_with_citations(query, retrieved_docs):5    # 1. Format the retrieved documents with explicit IDs6    formatted_context = ""7    doc_mapping = {}8    9    for i, doc in enumerate(retrieved_docs, start=1):10        doc_id = f"DOC_{i}"11        doc_mapping[doc_id] = doc["metadata"]["url"]12        formatted_context += f"[{doc_id}]: {doc['text']}\n\n"13        14    # 2. Instruct the LLM to use the IDs15    system_prompt = f"""16    You are a helpful assistant. Answer the user's query based ONLY on the provided context.17    Every factual claim you make MUST be followed by a citation using the exact ID of the 18    document you got the fact from, formatted as [DOC_X].19    20    Context:21    {formatted_context}22    """23    24    response = openai.chat.completions.create(25        model="gpt-4o",26        messages=[27            {"role": "system", "content": system_prompt},28            {"role": "user", "content": query}29        ]30    )31    32    raw_answer = response.choices[0].message.content33    34    # 3. (Optional) Parse the output to build a UI-friendly response35    # This regex finds all [DOC_X] patterns so the frontend can turn them into links36    cited_docs_used = set(re.findall(r'\[(DOC_\d+)\]', raw_answer))37    38    print(f"Raw Output: {raw_answer}\n")39    print("Sources Used:")40    for doc_id in cited_docs_used:41        print(f"- {doc_id}: {doc_mapping.get(doc_id)}")42
43# --- Execution ---44docs = [45    {"text": "The Q3 marketing budget was $50,000.", "metadata": {"url": "finance/q3.pdf"}},46    {"text": "The campaign drove 400 new leads.", "metadata": {"url": "marketing/leads.pdf"}}47]48
49generate_with_citations("How much did we spend in Q3 and what was the result?", docs)50
51# -> Raw Output: In Q3, the marketing budget was $50,000 [DOC_1]. This expenditure 52# -> resulted in the generation of 400 new leads [DOC_2].53# -> 54# -> Sources Used:55# -> - DOC_1: finance/q3.pdf56# -> - DOC_2: marketing/leads.pdf

Watch Out For

Citation Hallucination

LLMs are pattern-matching engines. If you tell them to put [DOC_X] at the end of every sentence, they will enthusiastically do it, even if the document they are citing has absolutely nothing to do with the sentence. You cannot blindly trust that an LLM's citation is accurate just because the token appeared in the output. Post-hoc verification is required for critical applications (like legal or medical RAG).

The Quick Version

RAG grounds answers in facts, but users need proof to trust the system.
Naively listing retrieved documents at the bottom of the chat is insufficient because the LLM might not have used them.
Prompt engineering can force the LLM to emit inline citations (e.g., [DOC_1]) next to specific claims.
The UI maps these inline markers to the original source URLs, allowing the user to click through and verify the AI's work.
Robust systems use a secondary verification step to ensure the LLM didn't hallucinate the citation itself.