Gen AI

Episodic and Semantic Memory

Agents need to remember things. Episodic memory is remembering exact past conversations (the transcript). Semantic memory is extracting facts from those conversations to build a permanent knowledge base about the user.

Episodic memory stores raw chat logs for immediate context. Semantic memory extracts the facts (User likes Python, User lives in NY) into a permanent graph or vector database.

Why Does This Exist?

By default, Large Language Models (LLMs) have the memory of a goldfish. Every time you send an API request, the model starts with a completely blank slate.

If you want a chatbot to remember what you said 5 minutes ago, you must pass the entire chat history back into the prompt. This works for a short conversation, but due to strict Context Budgets, you cannot pass a 3-year history of a user's conversations into the prompt. It would cost hundreds of dollars per message and crash the model.

To build true AI Agents (like an AI therapist, or an AI coding assistant) that know the user over months or years, you must implement a robust memory architecture. Cognitive science divides human memory into two types, and AI systems copy this exact structure: Episodic Memory and Semantic Memory.

1. Episodic Memory (The Transcript)

Episodic memory is the exact, chronological record of what happened. It is "episodes" of your life.

In AI, episodic memory is simply the raw chat logs. User: "I am moving to Seattle." Bot: "That is exciting! When are you moving?"

How it's implemented: You store these transcripts in a standard SQL or NoSQL database. When the user sends a new message, you use a "sliding window" to pull the last 10 messages from the database and inject them into the prompt. If you need to recall an episode from 2 years ago, you can use vector search over the chat logs (e.g., searching for "Seattle" to find that specific conversation block).

2. Semantic Memory (The Knowledge Base)

Semantic memory is the extraction of permanent facts and concepts, divorced from the exact conversation where they were learned. If someone asks you what the capital of France is, you know it is Paris (Semantic), but you probably don't remember the exact day in 3rd grade when your teacher told you (Episodic).

In AI, semantic memory is a continuously updated profile of the user. From the episode above, the Semantic Memory extracts: User Location = Seattle.

How it's implemented: Every time a conversation ends (or in the background asynchronously), a "Memory Extractor LLM" reads the Episodic transcript. It is prompted to extract any new facts, preferences, or rules about the user. These facts are stored in a Vector Database or a Knowledge Graph. When the user starts a new conversation a month later, the system queries the Semantic Database, pulls the user's profile, and injects those facts into the System Prompt.

Think of It Like This

The Doctor's Office

Episodic Memory (The Visit Transcript): The doctor records exactly what happened during your visit on Tuesday. "Patient walked in at 2pm, complained of a headache, I asked if they drank water, they said no, I told them to drink water."

Semantic Memory (The Medical Chart): The doctor updates your permanent file. "Patient suffers from chronic dehydration."

When you visit next year, the doctor doesn't read the 3-page transcript from Tuesday (Episodic). They just glance at your medical chart (Semantic) and instantly know you need water.

Show Me the Code

This code demonstrates how to extract Semantic Memory from an Episodic chat log.

1import openai2import json3
4def extract_semantic_memory(chat_transcript, current_semantic_profile):5    prompt = f"""6    You are a Memory Management Agent.7    Read the following conversation transcript (Episodic Memory).8    Extract any new, permanent facts about the user (preferences, location, job, constraints).9    10    Update the current Semantic Profile with these new facts. Output ONLY valid JSON.11    12    Current Profile:13    {json.dumps(current_semantic_profile, indent=2)}14    15    Transcript:16    {chat_transcript}17    """18    19    response = openai.chat.completions.create(20        model="gpt-4o-mini",21        messages=[{"role": "user", "content": prompt}],22        temperature=0.023    )24    25    try:26        return json.loads(response.choices[0].message.content)27    except json.JSONDecodeError:28        return current_semantic_profile # Fallback on error29
30# --- Execution ---31current_profile = {32    "name": "Alex",33    "occupation": "Software Engineer",34    "known_languages": ["Python"]35}36
37# The raw conversation log38episodic_transcript = """39User: I'm trying to build a web app for my dog walking business.40Bot: That sounds fun! Are you going to use Python since you know it?41User: Actually, I want to learn React and TypeScript for this one. I live in 42      New York so I want to build something modern to show local clients.43"""44
45new_profile = extract_semantic_memory(episodic_transcript, current_profile)46print("--- Updated Semantic Memory ---")47print(json.dumps(new_profile, indent=2))48
49# -> --- Updated Semantic Memory ---50# -> {51# ->   "name": "Alex",52# ->   "occupation": "Software Engineer",53# ->   "known_languages": [54# ->     "Python",55# ->     "TypeScript",56# ->     "React"57# ->   ],58# ->   "location": "New York",59# ->   "projects": [60# ->     "Dog walking web app"61# ->   ]62# -> }

Watch Out For

Memory Contradictions

Over a 3-year timespan, users change. In 2024, the user says "I hate JavaScript." In 2026, the user says "I am writing a JavaScript framework." If you just append facts to the Semantic Database, the database will contain two contradictory statements, confusing the LLM. Your Semantic Extraction prompt must be highly tuned to explicitly overwrite or update old facts when new contradictory evidence appears in the episodic logs. Libraries like Mem0 (formerly Embedchain) are built specifically to handle this complex merging logic.

The Quick Version

LLMs have no inherent memory. You must inject memory into the prompt.
Episodic Memory is the raw, chronological transcript of conversations. It provides immediate context but is too large to keep in the prompt forever.
Semantic Memory is the extraction of permanent facts and user preferences from the transcripts.
You use background LLM tasks to continually read episodic logs and update the semantic knowledge base.
In future conversations, you inject the condensed semantic profile into the System Prompt, making the bot appear to "remember" the user over years of interaction.