Agent Memory

Learn how agents remember: short-term memory and conversation history, long-term and semantic memory with retrieval, persistent state across restarts, and the design problems of what to store and what to forget.

28 minIntermediateCode Examples

The Core Idea: A model has no memory of its own. Agent memory is your code deciding what to store, what to bring back into the prompt, and what to forget: a short-term window, long-term facts found by meaning, and state saved to disk.

  1. Beginner: Why Agents Need Memory

Each model call is independent. If the agent should remember that your name is Ana or that you are allergic to peanuts, your code must save that and put it back in the prompt.

Memory typeLastsHoldsExample
Short-termOne task or conversationRecent messagesThe last few turns
Long-termAcross tasks and daysFacts and preferencesUser is allergic to peanuts
SemanticAcross tasksSearchable knowledge, found by meaningPast answers and documents
Persistent stateAcross restartsProgress of a taskWhich step was done

Analogy: Short-term memory is what you hold in your head during a conversation. Long-term memory is your notebook. Persistent state is the bookmark that lets you resume after a break.

  1. Beginner: Short-Term Memory and Conversation History

Conversation history is the list of recent messages and tool results sent with each model call. Because the context window is limited, you keep only a rolling window and drop or summarize the rest.

In the code: A rolling buffer keeps only the last 4 messages, so the oldest message disappears as new ones arrive. Without a long-term store, the agent forgets my name is Ana once it falls out of the window.

Trimming strategies
Keep the newest messages, summarize older ones into a short note, and pin important facts so they never fall out.

  1. Medium: Long-Term and Semantic Memory

Long-term memory stores facts outside the prompt. When the agent needs something, it recalls the few most relevant items and adds them to the prompt. Finding them by meaning instead of exact words is semantic memory.

1
Remember: Save a fact or a summary of an interaction.
2
Embed: Turn the text into a vector of numbers where similar meanings are close together.
3
Recall: Turn the question into a vector and fetch the closest stored items.
4
Use: Add only those items to the next prompt.

Honest note: The code uses simple word overlap as a stand-in for embeddings so it runs without any library. Real systems use embedding models and a vector database, as in the RAG lessons.

  1. Medium: Persistent State

Persistent state is the saved progress of a task: the goal, the steps completed, partial results. Saved to a file or database, it lets an agent resume after a crash, a restart or a long wait instead of starting over.

StoreGood for
JSON filePrototypes and single-user tools
Database (SQL)Many users, queries, reliability
Key-value storeFast session data
Vector databaseSemantic memory

In the code: The facts are saved as JSON, a new memory object loads them back, and recall still works. That is persistence in four lines.

  1. Advanced: The Hard Parts of Memory

ProblemWhat goes wrongMitigation
What to storeStoring everything fills memory with noiseSave only durable facts and useful summaries
Stale factsThe user moved but old data says otherwiseAdd dates, allow updates and expiry
Wrong recallThe most similar item is not the relevant oneRank, limit to a few items, show sources
PrivacyPersonal data kept longer than neededAsk consent, allow viewing and deleting
Memory poisoningA malicious page writes false facts into memoryOnly store from trusted sources, review writes
CostLarge memories inflate every promptRecall a few items, not everything

Privacy: Memory about people is personal data. Let users see, correct and delete what the agent remembers.

  1. Visual Explanation: Three Layers of Memory and the Recall Cycle

The figure compares the three layers of memory, then shows how a stored fact finds its way back into a prompt.

Part 1 · Three Layers of MemoryFrom fast to lasting
Short-termRolling window
Last few messages
↓
Always in the prompt
↓
Fast, small, forgets
Long-termRecalled by meaning
Facts and preferences
↓
Only the best matches enter the prompt
↓
Large, searchable
PersistentSaved to disk
Task progress and memory
↓
Loaded after a restart
↓
Survives crashes
Part 2 · The Recall CycleHow a fact comes back
Store
Remember
Save the fact.
Search
Recall
Find the closest items to the question.
Insert
Prompt
Add the few best items.
Update
Maintain
Correct, expire or delete.
Click to see why we recall only a few itemsCompare ▼
Every recalled item costs tokens and can distract the model. Recalling the top one to three relevant items keeps prompts short and answers focused.

Part 1 reads left to right from fast and small to lasting. Part 2 shows the recall cycle. Open the bar to see why only a few items are recalled.

  1. Python Implementation: Short-Term, Long-Term and Persistent Memory

This script shows a rolling short-term buffer, a long-term memory that recalls facts by similarity, and persistence to a JSON file that survives a restart. Word overlap stands in for embeddings so it runs with no libraries.

agent_memory.pyPython 3.10+ · standard library only
import json, math, os, tempfilefrom collections import Counter, deque # 1. Short-term memory: only the most recent messages (a rolling window)short_term = deque(maxlen=4)for message in ["hi", "my name is Ana", "I like hiking", "what is RAG?", "thanks"]:    short_term.append(message)print("Short-term memory keeps the last 4:", list(short_term)) # 2. Long-term semantic memory: store facts, recall by meaning# (Word overlap stands in for embeddings here. Real systems compare embedding vectors.)STOP = {"the", "a", "an", "is", "of", "in", "to", "does", "what", "which", "who", "where"} def vectorize(text):    words = [w.strip(".,?!").lower() for w in text.split()]    words = [w[:-1] if len(w) > 3 and w.endswith("s") else w for w in words]    return Counter(w for w in words if w not in STOP) def cosine(a, b):    dot = sum(a[w] * b[w] for w in a)    norm = math.sqrt(sum(v * v for v in a.values())) * math.sqrt(sum(v * v for v in b.values()))    return dot / norm if norm else 0.0 class LongTermMemory:    def __init__(self):        self.items = []     def remember(self, text):        self.items.append(text)     def recall(self, query, k=1):        ranked = sorted(self.items, key=lambda item: cosine(vectorize(query), vectorize(item)), reverse=True)        return ranked[:k] memory = LongTermMemory()for fact in ["The user is allergic to peanuts", "The user lives in the city of Pune",             "The user prefers short answers", "The user is building a recipe app"]:    memory.remember(fact) for question in ["What is the user allergic to?", "Which city does the user live in?", "What is the user building?"]:    print(question.ljust(36), "->", memory.recall(question)) # 3. Persistent state: survives a restart because it is saved to diskpath = os.path.join(tempfile.mkdtemp(), "memory.json")with open(path, "w") as f:    json.dump(memory.items, f) restored = LongTermMemory()with open(path) as f:    restored.items = json.load(f)print("After a restart:", len(restored.items), "facts restored | recall:", restored.recall("Does the user like short answers?"))
EXPECTED OUTPUT (your numbers may differ slightly)
Short-term memory keeps the last 4: ['my name is Ana', 'I like hiking', 'what is RAG?', 'thanks']
What is the user allergic to? -> ['The user is allergic to peanuts']
Which city does the user live in? -> ['The user lives in the city of Pune']
What is the user building? -> ['The user is building a recipe app']
After a restart: 4 facts restored | recall: ['The user prefers short answers']

The short-term buffer has already forgotten hi, but the long-term memory still returns the right fact for each question. After the save and reload, the restored memory answers a question about short answers correctly.

Knowledge Check

Question 1: Why does an agent need long-term memory in addition to the conversation history?
Question 2: What does persistent state allow?

Key Points

✓Models have no memory between calls. Your code saves and restores what matters.
✓Short-term memory is a rolling window of recent messages.
✓Long-term memory stores facts outside the prompt and recalls the few most relevant ones.
✓Semantic memory finds items by meaning using embeddings and a vector store.
✓Persistent state saves task progress so the agent can resume after a restart.
✓Decide what to store, keep facts fresh, protect privacy and guard against memory poisoning.

The Big Picture

Memory in One Line

Remember → search by meaning → insert the best few into the prompt → update and forget

Memory turns a one-shot responder into an assistant that gets to know its user and its work.

Up Next: Memory is one source of knowledge. Another is a document collection. In Agents with RAG you will connect agents to retrieval and compare RAG with agentic RAG.