πΎ Overcoming Context Limits with Persistent Agent Memory
Foundation large language models are fundamentally stateless. Without external memory architectures, an AI agent forgets conversational context and enterprise data as soon as an execution turn concludes. Memory and Knowledge Management provides agents with both short-term working memory (to track active multi-step tasks) and long-term episodic/semantic memory (to query enterprise document stores via Vector Databases and Retrieval-Augmented Generation). This chapter explores how persistent storage enables agents to maintain continuity across complex, multi-day operational workflows.
AGENT MEMORY ARCHITECTURE HIERARCHY:
ββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββββββββββββ
β MEMORY TIER β PURPOSE & SCOPE β TYPICAL TECHNOLOGY STACK β
ββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββββββββββ€
β 1. Working Memory β Retaining active conversation β Sliding window token buffers, Redis cache,β
β (Short-Term) β turns, tool logs, and sub-plans β in-memory Python state dicts. β
ββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββββββββββ€
β 2. Episodic Memory β Recalling past agent experiences β Relational databases (PostgreSQL) storing β
β β and historical execution outcomesβ past task logs and reflection summaries. β
ββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββββββββββ€
β 3. Semantic Memory β Storing external domain knowledgeβ Vector Databases (Pinecone, Chroma, Milvusβ
β (Long-Term / RAG) β and enterprise documentation β utilizing dense embeddings & cosine sim.) β
ββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββββββββββββ
AGENTIC RAG RETRIEVAL PIPELINE:
[ USER PROMPT / SUB-TASK ]
β
βΌ
[ EMBEDDING GENERATOR ] ββ> Converts prompt text into dense vector array
β
βΌ
[ VECTOR DATABASE SEARCH ] ββ> Performs Cosine Similarity lookup against document chunks
β
βΌ
[ RETRIEVED CONTEXT ] ββ> Injects top-k relevant document snippets into LLM prompt context
β
βΌ
[ GROUNDED LLM REASONER ] ββ> Generates factually accurate, hallucination-free response.
class AgentMemoryManager:
"""Manages short-term working memory and long-term semantic knowledge retrieval."""
def __init__(self, max_buffer_size: int = 3):
self.working_memory = []
self.max_buffer_size = max_buffer_size
self.knowledge_base = [
{"id": "DOC-101", "content": "Enterprise API security mandates OAuth 2.0 with JWT validation."},
{"id": "DOC-102", "content": "Database migrations require scheduled downtime during off-peak hours."},
{"id": "DOC-103", "content": "All agent tool executions must log correlation IDs in HTTP headers."}
]
def add_conversation_turn(self, role: str, message: str):
"""Maintains a sliding window working memory buffer."""
self.working_memory.append({"role": role, "message": message})
if len(self.working_memory) > self.max_buffer_size:
# Drop oldest memory to prevent context overflow
removed = self.working_memory.pop(0)
print(f"[*] Memory Buffer Overflow: Dropped oldest turn ({removed['role']})")
def query_semantic_knowledge(self, query_keyword: str) -> str:
"""Simulates vector database semantic retrieval."""
print(f"[*] Searching knowledge base for: '{query_keyword}'")
for doc in self.knowledge_base:
if query_keyword.lower() in doc["content"].lower():
return f"[Found Match ({doc['id']})]: {doc['content']}"
return "[Knowledge Base]: No matching enterprise document found."
if __name__ == "__main__":
memory = AgentMemoryManager(max_buffer_size=2)
# Test Short-Term Memory Sliding Window
memory.add_conversation_turn("user", "What is our API security policy?")
memory.add_conversation_turn("assistant", "We use OAuth 2.0.")
memory.add_conversation_turn("user", "Do we log requests?") # Exceeds buffer size (2)
print("\nCurrent Working Memory Buffer:")
for turn in memory.working_memory:
print(f"- {turn['role'].upper()}: {turn['message']}")
# Test Long-Term Knowledge Retrieval
print("\nLong-Term Knowledge Retrieval:")
result = memory.query_semantic_knowledge("migration")
print(result)
VECTOR EMBEDDING & RETRIEVAL CONCEPTS:
ββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββββββββββββ€
β CONCEPT β DESCRIPTION β MATHEMATICAL / TECHNICAL BASIS β
ββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββββββββββ€
β Embedding Vector β High-dimensional array of floats β Maps semantic meaning into dense space β
β β representing text semantics β (e.g., 1536 dimensions for OpenAI ada-002)β
ββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββββββββββ€
β Cosine Similarity β Measures angle between query β Scores ranging from -1.0 to 1.0; closer β
β β vector and document vectors β to 1.0 indicates higher semantic match. β
ββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββββββββββ€
β Chunking Strategy β Splitting large documents into β Fixed-size sliding windows (e.g., 500 β
β β manageable text segments β tokens) with 50-token overlaps for contextβ
ββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββββββββββββ
ENTERPRISE KNOWLEDGE MANAGEMENT CHECKLIST:
ββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β BEST PRACTICE β IMPLEMENTATION & OBJECTIVE β
ββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β Automated Document β Ingest CI/CD pipelines to automatically re-index updated wiki pages and API β
β Sync β documentation into the vector database daily. β
ββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β Hybrid Search β Combine dense vector similarity search with traditional keyword search (BM25) β
β β to ensure high recall for specific error codes or IDs. β
ββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β Data Privacy Guard β Scrub PII, API secrets, and sensitive tokens before generating document β
β β embeddings in third-party vector stores. β
ββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
PART 2 REVIEW CHECKLIST:
1. We explored agent types, internal architectures, reasoning loops, task planning, and memory management.
2. Short-term memory buffers manage active sessions while vector RAG pipelines power long-term semantic knowledge.
3. Next Module: Part 3 explores advanced Agentic AI Architectures, covering Single-Agent and Multi-Agent Systems (Chapters 11-15).