Types of Agent Memory and the
Best Azure Service for Each

Short-term memory is the immediate operational context of an agent: the current conversation, the messages exchanged in this session, the tool calls made and their results, and the transient workflow state that changes with every agent step. It is the AI’s working memory — useful for immediate tasks but limited in scope. Once the session ends, short-term memory is gone.

The context window is short-term memory’s primary implementation — but the context window has a ceiling. At 128K–200K tokens, it is generous but finite. As conversations grow, prior context must be either truncated, summarised, or offloaded to longer-term stores. Short-term memory management is therefore the discipline of determining what must remain in the context window for the current step and what can be safely evicted or compressed.

On Azure, Azure Foundry Agent Service provides built-in session state management — the agent framework handles context threading automatically, with session IDs that persist context across tool calls within a workflow. Azure Cache for Redis is the right backing store for sub-millisecond session state access in high-throughput agents that cannot afford the latency of a database round-trip on every step. Azure SQL Database handles session logging — the persistent record of short-term memory that can be queried for debugging, compliance, and episodic memory extraction after the session ends.

The critical engineering discipline: short-term memory must be actively managed, not passively accumulated. Naive intuition says an agent that remembers more is better. Operating experience says otherwise: an agent that never forgets accumulates contradictions and retrieves noise. Implement compaction — summarising resolved steps — and sliding windows — keeping only the N most recent turns — to prevent context overflow without losing critical continuity.

Long-term memory bridges the gap between isolated sessions. Without it, every conversation starts from zero: the agent has no knowledge of prior interactions, no accumulated understanding of the user’s preferences, and no ability to build on previous work. A coding assistant that remembers you prefer functional components over class components in React, or that your team uses Prettier with tabs — that personalisation requires long-term memory. Without it, you’re re-explaining preferences every session.

Long-term memory on Azure divides into two complementary concerns. Cross-session fact storage — user preferences, important decisions made, recurring patterns identified — is handled by Azure Cosmos DB with its globally distributed, low-latency read performance ideal for per-user memory retrieval on every session start. Vector-based semantic retrieval — finding the relevant memory from past sessions based on the current query — requires the embedding generation capabilities of Azure OpenAI Service and the vector search capabilities of Azure AI Search or Cosmos DB Vector Search.

Microsoft Foundry Memory is the Azure-native memory layer purpose-built for this use case in 2025: it provides a managed memory store that automatically extracts, embeds, and indexes information from agent interactions, with retrieval APIs that surface relevant memories at session start without requiring the application to manage the extraction-embedding-retrieval pipeline manually. This is the fastest path to production-grade long-term memory for Azure-native agent development.

The critical design decision: many production systems split long-term memory into user-specific (preferences, decisions, long-lived constraints) and environment-specific (system state, product catalogs, policies, organisational knowledge) partitions. These partitions have different update frequencies, retention policies, and access patterns — and should be stored and retrieved separately, even if they are ultimately composed into the same context window injection.

Knowledge memory is the RAG layer: the curated corpus of documents, policies, product specifications, and domain knowledge that grounds the agent’s responses in authoritative, current information rather than LLM parametric recall. Semantic memory provides the factual foundation — an organised repository of facts, concepts, and relationships. It is foundational for structured reasoning and consistency.

On Azure, the knowledge memory pipeline begins with Azure Blob Storage as the document corpus — PDFs, Word documents, HTML pages, structured files — and Azure AI Document Intelligence as the extraction layer that converts raw documents into structured, chunked text, handling complex layouts like tables, forms, and multi-column documents that naive text extraction fails on. Extracted content is vectorised using Azure OpenAI Service embedding models and indexed in Azure AI Search for hybrid retrieval combining vector similarity with BM25 keyword ranking.

The production architecture distinguishes between static knowledge (policies, product catalogues, legal documents — updated infrequently, indexed in bulk) and dynamic knowledge (news feeds, live pricing, inventory — updated continuously, indexed incrementally). Static knowledge benefits from full re-indexing on update cycles; dynamic knowledge requires streaming ingestion pipelines triggered on document change events. Azure Event Grid + Azure Functions can trigger re-indexing automatically when new documents land in Blob Storage.

Knowledge memory is the type most exposed to supply-chain risk. Every document in the corpus is a potential RAG poisoning vector — injected content that ranks highly for specific queries and manipulates agent responses without any model compromise. Implement document provenance tracking (source, ingestion timestamp, owner), access controls on indexed content, and periodic adversarial retrieval testing to verify that no injected content is ranking unexpectedly for sensitive query categories.

Episodic memory is the agent’s history of specific events — what it did, what happened as a result, and what it learned from each experience. Records of specific past experiences: what happened, when, in which session, with what outcome. The implementation form is session logs, conversation transcripts, and structured episodes. Unlike semantic memory (which stores generalised facts), episodic memory stores the specific instances that the agent can recall and reason about.

Context continuity across sessions works through episodic memory. “Can you update that function from earlier?” only works if the agent knows which function you mean. Short-term memory handles this within a session, but episodic memory extends it across sessions. An agent that tried three approaches to optimise a database query, and remembers which one actually worked, gets better over time.

On Azure, episodic memory requires two complementary stores. Azure Cosmos DB or Azure SQL Database handles structured episode storage — each episode has a schema: timestamp, agent ID, session ID, actions taken, tools invoked, outcomes observed, and a structured summary. Azure AI Search provides the retrieval layer — enabling semantic search over episode summaries (“find episodes where the user asked about authentication and the approach succeeded”) that pure SQL queries cannot handle.

The production engineering challenge: episodic memory accumulates indefinitely. A well-designed system implements episode compaction — distilling patterns from many episodes into semantic facts — and staleness management — tagging or evicting episodes whose context has been superseded. Stale index problems are particularly nasty for agents that update beliefs over time: you retrieve the old belief confidently, act on it, and only discover the error downstream.

Semantic memory is the agent’s long-term store of what it knows — not what it experienced (that is episodic), but the generalised facts, concepts, relationships, and preferences that hold across contexts. Semantic memory is an organised repository of facts, concepts, and relationships. It is foundational for structured reasoning and consistency. A user who stated their programming language preference six months ago should not need to restate it today — their preference lives in semantic memory.

Semantic memory splits into two architectural sub-types in production. User-specific semantic memory stores personal facts: preferences, constraints, profile attributes, established decisions (“this user always wants Python, uses pytest, works on Ubuntu”). Domain semantic memory stores world facts relevant to the agent’s task: product definitions, organisational terminology, policy facts, conceptual relationships between entities the agent reasons about.

The Azure implementation leverages the vector-native capabilities of Azure Cosmos DB Vector Search for low-latency per-user fact retrieval (facts are retrieved by semantic similarity to the current query context, not by keyword), and Azure AI Search for domain-level semantic retrieval with hybrid ranking. Azure OpenAI Service handles the embedding generation that converts both stored facts and query context into comparable vector representations.

The production challenge unique to semantic memory is belief updating. When a user’s preference changes — they now prefer TypeScript over JavaScript — the old fact must be identified, superseded, and the new fact stored with appropriate provenance. Naive systems accumulate contradictory facts and retrieve both, confusing the agent. Implement versioned facts with effective-from dates and a staleness check on retrieval to surface only the most current version of any given fact category.

Procedural memory is the most underinvested memory type in 2026 — and the one with the highest compounding return. In the assessment of the mem0 State of AI Agent Memory 2026 report, procedural memory tooling is described as “still early-stage” — and that immaturity is exactly why it is the highest-leverage layer to design deliberately: it is where an agent’s performance compounds, and where the ecosystem gives you the least off-the-shelf help. Semantic memory makes an agent informed; procedural memory makes it better at its job.

Think about how you’ve learned to touch type or drive a car. Initially, each action required focused attention. Over time, these skills became automatic. Procedural memory in AI agents works similarly. When a customer service agent encounters a password reset request for the hundredth time, procedural memory means it doesn’t need to reason through the entire workflow from scratch each time.

On Azure, procedural memory maps directly to Azure Logic Apps — where business workflows are defined as durable, versionable, observable process definitions that an agent can invoke, inspect, and progressively refine. Azure Functions implements the discrete procedural steps — stateless, independently testable, updatable without redeploying the full workflow. Microsoft Foundry Agent Service and Microsoft Agent Framework manage the agent’s own system instructions and skill definitions — the procedural knowledge that shapes how the agent approaches any given task class.

The most sophisticated procedural memory implementation in 2026 is self-improving procedures: agents that update their own system instructions based on episodic evidence. LangMem, launched in early 2025, supports procedural memory through agents updating their own system instructions. On Azure, this pattern is implemented by giving the agent write access to a designated procedure store, with a governance gate — human review required before any self-modified procedure becomes active in production.

Shared memory is the coordination layer that enables multi-agent systems to operate coherently. When an orchestrator agent delegates a sub-task to a specialist agent, that specialist needs access to the shared context: the current task state, the work completed so far, the constraints that apply, and the decisions already made. Without shared memory, every agent starts from its own isolated context — producing the definition conflicts and coordination failures that account for a large share of multi-agent system failures in production.

The fundamental design principle: shared memory must be readable by many agents simultaneously but writable through controlled interfaces. Free-for-all write access to shared memory is the distributed systems equivalent of multiple agents arguing over the same fact — last-write-wins semantics produce incoherent agent behaviour. Implement write governance: an orchestrator or a dedicated state service owns writes; specialist agents only read from shared memory and submit proposed updates through the orchestrator for validation.

Azure Cosmos DB handles the durable shared state store — its multi-region writes, optimistic concurrency controls, and change feed API are purpose-built for the multi-writer, high-read workload of multi-agent coordination. Azure AI Search provides shared semantic retrieval — multiple agents query the same indexed knowledge base with the same consistent results. Azure Cache for Redis handles the ephemeral shared state that must be synchronised at sub-millisecond latency across agents running in parallel — task locks, progress counters, and shared working variables.

Microsoft Entra ID is the governance layer: every agent in the shared memory system has its own identity with its own scoped read/write permissions. An agent that should only read financial data should not be able to write to it — and Microsoft Entra ID enforces this at the identity layer, not just at the application layer. This is the technical implementation of the zero-trust-for-AI principle applied to shared memory access.