1:1 mentoring with Big Tech AI engineers
LLM & Agentic

Memory Decisions & Worked Examples

When to use which memory type, with three real-world scenarios: customer support, sales agent, and code assistant.

Last updated

Production6 min readFirst readMemory Implementation & Patterns

After this section you can

  • Use the tradeoff matrix to pick a memory design for a concrete product
  • Work through support, sales, and coding-assistant scenarios and defend each choice
03

Memory Decisions & Worked Examples

Which memory type for which job — a decision tree you can defend, a tradeoff matrix, and three real systems (support, sales, code assistant) traced end to end from write to retrieval.

THE CENTRAL IDEA

Knowing the four layers and three types is table stakes. The skill that gets you hired is choosing — and, more often, choosing to not add memory at all. Most “we need memory” requests are actually “we need to pass state within a session” (short-term, cheap) or “we need company facts” (that’s RAG, already solved). True cross-session per-user memory is powerful, and it is the expensive, privacy-laden, hard-to-evaluate option. This section is about matching the mechanism to the actual need — and knowing when the answer is “none.”

The decision tree — start at “do you need memory at all?”
Depends on anything beyond this single request? NO MEMORY stateless call no Shared across all users? docs, policies, product facts yes ORG-WIDE (RAG) read-only, shared yes Persist beyond this session? need it next time they return? no SHORT-TERM session buffer + summary no SEMANTIC durable fact → long-term EPISODIC past event → long-term yes — fact or event?

Related

More in LLM & Agentic

Get full access to all 74+ sections with code examples, diagrams, and interactive animations.

Unlock Premium