Memory & State

"Three layers:

• (1)Short-term — the message history array. This IS the memory for the current conversation. It's just text appended to the prompt each turn.
• (2)Long-term — persist key facts to a vector store after each conversation. Before the next conversation, retrieve relevant past context. Example: 'User prefers formal tone' or 'User's team uses Python 3.11.'
• (3)Structured memory — for critical facts, store in a database (not just vector). User preferences, past decisions, account metadata. The challenge: what to remember and what to forget. I'd use an LLM-based summarizer at conversation end to extract durable facts, and TTL-based expiry for ephemeral ones."

"Context management strategies:

• (1)Observation summarization — after each tool call, summarize the result to 2-3 sentences instead of keeping the full response.
• (2)Sliding window — keep the system prompt + first user message + last N messages. Drop middle turns.
• (3)Context compaction — Claude's Agent SDK does this automatically: when approaching the limit, it summarizes older turns while preserving key facts.
• (4)Hierarchical delegation — offload sub-tasks to sub-agents with their own context windows. The orchestrator only sees summaries.
• (5)External scratchpad — write intermediate results to a file/DB, reference by ID instead of keeping in context."

“They’re three different things, and people use the words interchangeably, which is exactly why memory bugs are so hard to talk about.

The context window is a hard limit on the model — how many tokens it can attend to in one call. It belongs to the model, it costs money every call, and it is gone the moment the call returns. The model is stateless; nothing persists inside it.

Conversation history is what you choose to resend. It’s your buffer of prior turns, and it’s the thing that grows until it no longer fits. Windowing, summarising and eviction all live here.

Memory is what survives the conversation — preferences, facts, past decisions — stored outside the model and retrieved back into the window when it’s relevant. It is a database you own, with all the ordinary database problems.

The practical consequence is that ‘the agent forgot’ is three different bugs: it never wrote the fact, it wrote it but didn’t retrieve it, or it retrieved it and your history policy then trimmed it back out.”

“First I’d ask how it got there, because most bad memories are an inference written as a fact. The user asked about vegetarian restaurants once, and the agent stored ‘user is vegetarian’. That is a guess promoted to a permanent record, and no correction path fixes a write policy that does that.

For correction itself: memories have to be updatable and supersedable, not merely appendable. When the user says ‘I’m not vegetarian’, that has to supersede the old row — otherwise retrieval returns both and the model picks whichever ranked higher.

So I want three fields on every memory: where it came from, when, and whether it was stated or inferred. Then correction is a real operation — find the conflicting rows, mark them superseded, write the new one.

And the user needs to be able to see and delete what the agent thinks it knows. That’s a trust feature, and in a lot of jurisdictions it’s also a legal requirement.”

“There are two different problems hiding in that question.

If it’s two sessions of the same user — a phone and a laptop — it’s ordinary database concurrency and I’d treat it as such. Version every memory row, compare-and-set on update, and on conflict re-read and re-apply rather than overwrite. Last-write-wins is fine for a preference and catastrophic for a running total.

If it’s two agents writing shared state in a multi-agent system, the real answer is usually don’t. Shared mutable state between agents is what makes those systems impossible to debug. Give each agent its own scratch state, and have exactly one writer for anything shared — a coordinator, or an append-only log the others read.

And there’s a wrinkle specific to agents: a long run reads state at step one and acts at step seven. Whatever it read may be stale by then. So for anything that matters, re-validate at the point of action, not at the point of planning.”

“The request is ‘delete everything you know about me’, and the honest answer is that it lands in five places, not one.

The memory rows are easy. The vector index is not — deleting a row doesn’t remove its embedding unless the index really supports deletes, and plenty only tombstone, which means the vector still shapes the graph until you rebuild. Then the conversation logs, which contain the same facts in prose. Then traces and observability, which nobody remembers and which retain whole prompts. And then the model provider’s retention window, which is a contract question rather than an engineering one.

Which means the real design decision is upstream: tag everything with a subject id at write time, so erasure is a query instead of an excavation. If you can’t enumerate where a user’s data went, you can’t honour the request — and you’ll discover that while a clock is running.

And be careful about derived artifacts. A summary generated from deleted conversations is still derived from them.”