Memory is a judgement problem: what we learned building it for an AI assistant

An AI assistant needs a memory.

That seemed obvious when we started building Zoe. If you tell your assistant that you prefer aisle seats, that your daughter is called Hannah, or that you eat vegetarian food during the week, it is irritating to have to explain it again every time you speak to it.

Zoe is an AI executive assistant that works across your email and calendar and acts on your behalf.

So the first version of the problem looked straightforward. When Zoe learned something useful, we would save it. When a new request came in, we would search those saved facts and give the relevant ones back to the AI.

This worked surprisingly well.

Then we started finding all the ways it could go wrong.

The difficult part of memory turned out not to be storing things. Databases are very good at storing things. The difficult part was deciding what deserved to be remembered, what was still true, which old facts mattered to the question being asked now, and what to do when something new almost, but not quite, contradicted something Zoe already knew.

Over time, our view of memory changed quite a lot.

We started by thinking about recall.

We ended up thinking much more about judgement.

Building on other people's work

There is already a lot of good work on AI memory. Mem0, Zep, Letta/MemGPT and Honcho have each explored a different part of the problem, using conversation summaries, extracted facts, semantic search, episodic memory, profiles, reflection, knowledge graphs and combinations of all of them.

The most useful thing we took from them is that a model does not need to carry an entire conversation history around forever. You can extract what matters, search it later, and reconstruct a useful picture of what happened.

The field has also moved quickly. These systems increasingly combine semantic and lexical search, entity linking and temporal reasoning, and they pay much closer attention to how many tokens retrieval consumes.

Our question became slightly different, because Zoe is a SaaS product rather than a research agent or a process running permanently on someone's laptop.

That introduced some unglamorous constraints. There could be thousands of users. The computer doing a task for someone today might not be the computer doing it tomorrow. A company using Zoe would expect its employees' information to be separated. A user should be able to see what Zoe believes about them, correct it and delete it. And we wanted to be free to change the AI underneath Zoe without losing everything she had learned.

Those requirements started to change the shape of memory.

Our first version was deliberately boring

The early model was roughly this:

conversation
    ↓
identify something worth remembering
    ↓
save it
    ↓
search memories later
    ↓
give relevant memories back to the AI

We stored memories as claims. If someone said, "I don't eat meat during the week," we might store something close to that sentence rather than trying to build a grand model of the person's dietary identity.

When a new memory appeared, we compared it with existing ones. An exact duplicate could be ignored. If there seemed to be one closely related memory, the new statement might replace it. If several memories could plausibly be related, we could stop and ask for a decision rather than guessing.

This was a reasonable first design. It also contained assumptions that became increasingly uncomfortable.

One was that things we had recently stored, or had marked as important, were probably the things the AI most needed to know.

That sounds sensible until someone asks the right question. A minor fact from six months ago can suddenly be more useful than an important fact from yesterday. If the user asks about booking dinner for a particular friend, some obscure dietary restriction may be the only memory that matters.

Importance is not relevance.

Recency is not relevance either.

SaaS changed who was allowed to remember

The first big architectural decision was that Zoe's memory could not belong to whichever AI happened to be doing the work.

The AI could disappear after a task. We might restart it, replace it, or run another copy somewhere else. Zoe therefore had to keep her own record of what she knew about a person.

The AI doing a task can search those memories and use them. It can suggest that Zoe has learned something new. But it does not own them.

That sounds like a small implementation detail. It has turned out to be one of our most important decisions.

Once Zoe owned memory, we could attach things to it that matter in a real product: which user it belongs to, where it came from, whether it is sensitive, when it changed, whether it is still current and whether the person has corrected it. It also meant we could change the AI doing the reasoning without throwing away the relationship Zoe had built with the user.

Memory is separate from authority

There is another distinction that became important as Zoe started doing things rather than simply answering questions.

Remembering that someone usually prefers an aisle seat is useful. Remembering that someone normally schedules meetings with a particular colleague without asking is also useful. Neither gives Zoe permission to act.

Memory answers the question of what Zoe knows. Authority answers the question of what Zoe may do. We have kept those separate on purpose.

The stakes are different from a chat companion. If a companion app wrongly remembers your favourite restaurant, that is annoying. If an assistant that sends, books and schedules on your behalf wrongly remembers that you approve a certain kind of expenditure, or that a meeting can be accepted automatically, the consequences are real. That is why memory, provenance and delegated action have to be designed together.

It matters most inside a company, where personal preferences, team conventions and corporate policies can conflict. Say a user prefers aisle seats and flexible fares, and their employer's policy books the cheapest economy fare. Zoe should remember both. Which one wins when she books the flight should be decided by policy, not by whichever memory scores higher in search.

Then the token bill started to matter

There is another constraint that does not show up in AI demos. Models charge, directly or indirectly, for what they read.

Suppose Zoe has eventually learned 2,000 useful things about a person. That is not a ridiculous number for an assistant someone uses for years. We could put all 2,000 into every prompt, and it would be a terrible idea.

Even if each memory took only 15 tokens to describe, that is 30,000 tokens before the model has read the user's current request, their email, the calendar event it is working on, or anything else the task needs. And we would pay for the model to read those memories again on the next request, and the one after that.

We came to think about memory cost in three ways. Storing memories is cheap. Finding them costs a little more. Making the AI repeatedly read them can become expensive very quickly.

It can also make the AI worse. A larger context window helps when the extra information is relevant. Filling it with facts that happen to be true but have nothing to do with the current question gives the model more to reason about and more chances to get distracted.

So retrieval became a way of deciding what the model does not need to read.

Today Zoe builds a small memory packet for each request. It has two parts: a small core slice of high-importance memories that are useful across many conversations, and memories retrieved using the current user turn, with anything already in the core excluded from the retrieved set. Both sets are bounded, so knowing more about someone does not mean sending more and more information to the model. Each record also carries a trust label. Provenance comes from the run, and for mail, only a verified message from the account owner counts as first-party.

This matters commercially as well. A research prototype can afford three model calls to decide which memories matter, another to reconcile them, and then a large summary passed to the model doing the actual work. A SaaS assistant sold for a fixed monthly price eventually has to pay for all of those decisions.

So we became quite interested in doing the cheap things cheaply. Search with a database when a database can do the job. Use embeddings when semantic similarity helps. Use a model when understanding meaning genuinely requires one, and not merely because you can.

Why a graph is not the centre of Zoe's memory

At various points we considered turning Zoe's memory into a knowledge graph. It is an appealing idea. Instead of storing individual claims, you represent a network:

Nathan
  → knows Steve
  → prefers aisle seats
  → works on Zoe
  → eats vegetarian food on weekdays

Steve
  → works at Company X
  → knows Sarah

Now the assistant could move through a web of people, companies, preferences and events.

There are problems where this is extremely useful. Fraud detection and supply chains are obvious examples. If the relationships between entities are the thing you are trying to understand, a graph may be exactly the right data structure.

It is also not an all-or-nothing choice. Zep's Graphiti, for example, combines graph traversal with semantic and full-text search and tracks when relationships were true. Mem0 takes a somewhat different approach: its retrieval combines semantic similarity, keyword matching and entity matching, using entity linking to connect memories about the same people, places and concepts without requiring an external graph store. Its open-source and platform versions differ in how they handle graphs and entities, so it is worth checking which one you are comparing against. The field is exploring several points between flat vector memory and a full temporal knowledge graph, and we looked at that range carefully.

A graph does not remove the retrieval problem. You still have to decide where to enter it, which relationships to follow and how much of the resulting neighbourhood to put in front of the model. A richly connected graph also creates its own version of the context problem. If Nathan knows Steve, Steve works at a company, that company has a travel policy, and Nathan happens to ask about Steve, should all of that be handed to the model? One hop? Two? Three?

For Zoe today, entity-aware claims plus hybrid retrieval give us most of the value we need without making a graph the primary source of truth. The question that has proved most useful is:

Which small number of things Zoe knows about this person matter to this request?

That leaves us well placed to build a graph as a view over the underlying claims if we find a problem where those relationships genuinely improve the product. We just do not yet think the extra machinery pays its rent.

Search got more sophisticated, but not for the reason we expected

Early retrieval leaned too heavily on importance and recency, and on fairly literal text matching.

That has an obvious weakness. A memory might say:

Nathan doesn't eat meat.

While the current request says:

Find somewhere vegetarian for dinner.

A human sees the connection immediately. A simple keyword search may not. So we added semantic retrieval using embeddings, which let you find things with similar meaning even when they use different words. None of this is novel. Combining lexical and semantic search is standard practice now.

The choice we made on purpose was to keep lexical retrieval as a working fallback, rather than making the semantic layer part of what memory depends on. If the embedding service is unavailable, Zoe should not suddenly forget who your family members are.

The same principle applies to storage. The vector representation of a memory is useful derived information. It is not the memory itself. That distinction has made the system considerably easier to reason about.

Retrieval wasn't the dangerous part

One of our more useful failures came from a mail interaction. Zoe had been handed a bundled incident report from an email, and background extraction turned it into a pending preference in the user's memory. The interface then repeated the model's own scoring language back to the user as if it were something they had said. The underlying problem was that our memory layer would accept any non-empty prose. It was coherent text and potentially useful information, so a simple memory test let it through. It was not something the user had told Zoe about themselves at all.

The system had, in a technical sense, worked. It found text. It extracted something that looked useful. It proposed saving it.

A conversation with an assistant contains all sorts of information that should not become personal memory. A user may paste an email from someone else. They may ask Zoe to summarise a report, quote another person's opinion, issue a temporary instruction, describe an error, speculate, or change their mind halfway through.

The fact that a sentence contains useful information does not mean it belongs in the user's permanent memory.

So we added a step before normal memory processing: should this become memory at all?

Some things are rejected. Some belong in a more authoritative part of the user's profile rather than being inferred. Some weak inferences can be kept temporarily while we see whether they are corroborated. Sensitive inferences get particularly careful treatment. Only after passing that test does a potential memory go further.

This changed our view of the AI's role as well. Models are very good at understanding language. We are much less comfortable letting a model directly rewrite what Zoe believes to be true.

Our rule has gradually become:

The model can help decide. It should not be the database administrator.

Corrections are harder than new facts

Imagine Zoe has remembered:

Nathan's mobile number is 07123 456789.

Then Nathan says:

I've got a new mobile number. It's 07987 654321.

The simplest memory system edits the first record. That gives you the right answer today. It also destroys what happened yesterday.

We prefer to think of this as a change in the history of a claim. The old number was once believed to be correct, and a new number replaced it. So instead of silently rewriting the first memory, Zoe marks it as superseded and stores the new one as current. That gives us a history of what changed and why.

Real language makes this harder. Consider:

I eat vegetarian food during the week.

Then later:

I'm eating meat tonight.

Has the preference changed? Is tonight an exception? Was the original statement wrong?

Text similarity cannot answer those questions. Today Zoe would not automatically use the second statement to replace the first. It would first identify the earlier claim as potentially related, then ask the interpretation step whether the new statement supersedes it, adds a separate fact, or leaves the relationship uncertain. Only a validated result can change what is stored; an uncertain result falls back to review.

A model can help interpret that relationship, but Zoe validates the answer before anything permanent changes. Invalid, incomplete or low-confidence results fall back to review. That safeguard turned out to matter in testing: one low-reasoning JSON configuration missed a conditional preference and also failed to recognise an explicit correction. Both answers looked plausible enough that we would not want them silently changing the user's memory.

Cheap models were not automatically good enough

Memory maintenance looks like exactly the sort of background work you would hand to the cheapest possible model.

We tried that. The results were encouraging and slightly uncomfortable.

Faster models did very well on small sets of memory interpretation tasks, but their failures mattered. A model that misunderstands one ordinary chat response gives a bad answer. A model that misunderstands a correction may quietly change what the assistant believes about the person, and carry that error into every future conversation.

In a later eight-case correction set, a Groq-hosted model passed 14 of 16 checks in each of two prompt variants, with no false supersessions. The uncomfortable part was that the two runs did not fail on exactly the same cases. DeepSeek passed all eight cases for each prompt variant, but in a single run, and it was much slower. This was a small evaluation, not a benchmark, but it was enough to convince us not to optimise this part of the system aggressively before we have better evidence.

Some AI tasks are cheap because being approximately right is fine. Memory is often not one of them.

We became suspicious of memory decay

A popular idea in memory systems is that old memories should gradually become less important. Mathematically, it is attractive. The older a memory gets, the lower its score, and eventually it fades away.

The problem is that different memories age in completely different ways.

"My passport expires next April" has a clear useful lifetime.

"I am allergic to peanuts" should not quietly become less true because nobody has mentioned peanuts for twelve months.

"I like restaurant X" probably can become weaker evidence over time.

Those are three different problems hiding behind the word "old."

So Zoe does not apply one universal decay curve to personal memory. Some memories can explicitly expire. Some are replaced when better information appears. Recency can help break a tie. Temporary observations can disappear if they are never confirmed. But a fact does not become less true merely because it has been sitting in the database for a long time.

One way we think about it: age is the timestamp on the card, and retention is whether the card should stay in the deck. Whether Zoe can find the right card when she needs it is a separate question, and it belongs to retrieval.

We still have work to do here. Understanding "this was true last year, but isn't true today" is harder than putting an expiry date on a database row.

Other systems are taking more explicit temporal approaches, including tracking when a fact was true in the world separately from when the system learned it. We may get there. For now we have chosen explicit expiry, supersession and provenance over a universal decay rule, and we would rather add temporal structure when we can show it earns its place.

What the system looks like now

After all of this, Zoe's memory architecture is not exotic. That is partly the point.

At a high level it looks like this:

conversation or action
        ↓
possible memory
        ↓
is this actually something we should remember?
        ↓
is it new, duplicated, related or a correction?
        ↓
does policy allow us to keep it?
        ↓
save the claim, its source and its history
        ↓
search when it becomes relevant
        ↓
give the AI a small set of useful memories

Behind that simple flow are lexical search, semantic retrieval, lifecycle states, provenance, model-assisted interpretation and various safeguards. But the model doing the task still sees something deliberately simple: a short set of facts that might help it answer the question in front of it.

We originally imagined that a more capable assistant would need a richer and richer representation of everything it knew about a person. Increasingly, we think it needs to be very good at deciding what not to bring into the room.

There is plenty we still don't know

Some parts of this remain open for us.

Our retrieval is still imperfect when a query and a memory have very similar meanings but almost no words in common. We are still learning how aggressively similar memories should be consolidated. We have fields that represent when facts were valid, but we would not claim to have solved temporal reasoning. There may be a point where graph structures genuinely improve some classes of tasks.

And there is an economic question underneath all of it: how much background reasoning about memory is worth paying for?

Those are useful open questions because they stop us confusing complexity with progress. A more elaborate memory architecture is not necessarily a better one.

What memory is really about

We started with a simple goal: Zoe should remember things so people do not have to repeat themselves.

That remains true. But production memory looks less like a storage problem and more like a set of small judgement calls.

A forgotten fact can be inconvenient. A confidently remembered fact that is wrong can shape every conversation that follows.

So the thing we are trying to optimise is not how much Zoe can remember. It is whether she remembers the right things, at the right time, for the right reason, and can show you why.