Agent Memory in TypeScript: Short-Term, Long-Term, and What to Throw Away

• 
  Book: AI That Plans
  


• 
  The series: AI in TypeScript — 5 books, from your first LLM call to agents in production — all five here
  


• 
  My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools


• 
  Me: xgabriel.com | GitHub
  

"Agent memory" gets implemented as one array of messages that grows

until something truncates it. That single array is doing three jobs

that have different lifetimes, different access patterns, and

different failure modes — which is why the truncation always feels

arbitrary.

Separate them and the eviction question answers itself.

Three stores, three lifetimes

Working memory is the message window sent on this turn. Lifetime:

one run. Bounded by the context limit. It is the only one the model

sees directly.

Long-term memory is durable facts about the user or the domain.

Lifetime: indefinite. Not sent wholesale — retrieved selectively.

"Prefers metric units", "primary account is acct_x", "is on the

enterprise plan".

Episodic memory is the record of previous runs. Lifetime: audit

retention. Almost never sent to the model. Used for debugging,

analytics, and evals.

Conflating the first two produces the common failure: a fact the user

stated in message three is still in the window at message forty

because someone was afraid to drop it, and it is crowding out the

last ten turns that actually matter.

export interface WorkingMemory {

  window(): MessageParam[];

  append(m: MessageParam): void;

  compact(budget: number): Promise<void>;

}



export interface LongTermMemory {

  recall(query: string, k: number): Promise<Fact[]>;

  remember(f: NewFact): Promise<void>;

  forget(id: string): Promise<void>;

}



export interface EpisodicMemory {

  record(run: RunRecord): Promise<void>;

  find(filter: RunFilter): Promise<RunRecord[]>;

}

Three interfaces with almost nothing in common. That is the point —

a single Memory interface with get/set is how the three get

conflated in the first place.

Working memory: the window and its budget

export class Window implements WorkingMemory {

  private msgs: MessageParam[] = [];

  private summary: string | null = null;



  constructor(private readonly system: string) {}



  window(): MessageParam[] {

    const head: MessageParam[] = this.summary

      ? [{ role: "user", content: `Earlier context:\n${this.summary}` }]

      : [];

    return [...head, ...this.msgs];

  }



  append(m: MessageParam) {

    this.msgs.push(m);

  }

}

The summary sits at the front as a synthetic turn. Everything after

it is verbatim recent history.

Compaction is where the design decision lives:

async compact(budget: number) {

  if (estimateTokens(this.window()) <= budget) return;



  const keepRecent = 6;

  const older = this.msgs.slice(0, -keepRecent);

  const recent = this.msgs.slice(-keepRecent);

  if (!older.length) return;



  const res = await model.invoke([

    { role: "user", content: SUMMARISE_PROMPT(this.summary, older) },

  ]);



  this.summary = textOf(res.content);

  this.msgs = recent;

}

Two properties matter more than the summarisation prompt.

Keep the recent turns verbatim. Summarising the last exchange

loses exactly the detail the next turn depends on. Compress the

distant past, never the present.

Never split a tool pair. A tool_use block whose matching

tool_result was evicted produces a malformed request. Whatever your

eviction rule, it operates on complete exchanges:

function safeCut(msgs: MessageParam[], from: number): number {

  while (from < msgs.length && isOrphanedToolResult(msgs[from])) {

    from++;

  }

  return from;

}

That guard is one of the more common causes of a mysterious 400 from

the API after a long conversation.

What to summarise for

A generic "summarise the conversation" prompt produces prose that

reads well and loses what the agent needs. Ask for the operational

residue instead:

const SUMMARISE_PROMPT = (prev: string | null, msgs: MessageParam[]) => `

${prev ? `Previous summary:\n${prev}\n\n` : ""}

Summarise the exchange below. Preserve, as a list:

- decisions made and their stated reasons

- identifiers, values, and constraints the user supplied

- tools already called and what they returned

- anything the user explicitly asked to remember

Omit pleasantries and restatements. Under 200 words.



${render(msgs)}

`;

Feeding the previous summary back in is what stops the tenth

compaction from being a summary of a summary of a summary. Each pass

revises one document rather than re-compressing its own output.

Long-term memory is written, not scraped

The tempting implementation runs after each conversation, extracts

"facts", and stores them. It fills up with noise — restatements,

transient context, things the user said once about a specific

request.

Make writing an explicit action the agent takes:

const remember = tool({

  name: "remember",

  description: ""

    "Store a durable fact about this user. Use only for stable " +

    "preferences, identifiers, or constraints that will matter in " +

    "future conversations. Do not store transient request details.",

  schema: z.object({

    fact: z.string().max(200),

    category: z.enum(["preference", "identifier", "constraint"]),

  }),

  async run({ fact, category }, ctx) {

    await ctx.longTerm.remember({

      userId: ctx.userId,

      fact,

      category,

      sourceRunId: ctx.runId,

    });

    return { stored: true };

  },

});

Two benefits beyond precision. The write is attributable — every fact

carries the run that created it, so you can audit where a wrong

memory came from. And it is inspectable and deletable, which you need

the first time a user asks what you have stored about them.

Retrieval is selective, and belongs at the start of a run:

const facts = await longTerm.recall(task, 5);

const system = [BASE_SYSTEM, facts.length

  ? "Known about this user:\n" + facts.map((f) => `- ${f.fact}`).join("\n")

  : ""].join("\n\n");

Five, not fifty. Long-term memory injected wholesale is just a

context leak with extra steps.

Episodic memory stays out of the window

export type RunRecord = {

  runId: string;

  userId: string;

  startedAt: Date;

  outcome: "done" | "turn_limit" | "budget" | "error";

  turns: number;

  costUsd: number;

  toolCalls: { name: string; ok: boolean }[];

  finalSummary: string;

};

This is not memory the model uses. It is memory you use — to answer

which tool fails most, whether cost per run is drifting, which

outcomes correlate with turn limits.

Keeping it in a separate store with its own retention is what stops

it leaking into the context window, which is the one thing that would

make it expensive.

The eviction policy, stated

Working memory: compact when over budget, keep the last N turns

verbatim, never split a tool pair, feed the previous summary forward.

Long-term: written deliberately by a tool, retrieved top-k per run,

deletable by the user.

Episodic: never in the window, retained per your audit policy.

Testing the part that breaks

Compaction is where the bugs are, and it is testable without a model

if you inject the summariser.

it("never leaves an orphaned tool_result", async () => {

  const w = new Window(SYSTEM, fakeSummariser);

  for (const m of longConversationWithToolPairs) w.append(m);



  await w.compact(1000);



  const win = w.window();

  for (const [i, m] of win.entries()) {

    if (isToolResult(m)) {

      expect(hasMatchingToolUse(win, i)).toBe(true);

    }

  }

});

That test catches the malformed-request bug before a user has a

conversation long enough to trigger it — which, in practice, is

always a real user rather than anyone on the team.

If this was useful

AI That Plans covers memory

as separate systems — window management, compaction that keeps what

matters, durable facts written deliberately, and the run history you

keep for yourself.

The full series is at

xgabriel.com/ai-in-typescript.