MemoryRouter
All posts

Vercel AI SDK Memory: What Persistence You Actually Own

John Rood··6 min read

A chat built on the AI SDK forgets on reload. useChat holds the message array in React state, and when the browser comes back to the page that state is gone unless your own code fetched the messages and handed them back through initialMessages.

That is documented, intentional, and fine. The persistence guide in the AI SDK docs is about a chat store you write: create a chat, load its messages, save the finished UIMessage[] when the stream ends, and validate what you loaded with validateUIMessages before it goes back to the model. Swap the sample file for Postgres and you have the shape almost every production chat app ships.

I have built that shape. It stores conversations. It does not produce memory, and the gap appears the first time somebody opens their second chat.

A transcript is keyed to a thread

Rows in a chat store are keyed by chat id. Retrieval is where chatId = :id. Nothing in that model can answer "what does this person keep telling me across every thread they have opened," because the unit of storage is the thread, not the person.

The customer who canceled the enterprise plan says so in March. In April they open a fresh chat, and the assistant pitches annual billing. The message is in your database. It was never a candidate for that prompt.

Nothing in the pipeline decides what matters

Saving a UIMessage saves text, ids, parts, tool calls. The content stays content. There is no step where "I am only available after 2pm on Fridays" becomes a durable fact that later retrieval can find, and no step where Tuesday's decision gets separated from the six messages of setup around it.

The usual assumption is that the model does this. It cannot. The model never sees your store. Your route loads rows, converts them with convertToModelMessages, and sends them.

History becomes replay or truncation

Once a thread outgrows the window you have two moves. Send more of it and pay every turn for context that is mostly irrelevant. Cut it and keep the follow-up question while losing the decision that made the question make sense.

One is expensive, the other is lossy, and most teams end up maintaining a summarization job to split the difference. That job is a memory system with no retrieval.

Memory is a second write path

MemoryRouter's API documents two integration modes. Proxy mode sends the model call through MemoryRouter, which retrieves memory, calls the provider, and stores the result in one trip. Local inference mode leaves the model call in your code and uses two endpoints for memory only. For an AI SDK route, local inference mode is the natural fit, because streamText is already doing the provider work.

The keys are per user. A Memory Key looks like mk_xxxxxxxx and does two jobs in one string: credential, and the name of the vault it opens. It belongs on the server.

import { convertToModelMessages, streamText, type UIMessage } from "ai";

const key = user.memoryKey; // mk_..., server side only

async function prepare(messages: UIMessage[], sessionId: string) {
  const res = await fetch("https://api.memoryrouter.ai/v1/memory/prepare", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${key}`,
      "Content-Type": "application/json",
      "X-Session-ID": sessionId,
    },
    body: JSON.stringify({ messages, density: "default" }),
  });
  return (await res.json()) as { context: string | null; memories_found: number };
}

async function ingest(sessionId: string, asked: string, answered: string) {
  await fetch("https://api.memoryrouter.ai/v1/memory/ingest", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${key}`,
      "Content-Type": "application/json",
      "X-Session-ID": sessionId,
    },
    body: JSON.stringify({
      messages: [
        { role: "user", content: asked },
        { role: "assistant", content: answered },
      ],
    }),
  });
}

prepare runs before streamText. Hand it the exact message array your route is about to convert, and it derives the retrieval query from the last few user turns, so nobody writes a search string by hand. The payload carries context, a ready-to-inject string, next to memories_found. When the vault has nothing that matches, context comes back null. Paste it into the system prompt above the conversation.

ingest runs after the turn finishes. The response is 202 in milliseconds and the write happens after the request is done, so the user's stream never waits on it. The body carries one user message and one completed assistant message, which is all capture accepts.

The tuning knobs are in the docs and worth reading once: density from low to xhigh for how much context you want, context_limit as an explicit chunk override, and tiers for restricting recall to raw memories, reflections, or high-level reflections when you want the vault to sound less like a transcript.

The tools route, if you want the model to choose

@ai-sdk/mcp ships createMCPClient with an HTTP transport, so a remote MCP server is a config object in your agent code. https://mcp.memoryrouter.ai/mcp is one of those servers. The server speaks revision 2026-07-28, the stateless one, still negotiates the older handshake-era versions, authenticates with OAuth 2.1 and PKCE, and exposes ten tools including search_memories, date_search_memories, store_memory, memory_status, and consolidate_memories.

Use that route when the model should decide when to look something up. Skip it when recall has to happen every turn, because a tool call is a decision the model makes and sometimes it does not make it.

What this does not solve

Backfill. Nothing imports the chat rows you already have. Bulk loading exists as POST /v1/memory/upload with JSONL, one memory per line, and that migration is yours to write.

Completeness. Retrieval is relevance ranked, and prepare returns nothing when the query does not match. A gap in the vault looks exactly like a user who never said the thing. Keep your transcript store as the record of what was said and treat the vault as the thing that shows up in the prompt.

Latency. Typical vault recall runs 300 to 500 ms on average, which lands as one more network round trip ahead of the provider call. In a chat UI nobody feels it. Under a strict budget, measure before you commit.

Isolation. The key decides which vault opens. One per end user, server side only, and it separates users exactly as well as you separate keys. A Memory Key shipped to the browser is that user's entire memory, readable by anyone who opens devtools.

Deletion is available per memory id and for the whole vault, and key suffixes let you hand a read-only credential to anything that should not be writing.

The one route to convert first

Pick the route where the user is most likely to be annoyed at repeating themselves, usually the main chat, and add prepare and ingest around the existing stream call. Leave the chat store alone. Then test in the app rather than in a script: tell it something durable, open a new chat from the sidebar, and check whether the second conversation knows.

Start the free trial and wire one route, or read the API reference and the Vercel AI SDK page first.

All posts