LlamaIndex memory: composable blocks, operational gaps
LlamaIndex treats memory as composition: blocks of static and dynamic context, chat stores keyed per user, buffers with token limits. Flexible, and entirely your operation.
LlamaIndex. Sources checked 2026-09-25.
How LlamaIndex handles memory today
The docs show a Memory class that composes blocks, supports static context blocks built from files, and runtime manipulation, plus ChatMemoryBuffer with token limits and a chat store keyed by a user identifier.
Composition is power and responsibility. Token limits are a truncation strategy, a chat store is a database you run, and long-term recall across sessions is the part you assemble.
What LlamaIndex forgets
A token limit is not a memory policy.
- Anything past the buffer limit, silently, as conversations get long.
- Cross-session recall, unless you build it on top of the chat store.
- Ranking. Retrieved context is only as good as the search you put in front of it.
- Operational isolation, since the store boundary is your schema.
The fix with MemoryRouter
Use MemoryRouter as the durable layer behind LlamaIndex memory blocks and let the buffer keep handling the live conversation.
- 1
Back the dynamic block with the vault
A dynamic memory block reads from the vault for the user in question, so recalled decisions enter context the same way file-based blocks do.
- 2
Keep buffers for the session
ChatMemoryBuffer handles the live window. The vault handles everything that should still exist next week for that user.
API surface
# MemoryRouter: search, store, and time-window endpoints # docs.memoryrouter.ai/api-reference - 3
Provision a vault per user at signup
When your app creates a user, create their vault and store the key with the account. Memory then follows the account, not the server process.
Vaults, keys, and a memory API for apps built on LlamaIndex. 14-day free trial.
Related
LlamaIndex memory FAQ
Sources
Sources checked 2026-09-25. Tool behavior changes; check the vendor docs before relying on a detail.