Prism DB on BEAM 100K: compact, auditable memory for long-horizon agents.

Long-running agents accumulate history faster than it can be replayed. A single BEAM 100K conversation exceeds one hundred thousand tokens; a production agent handling support threads, engineering sessions, or account workflows crosses that threshold within days. The default strategy — replay everything into every model call — fails in four compounding ways: per-call cost grows with lifetime, latency grows with prompt size, relevant evidence gets diluted by irrelevant history, and answers become unauditable because no one can say which part of a hundred-thousand-token prompt produced them.

The alternative is retrieval: store history once, retrieve a small, question-relevant slice at answer time. The open question is whether compact retrieval preserves answer quality across the full range of memory behaviors — not just fact lookup, but updates, contradictions, preferences, ordering, cross-session integration, and knowing when not to answer.

BEAM is built to measure exactly that range. This report describes how Prism DB performs on it.

BEAM evaluates long-term conversational memory using official probing questions over synthetic multi-session conversations. We evaluated on the 100K tier, where each conversation is roughly 100,000 tokens of multi-turn dialogue. Questions span ten categories, each isolating a distinct memory behavior:

Scoring follows BEAM's canonical convention, using BEAM's official evaluator unmodified: an LLM judge scores nine categories; event ordering is scored by normalized Kendall's tau against the reference order — a strict rank-correlation metric, not a judge opinion. The overall score is the macro-average across the ten categories, so no category can hide behind another.

Prism DB is a memory-first retrieval database: structured memory capsules with text, vector, graph, and temporal indexes behind one query surface, where every stored item and every retrieved result carries a pointer to its source. For this evaluation Prism DB served as the complete memory layer between the raw conversations and the reader model.

Write path (ingestion)

Each conversation is ingested once, before any question is seen:

• Messages are chunked into bounded capsules — message pairs capped at ~1,000 tokens with overlap — each carrying its position in the conversation and a source pointer back to the exact messages it came from.
• Alongside raw chunks, the ingester builds three families of deterministic auxiliary capsules: a chronological timeline index that gives the retriever a map of the conversation; state-update capsules that isolate moments where a fact changed (“moved”, “replaced”, “now”, “changed to”); and rule-based summary capsules that pre-digest each region of the conversation in order. No LLM is used at write time — ingestion is deterministic and replayable.
• Every capsule is embedded and inserted into Prism DB, which indexes it for fused text and vector search.

Read path (retrieval)

Each probing question triggers category-aware retrieval:

• The question is expanded into multiple variants (embedded in one batch), and fused text + vector search runs over the conversation's capsules, merging scores across variants.
• Retrieval policy adapts to the question category: ordering and summarization questions retrieve more candidates under a larger token budget with chronological packing; contradiction questions boost state-update capsules; extraction questions run tighter, precision-first retrieval.
• Neighboring chunks of strong hits are pulled in (adjacent expansion), and the final evidence pack is assembled under a fixed token budget with each item retaining its source pointer and timestamp.

Answering

The reader model receives the packed evidence — and only the packed evidence — with a category-appropriate answering policy: strict evidence-grounding with explicit permission to abstain when evidence is absent, and best-effort synthesis for categories (summarization, ordering) whose answers must be assembled from many pieces rather than found in one.

The efficiency baseline is full-context replay: the token cost of handing the reader the entire conversation for every question, which is how long-context agents operate by default.

Per-category BEAM-canonical scores:

At 95.68% reduction, the reader consumed roughly one twenty-third of the tokens that full replay would have required per question — a few thousand tokens of packed evidence against conversations of a hundred thousand.

The retrieval-dependent categories are the strongest. Information extraction (87.5), instruction following (80.8), knowledge update (77.5), and preference following (72.5) all reduce to the same operation: find the right moments in a very long history and put them in front of the reader with their timestamps. This is the core competency of a retrieval engine, and it is where compact memory most directly replaces full replay — the fact stated once at token 40,000 is retrieved as precisely as if the conversation were forty turns long.

Abstention held at 77.5. This number deserves attention because it is where retrieval systems typically fail silently. A memory layer that always returns something pressures the reader into answering unanswerable questions. Prism DB's evidence packs are grounded — every item traces to source — and the answering policy treats absent evidence as a first-class outcome. A system that fabricates confident answers when memory is empty is worse than a system with no memory at all; this category is the guard on that failure mode.

Summarization at 75.8 challenges the assumption that synthesis requires full context. Summarization is the category most often cited as needing the whole transcript, since the answer integrates information spread across the entire history. The deterministic summary and timeline capsules — built once at ingestion, retrieved like any other memory — carry that global structure into a compact evidence pack. The reader synthesizes from pre-digested, source-linked material instead of a raw hundred-thousand-token transcript.

Temporal structure is the frontier. The three weakest categories — temporal reasoning (62.5), event ordering (60.4 tau), and multi-session reasoning (55.0) — share a common demand: reconstructing global structure (what happened before what, across session boundaries) from locally retrieved evidence. Chronological packing and the timeline index carry part of that structure; the remaining gap is concentrated in answer synthesis over ordered evidence rather than in retrieval itself. Event ordering is also scored by rank correlation, the harshest metric in the suite — partial credit decays quickly with each transposition. These categories are where our current work is focused, and they are tractable: the evidence is being retrieved; the ordering discipline in the final synthesis is what must improve.

The score and the reduction must be read together. 71.7 in isolation is a strong long-memory result. 95.68% in isolation is a strong efficiency result. The claim that matters is the conjunction: this score was achieved while withholding ~96% of the context from the model. Full-context replay is not a free alternative — it costs twenty-three times the input tokens on every single question, forever, and that ratio worsens as histories grow past the 100K tier, until replay stops being possible at all and retrieval becomes the only option.

For an agent answering questions against long histories, input-token spend scales with (questions × context-per-question). Retrieval changes the second factor from “everything so far” to “what this question needs”:

• Cost: per-question input cost drops by roughly the reduction factor. An agent workload spending $10,000/month on replayed context runs on hundreds of dollars of retrieved context.
• Latency: prompt-processing time scales with prompt size; packed evidence keeps time-to-first-token flat as history grows.
• Ceiling: replay hits the context-window wall and stops; retrieval does not. The 100K tier is where replay begins to strain — Prism DB's architecture is indifferent to whether the history is 100K or 10M tokens.
• Auditability: every item in an evidence pack carries a pointer to the source messages it came from. When an agent answers, the evidence for that answer is enumerable and inspectable — a property full-context replay cannot offer even in principle.

The evaluation harness treats runs as artifacts, not events. Every run writes a self-contained bundle: raw reader answers, unmodified official evaluator outputs, per-category aggregation, retrieved-token and baseline-token accounting, latency percentiles, model identifiers, the exact reproduction command, and a SHA-256 manifest over the bundle. Ingestion is deterministic — no LLM writes — so the memory a run builds can be reconstructed exactly. The harness, including the chunking, capsule construction, retrieval policies, and scoring aggregation, is versioned in-repo with its contract test suite.

The result supports a specific architectural claim: long-running agents do not need ever-larger prompts as their default memory strategy — they need retrieval that is compact, category-aware, and auditable. Our current focus is the remaining gap in temporal structure and cross-session reasoning, larger benchmark tiers, and lower retrieval latency, with the same reporting discipline: official evaluators, unmodified; artifacts, published; caveats, stated.

The economics point one direction. A model that reads twenty-three times fewer tokens to reach the same answer is not an optimization; it is a different cost structure for the agent era. Memory is how agents get one.