Substrate memory vs. a bigger context window

Context windows have grown by roughly two orders of magnitude in a few years, and several vendors now advertise windows in the millions of tokens. Set against a memory architecture built from typed entries and links in a shared graph, the reasonable objection is to ask why any of it is necessary: if a session can already hold the equivalent of a large codebase or years of correspondence, why maintain a separate structure for the same purpose?

The objection identifies a real trend and deserves a direct answer rather than a dismissal. Context windows are getting larger, and a larger window genuinely helps within the boundary where it applies.

A context window is the span of tokens a model can attend to within a single call or conversation. Everything inside it, the system prompt, the conversation so far, any documents pasted or retrieved, is visible to the model at once; everything outside it is not. When the session ends, so does the window’s content. Nothing about a larger window changes what happens after that point: the next conversation, started a minute or a month later, begins without it.

Several 2026 field commentators have converged on this distinction independently, in different words but stating the same fact: a million-token window is not a million tokens of memory, because a window’s contents disappear at the session boundary while memory, by definition, is expected to survive it. A context window is what a model can see right now, in this session; memory is a separate system, one that has to persist deliberately rather than simply be large enough to hold everything at once.

The practical usage pattern for very large windows bears this out. Coverage of 2026 usage describes million-token-plus context as concentrated among a smaller set of tasks: full codebase exploration, comprehensive literature review, large-document compliance review, work naturally bounded to a single session and a single document set. None of that argues against those use cases; a bigger window suits them well. It is a claim about scope: those are single-session tasks, and a comparison built around cross-session or multi-agent work is answering a different question than window size answers.

Before the session-boundary limit is even reached, advertised window size and usable window size diverge. NVIDIA’s RULER benchmark, designed to measure a model’s effective context length rather than its stated maximum, finds effective length is typically only 50–65% of what a model’s advertised window claims. On the MRCR v2 long-context evaluation, Claude Opus 4.6 retains roughly 76% accuracy at the full 1-million-token mark, the strongest recorded result on that benchmark; other evaluated models fall below 50% at the same length.

These are measurements of model behavior generally, independent of any particular memory architecture, and they narrow the practical case for window size as a substitute for structured memory before that comparison even starts: a window advertised as holding a codebase or a year of correspondence typically holds substantially less than that in practice, at exactly the point where skipping a separate memory layer would need it to hold all of it.

The window-size ceiling above is a measurement problem: even accounting for it, a sufficiently large window might still seem to solve the underlying need. It doesn’t, because the constraint a bigger window runs into is structural rather than a matter of degree.

A single orchestrating agent coordinating parallel subagents can generate well over a million tokens of intermediate reasoning across those subagents combined. That reasoning cannot be admitted into the orchestrator’s own context window regardless of how large that window is: doing so would bury the orchestrator’s attention under noise it has no way to prioritize, degrading the session a bigger window was meant to help. Runtimes that fan work out to subagents handle this by keeping intermediate reasoning outside the orchestrator’s context entirely and returning only a converged synthesis; a wider window changes the size of the synthesis that can be admitted, not the reasoning discarded to produce it.

The same shape recurs at three scales, each with its own boundary a wider window does not move:

Widening the context window helps only the first row, and only within a single open session; it does nothing for the second or third, because both are boundaries a session’s own size cannot cross. A related point: the same architecture that lets subagent reasoning persist outside the orchestrator’s window is also what keeps the cost of holding old context down over time, since cached, structured state reads far cheaper than the same volume of prose read fresh on every turn. That cache economics case is covered on its own page; the point here is narrower: it isn’t only that a wider window can’t hold cross-session or cross-agent state, it’s that holding all of it as fresh prose costs more than holding a structured equivalent, at any window size.

Vinculum’s architecture is built on a different bet about where capability comes from: that capability scales better through horizontal coordination of many small-context agents operating over structured shared state than through vertical scaling of one agent’s context window. More agents, each with a smaller working context, coordinating through a shared graph rather than each holding the whole picture in its own window.

Framed this way, a widely observed usage pattern — that real working sessions routinely run past a hundred thousand tokens of context once files, history, and tool output pile up — reads differently than a caution about user discipline. It reads as evidence that memory built as long prose forces sessions to carry more context than the task itself requires, and that the resulting compute cost is a symptom of that shape rather than an unavoidable cost of capability.

An early production demonstration of the horizontal bet, run in April 2026: a 48-hour multi-session orchestration through the substrate showed no observed context drift, in contrast to the same underlying model drifting within a single six-hour prose-memory conversation. In a separate run the same month, four independently spawned agents given one shared prompt claimed adjacent pieces of work and coordinated to avoid file-level collisions with no primitive enforcing that coordination; the structured shared state was sufficient on its own. A third comparison found that synthesizing roughly 15,000 tokens of structured closure summaries reconstructed a project’s trajectory more accurately than several hours of prose-memory conversation covering the same period. None of the three required a larger context window; each ran on the existing window size, coordinated through structure instead.

The full architectural case, including the three-timescale mechanism above, is set out at horizontal coordination, the deeper companion to this comparison.

None of the above claims that large context windows are unnecessary or that models don’t benefit from scaling them. A bigger window is a real improvement for the tasks it suits: single-session work over a single large document set, where the whole task fits inside one conversation and nothing needs to survive past it. The comparison here is narrower and specific to a different category of work, tasks spanning multiple sessions, multiple agents, or both, where the question isn’t how much a single window can hold but what persists once that window closes and how several agents reason over the same state without one of them holding everything.

It is also not a claim that structured shared state is the only viable answer to that category of work, only that it is one architecture built specifically to address it, with production use as its evidence rather than a specification alone.