Recursive Language Models (RLMs): Solving Problems Beyond One Context Window

Context windows keep growing — from a few thousand tokens to hundreds of thousands, and beyond. It's tempting to assume that trend eventually solves every "too much text" problem on its own. It doesn't. However large a context window gets, there is always a bigger version of the problem: a longer document, a larger codebase, a deeper research task with more sources to weigh. A single model call is fundamentally bounded by whatever window it was given, and no fixed window size is safe from being outgrown.

Recursive language models (RLMs) are one response to that limit — not a specific product or a single agreed-upon algorithm, but a general pattern for handling problems that don't fit in one call by having the model call itself, or a fresh copy of itself, on smaller pieces of the same problem.

The problem: some tasks just don't fit

Think about what it actually takes to summarize a book-length document, reason carefully across an entire codebase, or work through a multi-step problem whose intermediate steps alone would overflow a normal prompt. You can compress, truncate, or retrieve the "most relevant" slice — and often that's enough — but for a genuinely large task, compression eventually throws away exactly the details the task needed. The underlying issue isn't that today's context windows are too small; it's that any fixed-size window is, by definition, finite, while the problems people want solved are not.

The core idea: recurse instead of force-fit

Instead of trying to cram an oversized problem into one model call, the recursive pattern breaks the problem into smaller subproblems, invokes a model call — often a fresh, independent instance — on each one, and then combines those results back together into an answer to the original problem. If a subproblem is still too large, the same trick applies again: split it further and recurse another level down. The recursion only stops once a piece is small enough that a single model call can actually handle it.

This is, quite deliberately, the same shape as a classic recursive algorithm. A recursive merge sort splits a list in half, sorts each half (by recursing), and merges the two sorted halves back together. A recursive summarization pipeline splits a document into chunks, summarizes each chunk, and then summarizes the summaries — repeating that combine-and-reduce step until one summary remains. The recursive language model pattern generalizes exactly that divide-and-conquer, combine-the-results structure, except the "processing" step at each level is no longer a fixed deterministic function — it's a model call, deciding how to read, reason about, or summarize whatever piece it was handed.

def solve(problem, depth=0):
    # Base case: small enough to hand to a single model call
    if fits_in_context(problem):
        return llm.generate(problem)

    # Otherwise, split into smaller subproblems
    subproblems = split(problem)

    # Recurse on each piece — possibly with a fresh model instance
    sub_results = [solve(sp, depth + 1) for sp in subproblems]

    # Combine the recursive results back into one answer
    return combine(sub_results, problem)

What this buys you

The appeal is straightforward: each individual recursive call only ever has to deal with one sub-piece of the larger problem, plus whatever summary or state was handed down from the level above it. As long as that sub-piece comfortably fits in a single context window, the call itself is easy — it's no different from any other bounded prompt. What changes is the system built around those calls. By recursing, splitting, and recombining, the overall system can, in principle, work through something far larger than anything a single model call could ever see at once, without needing a model with an unbounded context window.

What this costs

None of this is free. Every extra level of recursion means more model calls, which means more latency and more cost — a problem that would have been one API call now might be dozens, spread across several layers. There's also a subtler cost: every time a subproblem's result gets compressed into a smaller summary before being passed back up a level, there's an opportunity to lose or distort information. A detail that mattered but didn't make it into the summary at one level simply isn't available to the levels above it — and because the combining step usually trusts the summaries it's given, an error or omission low in the recursion can propagate upward largely unnoticed.

That makes two design decisions unusually important: how the problem gets split in the first place, and how much context or state gets preserved (versus summarized away) at each level as results flow back up. Split badly, and subproblems end up incoherent or overlapping in unhelpful ways. Preserve too little at each combine step, and the final answer confidently reflects a distorted picture of the parts beneath it.

Where this fits among related ideas

The recursive pattern sits alongside — and overlaps with — a few other approaches people use to work around context limits: hierarchical summarization (summarize chunks, then summarize the summaries), and multi-agent decomposition (route different parts of a task to different agents and merge their outputs). Recursive language models can be thought of as a generalization that unifies those ideas under one recurring structure: split, recurse, combine. It's an active area of exploration rather than a finished technique, and it's reasonable to expect the specifics — how splits are chosen, how much state gets carried between levels, how results get merged — to keep changing as people experiment with what works.