The "Long-Horizon" Task: Keeping Gemini 3 Focused Over Months-Long Projects

A months-long project, a deep research effort, a self-improving codebase — these tasks outlast a single context window. The naive fix is a bigger window, but context growth brings context rot: performance degrades as the trajectory grows, even at near-million-token windows.

In this tutorial, you will learn the discipline of long-horizon engineering — keeping the active context small and the accessible state complete. We’ll cover execution-state memory (MAGE), agent-managed context tools (ACM), file-centric state (InfiAgent), append-and-search-with-code (PRO-LONG), and proactive memory intervention.

The Failure Modes of Long-Horizon Memory

1. State fragmentation. Similarity-based memory (vector stores, graph stores) organizes entries by content relevance, not execution-state dependency. Interdependent decisions get fragmented — the agent can’t reconstruct a coherent state.
2. Error contamination. Entries from different trajectories share one relevance space, so valid and erroneous traces mix and poison reasoning.
3. Behavioral state decay. Decision-relevant facts get buried in (or pushed beyond) the context window and stop influencing decisions.

Although modern LLMs can process context windows of near to a million tokens, performance remains expensive and vulnerable to degradation as trajectories grow, i.e., context rot. — PRO-LONG (https://arxiv.org/html/2607.20064)

Shift: From Similarity Archive to Execution-State Manager (MAGE)

MAGE treats memory as an execution-state structure, not a pool of retrievable facts. It organizes history as a two-layer hierarchical state tree:

• Bottom layer — the step-by-step action-observation trace.
• Top layer — summaries generated at subgoal/decision boundaries (boundary-aware compression).

The current execution state is the active root-to-current path: compressed state + recent raw state + execution hints from sibling branches. Four coupled operations maintain it:

• Grow — append new action-observation pairs.
• Compress — fold completed raw segments into top-layer summaries.
• Maintain — validate each new summary before it becomes trusted memory (catches errors before they propagate).
• Revise — restore state to a target boundary and resume on a new branch, isolating the flawed segment.

Similarity-driven organization leads to… state fragmentation that weakens execution state integrity… erroneous and valid traces can be surfaced together and contaminate subsequent reasoning. — MAGE (https://arxiv.org/html/2606.06090)

The results on MemoryArena: +7.8–20.4 pp task success over baselines, while cutting token consumption 55.1%.

class ExecutionState:
    def __init__(self):
        self.tree = StateTree()          # two-layer hierarchical tree
        self.path = self.tree.root

    def step(self, action, observation):
        self.path.grow(action, observation)            # Grow
        if self.path.at_subgoal_boundary():
            summary = self.path.compress()              # Compress
            if not self.maintain_validates(summary):   # Maintain
                self.path = self.tree.revise(           # Revise
                    to=summary.error_boundary, new_branch=True)

Let the Agent Manage Its Own Context (ACM)

Agentic Context Management (ACM) gives the agent purpose-built context tools and lets it decide when to use them — no external monitor, no heuristic timing:

• manage_context — compress previous turns into a concise summary and offload the raw messages to external storage, losslessly.
• query_memory — query the stored raw messages on demand.

The raw content is never discarded; compression is agent-initiated and lossless. Context management becomes an explicit agent action — the agent decides when compression follows its evolving reasoning state, not a fixed schedule.

Equipping the agent with a set of well-designed memory tools, we allow the agent itself to decide when and how to manage its context: identifying irrelevant information, summarizing and offloading it to external memory, and querying it on demand. — ACM (https://arxiv.org/html/2607.23809)

File-Centric State: The Filesystem as Truth (InfiAgent)

InfiAgent externalizes persistent state into a file-centric abstraction: the filesystem is the authoritative record of the agent’s actions, environment, and intermediate artifacts. At each step the agent rebuilds its reasoning context from:

1. A workspace state snapshot.
2. A fixed-size window of recent actions.

A dedicated thinking model periodically rewrites a thinking record — task to-do list, descriptions of reusable files, pinned carry-over state (rules, failures, content that must survive window resets), and next tool-level steps. Context stays strictly bounded regardless of task duration; interruption resumes from the last checkpoint, not a replay.

InfiAgent treats the file system as the authoritative and persistent record of the agent’s actions, environment, and intermediate artifacts… ensuring that the context size remains strictly bounded regardless of task duration. — InfiAgent (https://aclanthology.org/2026.findings-acl.1787.pdf)

Programmatic Memory: Append Everything, Search with Code (PRO-LONG)

The simplest discipline that works surprisingly well: write everything, search with code.

• Write — append every observation, action, and outcome to a log. No summarization, no learned decisions — lossless.
• Read — programmatic search over the log (regex, code), which is native to coding agents.

On ARC-AGI-3, PRO-LONG improves a base coding agent by 18.0 pp on average across frontier models, and matches specialized harnesses while using 4.2–5.8× fewer tokens.

PRO-LONG keeps a complete, structured interaction log and capitalizes on recent progress in coding agents to search this history efficiently. — PRO-LONG (https://arxiv.org/html/2607.20064)

Proactive Memory Intervention

A separate memory agent runs alongside the unmodified action agent. At fixed intervals it updates a structured memory bank and decides whether to inject a concise reminder into the next action call — or remain silent. Selective intervention beats passive retrieval, always-on injection, and advisor-only guidance, gaining +8.3 pp on Terminal-Bench and +6.8 pp on τ²-Bench.

The null intervention is an explicit action: memory should intervene only when a remembered item — a forgotten requirement, a failed attempt, a diagnosis — is likely to affect the next decision.

Effective memory for long-horizon agents is not only a write and retrieval problem, but also an intervention problem. — Remember When It Matters (https://arxiv.org/html/2607.08716)

Pattern Comparison

Pattern	Memory structure	Who decides to compress/retrieve	Context cost
MAGE	Two-layer state tree, path-based	Agent (Grow/Compress/Maintain/Revise)	−55.1% tokens
ACM	Lossless offload + on-demand query	Agent (tools)	Agent-initiated
InfiAgent	File-centric workspace + thinking record	Fixed-schema builder	Strictly bounded
PRO-LONG	Append-only log	Code search	4.2–5.8× fewer
Proactive memory	Structured bank + selective injection	Separate memory agent	Selective

Conclusion

Months-long tasks survive because state lives outside the context window. Organize memory as execution state, not semantic soup (MAGE). Let the agent own its context management with lossless tools (ACM). Treat the filesystem as the source of truth and keep the reasoning context bounded (InfiAgent). Append everything and search with code (PRO-LONG). And let a proactive memory agent decide when to remind.

Next steps: start with the cheapest discipline — an append-only log plus code search — then add boundary-aware compression and a validation step before promoting summaries. The common thread: keep the active context small, the accessible state complete, and the checkpoints forkable. Focus is a systems problem, and the system is the memory.

References

• MAGE — Memory as Execution State Management for Long-Horizon Agents (arxiv 2606.06090): https://arxiv.org/html/2606.06090
• ACM — Agentic Context Management for Long Horizon Tasks (arxiv 2607.23809): https://arxiv.org/html/2607.23809
• InfiAgent — An Infinite-Horizon Framework for General-Purpose Autonomous Agents (ACL Findings 2026): https://aclanthology.org/2026.findings-acl.1787.pdf
• PRO-LONG — Programmatic Memory Enables Long-Horizon Reasoning (arxiv 2607.20064): https://arxiv.org/html/2607.20064
• Remember When It Matters: Proactive Memory Agent for Long-Horizon Agents (arxiv 2607.08716): https://arxiv.org/html/2607.08716