Memory Pruning: Forgetting, Expiry, and Context Budgets for Agents

An agent that remembers everything eventually remembers nothing useful. As memory stores grow, irrelevant or outdated information crowds out what matters. Retrieval gets slower, context windows fill with noise, and the agent’s performance degrades. Memory pruning — the deliberate process of forgetting — is essential for keeping agents focused and efficient.

Why Pruning Matters

Token Budget Constraints

Every memory retrieved consumes tokens in the context window. Without pruning, you either hit the context limit or waste tokens on irrelevant information:

Without pruning:
  Retrieved memories: 50 (last 6 months)
  Tokens consumed: 15,000
  Relevant memories: 8
  Noise: 42 memories (84% waste)

With pruning:
  Retrieved memories: 10 (recent + relevant)
  Tokens consumed: 3,000
  Relevant memories: 7
  Noise: 3 memories (30% waste)

Retrieval Performance

More memories mean more vectors to search, more keys to index, and more edges to traverse. Pruning keeps retrieval fast.

Relevance Decay

Information loses relevance over time. A debugging session from yesterday is more relevant than one from six months ago. Pruning reflects this reality.

Pruning Strategies

Strategy 1: Time-to-Live (TTL) Expiry

The simplest strategy: memories expire after a fixed duration.

from datetime import datetime, timedelta
from typing import Optional

class TTLMemoryPruner:
    def __init__(self, default_ttl_days: int = 30):
        self.default_ttl = timedelta(days=default_ttl_days)

    def store_with_ttl(
        self,
        memory: Memory,
        ttl: Optional[timedelta] = None
    ):
        memory.expires_at = datetime.now() + (ttl or self.default_ttl)
        self.store(memory)

    def prune_expired(self):
        """Remove all memories past their expiry date."""
        now = datetime.now()
        expired = [
            m for m in self.store.get_all()
            if m.expires_at and m.expires_at < now
        ]
        for memory in expired:
            self.store.delete(memory.id)

        return len(expired)

    def get_active_memories(self) -> list[Memory]:
        """Retrieve only non-expired memories."""
        now = datetime.now()
        return [
            m for m in self.store.get_all()
            if not m.expires_at or m.expires_at > now
        ]

TTL Guidelines:

• Session state: 1 hour
• Conversation summaries: 7 days
• Extracted facts: 30 days
• Important decisions: 90 days
• Permanent knowledge: No TTL (managed by other strategies)

Strategy 2: Relevance Decay

Memories lose relevance over time, weighted by their type:

import math

class RelevanceDecayPruner:
    def __init__(self):
        # Half-life in days for different memory types
        self.half_lives = {
            "session_state": 0.5,      # 12 hours
            "conversation": 7,          # 1 week
            "extracted_fact": 30,       # 1 month
            "decision": 90,             # 3 months
            "permanent_knowledge": 365, # 1 year
        }

    def calculate_relevance(self, memory: Memory) -> float:
        """Calculate current relevance score with decay."""
        half_life = self.half_lives.get(memory.type, 30)
        days_old = (datetime.now() - memory.created_at).days

        # Exponential decay
        decay_factor = math.exp(-0.693 * days_old / half_life)

        # Apply access frequency boost
        access_boost = 1 + (memory.access_count * 0.1)

        return memory.base_relevance * decay_factor * min(access_boost, 2.0)

    def prune_by_relevance(self, threshold: float = 0.1):
        """Remove memories below relevance threshold."""
        all_memories = self.store.get_all()
        pruned = 0

        for memory in all_memories:
            relevance = self.calculate_relevance(memory)
            if relevance < threshold:
                self.store.delete(memory.id)
                pruned += 1

        return pruned

Strategy 3: Budget-Aware Selection

Select memories based on available token budget:

class BudgetAwareMemorySelector:
    def __init__(self, max_tokens: int = 4000):
        self.max_tokens = max_tokens

    def select_memories(
        self,
        candidates: list[Memory],
        query: str
    ) -> list[Memory]:
        """Select memories that fit within token budget."""
        # Score each candidate
        scored = [
            (memory, self._score(memory, query))
            for memory in candidates
        ]

        # Sort by score (highest first)
        scored.sort(key=lambda x: x[1], reverse=True)

        # Greedily select until budget is exhausted
        selected = []
        tokens_used = 0

        for memory, score in scored:
            memory_tokens = self._estimate_tokens(memory)
            if tokens_used + memory_tokens <= self.max_tokens:
                selected.append(memory)
                tokens_used += memory_tokens

        return selected

    def _score(self, memory: Memory, query: str) -> float:
        """Score memory relevance to query."""
        # Semantic similarity
        similarity = cosine_similarity(
            memory.embedding,
            self.embed(query)
        )

        # Recency factor
        days_old = (datetime.now() - memory.created_at).days
        recency = math.exp(-0.01 * days_old)

        # Access frequency factor
        frequency = min(memory.access_count / 10, 1.0)

        return similarity * 0.6 + recency * 0.3 + frequency * 0.1

    def _estimate_tokens(self, memory: Memory) -> int:
        """Estimate token count for a memory."""
        return len(memory.content.split()) * 1.3  # Rough estimate

Strategy 4: Semantic Deduplication

Remove memories that are too similar to each other:

import numpy as np

class SemanticDeduplicator:
    def __init__(self, similarity_threshold: float = 0.92):
        self.threshold = similarity_threshold

    def deduplicate(self, memories: list[Memory]) -> list[Memory]:
        """Remove near-duplicate memories."""
        if not memories:
            return []

        # Compute pairwise similarities
        embeddings = np.array([m.embedding for m in memories])
        similarity_matrix = np.dot(embeddings, embeddings.T)

        # Greedy selection of unique memories
        selected = [0]  # Always keep the first
        for i in range(1, len(memories)):
            # Check similarity to all selected memories
            max_sim = max(
                similarity_matrix[i][j]
                for j in selected
            )
            if max_sim < self.threshold:
                selected.append(i)

        return [memories[i] for i in selected]

Combining Strategies

Production systems typically combine multiple pruning strategies:

class CompositePruner:
    def __init__(self):
        self.ttl_pruner = TTLMemoryPruner()
        self.relevance_pruner = RelevanceDecayPruner()
        self.deduplicator = SemanticDeduplicator()
        self.budget_selector = BudgetAwareMemorySelector(max_tokens=4000)

    def full_pruning_cycle(self):
        """Run all pruning strategies in sequence."""
        stats = {}

        # Step 1: Remove expired memories
        stats["expired_removed"] = self.ttl_pruner.prune_expired()

        # Step 2: Remove low-relevance memories
        stats["low_relevance_removed"] = self.relevance_pruner.prune_by_relevance()

        # Step 3: Remove duplicates
        all_memories = self.store.get_all()
        deduplicated = self.deduplicator.deduplicate(all_memories)
        stats["duplicates_removed"] = len(all_memories) - len(deduplicated)

        return stats

    def select_for_context(self, query: str) -> list[Memory]:
        """Select memories for a specific query context."""
        candidates = self.store.get_active_memories()
        return self.budget_selector.select_memories(candidates, query)

Best Practices

1. Prune on a schedule — Run pruning cycles periodically (e.g., daily for active systems, weekly for quieter ones). Don’t wait for the store to grow unbounded.

2. Track pruning metrics — Monitor how many memories are pruned, store size over time, and retrieval quality. This helps tune pruning parameters.

3. Preserve high-value memories — Some memories (important decisions, user corrections) should survive pruning. Tag these as “protected” and exclude them from automatic expiry.

4. Use soft deletes first — Mark memories as “archived” before permanent deletion. This allows recovery if pruning was too aggressive.

5. Test retrieval quality — After pruning, verify that relevant memories are still accessible. Use evaluation queries to ensure pruning didn’t break important recall patterns.

Conclusion

Memory pruning is not about losing information — it’s about keeping what matters. By implementing TTL expiry, relevance decay, budget-aware selection, and semantic deduplication, you maintain a memory store that stays relevant, fast, and focused. The best pruning strategy combines multiple approaches, adapted to your specific memory patterns and access requirements.

References:

• LangGraph Memory Management
• OpenAI Agents SDK Memory