The Self-Improving Codebase: Gemini 3 Agents that Refactor Based on Performance Logs

The performance of an AI application is dictated less by the model than by its runtime harness — the execution logic, system prompts, memory management, and tool configuration that connect the model to the real world. And yet most teams still tune that harness by hand, tweaking a prompt to fix one edge case and silently breaking the tool-calling loop in another.

The 2026 answer is self-improvement. Research like SICA (arXiv:2504.15228) shows a coding agent that edits its own codebase, improving from 17% to 53% on SWE-Bench Verified. The Self-Harness framework (arXiv:2606.09498) turns this into a repeatable three-stage loop: mine weaknesses from execution traces, propose minimal patches, and validate against regression tests.

In this tutorial, you will learn how to build a safe, self-improving refactoring loop for a Gemini 3 agent codebase — one driven by performance logs and gated by strict validation.

The Harness Is the Lever

Manual prompt engineering scales poorly. As new models drop rapidly, hand-crafting harnesses per model doesn’t scale. The alternative is an iterative, autonomous loop where the agent improves its own scaffolding by mining its execution traces.

The key insight from SICA: the agent edits only the harness — prompts, tool config, control flow — not the model weights. This keeps recursive self-improvement risk low, because the change surface is the code around the model, not the model itself.

The Loop: Mine → Propose → Validate

flowchart LR
    A["Execution Traces<br/>(tool calls, errors, logs)"]
    B["Weakness Mining<br/>analyze failure patterns"]
    C["Harness Proposal<br/>minimal patch"]
    D["Regression Gate<br/>run tests + benchmarks"]
    E["Merge / Reject"]

    A --> B
    B --> C
    C --> D
    D -->|"pass"| E
    D -->|"fail"| A

    classDef mine fill:#e0f2fe,stroke:#0284c7,color:#0f172a;
    classDef gate fill:#fef3c7,stroke:#d97706,color:#0f172a;

    class B mine;
    class D gate;

Step 1: Instrument Execution Traces

You cannot mine what you don’t record. Emit structured logs for every tool call, error, and response:

import json
import logging

logger = logging.getLogger("agent.trace")


def trace_tool_call(tool: str, args: dict, result: dict, error: str | None = None) -> None:
    logger.info(json.dumps({
        "event": "tool_call",
        "tool": tool,
        "args": args,
        "result": result,
        "error": error,
        "ts": time.time(),
    }))


def trace_llm_call(prompt: str, output: str, latency_ms: float) -> None:
    logger.info(json.dumps({
        "event": "llm_call",
        "prompt_tokens": len(prompt.split()),
        "output": output,
        "latency_ms": latency_ms,
        "ts": time.time(),
    }))

Step 2: Weakness Mining

Feed the trace logs to a Gemini 3 agent that identifies model-specific failure patterns — not general software bugs. It analyzes what failed, where the logic pivoted, and what the harness should have handled differently:

from google import genai

client = genai.Client()
WEAKNESS_PROMPT = """
Analyze these execution traces and identify up to 3 weaknesses in the agent harness.
For each, output JSON:
{
  "weakness": "description of the failure pattern",
  "evidence": ["trace ids that demonstrate it"],
  "suggested_fix": "minimal prompt or code change to the harness"
}
Only report weaknesses you can point to specific evidence for.
"""

def mine_weaknesses(trace_log: str) -> list[dict]:
    response = client.models.generate_content(
        model="gemini-3-pro",
        config=types.GenerateContentConfig(response_mime_type="application/json"),
        contents=f"{WEAKNESS_PROMPT}\n\n{json.dumps(trace_log, indent=2)[:20000]}",
    )
    return json.loads(response.text)

Step 3: Proposal

The mining agent hands off to a proposer agent that writes a minimal, targeted patch to the harness — a prompt tweak, a tool-config change, or a control-flow fix. Minimality matters: small patches are easy to review and easy to revert.

def propose_fix(weakness: dict) -> str:
    """Generate a minimal diff against the harness files."""
    response = client.models.generate_content(
        model="gemini-3-pro",
        contents=(
            f"Weakness: {weakness['weakness']}\n"
            f"Evidence: {weakness['evidence']}\n"
            f"Produce a minimal unified diff that fixes this in the harness. "
            f"Do not touch unrelated code."
        ),
    )
    return response.text

Step 4: The Regression Gate

This is the non-negotiable safety step. A self-improving system is only as trustworthy as its validation. Before any self-proposed change merges, run the full test suite plus performance benchmarks, and reject the patch if any metric regresses:

import subprocess
from dataclasses import dataclass


@dataclass
class EvalResult:
    pass_rate: float
    p50_latency_ms: float
    token_cost: float


def validate(patch: str) -> EvalResult:
    # 1. Apply the patch to the harness
    subprocess.run(["git", "apply", "-"], input=patch, check=True)
    try:
        # 2. Run tests
        tests = subprocess.run(["pytest", "-q"], capture_output=True, text=True)
        # 3. Run benchmarks
        bench = subprocess.run(["python", "benchmarks/run.py"], capture_output=True, text=True)
        return parse_results(tests.returncode, bench.stdout)
    finally:
        subprocess.run(["git", "checkout", "--", "agent_harness/"])


result = validate(patch)
if result.pass_rate < BASELINE.pass_rate or result.p50_latency_ms > BASELINE.p50_latency_ms:
    reject(patch, reason="regression detected")
else:
    merge(patch)

Safety: Observability and Oversight

SICA’s core safety mitigation is observability: humans can and should oversee all aspects of a potentially self-improving system. Retain agent thinking, terminal output, and file modifications so you can always answer “why was this changed this way.”

Two additional guardrails:

1. An overseer — a separate LLM (different from the running agent) that reviews proposed changes and can cancel an agent mid-run.
2. Change budgets — limit how much self-improvement can merge in a given window, preventing runaway loops.

Common Failure Modes

• Overfitting to the eval set. The harness improves benchmark scores but breaks real tasks. Guardrail: keep a held-out production sample in the validation set.
• Patch chains. Each improvement looks fine, but five in a row drift the harness. Guardrail: review merged self-improvements periodically.
• Model-saturation. When the base model already performs well, self-improvement yields marginal gains (SICA observed this on reasoning benchmarks). Don’t force it.

Conclusion

The self-improving codebase turns performance logs into fuel for continuous refactoring. Mine execution traces for real weaknesses, propose minimal harness patches, and gate every change behind strict regression testing.

The developer role shifts from hand-tuning prompts to building the infrastructure that makes agent learning possible: trace logging, curated validation datasets, weakness mining, and automated eval gates. Your codebase won’t just ship features — it will learn to ship them better.