Building a Collaborative Gemini 3 Editor: Real-time Writing and Fact-Checking

Most people use AI on a document the hard way: copy a passage into a chat window, ask a question, paste the answer back. The loop is manual, lossy, and breaks the moment the document changes. A collaborative AI editor closes that gap — the AI sits beside the document, reads it directly, and edits it in real time, exactly like another person on the same screen.

The architectural key is recognizing that an AI agent is just another CRDT peer. Once you treat it that way, the agent can join a shared document the same way a second human does — and it receives none of the special-casing that makes most AI writing tooling bolted-on and fragile.

Why Treat the Agent as a Peer?

CRDTs are already how Google Docs-class collaboration works. The natural evolution is to make the agent a participant, not an assistant:

• The agent can edit whether or not any browser has the document open (it’s a server-side peer).
• Conflicts resolve mathematically — Yjs merges a human and an agent editing simultaneously without coordination.
• Presence and provenance track who wrote what, human and AI alike.

The gap: Word Copilot edits in private. Notion AI answers in a side panel. The fix: stop treating AI as an assistant. Give it a seat, a name, a thread. — Knovya real-time collaboration (https://knovya.com/features/real-time-collaboration)

The Architecture

flowchart LR
    A["Human editor"] --> D["Shared doc<br/>(Yjs CRDT)"]
    B["Gemini 3 agent"] --> D
    C["Second human"] --> D
    D --> E["prosemirror"]
    subgraph Server
      F["CRDT server"]
      G["Agent runtime"]
    end

Step 1 — Use a CRDT for the document

Yjs is the dominant toolkit. The document is a shared CRDT; edits from any client (human or agent) converge automatically. Wire the editor to the CRDT with a binding like y-prosemirror for a rich-text experience, and use y-protocols for presence (cursors, selections).

Step 2 — Make the agent a server-side peer

The agent opens its own Yjs document on the server and connects to the same shared state as the human editors. It’s just another participant with a name and identity:

// server/agent.js
import * as Y from 'yjs';

const doc = new Y.Doc();     // the agent's view of the shared document
const type = doc.getText('content');

Because the agent is server-side, it can work while everyone’s laptops are closed.

3 — Tools, not freehand edits

The agent doesn’t mutate the Yjs doc directly. It reasons, then makes tool calls that a runtime translates into CRDT operations. This keeps the agent’s editing deterministic and reviewable — and preserves the “agent as separate principal” contract.

const docTools = [
  get_document_snapshot,   // read a plain-text snapshot to see current state
  search_text,             // find text, returns stable match handles
  place_cursor,            // move agent cursor to a match handle
  insert_text,             // insert at the agent's cursor
  start_streaming_edit,    // reroute the model's token stream into the doc
  replace, select, format, delete,
];

The important pattern is search → place cursor → edit: the agent locates content by meaning (match handles backed by Yjs relative positions), not by brittle character indexes. This stays valid even when other users edit concurrently.

Stream the model output into the document

The trick that makes it feel live: the model generates in its native format (markdown), and a streaming pipeline converts tokens into rich-text nodes as they arrive. **bold** becomes a bold mark, ## heading becomes a heading node, - item a list item. The agent’s cursor moves through the document as it writes — the user sees text appear in real time, exactly like watching someone type.

A start_streaming_edit tool flips a switch: after it’s called, the model’s next text output is intercepted and redirected into the document instead of into the chat sidebar. The model just writes naturally; the infrastructure decides where it lands.

Fact-checking as a First-Class Tool

Real-time verification wires into the same tool loop. Give the agent tools like:

• web_search / grounding — check a claim against sources while writing.
• query_document — ask a question grounded in the doc’s own content.
• suggest_citation — attach a source to a claim.

With Gemini 3’s Grounding with Google Search, the agent can verify each section as it writes and flag or fix claims that don’t check out — turning a “writer” into a “writer who checks its work.”

Propose, Accept, Reject

Full autonomy is rarely right for prose. The pattern that wins is propose-then-commit — a seat of suggestions that the human decides to keep.

• Proposals / annotations surface as diff cards beside the document (accept / reject / reply).
• Block-level propose-apply-reject for humans and AI alike, with race-safe acceptance under concurrent reviewers.
• Smart merge renders a four-pane view when concurrent edits genuinely collide: auto-merged blocks above, conflicts paired side-by-side.

This keeps the agent’s power while leaving editorial control human.

Provenance & Audit

Because the agent is a named principal, provenance becomes meaningful:

• Hover a block to see who edited it last.
• Share breakdown across 7/30/90 days — agents honored earned their own row.
• Version history reconstructs every change with an author trail.

EditorZero treats this as a hard invariant: every mutation produces exactly one audit entry, and the audit log alone can reconstruct the final state (https://github.com/numman-ali/editorzero).

Getting the Document into the AI’s Context & Vise Versa

Two transport patterns keep the agent grounded:

1. Document → model: the agent reads a snapshot (or relevant range via a tool) to see the current state before acting.
2. Model → document: streamed edits bound to match handles that survive concurrent mutation.

A durable session layer (e.g., Durable Streams) repeats both — the document and the agent’s chat / tool stream share one resumable transport, so a refresh or reconnect picks up mid-generation without losing either state (https://electric.ax/blog/2026/04/08/ai-agents-as-crdt-peers-with-yjs.md).

Putting It All Together

A collaborative Gemini 3 editor:

1. Shared CRDT (Yjs) + rich-text binding (ProseMirror) for the document.
2. Server-side agent peer that opens its own document instance.
3. Tool-driven editing — search/place/insert, with streamed token → rich text.
4. Fact-checking tools — grounded in sources and the document itself.
5. Propose-commit UX with race-safe acceptance.
6. Provenance & audit — agents as first-class authors.

Conclusion & Next Steps

You’ve learned to go from a bolted-on AI sidebar to a true co-author: model the agent as a CRDT peer, let it read and edit the shared document through tools, stream its output in, and let the CRDT resolve conflicts with everyone else.

To go further:

• Multi-agent — let several named agents tag, reply, and coordinate on the same doc through a shared turn queue.
• Command surface — a @-mention agent in a thread; the agent replies in the same thread, not a sidebar.
• Export — round-trip real formats (e.g., .docx) with agent comments intact.

The next generation of collaborative documents won’t have humans and AI on separate planes. When the AI has a cursor, a name, and a CRDT-consistent view of what you’re writing — and it fact-checks as it goes — it stops being a tool and starts being a colleague.