Zero-Knowledge Agents: Processing Private Data with Gemini 3's Encrypted Reasoning

When an agent processes private data — health records, legal documents, payroll — today’s reality is that the data passes through someone’s API in cleartext. Zero-knowledge agents change two promises at once:

1. Privacy — the provider computes on your data without seeing it (FHE / TEE).
2. Integrity — you can verify the model you paid for actually ran your prompt and produced the returned transcript (ZK proofs over the inference), without trusting the provider.

In this tutorial, you will learn the integrity gap zero-knowledge closes, how ZK over LLM inference went from proving one token to proving a whole transcript, and how FHE + TEE split the privacy problem into practical pieces.

The Integrity Gap

Even when the agent loop runs on your device, the provider’s API executes the model and hosts the tools (web search, sandboxed code). Those steps are opaque. A malicious or economically incentivized provider can:

• Substitute the advertised model with a cheaper or backdoored one.
• Fabricate server-tool observations (forged search results, misreported sandbox outputs) to steer the agent.

Outsourcing introduces the challenge of verifying that the returned output is the genuine result of the specified model. — DeepProve (https://eprint.iacr.org/2026/1112)

A malicious or economically incentivized provider may deviate arbitrarily from the prescribed agent procedure: silently substituting the advertised model… or fabricating server-tool observations. — zkAgent (https://eprint.iacr.org/2026/199.pdf)

ZK over LLM Inference: From One Token to a Whole Transcript

Early zkLLM work proved a single Transformer forward pass. Real agents interleave many inferences with tool calls. The 2026 step-change:

• zkAgent proves the complete inference pipeline — token→embedding lookup, positional encoding, Transformer compute, decoding — and binds each tool observation to authenticated execution (zkTLS / zkVM), producing a single end-to-end proof. Its one-shot transcript proving exploits the Transformer’s causal attention mask to prove an entire multi-step agent transcript in a single forward pass.

The numbers matter: on GPT-2 with a 512-token transcript, 767× prover speedup over the state of the art and 10,384× faster verification (0.42s vs 4,361s). A real coding-assistant execution proves end-to-end in 99.74s with 0.28s verification — verifiable agent execution is now practical.

By exploiting the Transformer’s causal attention mask, zkAgent proves an entire multi-step agent transcript in a single forward pass… amortizing to 0.40 s/token, and reduces verification time by 10,384×. — zkAgent (https://eprint.iacr.org/2026/199.pdf)

• DeepProve certifies the correctness of the output sequence rather than encoding all compute in-circuit, reaching ~86–174 tokens/min with 1–3.7s verification.
• NanoZK decomposes inference into layerwise proofs linked by a SHA-256 commitment chain — constant ~3.5KB sub-circuit proofs that hide weights and activations from verifiers.

The Privacy Side: FHE + TEE

Privacy and integrity are complementary. Fully Homomorphic Encryption (FHE) lets an untrusted accelerator compute on ciphertexts; Trusted Execution Environments (TEEs) confine plaintext to hardware-isolated domains. Neither alone is sufficient — FHE is too expensive for non-linear operators, a CPU-only TEE forfeits the GPU.

Bifrost splits the difference: large linear layers run on accelerator-backed CKKS (FHE); non-linear ops, attention control, KV-state transitions, and ciphertext refresh execute inside the CPU TEE. Bifrost+ adds prefill/decode disaggregation, cutting time-to-first-token by 14.6–45.8× on GPT-2 (124M).

The lesson: selective encrypted execution — use FHE only where ciphertext-only accelerator delegation is required, and keep non-linear, refresh, and prompt-side work inside the CPU TEE.

FHE is therefore the secure delegation mechanism, not the sole trust anchor. — Bifrost (https://arxiv.org/html/2606.17421v1)

The Collaborative Pattern (CoTrust)

Running a full LLM inside a TEE is prohibitively expensive. CoTrust’s split: an SLM inside the TEE processes the full private input; a large LLM outside operates only on de-identified views. The external LLM produces a consensus scaffold — reasoning structure with no private entities — which the TEE SLM then grounds in the full-fidelity input. Privacy stays confined to the TEE; reasoning power comes from the outside model.

The external LLM produces a de-identified scaffold that captures high level reasoning and answer structure, while delegating all privacy-sensitive instantiation to a TEE resident SLM with full access to the original input. — CoTrust (https://aclanthology.org/2026.findings-acl.1078.pdf)

The Architecture

flowchart LR
    U["User private data"] --> T["TEE boundary"]
    T --> S["TEE-resident SLM<br/>(full-fidelity input)"]
    T --> D["De-identification<br/>(mask PII)"]
    D --> L["External LLM<br/>(Gemini 3, honest-but-curious)"]
    L --> SC["De-identified scaffold"]
    SC --> T
    S --> A["Private answer"]
    P["Model commitment c_W"] --> V["Verifier<br/>(client / auditor)"]
    L -. "(y, proof π)" .-> V

What’s In Scope and What Isn’t

Claim	Covered by	Notes
Model identity (no substitution)	ZK proofs over committed weights	Public Merkle root over weights
Tool observation provenance	zkTLS / zkVM subproofs	Provenance, not business-logic correctness
Prompt privacy from provider	FHE / TEE	Complementary to ZK; ZK alone doesn’t hide the prompt from the prover
Prompt injection defense	Not ZK	Injection appears in the transcript and is auditable, but filtering is an alignment/policy task
Replay protection	Timestamp / trace-id windows	Orthogonal layer

NanoZK hides weights and activations from verifiers and auditors but does not hide the prompt from the prover — this is complementary to HE/MPC. — NanoZK (https://www.arxiv.org/pdf/2603.18046)

Practical Deployment Guidance

• Minute-scale proving is fine for high-stakes async decisions (medical, legal, regulatory attestation), audit logging, and sampling-based attestation. Real-time chat is out of scope — pair ZK with TEE/MPC there.
• Verify a commitment, not a mystery: the provider publishes c_W; the auditor supplies the input, receives (y, Π), and accepts iff every layer proof verifies against c_W and the chain digests match.
• Compose the layers: ZK for integrity + FHE/TEE for confidentiality + per-user de-identification for cross-model collaboration.

Conclusion

Zero-knowledge agents are what happens when agents must touch data people aren’t allowed to expose. ZK proofs make inference verifiable — the provider can’t swap the model or forge tool results, and zkAgent’s one-shot transcript proofs made this practical. FHE + TEE make inference confidential — the accelerator never sees plaintext, thanks to Bifrost’s selective encryption. And SLM/LLM collaboration keeps cost sane while confining private data to the trust boundary.

Next steps: for a regulated workload, start with the verifiable end — publish a model commitment and verify a few sample transcript proofs — then layer TEE-based prefill for prompt confidentiality. Prove what you can, encrypt what you must, and never ship the secret. That’s encrypted reasoning in production.

References

• zkAgent — Verifiable LLM Agent Execution via One-Shot Transcript Proofs (eprint 2026/199): https://eprint.iacr.org/2026/199.pdf
• DeepProve — Verifiable End-to-End Large Language Model Inference (eprint 2026/1112): https://eprint.iacr.org/2026/1112
• NanoZK — Zero-Knowledge Proof System for Verifiable LLM Inference (arxiv 2603.18046): https://www.arxiv.org/pdf/2603.18046
• Bifrost — Hybrid TEE–FHE Inference for Privacy-Preserving Transformer and LLM Serving (arxiv 2606.17421): https://arxiv.org/html/2606.17421v1
• CoTrust — Privacy-Preserving Collaboration Between Large and Small Language Models in TEEs (ACL Findings 2026): https://aclanthology.org/2026.findings-acl.1078.pdf