Gemini 3 Robotics: Bridging the Gap Between Reasoning and Physical Motion

Reasoning and motion have historically lived in different worlds: an LLM plans, a low-level controller moves, and the two rarely talk. Gemini 3 robotics bridges the gap with a two-brain design — a vision-language-action (VLA) model that converts perception and language into motor control, and an embodied reasoning (ER) model that acts as the robot’s high-level brain, planning multi-step tasks and coordinating the VLA. The result: a robot that understands what you mean, works out how to do it, and moves its whole body to get there.

In this tutorial, you will learn the model family, how the ER brain runs as an agent loop with the body as tools, how progress tracking solves “is it done yet?,” and the layered safety model that keeps it all honest.

The Model Family

Model	Role	Deploy target
Gemini Robotics 2	Most advanced VLA; controls full humanoids feet-to-fingertips, bi-arms, 22-DoF hands	Cloud, early-access partners
Gemini Robotics ER 2	Embodied reasoning; plans multi-step tasks, coordinates VLAs, multi-robot teamwork	Public via Gemini API / AI Studio
Gemini Robotics On-Device 2	Most efficient VLA; runs locally on robotic hardware; adapts to new embodiments in a few hours	On-device, early-access

Our vision-language-action (VLA) model and embodied reasoning (ER) model work together to interact with the physical world. Each has a specialist role, but they operate as one powerful and versatile system. — Google DeepMind (https://deepmind.google/models/gemini-robotics/)

The Two-Brain Architecture

flowchart TD
    U["User instruction<br/>(natural language)"] --> ER["Gemini Robotics ER 2<br/>(embodied reasoning brain)"]
    ER --> P["Plans multi-step task<br/>(observe, decompose, track)"]
    P --> VLA["Gemini Robotics 2<br/>(vision-language-action)"]
    VLA --> M["Motor commands<br/>(whole-body, dexterous)"]
    V["Video feed<br/>(progress tracking)"] --> ER
    ER --> S["Tools<br/>(Google Search, VLA APIs, nav APIs)"]

• The ER model observes a room, decomposes the goal, coordinates the VLA, tracks progress via continuous video, and self-corrects when a step fails.
• The VLA executes — walking, crouching, reaching, tying knots, sealing bags, tight packing.
• The ER model can natively call tools (search, VLA control interfaces, navigation APIs), making the robot an agent with a body.

Our embodied reasoning (ER) model, Gemini Robotics ER 2, serves as the robot’s high-level brain, processing user instructions and communicating with humans… coordinates with the VLA to carry out the actions, and tracks progress until the task is done. — Google DeepMind (https://deepmind.google/blog/gemini-robotics-2-brings-whole-body-intelligence-to-robots/)

Reasoning Meets Control: The Agent Loop

Developers wire low-level control (VLA models, navigation APIs) as tools and stream multimodal video, audio, or text into the ER model over the Gemini Live API — a bidirectional streaming endpoint optimized for latency-sensitive tasks. The result is fluid orchestration without the jarring “stop-and-think” pauses. The ER model’s tool-orchestration performance beats prior versions across three control modes: real VLA, simulated VLA, and human tele-operation.

Developers can declare low-level control interfaces — like Vision-Language-Action (VLA) models or navigation APIs — as tools, and stream multimodal video, audio, or text directly into the model. — Google (https://blog.google/innovation-and-ai/models-and-research/google-deepmind/gemini-robotics-er-2/)

Progress Understanding: Knowing When a Task Is Done

One of robotics’ hardest problems is knowing when a task is finished. The ER model assigns each video frame into five progress buckets (0–20% … 80–100%), giving the robot real-time situational awareness to adjust, retry, or move on.

For moment-finding — the exact frame a critical event happens, like when to stop pouring coffee — it hits 91.3% accuracy with a 0.96s mean absolute distance, competing closely with much larger model categories but at a fraction of the compute cost and 4× the execution speed. That sub-second latency is what safely operating physical robotics in the real world actually requires.

Gemini Robotics ER 2 achieves 91.3% accuracy and a 0.96s mean absolute distance… delivering this precision at a fraction of the compute cost and 4x the execution speed. — Google (https://blog.google/innovation-and-ai/models-and-research/google-deepmind/gemini-robotics-er-2/)

Multi-Robot Collaboration

Different robots suit different tasks — a wheeled rover for indoors, a humanoid for uneven terrain. Gemini Robotics 2 enables diverse machines to communicate via shared semantic understanding, divide labor, and hand off to complete workflows a single robot couldn’t do alone. Demonstrated with Apptronik’s Apollo 2 and Franka F3 Duo collaborating on a shared task.

We are introducing multi-robot collaboration. This enables different types of robots to communicate and work together to solve complex workflows a single robot could not do alone. — Google DeepMind (https://deepmind.google/blog/gemini-robotics-2-brings-whole-body-intelligence-to-robots/)

On-Device Robotics: Motion Without the Cloud

Gemini Robotics On-Device 2 runs locally — critical where network latency or connectivity can’t be tolerated. It’s natively multi-embodiment, inheriting advanced motion-transfer techniques, and adapts to new bi-arm embodiments with a few hours of adaptation and typically under 200 examples — even for bodies with drastically different shapes, sensors, and degrees of freedom (Dexmate, SO101, Trossen platforms).

A VLA model designed for on-device deployment in robotics applications, offering competitive results to larger models on in-domain tasks. — Model card (https://deepmind.google/models/model-cards/gemini-robotics-on-device-2/)

Safety: A Layered, Not Bolted-On, Concern

Safety is a layer in the stack, not a feature:

• ER-as-safety-monitor: the ER model refuses unsafe tool calls from the VLA, predicts task feasibility, and proactively requests human intervention when uncertain (the ASIMOV-Agentic benchmark).
• Human proximity: the model detects nearby humans, triggers safety tool calls, and brings the robot to a safe stop, resuming only when the area is clear. The ER model is the safest to date on safety instruction following and human proximity benchmarks.
• Low-level controllers handle collision-free motion planning, balance, and force control; hardware functional safety is separate.

We recommend a layered approach to safety: interfacing VLAs with Embodied Reasoning models for high-level semantic safety; using low-level controllers for collision-free motion planning, balance and force control, and implementing hardware-specific functional safety mechanisms. — Model card (https://deepmind.google/models/model-cards/gemini-robotics-on-device-2/)

Developer Sketch: Wire a Robot Body as an Agent Tool

# Conceptual: treat a robot body as tools for an embodied-reasoning agent
from google import genai

client = genai.Client()  # Gemini Robotics ER 2 via Gemini API

tools = [
    {"function_declarations": [
        {"name": "navigate_to", "description": "Move the robot to a named location",
         "parameters": {"location": "string"}},
        {"name": "pick_up", "description": "Grasp an object at a location",
         "parameters": {"object": "string", "from": "string"}},
        {"name": "place_at", "description": "Put the held object at a location",
         "parameters": {"object": "string", "at": "string"}},
    ]}
]

stream = client.models.generate_content_stream(
    model="gemini-robotics-er-2",
    contents="Pick up the watering can and put it in the green bin on the bottom shelf.",
    tools=tools,
    config={"live": {"video": True}},  # stream robot camera feed
)

Conclusion

Gemini 3 robotics closes the reasoning-motion gap with two specialized brains: an embodied reasoning model that plans, tracks progress, and coordinates — and a VLA that moves. The ER brain runs as an agent loop with the body as tools and the camera as context, enabled by the Gemini Live streaming API. Progress tracking and moment-finding turn “is it done yet?” into a measurable classifier. Multi-robot teamwork emerges from shared semantic understanding. On-device VLAs bring motion where the cloud can’t reach. And safety is layered: the reasoner supervises the mover, proximity checks gate, and controllers handle physics.

Next steps: if you have a robot platform, expose its navigation and manipulation APIs as function declarations and stream its camera feed into ER 2. Start with a single pick-and-place task, add progress classification, then scale to multi-robot handoffs. The gap between thinking and doing is closing — one tool call at a time.

References

• Google DeepMind — Gemini Robotics 2 model page: https://deepmind.google/models/gemini-robotics/
• Google DeepMind — Gemini Robotics 2 brings whole body intelligence to robots: https://deepmind.google/blog/gemini-robotics-2-brings-whole-body-intelligence-to-robots/
• Google — Gemini Robotics ER 2: https://blog.google/innovation-and-ai/models-and-research/google-deepmind/gemini-robotics-er-2/
• Google DeepMind — Gemini Robotics On-Device 2 model card: https://deepmind.google/models/model-cards/gemini-robotics-on-device-2/
• Gemini Robotics 2 (VLA) page: https://deepmind.google/models/gemini-robotics/vla/