The Rise of Nano-3: Why Small Models are the New Standard for Mobile Intelligence

For years, “mobile AI” meant sending your keystrokes to a distant server and waiting for the answer to bounce back. That model carried two hidden costs: a 500ms-to-2-second latency that breaks interactive UX, and a token bill that scales with every user you add.

That era is ending. Small Language Models (SLMs) have become the new default for mobile and edge intelligence.

Gemini Nano v3, launched alongside Android 17 at Google I/O 2026, is described by Google as the most capable on-device model it has shipped inside a smartphone. It runs entirely on the device’s NPU, works offline, and keeps every byte of user data in local memory.

In this tutorial, you will learn what Nano-3 actually is, how it works through Android’s AICore service, and how to build privacy-first on-device features with the ML Kit GenAI APIs.

Why Mobile AI is Moving On-Device

Three forces push intelligence to the edge:

1. Latency. Cloud LLMs add 500ms-2s per round trip. On-device SLMs reach sub-100ms first-token latency — fast enough to feel native to the OS.
2. Privacy. Sending every keystroke to a server is no longer acceptable. On-device inference never lets data leave volatile memory.
3. Cost. Token-based cloud pricing scales poorly with high user volume. Local inference uses the user’s own hardware, eliminating recurring API costs for standard generative tasks.

This is why a secure messaging app can now summarize encrypted threads on-device — the plaintext never leaves the device to be decrypted on a server.

What Nano-3 Actually Is

Gemini Nano is the smallest member of the Gemini family, designed to run on mobile hardware without a network connection. It is not the Gemini you access via the API or the Gemini app — those are larger cloud-hosted models. Nano is smaller, quantized, and runs entirely on the device’s NPU or GPU.

Nano-3 ships in two hardware-matched sizes:

Variant	Parameters	Target hardware
Nano-1	1.8B	Low-memory devices
Nano-2	3.25B	Flagship hardware

Both use 4-bit quantization to fit mobile memory budgets. The models are created through knowledge distillation: a small student model is trained to imitate the probability distributions of a much larger Gemini teacher, carrying over a useful share of its behavior in a fraction of the parameter budget.

flowchart LR
    A["Gemini Teacher<br/>(large, cloud)"]
    B["Distillation + 4-bit Quantization"]
    C["Gemini Nano-2<br/>(3.25B)"]
    D["Gemini Nano-1<br/>(1.8B)"]
    E["Device NPU"]
    F["AICore System Service"]

    A --> B
    B --> C
    B --> D
    C --> F
    D --> F
    F --> E

    classDef teacher fill:#fef3c7,stroke:#d97706,color:#0f172a;
    classDef nano fill:#e0f2fe,stroke:#0284c7,color:#0f172a;
    classDef runtime fill:#dcfce7,stroke:#16a34a,color:#0f172a;

    class A teacher;
    class C,D nano;
    class F runtime;

AICore and the AI Edge SDK

Gemini Nano runs through Android’s AICore system service. This is the key architectural detail: the model is not bundled inside your APK.

AICore is a system-level service that manages the model lifecycle — downloads, updates, and safety features. Your app requests inference through the Google Play Services AI APIs. Because the model is several gigabytes, keeping it out of your APK keeps your app download small.

The API is intentionally similar to the Gemini cloud API, so switching between on-device and cloud is often just a model-name swap.

// Check device support and download if needed
val status = Generation.getClient().checkStatus()

when (status) {
    FeatureStatus.DOWNLOADABLE -> Generation.getClient().download().collect {
        // Progress updates
    }
    FeatureStatus.AVAILABLE -> println("Nano-3 ready to use")
    FeatureStatus.UNSUPPORTED -> println("Device does not support on-device Gemini")
}

Building with ML Kit GenAI APIs

The ML Kit GenAI APIs expose high-level, task-specific surfaces on top of Nano-3: summarization, proofreading, rewriting, and image description. Each API ships with a small task-specific LoRA adapter trained on representative data.

LoRA adapters load on demand once the base Nano model is on the device. They are worth taking seriously — feature-specific LoRA tuning lifts summarization quality from 77.2 to 92.1 and image description from 86.9 to 92.3 on Google’s internal raters.

import com.google.mlkit.genai.Summarization

suspend fun summarizeConversation(thread: String): String {
    val client = Summarization.getClient()
    return client.summarize(thread)
}

The LoRA blocks are also the integration point for safety: they are trained against app-specific safety data, and AICore runs additional input and output safety classifiers around each call.

Building a GeminiNano Model Directly

For more control, you can instantiate a GenerativeModel configured for Nano through the AI Edge SDK:

val model = GenerativeModel(
    modelName = "gemini-nano-2",
    generationConfig = GenerationConfig(
        temperature = 0.2f,
        topK = 1,
        maxOutputTokens = 256,
    ),
)

This gives you direct prompt-based inference for use cases beyond the fixed task APIs.

Cross-Platform with MediaPipe

For teams targeting both Android and iOS with shared logic, MediaPipe’s LLM Inference API loads any compatible model — Gemma, Phi, Falcon, and more — across mobile and web. It offers a more flexible pipeline for complex data types.

If you target web too, Chrome’s on-device GenAI (via LiteRT-LM) extends the same story to Chromebooks and Pixel Watch: summarizers, translation widgets, and structured-output JSON generators that run entirely on the user’s machine.

When to Go On-Device vs. Cloud

Task	On-device Nano	Cloud Gemini
Summarization	✅	—
Smart reply	✅	—
Proofreading / rewriting	✅	—
Image description	✅	—
Offline translation	✅	—
Massive multi-step reasoning	—	✅
Complex RAG / agentic loops	—	✅
Latest knowledge	—	✅

The hybrid future: use SLMs for 90% of tasks and reserve cloud LLMs for complex reasoning.

Conclusion

Gemini Nano v3 marks the point where on-device models became the standard for mobile intelligence — not a fallback. With sub-100ms latency, zero token cost, and data that never leaves the device, SLMs deliver the responsive, private, and affordable features that modern apps demand.

Start with the ML Kit GenAI task APIs for summarization and proofreading, drop to the AI Edge SDK when you need direct prompt control, and reserve cloud Gemini for the reasoning-heavy work. Your users get faster features and real privacy — and your cloud bill stops scaling with every user who hits the summarize button.