On-Device AI in Flutter with Gemini Nano and GenKit

The future of mobile AI isn’t in the cloud—it’s right in your users’ pockets. On-device AI eliminates latency, protects user privacy, and works without an internet connection. With Google’s Gemini Nano and GenKit, you can bring powerful AI capabilities directly into your Flutter applications.

In this guide, you’ll learn how to integrate Gemini Nano’s on-device inference engine with Flutter using GenKit as your orchestration layer. We’ll build a practical example that demonstrates text generation, classification, and summarization—all running locally on the device.

Prerequisites

Before diving in, ensure you have:

• Flutter 3.19 or later installed
• Android Studio with an emulator or physical device running Android 11+
• A Google AI Studio API key (for GenKit setup)
• Basic familiarity with Dart and Flutter widgets
• Gemini Nano-compatible device (most modern Android devices with 4GB+ RAM)

Add these dependencies to your pubspec.yaml:

dependencies:
  flutter:
    sdk: flutter
  firebase_core: ^3.0.0
  firebase_ai: ^1.0.0
  genkit: ^1.0.0
  genkit_rive: ^0.5.0

Run flutter pub get to install everything.

Understanding Gemini Nano

Gemini Nano is Google’s smallest Gemini model, optimized for on-device execution. Unlike cloud-based Gemini APIs, Nano runs entirely on the device’s hardware—no data leaves the phone.

Key capabilities:

• Text generation and completion
• Text classification
• Summarization
• Sentiment analysis
• Code generation (limited)

The model activates automatically when your app requests it through Firebase AI, consuming minimal battery while delivering instant responses.

Setting Up Firebase AI

Firebase AI is the bridge between your Flutter app and Gemini Nano. Initialize it in your app’s entry point:

import 'package:flutter/material.dart';
import 'package:firebase_core/firebase_core.dart';
import 'package:firebase_ai/firebase_ai.dart';

void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  await Firebase.initializeApp();

  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'On-Device AI Demo',
      theme: ThemeData(
        colorSchemeSeed: Colors.blue,
        useMaterial3: true,
      ),
      home: const AIAssistantScreen(),
    );
  }
}

Configuring GenKit

GenKit provides a structured way to define AI flows. Create a configuration file that maps your on-device operations:

import 'package:genkit/genkit.dart';
import 'package:firebase_ai/firebase_ai.dart';

class AIConfig {
  static late FirebaseAI geminiNano;

  static Future<void> initialize() async {
    geminiNano = FirebaseAI.googleAI(
      safetySettings: [
        SafetySetting(
          harmCategory: HarmCategory.hateSpeech,
          threshold: SafetySetting.blockMediumAndAbove,
        ),
        SafetySetting(
          harmCategory: HarmCategory.harassment,
          threshold: SafetySetting.blockLowAndAbove,
        ),
      ],
    );
  }
}

Building the Text Generator Flow

Let’s create a reusable text generation flow that taps into Gemini Nano:

class TextGeneratorFlow {
  static Future<String> generate({
    required String prompt,
    int maxTokens = 256,
    double temperature = 0.7,
  }) async {
    final model = AIConfig.geminiNano.generativeModel(
      model: 'gemini-nano',
      generationConfig: GenerationConfig(
        maxOutputTokens: maxTokens,
        temperature: temperature,
      ),
    );

    final response = await model.generateContent([
      Content.text(prompt),
    ]);

    return response.text ?? '';
  }

  static Future<String> summarize(String text) async {
    final prompt = 'Summarize the following text concisely:\n\n$text';
    return generate(prompt: prompt, maxTokens: 150);
  }

  static Future<String> classify(String text, List<String> categories) async {
    final categoryList = categories.join(', ');
    final prompt = '''
Classify the following text into one of these categories: $categoryList

Text: "$text"

Respond with ONLY the category name.
''';
    return generate(prompt: prompt, temperature: 0.2);
  }

  static Future<String> analyzeSentiment(String text) async {
    final prompt = '''
Analyze the sentiment of this text. Respond with:
- "positive" if sentiment is positive
- "negative" if sentiment is negative
- "neutral" if sentiment is neutral

Text: "$text"
''';
    return generate(prompt: prompt, temperature: 0.1);
  }
}

Creating the UI Layer

Build a clean interface that showcases the on-device AI capabilities:

class AIAssistantScreen extends StatefulWidget {
  const AIAssistantScreen({super.key});

  @override
  State<AIAssistantScreen> createState() => _AIAssistantScreenState();
}

class _AIAssistantScreenState extends State<AIAssistantScreen> {
  final TextEditingController _inputController = TextEditingController();
  String _result = '';
  bool _isLoading = false;
  String _selectedMode = 'generate';

  final Map<String, String> _modes = {
    'generate': 'Generate Text',
    'summarize': 'Summarize',
    'classify': 'Classify',
    'sentiment': 'Sentiment',
  };

  Future<void> _processInput() async {
    if (_inputController.text.isEmpty) return;

    setState(() {
      _isLoading = true;
      _result = '';
    });

    try {
      String response;

      switch (_selectedMode) {
        case 'summarize':
          response = await TextGeneratorFlow.summarize(_inputController.text);
          break;
        case 'classify':
          response = await TextGeneratorFlow.classify(
            _inputController.text,
            ['technology', 'sports', 'politics', 'entertainment', 'science'],
          );
          break;
        case 'sentiment':
          response = await TextGeneratorFlow.analyzeSentiment(_inputController.text);
          break;
        default:
          response = await TextGeneratorFlow.generate(
            prompt: _inputController.text,
          );
      }

      setState(() => _result = response);
    } catch (e) {
      setState(() => _result = 'Error: ${e.toString()}');
    } finally {
      setState(() => _isLoading = false);
    }
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: const Text('On-Device AI'),
        actions: [
          IconButton(
            icon: const Icon(Icons.info_outline),
            onPressed: () => _showDeviceInfo(context),
          ),
        ],
      ),
      body: Padding(
        padding: const EdgeInsets.all(16.0),
        child: Column(
          crossAxisAlignment: CrossAxisAlignment.stretch,
          children: [
            SegmentedButton<String>(
              segments: _modes.entries.map((entry) {
                return ButtonSegment<String>(
                  value: entry.key,
                  label: Text(entry.value),
                );
              }).toList(),
              selected: {_selectedMode},
              onSelectionChanged: (Set<String> selected) {
                setState(() => _selectedMode = selected.first);
              },
            ),
            const SizedBox(height: 16),
            TextField(
              controller: _inputController,
              maxLines: 4,
              decoration: InputDecoration(
                hintText: _getHintText(),
                border: const OutlineInputBorder(),
              ),
            ),
            const SizedBox(height: 16),
            FilledButton.icon(
              onPressed: _isLoading ? null : _processInput,
              icon: _isLoading
                  ? const SizedBox(
                      width: 20,
                      height: 20,
                      child: CircularProgressIndicator(strokeWidth: 2),
                    )
                  : const Icon(Icons.auto_awesome),
              label: Text(_isLoading ? 'Processing...' : 'Run AI'),
            ),
            const SizedBox(height: 16),
            if (_result.isNotEmpty) ...[
              const Text(
                'Result:',
                style: TextStyle(fontWeight: FontWeight.bold),
              ),
              const SizedBox(height: 8),
              Expanded(
                child: Card(
                  child: Padding(
                    padding: const EdgeInsets.all(16),
                    child: SingleChildScrollView(
                      child: Text(_result),
                    ),
                  ),
                ),
              ),
            ],
          ],
        ),
      ),
    );
  }

  String _getHintText() {
    switch (_selectedMode) {
      case 'summarize':
        return 'Paste text to summarize...';
      case 'classify':
        return 'Enter text to classify...';
      case 'sentiment':
        return 'Enter text to analyze sentiment...';
      default:
        return 'Enter a prompt for text generation...';
    }
  }

  void _showDeviceInfo(BuildContext context) {
    showDialog(
      context: context,
      builder: (context) => AlertDialog(
        title: const Text('Device Info'),
        content: const Column(
          mainAxisSize: MainAxisSize.min,
          crossAxisAlignment: CrossAxisAlignment.start,
          children: [
            Text('Model: Gemini Nano (On-Device)'),
            SizedBox(height: 8),
            Text('Processing: Local GPU/NPU'),
            SizedBox(height: 8),
            Text('Privacy: Data never leaves device'),
          ],
        ),
        actions: [
          TextButton(
            onPressed: () => Navigator.pop(context),
            child: const Text('Close'),
          ),
        ],
      ),
    );
  }
}

Putting It All Together

Combine everything in your main app initialization:

void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  await Firebase.initializeApp();
  await AIConfig.initialize();

  runApp(const OnDeviceAIDemo());
}

class OnDeviceAIDemo extends StatelessWidget {
  const OnDeviceAIDemo({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Gemini Nano Demo',
      theme: ThemeData(
        colorSchemeSeed: Colors.deepPurple,
        useMaterial3: true,
      ),
      home: const AIAssistantScreen(),
    );
  }
}

Advanced: Caching and Performance

Optimize your on-device AI with intelligent caching:

class AICache {
  static final Map<String, String> _cache = {};
  static const int _maxCacheSize = 100;

  static String? get(String key) => _cache[key];

  static void set(String key, String value) {
    if (_cache.length >= _maxCacheSize) {
      _cache.remove(_cache.keys.first);
    }
    _cache[key] = value;
  }

  static void clear() => _cache.clear();
}

class CachedAIFlow {
  static Future<String> generateCached({
    required String prompt,
    int maxTokens = 256,
  }) async {
    final cacheKey = '${prompt.hashCode}_$maxTokens';
    final cached = AICache.get(cacheKey);

    if (cached != null) return cached;

    final result = await TextGeneratorFlow.generate(
      prompt: prompt,
      maxTokens: maxTokens,
    );

    AICache.set(cacheKey, result);
    return result;
  }
}

Handling Errors Gracefully

On-device AI can fail due to model unavailability or device limitations. Implement robust error handling:

class SafeAIHandler {
  static Future<String> safeGenerate(String prompt) async {
    try {
      return await TextGeneratorFlow.generate(prompt: prompt);
    } on FirebaseException catch (e) {
      if (e.code == 'model-not-available') {
        return 'AI model not available on this device. Please check device compatibility.';
      }
      return 'Firebase error: ${e.message}';
    } catch (e) {
      return 'Unexpected error: ${e.toString()}';
    }
  }

  static Future<bool> checkDeviceCompatibility() async {
    try {
      final result = await TextGeneratorFlow.generate(
        prompt: 'Hello',
        maxTokens: 10,
      );
      return result.isNotEmpty;
    } catch (_) {
      return false;
    }
  }
}

Testing On-Device AI

Write tests to verify your AI flows work correctly:

import 'package:flutter_test/flutter_test.dart';

void main() {
  test('Text generation returns non-empty response', () async {
    final result = await TextGeneratorFlow.generate(
      prompt: 'Say hello',
      maxTokens: 50,
    );
    expect(result.isNotEmpty, true);
  });

  test('Classification returns valid category', () async {
    final result = await TextGeneratorFlow.classify(
      'The football team won the championship',
      ['sports', 'technology', 'politics'],
    );
    expect(result.toLowerCase(), contains('sports'));
  });

  test('Sentiment analysis detects positive text', () async {
    final result = await TextGeneratorFlow.analyzeSentiment(
      'This is amazing and wonderful!',
    );
    expect(result.toLowerCase(), 'positive');
  });
}

Performance Considerations

When deploying on-device AI in production:

1. Model warming: Pre-load the model during app startup to avoid cold-start delays
2. Batch processing: Group similar requests to reduce model switching overhead
3. Token limits: Keep prompts concise—Gemini Nano processes faster with shorter inputs
4. Device targeting: Set minimum SDK requirements to ensure Gemini Nano compatibility
5. Battery optimization: Use background processing for non-urgent AI tasks

Conclusion & Next Steps

You’ve built a complete on-device AI system in Flutter using Gemini Nano and GenKit. Your app now processes AI tasks locally, delivering instant results while keeping user data private.

Next steps to explore:

• Implement streaming responses for real-time text generation
• Add multimodal capabilities with image understanding
• Explore fine-tuning options for domain-specific tasks
• Build offline-first AI features with local storage integration
• Integrate with Firebase ML Model Downloader for automatic model updates

The on-device AI revolution is here—and with Gemini Nano, your Flutter apps are ready to lead it.