Skip to content
Blog

Retrieval Stack in Flutter: A Pragmatic, End-to-End Tutorial

Build a production agent + retrieval stack in Flutter: an ADK backend with SSE streaming, a hand-rolled Dart SSE parser, a ChangeNotifier chat controller, and visible citations.

Published on • November 14, 2026

AI Assistant

Why this stack, and why now

Three things matured at the same time, and that coincidence defines how we build AI apps in 2026:

  1. Agent frameworks stopped being toys. Google’s Agent Development Kit (ADK) is now available in five languages (Python, TypeScript, Go, Java, and Kotlin) and is explicitly positioned as “build production agents, not prototypes” — with multi-agent orchestration, graph workflows, and a managed runtime behind it [1].
  2. Retrieval became a solved default, not an experiment. Retrieval-Augmented Generation (RAG) — combining a model’s parametric memory with a dense vector index of your documents — was formalized in the RAG paper (Lewis et al., NeurIPS 2020) and remains the standard recipe for grounding LLM answers in real, citable data [2].
  3. Flutter grew first-class AI surfaces. The Flutter AI Toolkit ships multiturn chat, streaming responses, function calling, and a pluggable LLM-provider API; Flutter’s docs now route production AI calls through a backend rather than straight from the client [3].

The pragmatic result: Flutter is the client, an agent runtime is the brain, and a vector index is the memory. This tutorial builds exactly that, in one pass.


The architecture

┌────────────────────────────┐
│  Flutter app (iOS/Android/ │
│  web/desktop)              │
│  · chat UI, streaming text │
│  · citations, tool states  │
└──────────────┬─────────────┘
               │  POST /run_sse  (Server-Sent Events)
               ▼
┌────────────────────────────┐
│  ADK API server            │
│  · agent + session state   │
│  · tool calls, guardrails  │
└──────┬──────────────┬──────┘
       │              │
       ▼              ▼
┌────────────┐  ┌─────────────────┐
│ Gemini     │  │ Retrieval tool  │
│ (model)    │  │ embed → search  │
└────────────┘  │ → rerank → cite │
                └────────┬────────┘
                         ▼
                ┌─────────────────┐
                │ Vector index    │
                │ (sqlite-vec /   │
                │  managed DB)    │
                └─────────────────┘

The division of labor is the whole point:

LayerOwnsNever owns
Flutter clientUI, streaming rendering, citation display, session UXAPI keys, retrieval logic, prompt text
ADK agentOrchestration, tool calls, session/memory state, guardrailsVector storage details, UI concerns
Retrieval toolEmbedding, kNN search, reranking, source attributionConversation flow
Model (Gemini)Reasoning and generation over retrieved contextYour private data (unless grounded)

Two rules that keep this pragmatic rather than aspirational:

  • Keys live on the backend only. Flutter’s own AI docs warn that client-side calls with a committed firebase_options.dart let anyone reuse your configuration and burn quota — they recommend routing through a backend service for exactly the reason we do here [3].
  • The client speaks one protocol: SSE events. Everything else (retrieval, tools, memory) is invisible to the UI.

Step 1 — The agent backend (ADK)

Create an agent project and expose it with ADK’s API server:

pip install google-adk
adk api_server          # serves at http://localhost:8000

The API server gives you the exact contract the Flutter app will consume [4]:

  • POST /apps/{app}/users/{user}/sessions/{session} — create a session
  • POST /run — run the agent, return all events at once as a JSON array
  • POST /run_sse — run the agent and stream events as Server-Sent Events; set "streaming": true for token-level streaming
  • GET /list-apps, GET/PATCH/DELETE sessions — utility and session management

A minimal retrieval-equipped agent (Python):

# agent.py
from google.adk import Agent
from my_tools import search_knowledge_base   # embed → kNN → rerank → chunks

root_agent = Agent(
    name="docs_assistant",
    model="gemini-flash-latest",
    instruction=(
        "Answer only from the retrieved context. "
        "If the context is insufficient, say so. "
        "Cite sources by their [id] after every claim."
    ),
    tools=[search_knowledge_base],
)

The retrieval tool does the four-step pipeline every production RAG system converges on — parse → chunk → embed → index on ingest, and embed query → search → rerank → cite on query. The original RAG paper frames this as combining parametric and non-parametric memory, and notes that retrieval also gives you provenance for model decisions — which is why citations are free once your pipeline returns source ids [2].

For the index itself, pick by deployment target:

  • Server-side / self-hosted: sqlite-vec is a vector-search SQLite extension written in pure C with no dependencies — it runs anywhere SQLite runs (including WASM), and stores non-vector metadata alongside vectors [5]. That means your chunks, their titles, and their embeddings live in one file.
  • Managed: any vector DB that exposes a kNN endpoint (pgvector, managed Firestore vector search, etc.) behind your ADK tool.

Verify the contract before touching Flutter:

curl -X POST http://localhost:8000/apps/docs_assistant/users/u_1/sessions/s_1 \
  -H "Content-Type: application/json" -d '{}'

curl -X POST http://localhost:8000/run_sse \
  -H "Content-Type: application/json" \
  -d '{"appName":"docs_assistant","userId":"u_1","sessionId":"s_1",
       "newMessage":{"role":"user","parts":[{"text":"How do I reset a session?"}]},
       "streaming":true}'

Each SSE line is data: {...} carrying an event object with content.parts (text, functionCall, functionResponse), author, invocationId, and actions.stateDelta [4]. That JSON shape is what we parse in Dart.


Step 2 — The Flutter client: SSE streaming

Add the one dependency you need for HTTP:

# pubspec.yaml
dependencies:
  flutter:
    sdk: flutter
  http: ^1.2.0

Flutter’s networking guidance is deliberately simple: the http package is the cross-platform way to issue requests, supported on Android, iOS, macOS, Windows, Linux, and the web — just remember the platform bits (the INTERNET permission on Android, the com.apple.security.network.client entitlement on macOS) [6].

A client that turns /run_sse into a Dart stream of text deltas:

// lib/data/agent_client.dart
import 'dart:async';
import 'dart:convert';
import 'package:http/http.dart' as http;

class AgentEvent {
  final String? text;
  final String? toolName;
  final bool isFinal;
  const AgentEvent({this.text, this.toolName, this.isFinal = false});
}

class AgentClient {
  AgentClient({required this.baseUrl, http.Client? client})
      : _client = client ?? http.Client();

  final String baseUrl;
  final http.Client _client;

  Future<void> ensureSession({
    required String app, required String user, required String session,
  }) async {
    final res = await _client.post(
      Uri.parse('$baseUrl/apps/$app/users/$user/sessions/$session'),
      headers: const {'Content-Type': 'application/json'},
      body: '{}',
    );
    // 409/200 both mean "session exists" — ADK rejects duplicate ids.
    if (res.statusCode >= 400 && !res.body.contains('already exists')) {
      throw StateError('session create failed: ${res.statusCode}');
    }
  }

  /// Streams parsed events from POST /run_sse with streaming: true.
  Stream<AgentEvent> ask({
    required String app, required String user, required String session,
    required String message,
  }) async* {
    final req = http.Request('POST', Uri.parse('$baseUrl/run_sse'))
      ..headers['Content-Type'] = 'application/json'
      ..headers['Accept'] = 'text/event-stream'
      ..body = jsonEncode({
        'appName': app,
        'userId': user,
        'sessionId': session,
        'newMessage': {
          'role': 'user',
          'parts': [{'text': message}],
        },
        'streaming': true,
      });

    final res = await _client.send(req);
    if (res.statusCode >= 400) {
      throw StateError('run_sse failed: ${res.statusCode}');
    }

    var buffer = '';
    await for (final chunk in res.stream.transform(utf8.decoder)) {
      buffer += chunk;
      // SSE frames are separated by a blank line.
      final frames = buffer.split('\n\n');
      buffer = frames.removeLast();          // keep the partial frame
      for (final frame in frames) {
        for (final line in const LineSplitter().convert(frame)) {
          if (!line.startsWith('data:')) continue;
          final payload = line.substring(5).trim();
          if (payload.isEmpty) continue;
          final event = _parse(jsonDecode(payload) as Map<String, dynamic>);
          if (event != null) yield event;
        }
      }
    }
  }

  AgentEvent? _parse(Map<String, dynamic> json) {
    final parts = ((json['content']?['parts']) as List?) ?? const [];
    String? text;
    String? tool;
    for (final raw in parts) {
      final part = raw as Map<String, dynamic>;
      if (part['text'] is String) {
        text = (text ?? '') + part['text'] as String;
      } else if (part['functionCall'] is Map) {
        tool = part['functionCall']['name'] as String?;
      }
    }
    if (text == null && tool == null) return null;
    return AgentEvent(text: text, toolName: tool);
  }
}

Notes that matter in practice:

  • We parse SSE by hand instead of using EventSource, because EventSource only does GET, while ADK’s /run_sse is a POST with a JSON body [7]. The frame rules (data: lines, blank-line boundaries) are exactly the SSE spec.
  • utf8.decoder + a leftover partial frame is the difference between a demo and something that survives real network chunking.
  • Token-level streaming requires "streaming": true; without it you still get event-level SSE but each model turn arrives as one blob [4].

Step 3 — State: one ChangeNotifier, no surprises

For a chat screen, Flutter’s own state-management guidance says ephemeral state (scroll position, one field’s text) stays local, while app state shared across the screen belongs in a simple app-state holder [8]. A ChangeNotifier is enough here — no streams-of-streams, no framework.

// lib/state/chat_controller.dart
import 'dart:async';
import 'package:flutter/foundation.dart';
import '../data/agent_client.dart';

enum ChatStatus { idle, thinking, streaming, error }

class ChatMessage {
  final String role;          // 'user' | 'assistant'
  final String text;
  final List<String> toolsCalled;
  const ChatMessage(this.role, this.text, {this.toolsCalled = const []});
}

class ChatController extends ChangeNotifier {
  ChatController({required this.client, this.app = 'docs_assistant',
                  this.user = 'u_1', this.session = 's_1'});

  final AgentClient client;
  final String app, user, session;

  final messages = <ChatMessage>[];
  var status = ChatStatus.idle;
  var _streamed = '';

  /// Live partial text of the answer currently streaming in.
  String get streamedText => _streamed;

  Future<void> send(String prompt) async {
    if (prompt.trim().isEmpty || status == ChatStatus.streaming) return;
    messages.add(ChatMessage('user', prompt));
    _streamed = '';
    status = ChatStatus.thinking;
    notifyListeners();

    try {
      await client.ensureSession(app: app, user: user, session: session);
      final tools = <String>[];

      await for (final event in client.ask(
        app: app, user: user, session: session, message: prompt,
      )) {
        if (event.toolName != null) {
          tools.add(event.toolName!);
          status = ChatStatus.thinking;      // retrieval in progress
        }
        if (event.text != null) {
          status = ChatStatus.streaming;
          _streamed += event.text!;
        }
        notifyListeners();
      }

      messages.add(ChatMessage('assistant', _streamed, toolsCalled: tools));
      status = ChatStatus.idle;
    } catch (e) {
      status = ChatStatus.error;
    }
    notifyListeners();
  }
}

Two UI-facing states instead of one: thinking (the agent may be running retrieval — show a tool chip, not a fake typing cursor) and streaming (text is arriving — render it). This maps directly onto what the SSE events tell you: a functionCall part means “working”, a text part means “answering” [4].


Step 4 — The chat screen

// lib/ui/chat_page.dart
import 'package:flutter/material.dart';
import '../state/chat_controller.dart';

class ChatPage extends StatefulWidget {
  const ChatPage({super.key, required this.controller});
  final ChatController controller;

  @override
  State<ChatPage> createState() => _ChatPageState();
}

class _ChatPageState extends State<ChatPage> {
  final _input = TextEditingController();
  final _scroll = ScrollController();

  @override
  void initState() {
    super.initState();
    widget.controller.addListener(_follow);
  }

  void _follow() {
    if (_scroll.hasClients) {
      _scroll.jumpTo(_scroll.position.maxScrollExtent);
    }
  }

  Future<void> _send(ChatController c) async {
    final text = _input.text;
    _input.clear();
    await c.send(text);
  }

  /// Live text while streaming: rendered as the trailing bubble.
  String _liveText(ChatController c) => c.streamedText;

  @override
  void dispose() {
    widget.controller.removeListener(_follow);
    _input.dispose();
    _scroll.dispose();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    final c = widget.controller;
    return ListenableBuilder(
      listenable: c,
      builder: (context, _) {
        return Scaffold(
          appBar: AppBar(title: const Text('Docs Assistant')),
          body: Column(children: [
            Expanded(
              child: ListView.builder(
                controller: _scroll,
                itemCount: c.messages.length + (c.status == ChatStatus.streaming ? 1 : 0),
                itemBuilder: (_, i) {
                  if (i >= c.messages.length) {
                    // Trailing bubble: the answer currently streaming in.
                    return _Bubble(role: 'assistant', text: _liveText(c), busy: true);
                  }
                  final m = c.messages[i];
                  return _Bubble(role: m.role, text: m.text, tools: m.toolsCalled);
                },
              ),
            ),
            if (c.status == ChatStatus.error)
              const MaterialBanner(
                content: Text('Agent unreachable — is adk api_server running?'),
                actions: [SizedBox.shrink()],
              ),
            SafeArea(
              child: Padding(
                padding: const EdgeInsets.all(8),
                child: Row(children: [
                  Expanded(
                    child: TextField(
                      controller: _input,
                      decoration: const InputDecoration(hintText: 'Ask the docs…'),
                      onSubmitted: (_) => _send(c),
                    ),
                  ),
                  IconButton(
                    icon: const Icon(Icons.send),
                    onPressed: c.status == ChatStatus.streaming ? null : () => _send(c),
                  ),
                ]),
              ),
            ),
          ]),
        );
      },
    );
  }
}

/// Renders one message; `busy` adds a subtle blinking caret while streaming.
class _Bubble extends StatelessWidget {
  const _Bubble({required this.role, required this.text, this.busy = false,
                 this.tools = const []});
  final String role;
  final String text;
  final bool busy;
  final List<String> tools;

  @override
  Widget build(BuildContext context) {
    final mine = role == 'user';
    return Align(
      alignment: mine ? Alignment.centerRight : Alignment.centerLeft,
      child: Container(
        margin: const EdgeInsets.symmetric(horizontal: 12, vertical: 6),
        padding: const EdgeInsets.all(12),
        constraints: const BoxConstraints(maxWidth: 480),
        decoration: BoxDecoration(
          color: mine ? Theme.of(context).colorScheme.primaryContainer
                      : Theme.of(context).colorScheme.surfaceContainerHighest,
          borderRadius: BorderRadius.circular(16),
        ),
        child: Column(
          crossAxisAlignment: CrossAxisAlignment.start,
          children: [
            SelectableText(text + (busy ? '▍' : '')),
            if (tools.isNotEmpty)
              Padding(
                padding: const EdgeInsets.only(top: 6),
                child: Text('🔧 ${tools.join(', ')}',
                    style: Theme.of(context).textTheme.labelSmall),
              ),
          ],
        ),
      ),
    );
  }
}

ChatController.streamedText is the partial answer the SSE stream has produced so far, so the trailing bubble always shows exactly what has arrived — no buffering the whole response before painting [3].

The streaming bubble is the single most important UX decision: users read while generation happens. Flutter’s AI Toolkit lists streaming responses as a headline feature for the same reason [3].


Step 5 — Citations: make retrieval visible

A 2026 agent app without citations is a 2024 app. Since the agent instruction forces [id] markers after claims [4], the UI can surface sources cheaply:

class SourceStrip extends StatelessWidget {
  const SourceStrip({super.key, required this.ids});
  final List<String> ids;

  @override
  Widget build(BuildContext context) {
    if (ids.isEmpty) return const SizedBox.shrink();
    return Wrap(
      spacing: 6,
      runSpacing: 6,
      children: [
        for (final id in ids)
          ActionChip(
            avatar: const Icon(Icons.menu_book, size: 16),
            label: Text('Source $id'),
            onPressed: () => openSource(id),   // fetch chunk + jump to doc
          ),
      ],
    );
  }
}

Why this earns its place: the RAG paper’s stated motivation was that models struggle to access and precisely manipulate knowledge and that “providing provenance for their decisions” was an open problem — retrieval solves it by attaching real documents to real sentences [2]. Show the ids, and let a tap reveal the chunk.


Step 6 — Where each concern lives (the “pragmatic” part)

The most common failure mode isn’t the model — it’s putting a layer in the wrong place.

Do in the backend (ADK):

  • All tool definitions, prompts, and guardrails — so you can change behavior without an app-store release.
  • Session and state management (stateDelta, session CRUD) [4].
  • Evaluation and observability: ADK ships a dedicated evaluation area (criteria, user simulation, environment simulation, custom metrics) plus logging, metrics, and traces for the runtime [1][4].

Do in retrieval:

  • Chunking strategy and reranking — quality here beats prompt tricks.
  • Returning structured results (id, title, snippet, score) rather than a wall of text, so the Flutter layer can render a source strip.

Do in Flutter:

  • Rendering, optimistic input locking, retry UX, and accessibility (semantics for streaming text, hit targets for chips).
  • Offline/error states — SSE connections fail; the MaterialBanner above is the minimum viable recovery.

Don’t do in Flutter:

  • Embeddings, kNN search, prompt assembly, or holding an API key. Flutter’s AI guidance explicitly recommends routing AI requests through a backend service so that the backend — not the client — controls access [3].

Step 7 — Test it like production

  1. Contract test the server first. /list-apps, session create, /run, /run_sse — curl each one before writing a line of Dart [4]. Half of “Flutter streaming is broken” bugs are server-side.
  2. Widget-test the controller with a fake AgentClient. Feed a scripted sequence (functionCall → text chunks) and assert status transitions: thinking → streaming → idle.
  3. Unit-test the SSE parser with realistic frames, including a frame split across two network chunks and a data: line containing nested JSON.
  4. Evaluate the agent, not the UI. Use ADK’s evaluation tooling to score answer quality and grounding on a fixed question set before you tune the Flutter layer [1].
  5. Check the platform plumbing once: Android INTERNET permission and macOS network entitlement [6].

What you shipped

A complete agent + retrieval stack in a single pass:

  • Backend: ADK agent with a retrieval tool, served over a documented REST contract with SSE streaming [1][4].
  • Memory: a vector index (e.g. sqlite-vec — pure C, no dependencies, runs anywhere SQLite runs) with chunk metadata stored alongside embeddings [5].
  • Client: a Flutter app that parses SSE by hand, distinguishes thinking from streaming, and shows sources [6][7].
  • Grounding: RAG with visible provenance, following the parametric/non-parametric split from the original paper [2].
  • Operations: keys and prompts on the server, evaluation on the agent, widget tests on the UI [1][3].

That is the 2026 stack — not because it uses the newest nouns, but because every layer has one job, one contract, and one place where it can be tested.

Takeaway: Keep one job per layer — Flutter renders and streams, the ADK agent orchestrates and guards, and the vector index grounds and cites — and every layer becomes independently testable, replaceable, and safe to ship.

Sources

  1. Google Agent Development Kit (ADK) documentation — “Build production agents, not prototypes”; available in Python, TypeScript, Go, Java, and Kotlin; covers multi-agent orchestration, graph workflows, evaluation, and observability. https://google.github.io/adk-docs/
  2. Lewis, P., Perez, E., Piktus, A., et al. (2020). “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.” NeurIPS 2020; arXiv:2005.11401. https://arxiv.org/abs/2005.11401
  3. Flutter Docs. “Flutter AI Toolkit” — multiturn chat, streaming responses, function calling, pluggable LLM providers; security note on routing AI requests through a backend. https://docs.flutter.dev/ai/ai-toolkit
  4. Google ADK Docs. “Use the API Server” — adk api_server, session endpoints, POST /run, POST /run_sse with streaming: true, event JSON shape. https://google.github.io/adk-docs/runtime/api-server/
  5. Alex Garcia. sqlite-vec — a vector-search SQLite extension, written in pure C with no dependencies; stores metadata alongside vectors. https://github.com/asg017/sqlite-vec
  6. Flutter Docs. “Networking” — the http package across platforms; Android INTERNET permission and macOS network entitlement. https://docs.flutter.dev/data-and-backend/networking
  7. MDN Web Docs. “Server-sent events” — EventSource, event/data framing, and the HTML spec’s SSE section. https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events
  8. Flutter Docs. “State management — Introduction” — ephemeral vs. app state; choosing simple app state when a screen’s data is shared. https://docs.flutter.dev/data-and-backend/state-mgmt/intro
  9. Google ADK Docs. “Evaluation” — criteria, user simulation, environment simulation, custom metrics. https://google.github.io/adk-docs/evaluate/