Skip to content
Blog

Building a Mobile AI Chat App End-to-End

From backend proxy to streaming UI: the full architecture for a production Flutter AI chat app — API key security, token streaming, state management, and cost control.

Published on • August 10, 2026

AI Assistant

“Can we add AI to the app?” is the question on nearly every mobile project in 2026. Flutter is arguably the best-positioned mobile framework for it — one codebase, iOS and Android shipped simultaneously — but most teams get the architecture wrong before they write a single prompt. They ship an API key inside the app binary, watch their bill spike when someone extracts it, and rebuild the whole thing properly three months later.

In this post, you will learn the end-to-end architecture for a production Flutter AI chat app: a backend proxy for API key security, streaming token delivery to a Dart stream, a state-managed chat UI, and the cost controls that keep it affordable at scale.

The architecture decision before any code

The most important decision is where the LLM calls live. Never call the model API directly from Flutter. Your key is extractable from the app binary, and rate limiting, prompt tuning, and model swaps become impossible without a release. The production pattern is a thin backend proxy:

Flutter app  ──►  Backend proxy (Cloud Function / Genkit flow)
                        │
                        └──► Gemini API (key lives here)

The proxy keeps the key server-side, adds auth, rate limiting, logging, and caching — and because prompts live on the server, you can fix a bad prompt or swap models without an app-store release. On mobile, where a review cycle plus update lag means a fix can take a week to reach everyone, that alone justifies the proxy.

In 2026 the managed option is Firebase AI Logic (firebase_ai) — the successor to Vertex AI in Firebase — which gives you the proxy pattern, streaming, and per-user auth out of the box. For everything custom, a Cloud Function or Genkit flow works.

The backend: a streaming Cloud Function

The backend’s job: take the user’s message, open a streaming request to the model, and relay each token chunk back to the client. Genkit adds telemetry, schema enforcement, and caching on top:

// Firebase Cloud Function (TypeScript)
import { onRequest } from 'firebase-functions/v2/https';
import { initializeGenkit } from '@genkit-ai/firebase';

const ai = initializeGenkit();

export const chat = onRequest(async (req, res) => {
  const { messages } = req.body;

  res.writeHead(200, {
    'Content-Type': 'text/event-stream',
    'Cache-Control': 'no-cache',
    Connection: 'keep-alive',
  });

  const response = await ai.generateStream({
    model: 'gemini-1.5-flash',
    messages,
  });
  for await (const chunk of response.stream) {
    res.write(`data: ${JSON.stringify({ text: chunk.text })}\n\n`);
  }
  res.write('data: [DONE]\n\n');
  res.end();
});

Each data: line is an SSE event carrying one token chunk. The client consumes this stream and appends tokens as they arrive — that’s what makes the UI feel instant instead of a 3-second spinner.

Streaming in Flutter: Dart streams

Flutter is uniquely good at this because StreamBuilder maps directly onto token streaming. The HTTP response’s stream feeds a Dart StreamController, and the UI rebuilds per chunk:

class ChatRepository {
  Future<void> streamResponse(
    List<Message> history,
    void Function(String token) onToken,
  ) async {
    final res = await http.post(
      Uri.parse(apiUrl),
      headers: {'content-type': 'application/json'},
      body: jsonEncode({'messages': history}),
    );

    await for (final line in res.stream.transform(utf8.decoder).transform(const LineSplitter())) {
      if (!line.startsWith('data:')) continue;
      final data = line.substring(5).trim();
      if (data == '[DONE]') break;
      onToken(jsonDecode(data)['text'] as String);
    }
  }
}

In the widget layer, a simple approach keeps a StreamController<String> and uses StreamBuilder; a production app uses proper state management (BLoC or Riverpod) with the stream flowing into state:

// With Riverpod, the AI stream becomes an AsyncValue you can consume declaratively
final responseStreamProvider = StreamProvider<String>((ref) async* {
  // yield* tokens as they arrive
});

The chat screen holds three key states in one place — messages, streaming status, and error — so they can’t get out of sync. A typing indicator shows while waiting for the first token; a pulsing cursor while tokens stream.

The chat UI

The essentials for a chat UI that feels right: auto-scroll to the latest message, distinct user/assistant bubbles, markdown rendering for assistant replies (code blocks, lists), a typing indicator, and a Stop button that aborts the in-flight request.

ListView.builder(
  itemCount: messages.length + (isStreaming ? 1 : 0),
  controller: _scrollController,
  itemBuilder: (context, i) {
    if (i == messages.length) return const TypingIndicator();
    final m = messages[i];
    return MessageBubble(
      role: m.role,
      content: m.role == MessageRole.assistant
          ? MarkdownBody(data: m.content)
          : Text(m.content),
    );
  },
)

Abort handling matters: use an AbortController/token on the request so Stop cancels generation on both sides — no orphaned connections, no wasted tokens.

Cost control: the part founders learn the expensive way

At 10,000 daily active users, each making 5 AI interactions a day with ~1,500 tokens per interaction, you’re generating 75 million tokens per day — roughly $200 to $500 per day at 2026 frontier pricing. Cost architecture is not optional. The levers:

  1. Semantic caching — cache common queries by prompt hash; one production team cut repeat API calls by ~60%.
  2. Context window trimming — cap the conversation history you send; full transcripts bloat every request.
  3. Per-user caps and routing — limit daily interactions per user, and route simple tasks to cheaper or on-device models (see on-device Gemini Nano for lightweight classification).
  4. Streaming instead of buffering — streaming doesn’t reduce cost, but it makes the perceived latency of each request acceptable so you can use faster, cheaper models.

Putting It All Together

A complete production chat app: Flutter frontend with a StreamBuilder-driven chat UI, streaming responses over SSE, auth via Firebase, a GenKit Cloud Function as the proxy (key never on device), a Firestore response cache, and per-user rate limits. The stack ships in 1–2 weeks with Firebase AI Logic, or 2–4 weeks with a custom proxy. Compare against a production-grade RAG assistant, which is a 4–8 week build.

Conclusion & Next Steps

You now understand the end-to-end architecture: why the backend proxy is non-negotiable, how SSE tokens become Dart streams, how to build the streaming chat UI, and how to control cost before it controls you. Next steps: stand up a GenKit Cloud Function, wire StreamProvider in a test screen, and add a Firestore cache — the architecture decision from week one is what separates a weekend demo from a feature you can afford at scale.

References / Sources

flutter ai-chat streaming mobile firebase genkit