Machine Learning in Flutter Web: A Working Inference Pattern

Flutter web compiles to JavaScript or Wasm, but the model still has to run somewhere — and “somewhere” means a browser ML runtime you drive through JS interop. This is the umbrella post for that setup: which runtime to pick, how to load a model from a Flutter app, how to keep inference off the UI thread, and what the multi-megabyte model file does to your page metrics. The TensorFlow Lite and ONNX Runtime paths get their own posts; here we wire the general pattern.

The options in one pass

TensorFlow.js is the default answer: it runs pre-trained models, converts Python TensorFlow models into a browser format, does transfer learning in the browser, and switches between WebGL and WebGPU backends. Flutter’s own Holobooth demo is a Flutter web UI sitting on TensorFlow.js — proof the combination ships. ONNX Runtime Web covers the PyTorch side: export once, run in the browser (see the ONNX post). Transformers.js pulls Hugging Face models into the page, and MediaPipe handles vision tasks.

Pick by model origin, not preference: TensorFlow models go to TensorFlow.js, PyTorch exports go to ONNX, and a .tflite file has to be converted before it runs — that is the TensorFlow Lite post. The integration layer below is identical for all of them.

Load the model and call it through a bridge

Declare the model in your Flutter assets, then keep one thin JS bridge file that Dart can reach through dart:js_interop:

# pubspec.yaml
flutter:
  assets:
    - assets/models/model.json
    - assets/models/group1-shard1of1.bin

import 'dart:js_interop';
import 'dart:js_interop_unsafe';
import 'package:web/web.dart' as web;

Future<Float32List> runInference(Float32List input) async {
  final bridge = web.window.getProperty('inference'.toJS)! as JSObject;
  final pending = bridge.callMethod('run'.toJS, input.toJS)! as JSPromise;
  final output = await pending.toDart as JSFloat32Array;
  return output.toDart;
}

The inference global lives in a plain script under web/ that loads the runtime and the model once, so session creation happens a single time instead of per call.

Keep inference off the main thread

A model run on the UI thread drops frames — Flutter’s rendering pipeline is already using the main thread. Move the model into a Web Worker and talk to it with postMessage, copying small results and transferring large buffers:

// web/inference-worker.js
self.onmessage = async ({ data: { id, input } }) => {
  self.model ??= await self.tf.loadGraphModel('/assets/models/model.json');
  const output = self.model.execute(input);
  self.postMessage({ id, data: await output.data() });
};

// main thread
const worker = new Worker('/inference-worker.js');
worker.postMessage({ id: 1, input: tensorData }, [tensorBuffer]);

If you need zero-copy sharing between worker and page, SharedArrayBuffer is available — but only under cross-origin isolation (COOP and COEP headers). Without those headers, plan on structured-clone copies and keep payloads small.

Give the worker a timeout and a recovery path: a model that throws mid-session should not leave the UI thread waiting forever. Recreate the worker on failure and surface a retry affordance to the user rather than freezing the feature.

Budget the model like you budget the bundle

A 30 MB float32 model on a mobile network is a page-load event, not a download. Three levers matter:

# versioned model files: cache them forever
location /assets/models/ {
  add_header Cache-Control "public, max-age=31536000, immutable";
}

Quantize first: float32 to uint8 typically cuts the file 4x with a small accuracy cost. Cache aggressively second — the model should be a repeat-visit non-event. Warm up third: run one dummy inference right after load so the first real request does not pay for shader compilation and graph setup.

Ship it and measure

flutter run -d chrome gives you the debug toolchain for wiring work; flutter build web --release is what your users get, with tree-shaking and minification applied. Either way, load the model after first paint — a multi-MB model fetched during startup shows up directly in LCP and INP, and no amount of interop fixes a slow first frame:

flutter run -d chrome          # wiring and debugging
flutter build web --release    # tree-shaking + minified output

Log the numbers that matter: model download time, time-to-first-inference (the warm-up run), and per-inference latency inside the worker. If inference latency dominates, the fix is a smaller or quantized model, not faster Dart.

Ship a degraded path as well: if the model download fails or times out, fall back to a non-ML feature rather than a spinner that never resolves — the model is an enhancement to the page, not the page itself.

Takeaway: Choose the runtime by model origin, load the model through a thin JS bridge with dart:js_interop, run inference in a Web Worker, and treat quantization plus long-lived caching as part of the feature, not an optimization for later.

Sources

• https://www.tensorflow.org/js
• https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API
• https://onnxruntime.ai/docs/tutorials/web/
• https://docs.flutter.dev/platform-integration/web