Bringing Agentic Skills to the Edge with Gemma 4

With Gemma 4, Google DeepMind redefined what “open model” means. Released under the Apache 2.0 license, the Gemma 4 family brings state-of-the-art agentic capabilities to your own hardware: multi-step planning, autonomous action, offline code generation, and audio-visual processing—all without specialized fine-tuning, and with support for over 140 languages.

The big shift isn’t the model sizes. It’s that Gemma 4 lets you go beyond chatbots to build agents that run entirely on-device. Today you can bring these agentic skills to Android via AICore Developer Preview, to mobile, desktop, and edge devices through Google AI Edge, and to Flutter apps through the LiteRT-LM runtime.

In this tutorial, you will learn how to build a truly agentic on-device experience with Gemma 4: how to define skills, how the tool-calling loop works, and how to wire it all together with real code.

Prerequisites

• A recent Flutter SDK or access to the LiteRT-LM runtime (Python/CLI also work)
• An Android device (Gemma 4 E2B/E4B works on mid-range hardware with 4GB+ RAM)
• A model file: gemma-4-E2B-it.litertlm or gemma-4-E4B-it.litertlm from the LiteRT Hugging Face community
• Familiarity with function calling in LLMs

What Makes a Model “Agentic”

An agentic model doesn’t just generate text—it acts. The critical enabler is native function calling, introduced in FunctionGemma and perfected in Gemma 4. The runtime pauses generation, returns a structured tool-call request to your application layer, and resumes once the tool’s output is available.

User: "Plan my day and notify me before lunch."

Gemma 4: {tool: "get_calendar", args: {date: "today"}}
App:     [calls calendar API, returns events]
Gemma 4: {tool: "schedule_notification", args: {time: "11:45", title: "Lunch prep"}}
App:     [schedules notification]
Gemma 4: "Done. You have 3 meetings; I'll remind you before lunch."

Notice what happened: the model planned multiple steps, delegated each to a tool, and only produced its final answer once the workflow completed. That planning + tool-use + action loop is the essence of agentic AI.

What Is a “Skill”?

In the Google AI Edge world, a skill is a unit of capability the agent can invoke. Skills come in different mechanisms:

• Text-only skills: a persona or instruction the model follows directly.
• JavaScript skills: real code that runs in a sandboxed webview and returns a result.
• Native-intent skills: open an OS surface—mail, SMS, calendar, notifications.
• MCP skills: call remote Model Context Protocol tools over Streamable HTTP.

This design matters: the model stays small and general, while skills give it a toolbox that grows with your app. The same skill catalog that powers the Google AI Edge Gallery app can be reused in your own product.

Installing the Model and Runtime

Let’s build an on-device agent in Flutter using flutter_gemma plus the agent skills package:

dependencies:
  flutter_gemma: ^0.4.0
  flutter_gemma_agent: ^0.1.0

Initialize the inference engine and load a function-calling model:

import 'package:flutter_gemma/flutter_gemma.dart';
import 'package:flutter_gemma_agent/flutter_gemma_agent.dart';

const gemma4E4BUrl =
    'https://www.kaggle.com/models/google/gemma-4/liteRt/gemma-4-E4B-it-litertlm/1';

Future<void> main() async {
  WidgetsFlutterBinding.ensureInitialized();

  await FlutterGemma.initialize(
    inferenceEngines: [LiteRtLmEngine()],
  );

  await FlutterGemma
      .installModel(modelType: ModelType.gemma4, fileType: ModelFileType.litertlm)
      .fromNetwork(gemma4E4BUrl)
      .install();

  final model = await FlutterGemma.getActiveModel(maxTokens: 4096);
  // ... build the agent session (below)
}

For a truly agentic experience, prefer the E4B variant: it delivers noticeably stronger reasoning and frontier-level edge performance, ideal for complex multi-step planning.

Loading Skills into the Agent

The agent skills package ships a SkillRegistry that discovers available skills, plus executors for each mechanism. Load the bundled starter skills from assets:

final source = AssetSkillSource();
final registry = SkillRegistry()
  ..addAll(await source.load(), selected: true);

final session = await AgentSession.fromModel(
  model,
  registry: registry,
  executors: [
    TextSkillExecutor(),
    JsSkillExecutor(sourceFor: source.jsSkillSourceFor),
    NativeIntentExecutor(),
    // McpSkillExecutor(...) to also call remote MCP tools.
  ],
);

The starter catalog includes skills that demonstrate every mechanism:

Skill	Mechanism	What it does
calculate-hash	JS	Hash a piece of text
qr-code	JS	Generate a QR code
query-wikipedia	JS	Summarize a Wikipedia topic
interactive-map	JS (webview)	Show a location on an embedded map
send-email	intent	Open the OS mail composer
create-calendar-event	intent	Open the calendar event editor
get-current-time	intent	Report current local time
kitchen-adventure	text-only	A text-adventure persona

Two-Stage Skill Discovery

Naively, you could dump every skill description into the system prompt. That burns tokens and confuses the model. The skills package uses two-stage discovery: the agent first sees only the cheap name + description list, picks the relevant skills, and only then loads the full details of what it needs.

class SkillRegistry {
  String buildDiscoveryString() {
    return _skills.values
        .where((s) => s.selected)
        .map((s) => '- ${s.name}: ${s.description}')
        .join('\n');
  }

  Skill? resolve(String name) => _skills[name];
}

This keeps prompts small even with dozens of installed skills, which directly improves tool-call reliability on small models.

The Agent Loop in Action

Here’s the core loop the AgentSession runs. Understanding it is essential for debugging:

Future<SkillResult?> executeLoop(String userMessage) async {
  final conversation = model.createConversation(
    systemInstruction:
        'You are an on-device agent. Choose a skill for each step and call it via function calling.',
  );

  var lastInput = userMessage;

  for (var step = 0; step < 6; step++) {
    final response = await conversation.sendMessage(lastInput);

    final toolCall = response.toolCalls?.firstOrNull;
    if (toolCall == null) {
      return null; // final answer reached
    }

    final skill = registry.resolve(toolCall.name);
    if (skill == null) {
      lastInput = 'Unknown skill: ${toolCall.name}. Choose from ${registry.buildDiscoveryString()}';
      continue;
    }

    final result = await skillExecutor.execute(skill, toolCall.arguments);
    lastInput = result.toAgentInput(); // feeds tool output back
  }

  return const SkillResult.error('Max steps reached');
}

Three details are worth highlighting:

1. The model never executes code. It requests a skill by name and arguments; your code decides whether to run it. That is your safety boundary.
2. Parse errors are expected. If the model emits malformed JSON, return a corrective message and let it retry rather than crashing.
3. Termination is explicit. The loop ends when the model stops emitting tool calls.

Security: Secrets Never Enter the Prompt

Skills often need secrets—API tokens, service URLs. The critical rule is: secrets are injected into the skill’s runtime, never into the model prompt. The JS executor serves skill assets over a loopback HTTP server (http://127.0.0.1), a W3C “potentially trustworthy” origin, so crypto.subtle and other secure-context Web APIs work. Secrets are passed as the JS secret argument:

<!-- skill/scripts/index.html -->
<script>
  window.ai_edge_gallery_get_result = async (data, secret) => {
    const headers = { Authorization: `Bearer ${secret}` };
    const res = await fetch(data.endpoint, { headers });
    return JSON.stringify({ result: await res.text() });
  };
</script>

Because the secret never appears in the model prompt, a prompt-injection attack can’t exfiltrate it through the model’s output.

Building the Chat UI

The package ships an adaptive AgentChatView so you can go from zero to a working agent UI quickly:

import 'package:flutter/material.dart';
import 'package:flutter_gemma_agent/flutter_gemma_agent.dart';

class AgentScreen extends StatelessWidget {
  final AgentSession session;

  const AgentScreen({super.key, required this.session});

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(title: const Text('On-Device Agent')),
      body: AgentChatView(session: session),
    );
  }
}

If you prefer a custom UI, session.send(...) returns a Stream that emits skill loads, tool calls, inline results, and streamed text—so you can render each event distinctly.

Thinking Mode for Better Multi-Step Tasks

Gemma 4’s thinking mode dedicates a scratchpad to step-by-step reasoning before committing to an action. Enable it for complex planning, but be aware of a hard-won lesson from production apps: thinking mode can crash when combined with vision input on some devices. Detect multimodal content and disable thinking for those requests.

final wantsThinking = !hasImageInput;
final response = await conversation.sendMessage(
  lastInput,
  thinking: wantsThinking,
);

When enabled, stream the reasoning to a collapsible panel so users can inspect the model’s plan without the main answer jumping around.

Putting It All Together

Assemble the complete flow:

Future<void> run() async {
  await FlutterGemma.initialize(inferenceEngines: [LiteRtLmEngine()]);
  await FlutterGemma
      .installModel(modelType: ModelType.gemma4, fileType: ModelFileType.litertlm)
      .fromNetwork(gemma4E4BUrl)
      .install();

  final model = await FlutterGemma.getActiveModel(maxTokens: 4096);
  final source = AssetSkillSource();
  final registry = SkillRegistry()
    ..addAll(await source.load(), selected: true);

  final session = await AgentSession.fromModel(
    model,
    registry: registry,
    executors: [
      TextSkillExecutor(),
      JsSkillExecutor(sourceFor: source.jsSkillSourceFor),
      NativeIntentExecutor(),
    ],
  );

  runApp(MaterialApp(home: AgentScreen(session: session)));
}

Now your app can do things like “summarize this Wikipedia topic,” “show me Paris on a map,” or “send an email to the team”—all decided and executed locally.

Conclusion & Next Steps

You’ve turned a general-purpose on-device model into an agent with a toolbox: skills defined once, discovered in two stages, executed behind a security boundary, with secrets kept out of the prompt. This is the pattern behind the Google AI Edge Gallery’s Agent Skills, and it’s now available to your app.

Next steps:

• Write your own SKILL.md files and register a custom AssetSkillSource.
• Add an McpSkillExecutor to let the agent reach remote tools when online.
• Benchmark E2B vs E4B on your target device to balance speed and reasoning.
• Apply the security rules strictly: only load skills you trust, and keep require-secret keys in memory.

The era of agentic experiences on-device is here. With Gemma 4, your apps can plan, call tools, and act—without ever needing a network connection.

References

• Bring state-of-the-art agentic skills to the edge with Gemma 4
• Gemma 4 on Google AI Edge
• flutter_gemma_agent package
• Google AI Edge Gallery