Orchestrix

Orchestrix is a fine-tuned FunctionGemma 270M model designed for on-device UI orchestration using function calling.
It converts natural language commands into deterministic UI actions, running fully offline via MediaPipe.

Model Description

This is a fine-tuned version of FunctionGemma 270M Instruct specifically trained for natural language-based UI control through function calling. The model interprets user commands and generates structured function calls to manipulate application UI elements.

Base Model: google/functiongemma-270m-it
Fine-tuning Task: Function calling for UI control
Format: MediaPipe compatible formats (.task, .tflite, .litert)
Deployment: On-device inference via Flutter

Intended Use

Primary Use Cases

Mobile/Desktop Applications: Natural language UI control in Flutter apps
On-Device AI: Privacy-focused, offline AI interactions
Function Calling: Demonstrating LLM tool use capabilities
Accessibility: Voice-controlled UI modifications

Supported Functions

The model is trained to call the following 6 functions:

Function	Description	Parameters
`change_theme`	Switch app theme	`theme`: "light", "dark", "system", "blue", "green", "purple", "orange", "red", "material", "cupertino", "custom"
`change_background_color`	Modify background color	`color`: "red", "green", "blue", "yellow", "purple", "orange", "white", "black", "pink", "cyan", "amber", "teal", "indigo", "lime", "brown", "grey"
`change_text_color`	Change text color	`color`: "red", "green", "blue", "yellow", "purple", "orange", "white", "black", "pink", "cyan", "amber", "teal", "indigo", "lime", "brown", "grey"
`change_text_size`	Adjust text size	`size`: "small", "medium", "large", "extra large", "tiny", "12", "14", "16", "18", "20", "22", "24", "28", "32", "36", "40", "48"
`change_app_title`	Change application title	`title`: Any app name (200+ variations including "Dashboard", "Settings", "Task Manager", etc.)
`show_alert`	Display notification/alert	`title`: Alert title, `message`: Alert message

Training Details

Training Data

The model was fine-tuned on a custom dataset of 3,300 examples mapping natural language commands to function calls. The dataset was synthetically generated with diverse phrasing patterns to ensure robust understanding.

Training data statistics:

Total examples: 3,300
Functions: 6 (change_theme, change_background_color, change_text_color, change_text_size, change_app_title, show_alert)
Pattern variations: 10-12 unique phrasings per function
Color options: 16 colors
Theme options: 11 themes
Text size options: 17 size variations
App titles: 200+ predefined titles
Alert variations: 100+ titles and 150+ messages

Training data format (JSONL):

{
  "user_content": "make the background purple",
  "tool_name": "change_background_color",
  "tool_arguments": "{\"color\": \"purple\"}"
}

Training Procedure

Base Model: FunctionGemma 270M Instruct
Fine-tuning Method: Supervised fine-tuning on function calling examples
Training Examples: 3,300 user command → function call pairs
Data Generation: Synthetic data with template-based variation
Optimization: Focused on accurate function name and parameter extraction

Usage

Flutter Integration

import 'package:flutter_gemma/flutter_gemma.dart';

// Initialize Flutter Gemma
await FlutterGemma.initialize();

// Create model instance
final model = await FlutterGemma.getActiveModel(
  maxTokens: 1024,
  preferredBackend: PreferredBackend.cpu,
);

// Define tools (must match training data)
final tools = [
  Tool(
    name: 'change_theme',
    description: 'Change application theme',
    parameters: {
      'type': 'object',
      'properties': {
        'theme': {
          'type': 'string',
          'enum': ['light', 'dark', 'system', 'blue', 'green', 'purple', 
                   'orange', 'red', 'material', 'cupertino', 'custom'],
        },
      },
      'required': ['theme'],
    },
  ),
  Tool(
    name: 'change_background_color',
    description: 'Change the background color of the app',
    parameters: {
      'type': 'object',
      'properties': {
        'color': {
          'type': 'string',
          'enum': ['red', 'green', 'blue', 'yellow', 'purple', 'orange', 
                   'white', 'black', 'pink', 'cyan', 'amber', 'teal', 
                   'indigo', 'lime', 'brown', 'grey'],
        },
      },
      'required': ['color'],
    },
  ),
  Tool(
    name: 'change_text_color',
    description: 'Change the text color in the app',
    parameters: {
      'type': 'object',
      'properties': {
        'color': {
          'type': 'string',
          'enum': ['red', 'green', 'blue', 'yellow', 'purple', 'orange', 
                   'white', 'black', 'pink', 'cyan', 'amber', 'teal', 
                   'indigo', 'lime', 'brown', 'grey'],
        },
      },
      'required': ['color'],
    },
  ),
  Tool(
    name: 'change_text_size',
    description: 'Adjust the text size',
    parameters: {
      'type': 'object',
      'properties': {
        'size': {
          'type': 'string',
          'enum': ['small', 'medium', 'large', 'extra large', 'tiny', 
                   '12', '14', '16', '18', '20', '22', '24', '28', 
                   '32', '36', '40', '48'],
        },
      },
      'required': ['size'],
    },
  ),
  Tool(
    name: 'change_app_title',
    description: 'Change the application title',
    parameters: {
      'type': 'object',
      'properties': {
        'title': {
          'type': 'string',
          'description': 'The new title for the application',
        },
      },
      'required': ['title'],
    },
  ),
  Tool(
    name: 'show_alert',
    description: 'Display an alert or notification',
    parameters: {
      'type': 'object',
      'properties': {
        'title': {
          'type': 'string',
          'description': 'The alert title',
        },
        'message': {
          'type': 'string',
          'description': 'The alert message',
        },
      },
      'required': ['title', 'message'],
    },
  ),
];

// Create chat session
final chat = await model.createChat(
  tools: tools,
  supportsFunctionCalls: true,
);

// Send user message
await chat.sendMessage('Switch to dark mode');

Example Interactions

Example 1: Theme Change

User: "Switch to dark mode"
Model Output: {
  "tool_calls": [{
    "name": "change_theme",
    "arguments": {"theme": "dark"}
  }]
}

Example 2: Background Color

User: "make it purple"
Model Output: {
  "tool_calls": [{
    "name": "change_background_color",
    "arguments": {"color": "purple"}
  }]
}

Example 3: Text Styling

User: "make text large and cyan"
Model Output: {
  "tool_calls": [
    {
      "name": "change_text_size",
      "arguments": {"size": "large"}
    },
    {
      "name": "change_text_color",
      "arguments": {"color": "cyan"}
    }
  ]
}

Example 4: App Title

User: "rename app to Task Manager"
Model Output: {
  "tool_calls": [{
    "name": "change_app_title",
    "arguments": {"title": "Task Manager"}
  }]
}

Example 5: Alert Notification

User: "show alert Success with message File uploaded successfully"
Model Output: {
  "tool_calls": [{
    "name": "show_alert",
    "arguments": {
      "title": "Success",
      "message": "File uploaded successfully"
    }
  }]
}

Limitations

Limited Function Set: Only supports 6 predefined functions
Parameter Constraints: Colors and themes are restricted to predefined enums
Language: English only
Context: Optimized for UI control tasks, not general conversation
Ambiguity: May struggle with highly ambiguous commands not seen during training
Synthetic Training: Trained on template-generated data, may differ from real user phrasing

Bias and Ethical Considerations

The model is trained on synthetic data and may not represent diverse user populations
Function calling is deterministic and limited to safe UI operations
No personal data is collected or transmitted (fully on-device)
Color names follow Western conventions and may not align with all cultural interpretations

Model Card Authors

Vijay K

Model Card Contact

For questions or issues, please contact: [email protected]

License

This model is released under the Gemma License.

Acknowledgments

Google DeepMind for the Gemma base model
MediaPipe for on-device inference capabilities
Flutter Gemma package maintainers

Repository

Training scripts and dataset generation code available in the project repository:

generate_data.py: Synthetic dataset generation script
training_data.jsonl: 3,300 training examples
train.jsonl: Converted training data for fine-tuning

Downloads last month: 7

Model tree for vijayk-huggingface/orchestrix-actions

Base model

google/functiongemma-270m-it

Finetuned

(205)

this model