Orchestration
Agent Orchestration is at the core of the product and it uses multiple small models (SLMs) to break down a user request to produce comparable results. This includes handling skills, processing complex prompts, and making external tool calls
Model selection is done based on capabilities. The main orchestration model, slp-orchestra-mini, handles the routing based on tasks.
You can select/deselect a model from the settings page of the desktop app or using the /model command in the terminal user interface (TUI). The orchestrator selects models based on the workload and capability for a given task
How it works
For example, if you have selected gemma3-4B and qwen2-vl-2B from the list:
When you upload an OCR document or a document with images that contain information, it will select a vision-capable model to process the document. Similarly, if you have a complex skill with multiple steps, it will use a similarly capable model to process and break down the skill into steps and produce the result
Memory
The orchestration process is heavily memory-driven on Apple Silicon-based Macs where GPU memory is shared to take full advantage of all the models selected. For example, if you have enabled both models from the settings and your orchestration is a tiny 1B fine-tuned model, the total memory pressure would be 2 + 4 + 1 + 1 (buffer/KV) = 8GB.
We have tested orchestration using NVIDIA RTX 4060 GPU with 8GB memory and it worked comfortably. If your task scope is small and you don't need a larger model, you can turn models on/off based on your needs. Otherwise, the orchestrator will fall back to the default model automatically, which will reduce response quality but works for most regular use cases unless you are processing documents with diagrams, complex math, and images or skills with multiple steps.
Available Models
| Model | Capabilities | Use Case |
|---|---|---|
| Gemma3-4B | SKILLS, DP | Multi-modal model for processing skills, making tool calls, and handling a wide range of tasks |
| Qwen2-VL-2B | VISION | Vision-language model for OCR documents, visual understanding, video comprehension, and image generation from text |
Next Steps
We generally use q4_k_m quantization to run models efficiently without much loss on resource-constrained devices.
We will include new models based on use-case and capabilities. If you want us to list a model, reach out to us at hello@smartloop.ai.