Core Concepts¶
This section explains the fundamental concepts behind the Oblix SDK and how it works to provide intelligent AI model orchestration between edge and cloud environments.
What is Oblix Orchestration?¶
Oblix orchestration is the process of intelligently managing and coordinating multiple AI models between edge (local) and cloud environments to optimize for factors such as:
- Performance and latency
- Resource utilization
- Cost efficiency
- Reliability
- Connectivity status
Rather than manually deciding which model to use in different situations, Oblix automatically orchestrates this decision-making process based on real-time system metrics and connectivity data, providing seamless switching between environments.
How Oblix Works¶
At its core, Oblix works by:
- Providing a unified interface for multiple model providers (OpenAI, Claude, Ollama)
- Monitoring system metrics through specialized agents
- Evaluating orchestration policies based on collected metrics
- Intelligently orchestrating executions between edge and cloud models
- Managing session context for conversational interactions
Key Components¶
OblixClient¶
The main entry point for the SDK provides methods for:
- Hooking models from different providers
- Adding monitoring agents for orchestration
- Executing prompts with seamless switching
- Managing chat sessions
Models¶
Oblix supports multiple model types:
- OLLAMA: Edge models running locally via Ollama
- OPENAI: Cloud models from OpenAI (GPT series)
- CLAUDE: Cloud models from Anthropic (Claude series)
- CUSTOM: Custom model implementations
Agents¶
Specialized components that monitor system state and enable intelligent orchestration:
- ResourceMonitor: Tracks CPU, memory, and GPU usage for edge execution decisions
- ConnectivityAgent: Monitors network connectivity quality for cloud execution decisions
Orchestration Flow¶
When you execute a prompt with Oblix:
- All registered agents perform checks on system resources and connectivity
- Orchestration policies evaluate the collected metrics
- The optimal model is selected based on the current state
- The prompt is sent to the selected model
- The response is returned along with execution metadata
Edge-Cloud Execution Strategy¶
Oblix implements an intelligent edge-cloud execution strategy that seamlessly switches between:
- Edge execution for better privacy, lower latency, and offline capability
- Cloud execution for more powerful models and when local resources are constrained
Benefits¶
This orchestration approach provides several key benefits:
- Resilience: Continue operating even when connectivity is limited or lost
- Optimization: Use the most cost-effective model for each task
- Performance: Select models based on current system capabilities
- Simplicity: Provide a single interface for all model interactions
- Adaptability: Automatically adjust to changing conditions
In the following sections, we'll dive deeper into each of these core concepts:
- Orchestration: How the orchestration system works
- Agents: Monitoring system state for orchestration decisions
- Session Management: Handling conversations
Understanding these concepts will help you get the most out of the Oblix SDK in your applications.