Basic Usage¶
This page provides basic examples to help you get started with Oblix. These examples cover installation, initialization, model hooking, and executing prompts with the SDK's orchestration capabilities.
Installation¶
Oblix is available exclusively for macOS at this time:
Note: Currently, only macOS is supported. Windows and Linux versions are planned for future releases. Oblix supports Python versions 3.9, 3.10, 3.11, 3.12, and 3.13.
Prerequisites¶
Before using Oblix with edge models, you'll need to:
- Install Ollama on your machine
- Download at least one model using Ollama (e.g.,
ollama pull llama2
)
Initialization¶
Initialize the Oblix client with your API key:
import asyncio
import os
from dotenv import load_dotenv
from oblix import OblixClient
async def main():
# Load environment variables from .env file
load_dotenv()
# Initialize with API key from environment
oblix_api_key = os.getenv('OBLIX_API_KEY')
client = OblixClient(oblix_api_key=oblix_api_key)
# Rest of your code here...
# Clean up resources when done
await client.shutdown()
# Run the async function
if __name__ == "__main__":
asyncio.run(main())
Hooking Models for Orchestration¶
Add both edge and cloud models to enable orchestration:
import asyncio
import os
from dotenv import load_dotenv
from oblix import OblixClient, ModelType
async def main():
# Load environment variables from .env file
load_dotenv()
oblix_api_key = os.getenv('OBLIX_API_KEY')
client = OblixClient(oblix_api_key=oblix_api_key)
# Hook an edge model via Ollama (replace with any model you've pulled)
await client.hook_model(
model_type=ModelType.OLLAMA,
model_name="llama2", # Use any available Ollama model
endpoint="http://localhost:11434" # Default Ollama endpoint
)
# Hook a cloud model via OpenAI
openai_api_key = os.getenv('OPENAI_API_KEY')
await client.hook_model(
model_type=ModelType.OPENAI,
model_name="gpt-3.5-turbo",
api_key=openai_api_key
)
# Optionally hook another cloud model via Claude
anthropic_api_key = os.getenv('ANTHROPIC_API_KEY')
await client.hook_model(
model_type=ModelType.CLAUDE,
model_name="claude-3-opus-20240229",
api_key=anthropic_api_key
)
# Clean up resources when done
await client.shutdown()
if __name__ == "__main__":
asyncio.run(main())
Adding Monitoring Agents for Orchestration¶
Add monitoring agents to enable intelligent orchestration between edge and cloud:
import asyncio
import os
from dotenv import load_dotenv
from oblix import OblixClient, ModelType
from oblix.agents import ResourceMonitor, ConnectivityAgent
async def main():
# Load environment variables from .env file
load_dotenv()
oblix_api_key = os.getenv('OBLIX_API_KEY')
client = OblixClient(oblix_api_key=oblix_api_key)
# Hook edge and cloud models for orchestration
await client.hook_model(
ModelType.OLLAMA,
"llama2",
endpoint="http://localhost:11434"
)
openai_api_key = os.getenv('OPENAI_API_KEY')
await client.hook_model(
ModelType.OPENAI,
"gpt-3.5-turbo",
api_key=openai_api_key
)
# Add resource monitoring for edge execution decisions
resource_monitor = ResourceMonitor(name="resource_monitor")
client.hook_agent(resource_monitor)
# Add connectivity monitoring for cloud execution decisions
connectivity_monitor = ConnectivityAgent(name="connectivity_monitor")
client.hook_agent(connectivity_monitor)
# Clean up resources when done
await client.shutdown()
if __name__ == "__main__":
asyncio.run(main())
Executing Prompts with Orchestration¶
Execute prompts with automatic orchestration between edge and cloud:
import asyncio
from oblix import OblixClient, ModelType
from oblix.agents.resource_monitor import ResourceMonitor
from oblix.agents.connectivity import ConnectivityAgent
async def main():
client = OblixClient(oblix_api_key="your_oblix_api_key")
# Hook edge and cloud models for orchestration
await client.hook_model(ModelType.OLLAMA, "llama2")
await client.hook_model(
ModelType.OPENAI,
"gpt-3.5-turbo",
api_key="your_openai_api_key"
)
# Add monitoring agents for orchestration
client.hook_agent(ResourceMonitor())
client.hook_agent(ConnectivityAgent())
# Execute a prompt with intelligent orchestration
response = await client.execute("Explain quantum computing in simple terms")
# Print the response
print(response["response"])
# Print which model was used by the orchestration
print(f"Model used by orchestration: {response['model_id']}")
if __name__ == "__main__":
asyncio.run(main())
Edge-Cloud Orchestration¶
Oblix's core strength is in orchestration between edge and cloud models:
# Let Oblix handle orchestration based on system conditions
response = await client.execute(
"Explain quantum computing in simple terms"
)
# Learn which model was selected by the orchestration
print(f"Model used by orchestration: {response['model_id']}")
# You can also override orchestration when needed
response = await client.execute(
"Explain quantum computing in simple terms",
model_id="openai:gpt-3.5-turbo" # Explicitly use cloud model
)
Note: For Oblix orchestration to work properly, ensure you've hooked at least one edge model (Ollama), one cloud model (OpenAI/Claude), and the appropriate monitoring agents. You can still override orchestration when needed using the
model_id
parameter.
Customizing Generation Parameters¶
Customize parameters for text generation:
# Execute with custom parameters
response = await client.execute(
"Write a short story about a robot",
temperature=0.8, # Controls creativity (0.0-1.0)
max_tokens=500 # Limits response length
)
Working with Sessions¶
Create and use sessions for conversational interactions with orchestration:
import asyncio
from oblix import OblixClient, ModelType
async def main():
client = OblixClient(oblix_api_key="your_oblix_api_key")
# Hook models for orchestration
await client.hook_model(ModelType.OLLAMA, "llama2")
await client.hook_model(
ModelType.OPENAI,
"gpt-3.5-turbo",
api_key="your_openai_api_key"
)
# Create a new session
session_id = await client.create_session(title="My Conversation")
# Set as current session
client.current_session_id = session_id
# Send messages in the session with orchestration
response1 = await client.execute("Hello, how are you today?")
print(f"Assistant ({response1['model_id']}): {response1['response']}")
response2 = await client.execute("Tell me about yourself")
print(f"Assistant ({response2['model_id']}): {response2['response']}")
# List all sessions
sessions = client.list_sessions()
print(f"Number of sessions: {len(sessions)}")
if __name__ == "__main__":
asyncio.run(main())
Simplified Chat Interface with Orchestration¶
Use the simplified chat interface for conversational applications:
import asyncio
from oblix import OblixClient, ModelType
async def main():
client = OblixClient(oblix_api_key="your_oblix_api_key")
# Hook models for orchestration
await client.hook_model(ModelType.OLLAMA, "llama2")
await client.hook_model(
ModelType.OPENAI,
"gpt-3.5-turbo",
api_key="your_openai_api_key"
)
# Send a chat message with orchestration (creates a new session)
response = await client.chat_once("Hello, how are you today?")
session_id = response["session_id"]
print(f"Assistant ({response['model_id']}): {response['response']}")
# Continue the conversation in the same session
response = await client.chat_once("What's the weather like?", session_id)
print(f"Assistant ({response['model_id']}): {response['response']}")
if __name__ == "__main__":
asyncio.run(main())
Interactive Terminal Chat with Orchestration¶
Start an interactive chat session in the terminal with orchestration:
import asyncio
import os
from dotenv import load_dotenv
from oblix import OblixClient, ModelType
from oblix.agents import ResourceMonitor, ConnectivityAgent
async def main():
# Load environment variables from .env file
load_dotenv()
oblix_api_key = os.getenv('OBLIX_API_KEY')
client = OblixClient(oblix_api_key=oblix_api_key)
# Hook edge and cloud models for orchestration
await client.hook_model(
ModelType.OLLAMA,
"llama2",
endpoint="http://localhost:11434"
)
openai_api_key = os.getenv('OPENAI_API_KEY')
await client.hook_model(
ModelType.OPENAI,
"gpt-3.5-turbo",
api_key=openai_api_key
)
# Add monitoring agents for orchestration
resource_monitor = ResourceMonitor(name="resource_monitor")
connectivity_monitor = ConnectivityAgent(name="connectivity_monitor")
client.hook_agent(resource_monitor)
client.hook_agent(connectivity_monitor)
# Start interactive streaming chat session with orchestration
print("Starting chat session with edge-cloud orchestration. Type 'exit' to quit.")
print("\nSpecial commands:")
print(" /metrics - Show current system metrics")
print(" /status - Show orchestration status")
print(" /edge - Force next response to use edge model")
print(" /cloud - Force next response to use cloud model")
await client.chat_streaming()
# Clean up resources
await client.shutdown()
if __name__ == "__main__":
try:
asyncio.run(main())
except KeyboardInterrupt:
print("\nProgram interrupted by user. Exiting...")
except Exception as e:
print(f"\nError: {e}")
Error Handling¶
Handle potential errors gracefully:
import asyncio
from oblix import OblixClient, ModelType
async def main():
try:
client = OblixClient(oblix_api_key="your_oblix_api_key")
# Hook edge model for orchestration
await client.hook_model(ModelType.OLLAMA, "llama2")
# Execute a prompt
response = await client.execute("Explain quantum computing")
print(response["response"])
except Exception as e:
print(f"Error: {e}")
finally:
# Clean up resources
if 'client' in locals():
await client.shutdown()
if __name__ == "__main__":
asyncio.run(main())
Complete Example¶
Here's a complete example combining multiple features with orchestration:
import asyncio
from oblix import OblixClient, ModelType
from oblix.agents.resource_monitor import ResourceMonitor
from oblix.agents.connectivity import ConnectivityAgent
async def main():
try:
# Initialize client
client = OblixClient(oblix_api_key="your_oblix_api_key")
# Hook edge and cloud models for orchestration
await client.hook_model(ModelType.OLLAMA, "llama2")
await client.hook_model(
ModelType.OPENAI,
"gpt-3.5-turbo",
api_key="your_openai_api_key"
)
# Add monitoring agents for orchestration
client.hook_agent(ResourceMonitor())
client.hook_agent(ConnectivityAgent())
# Create a session
session_id = await client.create_session(title="AI Assistant Chat")
client.current_session_id = session_id
# Execute prompts in the session with orchestration
print("Asking about quantum computing...")
response1 = await client.execute("Explain quantum computing briefly")
print(f"Response from {response1['model_id']}:")
print(response1["response"])
print("\n" + "-"*50 + "\n")
print("Asking a follow-up question...")
response2 = await client.execute("What are some practical applications?")
print(f"Response from {response2['model_id']}:")
print(response2["response"])
# Print orchestration decision information
print("\nOrchestration decisions:")
for agent_name, check_result in response2["agent_checks"].items():
print(f"- {agent_name}: {check_result.get('state')}, target: {check_result.get('target')}")
except Exception as e:
print(f"Error: {e}")
finally:
# Clean up resources
if 'client' in locals():
await client.shutdown()
if __name__ == "__main__":
asyncio.run(main())
This example demonstrates: - Initializing the client - Hooking multiple models for orchestration - Adding monitoring agents for orchestration decisions - Creating and using a session - Executing multiple prompts with orchestration - Accessing model information and orchestration decisions - Proper error handling and resource cleanup
macOS-Specific Considerations¶
Since Oblix is currently only available for macOS, here are some platform-specific notes:
- Oblix leverages macOS-specific optimizations for resource monitoring
- For Apple Silicon Macs, Oblix can detect and utilize Metal-compatible GPUs
- The built-in agents are tuned for optimal orchestration on macOS
- All examples in this documentation will work on macOS 10.15 (Catalina) or newer
For more advanced examples, see Hybrid Execution and specific use cases in the examples section.
Need help? Join our community on Discord: https://discord.gg/v8qtEVuU