Providers - switchAILocal

Overview

Providers are the backbone of switchAILocal, enabling access to various AI services through a unified interface. Each provider type has unique characteristics and authentication requirements.

Provider Types

switchAILocal supports three distinct categories of providers:

CLI Providers

Local command-line tools running on your machine

Cloud Providers

Remote API services accessed via HTTP

OAuth Providers

Services requiring OAuth2 authentication flows

CLI Providers

CLI providers execute locally-installed AI tools and expose them through the proxy.

Ollama

Local model server for running open-source LLMs.

ollama:
  enabled: true
  base-url: "http://localhost:11434"
  auto-discover: true  # Automatically fetch available models

Implementation Details

The OllamaExecutor (internal/runtime/executor/ollama_executor.go) translates OpenAI format to Ollama’s native API:

type OllamaExecutor struct {
    cfg     *config.Config
    baseURL string
    client  *http.Client
}

Key features:

Auto-discovery: Queries /api/tags for available models
Vision support: Handles base64-encoded images
Streaming: Real-time response chunks
No authentication: Direct HTTP access

Example models:

ollama:llama3.2
ollama:mistral
ollama:qwen3-vl:235b-instruct-cloud

OpenCode

Local AI coding agent for development tasks.

opencode:
  enabled: true
  base-url: "http://localhost:4096"
  default-agent: "build"

OpenCode agents are specialized for different tasks: build, debug, test, refactor.

LM Studio

GUI application for running local models with OpenAI-compatible API.

lmstudio:
  enabled: true
  base-url: "http://localhost:1234/v1"
  auto-discover: true

Gemini CLI

Access Google’s Gemini via OAuth credentials without API keys.

# Use OAuth login instead of API keys
# Run: switchAILocal -login

Authentication Flow

The GeminiCLIExecutor uses OAuth2 with device flow:

type GeminiCLIExecutor struct {
    cfg *config.Config
}

func prepareGeminiCLITokenSource(ctx, cfg, auth) (TokenSource, error) {
    // Exchange refresh token for access token
    // Endpoint: https://cloudcode-pa.googleapis.com
}

Scopes required:

https://www.googleapis.com/auth/cloud-platform
https://www.googleapis.com/auth/userinfo.email

Cloud Providers

Cloud providers access remote APIs using API keys or OAuth tokens.

Gemini API

Google’s Gemini models via REST API.

gemini-api-key:
  - api-key: "AIzaSy..."
    prefix: "google"
    base-url: "https://generativelanguage.googleapis.com"

Supported endpoints:

/v1beta/models/{model}:generateContent
/v1beta/models/{model}:streamGenerateContent
/v1beta/models/{model}:countTokens

Use the prefix to disambiguate when you have multiple Gemini providers configured. Access as google/gemini-2.0-flash.

Claude (Anthropic)

Anthropic’s Claude models.

claude-api-key:
  - api-key: "sk-ant-..."
    models:
      - name: "claude-3-5-sonnet-20241022"
        alias: "sonnet"
      - name: "claude-3-opus-20240229"
        alias: "opus"

Message Format Translation

The ClaudeExecutor translates between OpenAI and Claude formats:

// OpenAI format
{
  "model": "gpt-4",
  "messages": [{"role": "user", "content": "Hello"}]
}

// Claude format (translated)
{
  "model": "claude-3-5-sonnet-20241022",
  "messages": [{"role": "user", "content": "Hello"}],
  "max_tokens": 4096
}

Note: Claude requires explicit max_tokens parameter.

OpenAI / Codex

OpenAI’s GPT models.

codex-api-key:
  - api-key: "sk-..."
    base-url: "https://api.openai.com/v1"

Available models:

GPT-4 series: gpt-4, gpt-4-turbo, gpt-4o
GPT-3.5 series: gpt-3.5-turbo
O-series: o1, o1-mini, o1-preview

SwitchAI Cloud

Unified access to 100+ cloud models through single API key.

switchai-api-key:
  - api-key: "sk-lf-..."
    base-url: "https://switchai.traylinx.com/v1"
    models:
      - name: "openai/gpt-oss-120b"
        alias: "switchai-fast"
      - name: "deepseek-reasoner"
        alias: "switchai-reasoner"

SwitchAI provides access to models from OpenAI, Anthropic, Google, DeepSeek, and more through a single endpoint.

OpenAI-Compatible Providers

Many providers offer OpenAI-compatible endpoints that can be configured using openai-compatibility.

openai-compatibility:
  - name: "groq"
    prefix: "groq"
    base-url: "https://api.groq.com/openai/v1"
    api-key-entries:
      - api-key: "gsk_..."
  
  - name: "openrouter"
    prefix: "or"
    base-url: "https://openrouter.ai/api/v1"
    api-key-entries:
      - api-key: "sk-or-v1-..."
  
  - name: "together"
    prefix: "together"
    base-url: "https://api.together.xyz/v1"
    api-key-entries:
      - api-key: "..."

OpenAICompatExecutor Implementation

The OpenAICompatExecutor is a generic executor for OpenAI-compatible APIs:

type OpenAICompatExecutor struct {
    provider string  // e.g., "groq", "openrouter"
    cfg      *config.Config
}

func (e *OpenAICompatExecutor) Identifier() string {
    return e.provider
}

Supports:

Chat completions: /chat/completions
Image generation: /images/generations
Audio transcription: /audio/transcriptions
Audio speech: /audio/speech

Supported Services

Groq

Ultra-fast inference with LPU technologyModels: Llama, Mixtral, Gemma

OpenRouter

Access to 100+ models with unified billingModels: GPT-4, Claude, Gemini, and more

Together AI

Open-source models with fast inferenceModels: Llama, Mistral, Qwen

Anyscale

Serverless endpoints for OSS modelsModels: Llama, Mixtral, CodeLlama

Provider Lifecycle

Registration

Providers are registered during service initialization:

// Register executors in service builder
func (b *ServiceBuilder) Build() (*Service, error) {
    // Register Gemini CLI
    geminiCLIExec := executor.NewGeminiCLIExecutor(b.cfg)
    b.coreManager.RegisterExecutor(geminiCLIExec)
    
    // Register Ollama
    ollamaExec := executor.NewOllamaExecutor(b.cfg)
    b.coreManager.RegisterExecutor(ollamaExec)
    
    // Register OpenAI compat providers
    for _, compatCfg := range b.cfg.OpenAICompatibility {
        exec := executor.NewOpenAICompatExecutor(compatCfg.Name, b.cfg)
        b.coreManager.RegisterExecutor(exec)
    }
}

Discovery

Providers with auto-discover: true fetch available models:

// Ollama auto-discovery
func (e *OllamaExecutor) DiscoverModels() ([]string, error) {
    resp, err := e.client.Get(e.baseURL + "/api/tags")
    // Parse and return model list
}

Health Checking

Providers can implement health checks:

type ProviderHealthChecker interface {
    CheckHealth(ctx context.Context) error
}

Model Naming Conventions

Providers use prefixes to avoid naming conflicts:

# Without prefix (uses first matching provider)
model: gpt-4o

# With provider prefix
model: openrouter/gpt-4o
model: groq/llama-3.2-90b
model: ollama:mistral

# With custom alias
model: sonnet  # Maps to claude-3-5-sonnet-20241022

Use force-model-prefix: true in config to require explicit prefixes for all requests.

Provider Metadata

Each provider auth includes metadata:

type Auth struct {
    Provider   string
    Metadata   map[string]any
    Attributes map[string]string
}

// Example metadata
metadata: {
    "source": "config_yaml",
    "base_url": "https://api.groq.com/openai/v1",
    "prefix": "groq",
    "auto_discover": true
}

Proxy Configuration

Providers can use HTTP proxies:

# Global proxy for all providers
proxy-url: "socks5://user:pass@192.168.1.1:1080/"

# Per-provider proxy
gemini-api-key:
  - api-key: "AIzaSy..."
    proxy-url: "http://proxy.example.com:8080"

Supported proxy protocols: http://, https://, socks5://

Error Handling

Providers implement standardized error responses:

type Error struct {
    Code       string  // "auth_failed", "quota_exceeded", etc.
    Message    string
    HTTPStatus int     // 401, 429, 500, etc.
    Retryable  bool
}

Common error codes:

401: Invalid API key or expired token
402/403: Payment required or forbidden
404: Model not found
429: Rate limit exceeded (triggers cooldown)
500/502/503/504: Transient server errors

Provider Quotas

Quota management prevents excessive retry storms:

type QuotaState struct {
    Exceeded      bool
    Reason        string
    NextRecoverAt time.Time
    BackoffLevel  int  // Exponential backoff level
}

Backoff schedule:

Level 0: 1 second
Level 1: 2 seconds
Level 2: 4 seconds
…
Max: 30 minutes

Quota Recovery

When a provider returns 429 Too Many Requests:

Mark the credential as quota-exceeded
Set NextRecoverAt based on Retry-After header or exponential backoff
Skip this credential during selection until recovery time
Reset quota state on successful request

case 429:
    var next time.Time
    if result.RetryAfter != nil {
        next = now.Add(*result.RetryAfter)
    } else {
        cooldown, nextLevel := nextQuotaCooldown(backoffLevel)
        next = now.Add(cooldown)
        backoffLevel = nextLevel
    }
    state.Quota = QuotaState{
        Exceeded:      true,
        NextRecoverAt: next,
        BackoffLevel:  backoffLevel,
    }

Next Steps

Add Provider

Configure a new provider in your setup

Routing Strategies

Learn how requests are routed to providers

Authentication

Understand authentication flows

Model Discovery

Automatic model discovery and registration

Get Started

Core Concepts

Configuration

Intelligent Systems

Advanced Features

Guides

​Overview

​Provider Types

CLI Providers

Cloud Providers

OAuth Providers

​CLI Providers

​Ollama

​OpenCode

​LM Studio

​Gemini CLI

​Cloud Providers

​Gemini API

​Claude (Anthropic)

​OpenAI / Codex

​SwitchAI Cloud

​OpenAI-Compatible Providers

​Supported Services

Groq

OpenRouter

Together AI

Anyscale

​Provider Lifecycle

​Registration

​Discovery

​Health Checking

​Model Naming Conventions

​Provider Metadata

​Proxy Configuration

​Error Handling

​Provider Quotas

​Next Steps

Add Provider

Routing Strategies

Authentication

Model Discovery

Overview

Provider Types

CLI Providers

Ollama

OpenCode

LM Studio

Gemini CLI

Cloud Providers

Gemini API

Claude (Anthropic)

OpenAI / Codex

SwitchAI Cloud

OpenAI-Compatible Providers

Supported Services

Provider Lifecycle

Registration

Discovery

Health Checking

Model Naming Conventions

Provider Metadata

Proxy Configuration

Error Handling

Provider Quotas

Next Steps