LiteLLM Gateway for ArgoBox

ArgoBox can now route supported chat models through the permanent LiteLLM gateway on <internal-host> instead of talking to each hosted provider separately.

What It Does

When LITELLM_API_URL is set, ArgoBox prefers LiteLLM for these model families:

anthropic/*
openai/*
google/* mapped to gemini/*
groq/*
openrouter/*
perplexity/*

It also supports friendly aliases configured in the LiteLLM proxy:

dragon-qwen3-8b
gpt-5.4
gpt-5.4-mini
claude-sonnet-4.6
claude-opus-4.6
gemini-2.5-flash
groq-qwen3-32b
sonar-reasoning-pro
openrouter-claude-sonnet-4.6

Node-local Ollama stays on its hardware. ArgoBox should normally reach through LiteLLM, not through the raw Ollama endpoint.

Why Use It

One OpenAI-compatible endpoint for local and hosted inference
Easy model swaps without touching most Argobox call sites
's RTX 2080 can sit behind the same interface as cloud models
Cleaner local development and experimentation

Environment

Set these for ArgoBox development and runtime services that should use the gateway:

LITELLM_API_URL=http://<tailnet-address>.88:14000/v1
LITELLM_API_KEY=<value from the <internal-host> LiteLLM environment>

For services running on <internal-host> itself, LITELLM_API_URL=http://127.0.0.1:14000/v1 is also valid.

LITELLM_API_KEY is required. The gateway intentionally returns 401 for unauthenticated API calls.

Routing Behavior

With LITELLM_API_URL set:

getOpenAI() from src/lib/voice-engine.ts points to LiteLLM
model IDs like google/gemini-2.5-flash-preview are rewritten to gemini/gemini-2.5-flash-preview
admin chat endpoints can stream through LiteLLM directly

Without LITELLM_API_URL:

ArgoBox falls back to its existing direct-provider routing

Recommended Model Strategy

Default fast model: gemini-2.5-flash
Private/local testing: dragon-qwen3-8b
Cheap fast fallback: groq-qwen3-32b
Premium backup: openrouter-claude-sonnet-4.6

Important Limits

This gateway lives on <internal-host> and is exposed only on localhost plus Tailscale <tailnet-address>.88:14000
OpenAI and Anthropic may still fail if those provider accounts are quota-limited
LiteLLM is for inference routing, not provider billing or management APIs