Routing Rules

1. Cost control — downgrade simple prompts to a cheaper model.
2. Vendor pinning — force a specific tenant onto a specific provider, e.g. for compliance or data-residency reasons.
3. Resilience — name the models to fall back to when a model is rate limited or down, without changing your code.

​

Anatomy

{
  "name": "downgrade short prompts",
  "description": "Anything under 500 chars goes to gpt-5-nano",
  "priority": 100,
  "condition": {
    "max_tokens": 500
  },
  "action": {
    "provider": "openai",
    "model": "gpt-5-nano"
  }
}

• priority — lower numbers fire first. Use 100, 200, 300, … so you can wedge new rules between old ones without renumbering.
• condition — JSON object describing what to match. See the conditions reference below.
• action — JSON object describing the rewrite. Sets provider, model or fallback_models (at least one), and optionally failover_provider.

​

Conditions reference

​

Actions reference

​

Smart failover

1. Re-authenticates against the failover provider’s stored Provider Key. With no stored key for that provider, there is no retry and the caller sees the primary’s error.
2. Issues exactly one retry. No exponential backoff, no recursion — if the failover also 5xx’s, the caller sees that error.

​

Fallback models

{
  "name": "keep GPT-4o answering",
  "priority": 100,
  "condition": { "model": "gpt-4o" },
  "action": { "fallback_models": ["gpt-4o-mini", "claude-3-5-haiku-latest"] }
}

• A chain in the request replaces the rule’s. A request that sends its own fallback_models field or X-PromptGuard-Fallback-Models header uses that chain and never the rule’s. The rest of the rule (its provider and model) still applies.
• The chain follows the rule’s rewrite. When the rule also sets model, the fallbacks are for that model.
• With failover_provider too, a non-streamed request’s model gets its retry on the failover provider first, and the fallback models are tried after it.
• A model on a vendor you have not consented to — no stored provider key for it — is skipped for every request the rule matches, and the request goes ahead with its own model and the rule’s other fallback models. So is a model on another vendor on /api/v1/messages or /api/v1/responses, whose bodies only one vendor can read. A rule’s chain never stops a request that would otherwise have been answered. Saving the rule warns you about each model on a vendor with no stored key, and saves it anyway.
• Saving the rule refuses a chain of more than 3 models, one that names a model twice, or a model no provider serves.

​

Match counters

-- Quick health check from the SQL editor
SELECT name, priority, match_count, last_matched_at
FROM routing_rules
WHERE user_id = $1
ORDER BY match_count DESC;

​

Worked example

[
  {
    "name": "internal team always Sonnet",
    "priority": 50,
    "condition": { "header_match": { "x-tenant": "internal" } },
    "action": { "provider": "anthropic", "model": "claude-sonnet-4-5" }
  },
  {
    "name": "downgrade short prompts",
    "priority": 100,
    "condition": { "max_tokens": 500 },
    "action": { "provider": "openai", "model": "gpt-5-nano" }
  },
  {
    "name": "GPT keeps answering",
    "priority": 1000,
    "condition": { "model": "gpt-*" },
    "action": {
      "provider": "openai",
      "fallback_models": ["claude-3-5-haiku-latest"]
    }
  }
]