Skip to main content

​
Overview

PromptGuard’s proxy routes your LLM requests through the security pipeline before forwarding to providers. You can configure retry, fallback, timeout, and circuit breaker behavior per project.

​
Configuration Schema

Set these in your project settings via the dashboard or API: 
{
  "retry": {
    "max_attempts": 3,
    "delay_ms": 200,
    "backoff_multiplier": 2.0
  },
  "timeout": {
    "request_timeout_s": 30,
    "connect_timeout_s": 5
  },
  "fallback": {
    "strategy": "failover",
    "providers": ["openai", "anthropic", "groq"]
  },
  "circuit_breaker": {
    "enabled": true,
    "threshold": 5,
    "recovery_timeout_s": 30
  }
}

​
Retry

Control how failed requests are retried before falling back to alternative providers.
ParameterTypeDefaultRangeDescription
max_attemptsinteger31–5Total attempts including the initial request
delay_msinteger2000–5000Base delay between retries in milliseconds
backoff_multiplierfloat2.01.0–10.0Exponential backoff multiplier (delay × multiplier per retry)

​
Timeout

Set time limits for upstream provider connections and responses.
ParameterTypeDefaultRangeDescription
request_timeout_sinteger305–120Total request timeout in seconds
connect_timeout_sinteger51–30TCP connection timeout in seconds

​
Fallback Strategy

Control how PromptGuard selects and fails over between LLM providers.
ParameterTypeDefaultDescription
strategystring"failover"Routing strategy: failover, round_robin, cost, latency
providersstring[][]Ordered list of provider names for fallback

​
Strategies

  • failover — Try the primary provider first, fall back to alternatives on failure
  • round_robin — Distribute requests evenly across healthy providers
  • cost — Route to the cheapest provider that can serve the model
  • latency — Route to the provider with lowest observed latency

​
Circuit Breaker

Prevent cascading failures by temporarily removing unhealthy providers from rotation.
ParameterTypeDefaultRangeDescription
enabledbooleantrue—Enable/disable circuit breaker
thresholdinteger51–50Consecutive failures before opening the circuit
recovery_timeout_sinteger305–600Seconds before retrying an open circuit
When a provider hits the error threshold, it enters an open state and is skipped for recovery_timeout_s seconds. After recovery, a single probe request is sent — if it succeeds, the circuit closes and the provider returns to rotation.

​
Example: High-Availability Setup

{
  "retry": {
    "max_attempts": 3,
    "delay_ms": 500,
    "backoff_multiplier": 2.0
  },
  "timeout": {
    "request_timeout_s": 60,
    "connect_timeout_s": 10
  },
  "fallback": {
    "strategy": "failover",
    "providers": ["openai", "anthropic", "groq"]
  },
  "circuit_breaker": {
    "enabled": true,
    "threshold": 3,
    "recovery_timeout_s": 60
  }
}
This configuration:
  1. Tries OpenAI first, then Anthropic, then Groq on failure
  2. Retries up to 3 times with exponential backoff (500ms → 1s → 2s)
  3. Allows 60s for long completions
  4. Opens the circuit after 3 consecutive failures, waits 60s before probing

​
Example: Cost-Optimized Setup

{
  "retry": {
    "max_attempts": 2,
    "delay_ms": 100,
    "backoff_multiplier": 1.5
  },
  "timeout": {
    "request_timeout_s": 15,
    "connect_timeout_s": 3
  },
  "fallback": {
    "strategy": "cost",
    "providers": ["groq", "anthropic", "openai"]
  },
  "circuit_breaker": {
    "enabled": true,
    "threshold": 5,
    "recovery_timeout_s": 30
  }
}
This routes to the cheapest provider first while maintaining reliability through automatic failover.