Skip to main content
The proxy accepts stream: true and answers with Server-Sent Events, but it does not pass tokens through as the provider generates them. It reads the provider’s whole stream, scans the completed text, and only then sends it to you. Time to first token is therefore the time to the full completion, plus the scan.

How Streaming Works

  1. Your request is sent to PromptGuard
  2. PromptGuard scans the input. A blocked input gets an HTTP 403 before anything is forwarded, exactly as without streaming
  3. If the input passes, the request is forwarded to the LLM provider with stream: true
  4. PromptGuard reads the provider’s stream to the end and holds it
  5. In enforce mode, the completed text is scanned as output
  6. If the output passes, the held SSE events are sent to you in one burst. If it is blocked, you receive a single error event and [DONE], and none of the completion
In monitor mode step 5 is skipped, but the stream is still held until the provider finishes. Why: a stream that is forwarded as it arrives cannot be recalled. If the violation is in token 400, tokens 1 to 399 have already reached your user. Holding the stream is what lets a blocked response deliver nothing.

Using the OpenAI SDK

Your existing streaming code works unchanged: point the client at the proxy and add the X-API-Key header. Your provider key stays the client’s API key.

Using the PromptGuard SDK

The Python PromptGuard client streams: stream=True returns an iterator of parsed SSE chunks as dicts. It sends only your PromptGuard key, so the request reaches the provider only if you have stored a provider key for it.
The Node.js PromptGuard client does not stream: stream: true throws a PromptGuardError with code streaming_not_supported. In Node, stream with the OpenAI SDK pointed at the proxy, as above.

Server-Sent Events (SSE)

When streaming, the API returns Server-Sent Events. Each event contains a JSON chunk:

Framework Integration

FastAPI (Python)

Express (Node.js)

Next.js (React)

Error Handling During Streaming

A streamed request can be blocked at two points, and each arrives differently: The OpenAI SDKs raise both as APIError, so one handler covers them:
Because the output is scanned before anything is sent, a blocked response never puts partial text on screen: the error is the first and only event.

Streaming with auto-instrumentation

With auto-instrumentation (init()), your client talks to the provider directly and only the input is scanned for a streamed call. scan_responses / scanResponses does not apply to streams: the Node.js SDK skips the output scan for streaming calls on purpose, and the Python SDK finds no completion text on a stream object to scan. To scan streamed output in your own code, collect the text as it arrives and scan it with the Guard API once the stream ends:
If you show tokens to the user while collecting them, they have seen the text before the verdict arrives. Hold the text until the scan returns, or route the call through the proxy, which holds it for you.

Performance

Streaming through the proxy therefore does not lower perceived latency the way it does against the provider directly: your user waits for the whole answer either way. Keep stream: true if your client code already handles SSE, since nothing else changes, but do not choose it for a faster first token. The scans’ own cost depends on how often your traffic escalates past the deterministic layer; measure it with the latency budgets guide.

Next steps

Streaming protection

Three ways to protect a streamed response

Latency budgets

Per-detector budgets and how to keep the guard call fast

Error handling

Every error code the API returns and how to handle it
Last modified on October 31, 2026