This cookbook compares the ways to protect a streamed response. They trade time to first token against how much unscanned text your user can see.
Overview
Through the proxy
Point your client at the proxy and every streamed response is held and scanned as a whole before any of it is sent. A blocked response arrives as a single error event, so no partial text reaches the user. The cost is time to first token: it equals the time to the full completion plus the scans. Streaming explains the behaviour and the error shapes.Auto-instrumentation
Withinit(), your client talks to the provider directly. For a streamed call PromptGuard scans the input before the request is sent; the output is not scanned. scan_responses / scanResponses has no effect on streams: the Node.js SDK skips the output scan for streaming calls on purpose, and the Python SDK finds no completion text on a stream object.
Vercel AI SDK
Wrap the model withpromptguard(). With scanResponses: true, streamed text reaches the client only after it has been scanned, a window at a time. A blocked prompt reaches onError and the client gets an empty response. Blocked output ends the response body with an error on ai 6 and 7, without calling onError; on ai 5 it reaches onError, the body simply ends, and finishReason is 'error'. The Node.js SDK guide has the full table.
Window scanning in your code
To show text sooner than the proxy does while still scanning everything you send, collect the stream into windows and scan each one withGuardClient before passing it on. Each window costs one Guard API call, and a window is scanned on its own, so a threat split across two windows can be missed. Larger windows miss less and delay more.
Choosing
- Nothing unscanned may reach the user, and the wait is acceptable: stream through the proxy.
- Nothing unscanned may reach the user, and the first words must come sooner: scan windows in your code, and accept a Guard API call per window and the cross-window gap.
- Only the input needs scanning: auto-instrumentation.
Next Steps
Streaming Reference
How the proxy handles streamed responses
Error Handling
Handle errors gracefully