Skip to main content
What you prove. On one device, before you roll anything out: that Shadow AI stops a test secret in an AI tool people really use, that you can see it centrally, and that a policy you change in the dashboard reaches the device within a minute. Allow about 30 minutes. The hands-on device steps live in Test it yourself; this page adds the admin side and the pass/fail criteria.

​
Before you start

  • One test device you can install software on, with admin rights for the one-time certificate approval. Supported platforms lists the operating systems and AI tools.
  • A PromptGuard account. Personal Shadow AI is on every plan; rolling out to a team needs a Shadow plan, and Shadow Team and Shadow Business start with a trial (Pricing).
  • For step 4 only: a plan that includes custom policies (Pricing).

​
Step 1: Put the agent on the test device

On a Mac, follow Test it yourself through Step 3 — Turn on protection: install, connect the device to your account, and approve the certificate. On Windows or Linux, follow Get started. The menu-bar shield reads “You’re protected.” To test the path your fleet will use instead, enroll the device with a token from the Shadow AI page in the dashboard (Fleet enrollment).

​
Step 2: Send a test secret through a captured tool

Open claude.ai in a browser on the test device and send the test prompts in Test it yourself (Step 4), including the one with the example AWS key AKIAIOSFODNN7EXAMPLE. That key is AWS’s published documentation example: it matches the credential pattern and opens nothing. Expected on the device: the prompt with the key is blocked and never reaches Claude, the phone number is redacted, and the plain question is allowed. The shield’s Activity list shows each decision with its reason and a masked preview.

​
Step 3: Find the event in the dashboard

  1. Open the Shadow AI page. The test device is listed with a recent check-in.
  2. Open Projects and the project named Shadow AI Fleet (on a personal plan, Personal Shadow AI). Every enrolled device scans against it.
  3. Open its Interactions. The blocked, redacted and allowed prompts from step 2 are listed with their decisions. Open the block to see why it was blocked.
If you set up an alert destination (alert routing), the block arrives there too: a block always raises an alert (which events alert).

​
Step 4: Change a policy and time it

  1. In the Shadow AI Fleet project, open Guardrails → Policies tab → Create policy. Policy Name Block codename (evaluation), Policy Type Input Filter; one rule with Condition Contains Text (exact), value Project Falcon, Action Block. Save, and note the time.
  2. On the test device, send Write the Project Falcon launch announcement. in claude.ai.
  3. Expected: blocked on the first try. A change applies to the next scan; the only delay is a device reusing a verdict for an identical request for up to 60 seconds, which a prompt it has not sent before never hits (When a policy change takes effect).
  4. Delete the policy and send a reworded prompt that still names Project Falcon. It goes through.
Changing a built-in check instead (its strictness, or which kinds of PII it looks for) behaves the same way; Customise what Shadow AI detects lists every lever.

​
Success criteria

​
Before production

  • Rollout by enrollment token, not personal sign-in. Tokens give each device its own scan-only credential you can revoke alone; set a use limit and an expiry (Fleet enrollment). Rolling out for clients as an MSP: MSP guide.
  • Fail mode chosen on purpose. By default a device that cannot reach the engine sends the prompt with only its on-device masking; hold stops it instead (If the engine cannot be reached). Either way, have alerts for devices sending prompts unchecked (Device fail-open alerts).
  • Deployment mode decided. Cloud, hybrid or air-gapped, depending on where prompts may be scanned (Deployment modes).
  • Scan volume understood. Device scans count against your plan’s quota but are never refused for it or rate limited (Pricing).
  • Alerting routed. Blocks reach a person or your SIEM, not just the dashboard (alert routing).
  • Retention and privacy reviewed with whoever signs off on employee monitoring: what leaves the device, what is stored and for how long (Privacy and data handling).
  • Admin access secured. SSO and Directory Sync where your plan includes them, least-privilege roles, and policy editing limited to admins (security-admin path).
  • Evaluation leftovers removed. The codename policy is deleted, and the test device is revoked on the Shadow AI page or uninstalled.

​
Next steps

Security-admin path

Policies, SSO, roles, alerts, SIEM and audit, all from the dashboard.

Evaluate as a developer

The same kind of proof for an application calling the API.
Last modified on October 31, 2026