> ## Documentation Index
> Fetch the complete documentation index at: https://docs.promptguard.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Evaluate as an MSP

> A proof of concept for a managed service provider with pass/fail criteria: two test clients, one device each, a policy and events that stay in their own client, alerts reaching your SIEM tagged by client, and access that stops when a grant ends. Ends with a production-readiness checklist.

> [!NOTE]
>
> **What you prove.** Before you put a real client on PromptGuard: that each
> client is its own tenant, that you can run all of them from your own
> organization, and that ending a client's access really ends it. Allow about
> an hour. Each step names the dashboard screen and the API route behind it.
> The full procedure for real clients is the [MSP guide](https://docs.promptguard.co/platform/msp.md).

## Before you start

* **Your organization already manages a client.** The first client
  relationship is set up with us ([contact sales](mailto:sales@promptguard.co));
  until then, creating a client is refused with `403`.
* **A Plan with fleet management** (a Shadow AI plan, see [Pricing](https://docs.promptguard.co/pricing.md#shadow-ai)).
  Your test clients run on your Plan. For step 3 your Plan must also include
  custom policies ([Pricing](https://docs.promptguard.co/pricing.md#feature-comparison)).
* **Two test devices** (virtual machines will do) you can install software on.
  [Supported platforms](https://docs.promptguard.co/shadow-ai/supported-tools.md) lists the operating
  systems.
* **A place to receive alerts**: your SIEM, or any HTTPS endpoint whose
  requests you can read.
* The **Admin** role in your own organization.

## Step 1: Create two test clients

On **Dashboard → Clients**, click **Add client** twice: `PoC Client A` and
`PoC Client B`. Leave **Client's owner** empty, so nobody at a client is
invited.

Behind it: `POST /dashboard/organizations/clients`, acting as your own
organization ([details](https://docs.promptguard.co/platform/msp.md#1-create-a-client-organization)).

**Pass:** both clients are rows on **Dashboard → Clients**, each with an
**Act as** button.

## Step 2: Enroll one device into each client

1. On Client A's row, click **Act as**. The dashboard switches to Client A
   and opens its **Shadow AI** page.
2. Click **Add a device (enrollment token)**, and enroll the first test
   device with it ([Fleet enrollment](https://docs.promptguard.co/shadow-ai/fleet-enrollment.md#roll-it-out)).
3. Switch back to your organization with the organization selector, click
   **Act as** on Client B, and enroll the second device with a token minted
   there.

Behind it: `POST /dashboard/fleet/enrollment-tokens` and
`GET /dashboard/fleet/devices`, each sent with `X-Organization-Id` naming the
client. A token enrolls a device into the organization it was minted in.

**Pass:** acting as Client A, its **Shadow AI** page lists the first device
only; acting as Client B, the second device only.

## Step 3: Give one client a policy the other does not have

1. Acting as Client A, open **Projects → Shadow AI Fleet → Guardrails →
   Policies** → **Create policy**: **Policy Name** `Block codename (PoC)`,
   **Policy Type** **Input Filter**, one rule with **Condition** **Contains
   Text (exact)**, value `Project Falcon`, **Action** **Block**.
2. Still acting as Client A, open **Settings → Security** and turn on **Only
   admins can edit policies**.
3. On each device, send `Write the Project Falcon launch announcement.` in
   claude.ai.

Behind it: `POST /dashboard/policies` with Client A's project id, and
`PATCH /dashboard/organizations/{client_id}` setting `policies_admin_only`
([Set and lock each client's policy](https://docs.promptguard.co/platform/msp.md#4-set-and-lock-the-policy-for-each-client)).

**Pass:** the prompt is blocked on Client A's device and goes through on
Client B's.

## Step 4: Confirm events stay in their client

1. On each device, send the test prompt with the example AWS key
   `AKIAIOSFODNN7EXAMPLE` from [Test it yourself](https://docs.promptguard.co/shadow-ai/test-drive.md)
   (Step 4). It is AWS's published documentation example and opens nothing.
2. Acting as Client A, open **Projects → Shadow AI Fleet → Interactions**.
   Then do the same acting as Client B.
3. Back in your own organization, open **Dashboard → Clients → Client
   alerts** and filter by each client in turn.

Behind it: the Client Alerts queue is
`GET /dashboard/organizations/clients/alerts`, with `client_id` to narrow it
to one client ([Triage](https://docs.promptguard.co/platform/msp.md#6-triage-in-the-client-alerts-queue)).

**Pass:** acting as a client, you see that client's device's events and none
of the other's. The Client Alerts queue shows both blocks, each under its own
client.

## Step 5: Route alerts to your SIEM, tagged by client

1. Acting as Client A, open **Projects → Shadow AI Fleet → Settings → Alert
   delivery** and add your SIEM or endpoint as a destination. Do the same
   acting as Client B, with the same destination.
2. Send the AWS-key prompt again on each device. A block always raises an
   alert ([which events alert](https://docs.promptguard.co/integrations/siem-export.md#which-events-raise-an-alert)).

Behind it: `PATCH /dashboard/projects/{project_id}/webhook` on each client's
project ([Send every client's alerts to your SIEM](https://docs.promptguard.co/platform/msp.md#5-send-alerts-from-every-client-to-your-siem)).

**Pass:** two deliveries arrive, and their `organization_id` and
`organization_name` fields name Client A and Client B respectively
([the SIEM event](https://docs.promptguard.co/integrations/siem-export.md#the-siem-event)). Key your SIEM's
routing on `organization_id`.

## Step 6: See each client on the billing roster

Open **Dashboard → Clients**.

Behind it: `GET /dashboard/organizations/clients/endpoint-usage`
([Reconcile client billing](https://docs.promptguard.co/platform/msp.md#7-reconcile-client-billing)).

**Pass:** each client's **Live endpoints** is 1. **Counted** and **Billed**
fill in only after a billing period closes, so they may be empty during a
proof of concept.

## Step 7: End one client's access and confirm it stops

1. Acting as Client B, revoke its device on the **Shadow AI** page and revoke
   its enrollment token. Ending a grant does not revoke them, and nobody can
   sign in to Client B afterwards to do it, because nobody there was invited.
2. Back in your own organization, on Client B's row in **Dashboard →
   Clients**, click **End**.
3. Try to act as Client B through the API:

```bash theme={"system"}
curl "https://api.promptguard.co/dashboard/fleet/devices" \
  -H "Authorization: Bearer YOUR_SESSION_TOKEN" \
  -H "X-Organization-Id: CLIENT_B_ORG_ID"
```

Behind it: `DELETE /dashboard/fleet/devices/{device_id}`,
`DELETE /dashboard/fleet/enrollment-tokens/{token_id}`, then
`DELETE /dashboard/organizations/clients/{client_id}/grant`
([End access to a client](https://docs.promptguard.co/platform/msp.md#9-end-access-to-a-client)).

**Pass:** Client B leaves the roster, drops off the Client Alerts queue at its
next refresh, and the request above is refused with `403` and the code
`ORG_ACCESS_DENIED`.

## Success criteria

| # | Check | Pass when |
| - | - | - |
| 1 | Clients are created self-serve | Both test clients appear on **Dashboard → Clients** with **Act as** |
| 2 | Devices land in their own client | Each client's **Shadow AI** page lists only its own device |
| 3 | Policy is per client | The codename prompt is blocked on Client A's device and allowed on Client B's |
| 4 | Events are isolated | Acting as a client shows only that client's events; the Client Alerts queue shows both, each under its client |
| 5 | Alerts reach your SIEM tagged by client | Each delivery's `organization_id` and `organization_name` name the client it came from |
| 6 | Billing is per client | Each client shows 1 live endpoint on the roster |
| 7 | Ending a grant ends access | Client B leaves the roster and the queue, and acting as it returns `403 ORG_ACCESS_DENIED` |

## Before production

* [ ] **Capacity per client understood.** API keys in a client you created
  spend that client's own per-minute budget at your Plan's limit, so one busy
  client cannot use up another's; once a client has an owner of its own, its
  keys spend that owner's budget. Device scans are never rate limited
  ([Rate limits](https://docs.promptguard.co/production/rate-limits.md#rate-limiting)). On a contract that
  pools scans across clients, watch `GET /dashboard/organizations/clients/scan-allowance`
  ([Reconcile client billing](https://docs.promptguard.co/platform/msp.md#7-reconcile-client-billing)).
* [ ] **A policy baseline per client.** Set each client's policy and turn on
  **Only admins can edit policies** in each. There is no copy between clients,
  so keep your baseline in version control and apply it per client through
  the API ([Copying a policy to another client](https://docs.promptguard.co/platform/msp.md#copying-a-policy-to-another-client)).
* [ ] **Rollout tokens per client, named.** One rollout token per client,
  minted while acting as it and named after it, given only to that client's
  MDM and revoked when the rollout is done
  ([Roll out Shadow AI](https://docs.promptguard.co/platform/msp.md#3-roll-out-shadow-ai-one-fleet-per-client)).
* [ ] **Alert routing in place for every client.** Each client's **Shadow AI
  Fleet** project delivers to your SIEM, with a signing secret, and your SIEM
  routes on `organization_id`. Someone owns the Client Alerts queue for
  clients without a SIEM route
  ([SIEM export](https://docs.promptguard.co/integrations/siem-export.md#verify-that-a-delivery-came-from-promptguard)).
* [ ] **Device fail mode decided per client.** What a device does when it
  cannot reach the engine, and alerts for devices sending prompts unchecked
  ([Device fail-open alerts](https://docs.promptguard.co/integrations/siem-export.md#device-fail-open-alerts)).
* [ ] **Clients told what they will see.** PromptGuard does not offer
  white-labelling: the dashboard, emails and the device agent are
  PromptGuard-branded ([What your clients see](https://docs.promptguard.co/platform/msp.md#what-your-clients-see)).
* [ ] **Billing reconciled monthly.** Compare **Counted** and **Billed** for
  each client on the roster after each period closes.
* [ ] **Offboarding written down.** Before you end a grant: invite the
  client's owner if they are to keep the environment, and revoke the
  enrollment tokens, devices and API keys that should stop working. Use
  **End date** for a planned handover, **End** for an immediate one
  ([End access to a client](https://docs.promptguard.co/platform/msp.md#9-end-access-to-a-client)).
* [ ] **Proof-of-concept leftovers removed.** Client A's device and token are
  revoked, and its grant ended.

## Next steps

> [card] [MSP guide](https://docs.promptguard.co/platform/msp.md)
>
> The full procedure for real clients.

> [card] [Evaluate Shadow AI](https://docs.promptguard.co/guides/evaluate-shadow-ai.md)
>
> The single-organization proof, on one device.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.