> ## Documentation Index
> Fetch the complete documentation index at: https://docs.promptguard.co/llms.txt
> Use this file to discover all available pages before exploring further.

# vLLM

> Scan traffic to your own vLLM server through a self-hosted PromptGuard deployment

The `vllm/` namespace sends a request to the vLLM server named by the `VLLM_BASE_URL` environment variable **of the PromptGuard deployment**, default `http://localhost:8000/v1`. That variable is set by whoever runs PromptGuard, not by the caller, so this route is for a [self-hosted](/production/self-host-vs-cloud) PromptGuard running next to your vLLM server.

<Warning>
  On PromptGuard Cloud, `vllm/` models have nowhere to go: Cloud's `VLLM_BASE_URL` is not set to your server, and Cloud cannot reach your network. To scan a vLLM server from Cloud, expose it over `https` and use the [`openai-compatible/` namespace](/integrations/providers/openai-compatible), which sends requests to the host stored with your key.
</Warning>

## Configure the deployment

Set `VLLM_BASE_URL` on the PromptGuard API to your vLLM server's OpenAI-compatible base URL, including `/v1`:

```bash theme={"system"}
VLLM_BASE_URL=http://vllm.internal:8000/v1
```

## Send the request

Prefix the model id with `vllm/`. PromptGuard removes the prefix before forwarding, so the rest must be the model name your vLLM server knows: the `--model` value the server was started with.

PromptGuard refuses a proxy request with an empty `Authorization` header, and vLLM checks it only when started with `--api-key`, so send any non-empty value if your server has no key.

<CodeGroup>
  ```python Python theme={"system"}
  import os
  from openai import OpenAI

  client = OpenAI(
      base_url=os.environ["PROMPTGUARD_BASE_URL"],  # your deployment, e.g. https://promptguard.internal/api/v1
      api_key="your-vllm-api-key",  # forwarded to vLLM: the --api-key value, if you set one
      default_headers={"X-API-Key": os.environ["PROMPTGUARD_API_KEY"]},
  )

  response = client.chat.completions.create(
      model="vllm/meta-llama/Llama-3.1-8B-Instruct",
      messages=[{"role": "user", "content": "Hello!"}],
  )
  print(response.choices[0].message.content)
  ```

  ```typescript Node.js theme={"system"}
  import OpenAI from 'openai';

  const client = new OpenAI({
    baseURL: process.env.PROMPTGUARD_BASE_URL, // your deployment, e.g. https://promptguard.internal/api/v1
    apiKey: 'your-vllm-api-key', // forwarded to vLLM: the --api-key value, if you set one
    defaultHeaders: { 'X-API-Key': process.env.PROMPTGUARD_API_KEY },
  });

  const response = await client.chat.completions.create({
    model: 'vllm/meta-llama/Llama-3.1-8B-Instruct',
    messages: [{ role: 'user', content: 'Hello!' }],
  });
  ```
</CodeGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.