> For the complete documentation index, see [llms.txt](/docs/llms.txt). Markdown versions of all docs pages are available by appending .md to any docs URL.

# agentgateway

[agentgateway](https://agentgateway.dev/docs/) is an AI-native proxy that adds traffic management, observability, and security to large language model calls. It serves an OpenAI-compatible API, so a ModelConfig for agentgateway sets `provider: OpenAI` and points `openAI.baseUrl` at the Gateway service.

## Set up agentgateway model routing

> [!NOTE]
> The `AgentgatewayModel` feature is experimental and disabled by default. Enable it when you install agentgateway by passing `--set agentgatewayModels.enabled=true` to the control plane Helm chart.

1. Install agentgateway in your cluster, adding `--set agentgatewayModels.enabled=true` to the Helm command for the control plane. For more information, see the [agentgateway documentation](https://agentgateway.dev/docs/kubernetes/latest/documentation/setup/).
2. Create a `Gateway` resource for model routing.
   
   ```yaml
   kubectl apply -f - <<EOF
   apiVersion: gateway.networking.k8s.io/v1
   kind: Gateway
   metadata:
     name: agentgateway-proxy
     namespace: agentgateway-system
   spec:
     gatewayClassName: agentgateway
     listeners:
     - name: http
       protocol: HTTP
       port: 80
       allowedRoutes:
         namespaces:
           from: All
         kinds:
         - group: gateway.networking.k8s.io
           kind: HTTPRoute
         - group: agentgateway.dev
           kind: AgentgatewayModel
   EOF
   ```
3. Store the provider credentials that agentgateway uses to call the model. agentgateway authenticates to the provider on your agents' behalf, so this key belongs to the gateway rather than to kagent. Read the value from the `Authorization` key of the Secret by default.
   
   ```bash
   kubectl create secret generic openai-key -n agentgateway-system \
     --from-file=Authorization=<path-to-a-file-holding-your-api-key>
   ```
4. Create an `AgentgatewayModel` resource for each model that kagent should reach. The resource name becomes the model name that kagent sends in its requests, so it must match the `model` field of the ModelConfig that you create later. The following example routes requests for `gpt-4o-mini` to OpenAI. For more provider and authentication options, see the [agentgateway model documentation](https://docs.solo.io/agentgateway/kubernetes/latest/documentation/llm/models/).
   
   ```yaml
   kubectl apply -f - <<EOF
   apiVersion: agentgateway.dev/v1alpha1
   kind: AgentgatewayModel
   metadata:
     name: gpt-4o-mini
     namespace: agentgateway-system
   spec:
     parentRefs:
     - group: gateway.networking.k8s.io
       kind: Gateway
       name: agentgateway-proxy
       sectionName: http
     provider: OpenAI
     policies:
       auth:
         secretRef:
           name: openai-key
   EOF
   ```
   
   > [!NOTE]
   > `policies.auth` gives the gateway the credentials that it needs to reach the provider. A model with no `auth` is still accepted and programmed, and every request through it returns the provider's own `401`, because agentgateway forwards the call with no credentials.
5. Save the agentgateway Gateway service address in an environment variable. The `/v1` suffix is required: kagent appends `/chat/completions` to this value, and agentgateway serves that endpoint at `/v1/chat/completions`. Without the suffix, every model call returns `404 Not Found`.
   
   ```bash
   export AGENTGATEWAY_URL=http://agentgateway-proxy.agentgateway-system.svc.cluster.local/v1
   ```

## Create the ModelConfig

Choose the tab that matches how your agentgateway deployment authenticates callers.

You can choose from the following options.

**Option: No authentication**

When your agentgateway deployment does not enforce API key authentication, the ModelConfig does not need a Secret.

```yaml
kubectl apply -f - <<EOF
apiVersion: api.kagent.dev/v1alpha3
kind: ModelConfig
metadata:
  name: agentgateway-model-config
  namespace: kagent
spec:
  model: gpt-4o-mini
  provider: OpenAI
  openAI:
    baseUrl: "$AGENTGATEWAY_URL"
EOF
```

| Field | Description |
| --- | --- |
| `model` | The model name to request from agentgateway. This must match the name of an `AgentgatewayModel` resource in your agentgateway deployment. |
| `provider` | The provider to use, `OpenAI`, because agentgateway serves an OpenAI-compatible API. |
| `openAI.baseUrl` | The Kubernetes Service address of your agentgateway Gateway, including the `/v1` path. |

**Option: API key authentication**

When your agentgateway deployment applies an `apiKeyAuthentication` policy with `mode: Strict`, supply the API key that callers send in the `Authorization` header.

1. Create a Kubernetes Secret that stores the API key, in the same namespace as the AgentTemplates that use it.
   
   ```bash
   export AGENTGATEWAY_API_KEY=<your-api-key>
   kubectl create secret generic kagent-agentgateway-key -n kagent \
     --from-literal AGENTGATEWAY_API_KEY=$AGENTGATEWAY_API_KEY
   ```
2. Create the `ModelConfig`.
   
   ```yaml
   kubectl apply -f - <<EOF
   apiVersion: api.kagent.dev/v1alpha3
   kind: ModelConfig
   metadata:
     name: agentgateway-model-config
     namespace: kagent
   spec:
     apiKeySecret: kagent-agentgateway-key
     apiKeySecretKey: AGENTGATEWAY_API_KEY
     model: gpt-4o-mini
     provider: OpenAI
     openAI:
       baseUrl: "$AGENTGATEWAY_URL"
   EOF
   ```
   
   | Field | Description |
| --- | --- |
| `apiKeySecret` | The name of the Kubernetes Secret that stores the agentgateway API key. |
| `apiKeySecretKey` | The key within that Secret that holds the API key. |
| `model` | The model name to request from agentgateway. This must match the name of an `AgentgatewayModel` resource in your agentgateway deployment. |
| `provider` | The provider to use, `OpenAI`, because agentgateway serves an OpenAI-compatible API. |
| `openAI.baseUrl` | The Kubernetes Service address of your agentgateway Gateway, including the `/v1` path. |

For every `openAI` field, including its type, default, and validation rules, see the [API reference](https://kagent.dev/docs/kagent/1.x/reference/api-ref#openaiconfig).

## Use the ModelConfig

Reference the ModelConfig by name from an AgentTemplate in the same namespace.

```yaml
spec:
  modelConfig:
    name: agentgateway-model-config
```

## Next steps

[Your first agent](https://kagent.dev/docs/kagent/1.x/get-started/your-first-agent/)

Create an agent that uses this model, and hold a conversation with it.

[BYO OpenAI-compatible endpoint](https://kagent.dev/docs/kagent/1.x/setup/model-providers/byo-openai/)

Point the OpenAI provider at any other compatible service.