For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.

Route kagent model requests through an agentgateway deployment for traffic management, observability, and security.

agentgateway is an AI-native proxy that adds traffic management, observability, and security to large language model calls. It serves an OpenAI-compatible API, so a ModelConfigModelConfigA Kubernetes custom resource naming one model at one provider, along with the credentials to reach it. An AgentTemplate references one by name, and every agent compiled from that template calls the model that it names.Learn more for agentgateway sets provider: OpenAI and points openAI.baseUrl at the Gateway service.

Set up agentgateway model routing

1. Install agentgateway in your cluster, adding --set agentgatewayModels.enabled=true to the Helm command for the control plane. For more information, see the agentgateway documentation.

2. Create a Gateway resource for model routing.

   kubectl apply -f - <<EOF
   apiVersion: gateway.networking.k8s.io/v1
   kind: Gateway
   metadata:
     name: agentgateway-proxy
     namespace: agentgateway-system
   spec:
     gatewayClassName: agentgateway
     listeners:
     - name: http
       protocol: HTTP
       port: 80
       allowedRoutes:
         namespaces:
           from: All
         kinds:
         - group: gateway.networking.k8s.io
           kind: HTTPRoute
         - group: agentgateway.dev
           kind: AgentgatewayModel
   EOF

3. Store the provider credentials that agentgateway uses to call the model. agentgateway authenticates to the provider on your agents’ behalf, so this key belongs to the gateway rather than to kagent. Read the value from the Authorization key of the Secret by default.

   kubectl create secret generic openai-key -n agentgateway-system \
     --from-file=Authorization=<path-to-a-file-holding-your-api-key>

4. Create an AgentgatewayModel resource for each model that kagent should reach. The resource name becomes the model name that kagent sends in its requests, so it must match the model field of the ModelConfig that you create later. The following example routes requests for gpt-4o-mini to OpenAI. For more provider and authentication options, see the agentgateway model documentation.

   kubectl apply -f - <<EOF
   apiVersion: agentgateway.dev/v1alpha1
   kind: AgentgatewayModel
   metadata:
     name: gpt-4o-mini
     namespace: agentgateway-system
   spec:
     parentRefs:
     - group: gateway.networking.k8s.io
       kind: Gateway
       name: agentgateway-proxy
       sectionName: http
     provider: OpenAI
     policies:
       auth:
         secretRef:
           name: openai-key
   EOF

   Note

   policies.auth gives the gateway the credentials that it needs to reach the provider. A model with no auth is still accepted and programmed, and every request through it returns the provider’s own 401, because agentgateway forwards the call with no credentials.

5. Save the agentgateway Gateway service address in an environment variable. The /v1 suffix is required: kagent appends /chat/completions to this value, and agentgateway serves that endpoint at /v1/chat/completions. Without the suffix, every model call returns 404 Not Found.

   export AGENTGATEWAY_URL=http://agentgateway-proxy.agentgateway-system.svc.cluster.local/v1

Create the ModelConfig

Choose the tab that matches how your agentgateway deployment authenticates callers.

kubectl apply -f - <<EOF
apiVersion: kagent.dev/v1alpha3
kind: ModelConfig
metadata:
  name: agentgateway-model-config
  namespace: kagent
spec:
  model: gpt-4o-mini
  provider: OpenAI
  openAI:
    baseUrl: "$AGENTGATEWAY_URL"
EOF

For every openAI field, including its type, default, and validation rules, see the API reference.

Use the ModelConfig

Reference the ModelConfig by name from an AgentTemplate in the same namespace.

spec:
  modelConfig:
    name: agentgateway-model-config

Next steps