Using agentgateway with kagent#

As your kagent deployment grows, you might need governance over how your agents communicate with LLM providers. Agentgateway is a proxy purpose-built for AI workloads that sits between your kagent agents and your LLM provider. This way, you can apply AgentgatewayPolicy for things like access control, rate limiting, audit logging, and observability.

Prerequisites#

1. A running kagent installation. If you haven't installed kagent yet, follow the quick start guide first.
2. Follow the agentgateway installation guide to install agentgateway in your cluster.
3. Set up an LLM provider with agentgateway. This guide uses the Ollama setup as an example.

Architecture#

Once set up, kagent agent pods route all LLM requests through agentgateway (running in the agentgateway-system namespace). Agentgateway enforces your policies, auth/authz, rate limiting, audit logging, and observability. Before forwarding requests to Ollama on the host.

Configure kagent to use agentgateway#

With agentgateway installed, point kagent at the agentgateway proxy instead of directly at Ollama.

1. If you installed kagent without agentgateway, upgrade your installation to route through the proxy.

   helm upgrade kagent oci://ghcr.io/kagent-dev/kagent/helm/kagent \
     --reuse-values \
     --namespace kagent \
     --set providers.default=ollama \
     --set providers.ollama.baseUrl=http://agentgateway-proxy.agentgateway-system.svc.cluster.local/v1 \
     --set providers.ollama.apiKey=dummy

2. Create a ModelConfig that points to Ollama via the agentgateway proxy.

   kubectl apply -f- <<EOF
   apiVersion: kagent.dev/v1alpha2
   kind: ModelConfig
   metadata:
     name: llama3-model-config
     namespace: kagent
   spec:
     model: llama3
     provider: Ollama
     ollama:
       host: agentgateway-proxy.agentgateway-system.svc.cluster.local
   EOF

3. Verify that kagent is still accessible and correctly functioning.

   Cloud Provider LoadBalancerPort-forward for local testing
   export INGRESS_GW_ADDRESS=$(kubectl get svc -n kagent kagent-ui -o jsonpath="{.spec.clusterIP}")
   echo $INGRESS_GW_ADDRESS

   kubectl port-forward -n kagent service/kagent-ui 8082:8080

4. Open the kagent UI.

5. Start a chat with an agent such as k8s-agent to confirm that requests flow through agentgateway.

   

Apply governance policies#

With agentgateway in place, you can now apply policies to govern how your kagent agents interact with your LLM provider.

Block requests with PII#

1. Create an AgentgatewayPolicy resource to reject any request that contains PII, such as an email address. For more policy examples, see the agentgateway guardrails docs.

   kubectl apply -f - <<EOF
   apiVersion: agentgateway.dev/v1alpha1
   kind: AgentgatewayPolicy
   metadata:
     name: prompt-guard
     namespace: agentgateway-system
   spec:
     targetRefs:
     - group: gateway.networking.k8s.io
       kind: HTTPRoute
       name: ollama
     backend:
       ai:
         promptGuard:
           request:
           - response:
               message: "Rejected due to inappropriate content"
             regex:
               action: Reject
               matches:
               - "email"
   EOF

2. Verify the policy by sending a prompt to your agent through the kagent UI that includes the word email. You get a 403 response.

   

Cleanup#

1. Remove agentgateway-related resources from your kagent setup.

   kubectl delete agentgatewaypolicy prompt-guard -n agentgateway-system
   kubectl delete modelconfig llama3-model-config -n kagent

2. If you no longer need agentgateway, uninstall agentgateway.

3. Upgrade your kagent installation to unset agentgateway as the proxy in front of your LLM provider.