Documentation

Configuring Custom Models

Often you may want to use a custom model with an existing provider, for example if you're using a self-hosted model exposed via the OpenAI API. Or if you're using a proxy to the OpenAI API.

We require certain information about any model to be able to use it. We ship with a default set of models, but you can also add custom models.

The example below will add an API key, but given that this is a custom provider, the key is optional.

  1. Create a Kubernetes Secret that stores the API key, replace <your_api_key> with an actual API key:
export OPENAI_API_KEY=<your_api_key>
kubectl create secret generic kagent-openai -n kagent --from-literal OPENAI_API_KEY=$OPENAI_API_KEY
  1. Create a ModelConfig resource that references the secret and key name, and specify the model you want to use:
apiVersion: kagent.dev/v1alpha1
kind: ModelConfig
metadata:
  name: custom-openai-model-config
  namespace: kagent
spec:
  apiKeySecretRef: kagent-openai
  apiKeySecretKey: OPENAI_API_KEY
  model: <insert-model-name> # This should match the model name passed to the provider.
  provider: OpenAI
  openAI:
    baseUrl: https://your-custom-openai-provider.com
  modelInfo:
    functionCalling: true
    family: unknown

If you don't need an API key you can elide the apiKeySecretRef and apiKeySecretKey fields.

  1. Apply the above resource to the cluster.

Once the resource is applied, you can select the model from the Model dropdown in the UI when creating or updating agents.