Operational considerations#

Review the following operational considerations when running kagent in production environments, including database configuration, high availability, and secret management.

Automatic agent restart on secret updates#

Kagent automatically restarts agents when you update the secrets that the agents reference. This restart ensures that agents pick up new API keys, TLS certificates, and other secret values without manual intervention.

The following secret updates trigger automatic agent restarts:

• API keys: Secrets referenced in ModelConfig resources (e.g., OPENAI_API_KEY, ANTHROPIC_API_KEY)
• TLS certificates: Secrets referenced in ModelConfig TLS configuration (e.g., CA certificates)
• Environment variables: Any secrets referenced via secretKeyRef in agent deployment specifications

Leader election when controller is scaled#

When you scale the kagent controller to multiple replicas for high availability, leader election is automatically enabled. This ensures that only one controller instance actively reconciles resources at a time, preventing conflicts and duplicate operations.

Leader election scenarios#

• Single replica: No leader election needed; the single controller instance handles all operations.
• Multiple replicas: Leader election is automatically enabled when controller.replicas > 1.
• Active leader: Only the elected leader performs reconciliation operations.
• Standby replicas: Other replicas remain ready but do not perform reconciliation until they become the leader.

Enable high availability#

You can set the number of controller replicas to enable high availability.

Helm --set flag:

helm upgrade kagent oci://ghcr.io/kagent-dev/kagent/helm/kagent \
  --namespace kagent \
  --set controller.replicas=3

Helm values file:

controller:
  replicas: 3

More considerations for HA#

• Database requirement: When using multiple controller replicas, use PostgreSQL as the database backend. The default in-memory database does not support multiple replicas.
• Leader election: Leader election uses Kubernetes leases and is handled automatically.
• Failover: If the leader fails, another replica automatically becomes the leader.

Use PostgreSQL for scaling#

To scale the controller to multiple replicas, configure PostgreSQL as the database backend. You can enable PostgreSQL by using the Helm --set flag or values file.

Helm --set flag:

helm upgrade kagent oci://ghcr.io/kagent-dev/kagent/helm/kagent \
  --namespace kagent \
  --set database.type=postgres \
  --set database.postgres.url=postgres://user:password@postgres-host:5432/kagent \
  --set controller.replicas=3

Helm values file:

database:
  type: postgres
  postgres:
    url: postgres://user:password@postgres-host:5432/kagent
controller:
  replicas: 3