For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.

Dive into the concept of AI agents in kagent, their components (instructions, tools, skills), and how they can even use other agents as tools.

An AI agent is an application that can interact with users in natural language. Agents use LLMs to generate responses to user queries and can also execute actions on behalf of the user.

Each agent consists of the following components:

• Agent instructions: A set of instructions that define the agent’s behavior and capabilities. This is also called a system prompt.
• Tools: Functions that the agent can use to interact with its environment. kagent features built-in tools and has support for accessing tools over the MCP.
• Skills: Descriptions of capabilities that help the agent act more autonomously and guide its tool usage and planning.

Agent Instructions

Agent instructions tell the agent what its role is, how to interact with the user, what actions it can take, how to behave and respond to user queries, and how to interact with other agents. The following example shows simple agent instructions:

You're a Kubernetes agent that can help users manage their Kubernetes resources.
Your responses should be clear and concise; you should provide helpful information and guidance to users.

Instructions are an important part of the agent’s behavior. They define the agent’s role and capabilities and help the agent understand its environment and the tasks it can perform.

Writing good instructions is an art and a science. It requires a good understanding of the task at hand, the tools available, and the user’s needs. To help you write good instructions, see the system prompt tutorial.

Prompt templates

You can use Go text/template syntax in system messages to compose reusable fragments stored in ConfigMaps. Instead of duplicating safety guidelines and tool usage instructions across every agent, store them once and reference them with {{include "alias/key"}} syntax. The controller resolves templates during reconciliation, so the final system message is fully expanded before reaching the agent runtime.

apiVersion: kagent.dev/v1alpha2
kind: Agent
metadata:
  name: k8s-agent
  namespace: kagent
spec:
  type: Declarative
  declarative:
    modelConfig: default-model-config
    promptTemplate:
      dataSources:
        - kind: ConfigMap
          name: kagent-builtin-prompts
          alias: builtin
        - kind: ConfigMap
          name: my-custom-prompts
    systemMessage: |
      You are a Kubernetes management agent named {{.AgentName}}.

      {{include "builtin/safety-guardrails"}}
      {{include "builtin/tool-usage-best-practices"}}
      {{include "my-custom-prompts/k8s-specific-rules"}}

      Your tools: {{.ToolNames}}
      Your skills: {{.SkillNames}}

The kagent-builtin-prompts ConfigMap ships with five reusable templates.

The following template variables are available in system messages.

Security Note: Only ConfigMaps are supported as data sources. Secret references are intentionally excluded to avoid leaking sensitive data into prompts sent to LLM providers.

Tools

Tools are functions that the agent can use to interact with its environment. For example, a Kubernetes agent might have tools to list pods, get pod logs, and describe services.

Tools definitions and their descriptions are made available to the agent and are sent to the LLMs together with the instructions. Based on the user query, the agent can use the tools to interact with the environment and generate responses.

For example, add the list_resources tool to your agent to allow it to list resources in the Kubernetes cluster. The agent determines, based on user input, whether to invoke any available tools.

If the user asks “List all pods in the cluster”, the agent can use the list_resources tool to list all pods in the cluster. Depending on how the instructions and tools are configured, the agent might list all namespaces first, then list all pods in each namespace. Alternatively, if the list_resources tool allows listing resources across namespaces, the agent picks that option.

Some tools support additional configuration that you set when adding the tool to the agent. For example, any Grafana or Prometheus tools will require an API endpoint URL to be set.

kagent comes with a set of built-in tools that you can use to interact with your environment. kagent also supports MCP (Model Context Protocol) tools. Using MCP, you can bring any external tool into kagent and make it available for your agents to run.

Human-in-the-Loop

kagent supports Human-in-the-Loop (HITL) to keep humans in control of agent actions. You can require user approval before an agent executes sensitive tools, and agents can ask users questions when they need clarification.

For a hands-on tutorial that walks through setting up HITL with tool approval and the ask_user tool, see the Human-in-the-Loop example.

Tool approval

Add requireApproval to your agent’s tool specification to gate destructive operations. Tools listed in requireApproval pause execution and present Approve/Reject buttons in the UI. Tools not listed run without waiting for approval.

tools:
  - type: McpServer
    mcpServer:
      name: kagent-tool-server
      kind: RemoteMCPServer
      apiGroup: kagent.dev
      toolNames:
        - k8s_get_resources       # runs immediately
        - k8s_describe_resource   # runs immediately
        - k8s_delete_resource     # pauses for approval
        - k8s_apply_manifest      # pauses for approval
      requireApproval:
        - k8s_delete_resource
        - k8s_apply_manifest

When you reject a tool call, you can provide a reason. This reason is passed back to the LLM as context, so the agent can adjust its approach.

Ask User

Every agent automatically has the built-in ask_user tool. It allows agents to pause and ask users questions with optional predefined choices, which is useful for clarifying ambiguous requests or collecting configuration preferences. No configuration for the ask_user tool is required.

Skills

Skills are descriptions or even executable implementations of the capabilities that an agent has to act more autonomously. They make the LLM’s responses more than just reactions to prompts by orienting them toward goals.

In some frameworks, skills are expressed as wrapped functions, reusable prompt templates, or even a synonym for a tool.

Think of skills like a catalog that expresses what the agent is capable of doing for a user. Unlike tools, skills are not a specific function that produces an output, like “fetch a website” or “tell the weather.” Unlike system instructions, they are not rules that apply to all interactions, such as “Follow my company’s style guide.”

Instead, skills are building blocks that guide the agent’s tool usage and planning. They help the agent understand what its goals are, and when and how to use tools effectively.

For example, two agents may share the same tools but use them differently based on their skills:

• A troubleshooting agent might use a describe tool to check the events of a crashing pod before taking a recovery action, such as restarting the pod.
• A research agent might use the same describe tool to gather details about a pod in order to answer a user’s question.

Skills can refer to two broad types:

• A2A skills: Metadata-based skills that are defined inline in the agent specification.
• kagent’s container-based skills: Executable skills packaged as container images and loaded from registries, for reuse across agents. You can use the agentregistry project to build and push skills to a registry.

A2A skills metadata

Actions-to-actions (A2A) skills are metadata: structured descriptions of capabilities, not executable code. Think of A2A skills as a machine-readable catalog entry about what a tool can do.

A2A skills metadata describes:

• What a capability is
• What the capability can do
• The inputs and outputs
• Safety requirements
• How a model should think about the capability

A2A skills metadata does not:

• Provide runtime code
• Execute actions
• Bundle actual logic for performing the skill

In kagent, you define A2A skills in a2aConfig.skills. These consist of a description, examples, ID, and tags that help guide the agent’s behavior. The description and examples provide context for both humans and the agent itself, often incorporated into system instructions. The ID and tags help you manage skills, such as by making it simpler to compare and reuse them across agents.

A2A skills are instructions about capabilities, not the capabilities themselves. That is why A2A is sometimes described as “just metadata”—it’s descriptive information, not executable code.

Container-based skills

kagent’s container-based skills are executable skill implementations packaged as container images. These are runnable procedural logic that the agent can use as an extension of itself.

Container-based skills include:

• Executable code snippets or procedures
• Behavior modules that the agent can call at inference time
• Validation and step-by-step behaviors
• Reusable functions
• Direct execution capabilities

These skills contain instructions, scripts, and resources that are loaded from container registries and made available to the agent at runtime. You can build and push skills as container images, then reference them in spec.skills.refs to load them into your agents.

Container-based skills are actual, callable capabilities—not just descriptions of capabilities.

kagent’s skills are similar to Claude’s Agent Skills, but with a key advantage: you can use kagent’s skills with any LLM provider, not just Anthropic Claude. This means your agents can use skills with OpenAI, Google Vertex AI, Azure OpenAI, Ollama, and any other LLM provider that kagent supports.

Git-based skills

You can also load skills directly from Git repositories, as an alternative to OCI images.

skills:
  gitRefs:
    - url: https://github.com/myorg/agent-skills.git
      ref: main
    - url: https://github.com/myorg/monorepo.git
      ref: main
      path: skills/kubernetes  # Use a subdirectory

For private repositories, configure authentication via HTTPS token or SSH key.

skills:
  gitAuthSecretRef:
    name: git-credentials  # Secret containing a `token` key
  gitRefs:
    - url: https://github.com/myorg/private-skills.git
      ref: main

A single gitAuthSecretRef applies to all Git repositories in the agent. You can combine Git and OCI skills in the same agent by specifying both refs and gitRefs.

S3-based skills

You can load skills directly from S3 — either a folder prefix (a path ending with / that contains a SKILL.md and sibling files) or a single .zip archive. Credentials use the AWS SDK default credential chain, which you supply as environment variables on the skills init container.

skills:
  s3Refs:
    - uri: s3://kagent-skills-bucket/team-a/kebab-maker   # folder prefix
      name: kebab-maker
    - uri: s3://kagent-skills-bucket/bundles/ops.zip       # zip archive
      region: us-east-1
  initContainer:
    env:
      - name: AWS_ACCESS_KEY_ID
        valueFrom:
          secretKeyRef:
            name: aws-creds
            key: AWS_ACCESS_KEY_ID
      - name: AWS_SECRET_ACCESS_KEY
        valueFrom:
          secretKeyRef:
            name: aws-creds
            key: AWS_SECRET_ACCESS_KEY
      - name: AWS_REGION
        value: us-west-2

You can combine S3 skills with OCI and Git skills in the same agent by specifying refs, gitRefs, and s3Refs together. For the full field reference, see S3SkillRef.

Best practices for skills

Containerize and store your skills in a specialized registry so that you can reuse them across agents. You can use the agentregistry project to build and push skills to a registry.

When creating skills for your agents, consider the following best practices. Agentregistry also has a repo of example skills based on Claude Skills that you can use as a starting point.

1. Be specific: Each skill should represent a distinct capability.
2. Provide good examples: Include diverse examples that cover different ways users might express the need for that skill.
3. Use descriptive tags: Tags help organize skills and make them easier to manage.
4. Align with tools: Ensure your skills align with the tools available to the agent. If you have a skill that centers around writing docs in markdown, you might want to align it with the write-markdown tool (as opposed to a generate-pdf tool).
5. Keep skills focused: Each skill should have a clear, focused purpose. For example, a document-generating skill might be too broad, but a skill that focuses on creating a specific type of document, such as a .docx file or alternatively a genre like a getting started guide, might be more appropriate.

To learn more about using skills in your agents, see the Skills example guide.

Runtime

You can choose between two Agent Development Kit (ADK) runtimes for declarative agents: Go (default) and Python.

Select the runtime via the runtime field in the declarative agent spec.

spec:
  type: Declarative
  declarative:
    runtime: go  # or "python"
    modelConfig: default-model-config
    systemMessage: "You are a helpful agent."

Choose Go when fast startup matters (autoscaling, cold starts), lower resource consumption is important, or you do not need Python-specific framework integrations.

Choose Python when you need Google ADK-native features, CrewAI/LangGraph/OpenAI framework integrations, or Python-based custom tools.

For more benchmarks and details, see the Go vs Python runtime blog post.

Deployment configuration

Control how the agent’s Kubernetes Deployment is configured in the spec.declarative.deployment stanza.

Environment variables

Use env to set individual environment variables, or envFrom to bulk-inject all keys from a ConfigMap or Secret.

spec:
  declarative:
    deployment:
      env:
        - name: LOG_LEVEL
          value: debug
      envFrom:
        - configMapRef:
            name: my-agent-config
        - secretRef:
            name: my-agent-secrets

Deployment annotations

Use deploymentAnnotations to add annotations to the Deployment object itself. This field is distinct from the annotations field, which targets pod template metadata only.

spec:
  declarative:
    deployment:
      deploymentAnnotations:
        argocd.argoproj.io/sync-wave: "5"
        notifications.argoproj.io/subscribe.on-degraded.slack: my-channel
      annotations:
        prometheus.io/scrape: "true"   # pod template only

deploymentAnnotations is useful for GitOps tooling such as Argo CD sync waves and Flux annotations, which key off Deployment-level metadata rather than pod metadata.

Memory

Your agents can save and retrieve relevant context across conversations using vector similarity search. When you enable memory on an agent, it receives three additional tools (save_memory, load_memory, prefetch_memory) and automatically extracts key information every 5th user message.

For configuration details, supported storage backends, API endpoints, and limitations, see Agent Memory.

Context Management

Long conversations can exceed LLM context windows. The context window is the maximum amount of text (tokens) that an LLM can process in a single request. You can enable event compaction to automatically summarize older messages while preserving key information.

spec:
  type: Declarative
  declarative:
    modelConfig: default-model-config
    systemMessage: "You are a helpful agent for extended sessions."
    context:
      compaction:
        compactionInterval: 5  # Compact every 5 user invocations

Compaction removes older conversation events to free up space in the context window. By default, compacted events are discarded. To preserve a summary of compacted events, configure the summarizer field with a modelConfig reference. Enable compaction for agents that handle long-running conversations, call many tools with large outputs, or need to support extended interactions.

Sandboxed Agents

You can run a declarative agent in an isolated sandbox by creating a SandboxAgent resource instead of a regular Agent. A SandboxAgent runs on Agent Substrate: the kagent controller runs it as a gVisor-sandboxed actor instead of a Deployment, snapshotting it to object storage when idle and rehydrating it on demand. The spec mirrors the Agent spec. All three runtimes are supported: Go (default), Python, and BYO. For Go and Python agents, session history is persisted to a local SQLite database in the agent’s durableDir volume, so conversation state survives pod restarts and Deployment rollouts. BYO agents do not get local session storage automatically. Configure substrate placement with the optional spec.substrate field (for example, workerPoolRef).

For setup steps, see the Agent Substrate example.

A2A AgentCard metadata

When another agent or client discovers your agent over the A2A protocol, it reads a machine-readable AgentCard from your agent’s /.well-known/agent.json endpoint. You can enrich that card with optional metadata fields on the Agent spec.

spec:
  iconUrl: https://example.com/icons/my-agent.png
  documentationUrl: https://docs.example.com/my-agent/
  version: "1.0.0"
  provider:
    organization: My Organization
    url: https://example.com

A2A AgentCard metadata

When another agent or client discovers your agent over the A2A protocol, it reads a machine-readable AgentCard from your agent’s /.well-known/agent.json endpoint. You can enrich that card with optional metadata fields on the Agent spec.

spec:
  iconUrl: https://example.com/icons/my-agent.png
  documentationUrl: https://docs.example.com/my-agent/
  version: "1.0.0"
  provider:
    organization: My Organization
    url: https://example.com

Agents as Tools

kagent also supports using agents as tools. Any agent you create can be referenced and used by other agents you have. An example use case would be to have a PromQL agent that knows how to create PromQL queries from natural language. Then you’d create a second agent that would use the PromQL agent whenever it needs to create a PromQL query.

The following example shows how to reference an existing agent (promql-agent) as a tool:

...
    # Referencing existing tools
    tools:
    - type: McpServer
      mcpServer:
        name: kagent-tool-server
        # Or a Kubernetes Service with "appProtocol: mcp", labels, and annotations for MCP
        # Or an MCPServer
        kind: RemoteMCPServer
        toolNames:
        - k8s_get_resources
        - k8s_get_available_api_resources
    # Referencing an existing agent as a tool (same namespace)
    - type: Agent
      agent:
        name: promql-agent
    # Referencing an agent in another namespace
    - type: Agent
      agent:
        name: promql-agent
        namespace: other-namespace

Per-call session isolation

By default, all calls to the same sub-agent share a single session, which preserves stateful continuity across calls. When a coordinator agent calls the same sub-agent in parallel, shared sessions can cause calls to interfere with each other.

Set isolateSessions: true on the Agent-type tool to give each call its own fresh session, enabling safe parallel fan-out.

spec:
  declarative:
    tools:
      - type: Agent
        agent:
          name: worker-agent
        isolateSessions: true

MCP server endpoint

A2A-enabled agents are automatically exposed as an MCP server on the kagent controller. The MCP endpoint is available at /mcp on the same port as the A2A endpoint (default 8083).

For more information, see the MCP tools guide.