Automation Platform > Deployment & hosting

Managed: Kubernetes backend

Deploy the Automation Platform managed worker into a Kubernetes cluster with the included Helm chart. Each agent task runs as a Kubernetes Job in your cluster.

Deploy the oz-agent-worker daemon into a Kubernetes cluster using the included Helm chart. Each agent task runs as a Kubernetes Job in your cluster. The Automation Platform orchestrates runs end to end (Slack, Linear, schedules, API, oz agent run-cloud); your cluster provides the compute, scheduling, and policy enforcement.

When to use the Kubernetes backend

You already operate a Kubernetes cluster and want agents to run there.
You need Kubernetes-native scheduling, resource management, or policy enforcement.
You want to use Kubernetes Secrets, ServiceAccounts, and admission policies to control task behavior.

How it works

The worker connects to the Kubernetes API server (using in-cluster auth by default, or an explicit kubeconfig).
On startup, the worker creates a short-lived preflight Job with the configured task pod shape. This catches Job creation, admission policy, Pod Security, and some image-volume failures before the worker accepts tasks.
For each assigned task, the worker creates a Kubernetes Job in the configured namespace.
The worker monitors the Job and Pod status via Kubernetes Watch (with a 30-second safety-net poll for watch disconnects).
After the task completes, the worker removes successful Jobs. Failed Jobs remain available for diagnosis for 24 hours by default.

Prerequisites

Enterprise plan with self-hosting enabled — Contact sales if self-hosting is not yet enabled for your team.
A Kubernetes cluster with the worker process able to reach the API server. The cluster must:
- Allow the task namespace to create Jobs with a root init container, unless you enable native image volumes with kubernetesBackend.useImageVolumes=true.
- Grant the worker these namespace-scoped permissions: create, get, list, watch, delete on jobs; get, list, watch on pods; get on pods/log; list on events.
Helm installed locally, plus kubectl authenticated against the target cluster.
An agent API key — Create one in the Oz web app so the worker can authenticate to the Automation Platform. You can bind the key to any cloud agent — that choice doesn’t restrict which agents can run on the worker. See API Keys for the full creation flow.

Install with the Helm chart

The oz-agent-worker repository includes a namespace-scoped Helm chart at charts/oz-agent-worker. This is the recommended way to deploy the worker into a cluster.

What the chart deploys

A long-lived Deployment running oz-agent-worker with the Kubernetes backend.
A namespaced ServiceAccount for the worker.
A namespaced Role / RoleBinding with the minimum permissions needed to manage task Jobs and Pods.
A ConfigMap containing the worker config YAML.
An optional Secret for WARP_API_KEY (or a reference to an existing Secret).

The chart does not create CRDs or cluster-scoped RBAC resources.

1. Set your API key and namespace

export WARP_API_KEY="your_agent_api_key"

Create the namespace if it doesn’t exist:

kubectl create namespace warp-oz

2. Create the API key Secret

If you’re not using an existing Secret, create one with the API key:

kubectl create secret generic oz-agent-worker \
  --from-literal=WARP_API_KEY="$WARP_API_KEY" \
  --namespace warp-oz

Expected outcome: kubectl get secret -n warp-oz oz-agent-worker shows the Secret.

3. Install the chart

Clone the worker repo and install the chart:

git clone https://github.com/warpdotdev/oz-agent-worker.git

helm install oz-agent-worker ./oz-agent-worker/charts/oz-agent-worker \
  --namespace warp-oz \
  --set worker.workerId=oz-k8s-worker \
  --set image.tag=<version>

Expected outcome: kubectl get pods -n warp-oz shows the worker Deployment pod as Running, and the worker logs include Successfully connected to server.

To scale horizontally, deploy multiple Helm releases with distinct worker IDs rather than increasing replicas on a single release.

Key chart values

Required:

worker.workerId — The worker ID (same as --worker-id).
image.tag — The worker image tag to deploy.

Worker configuration:

worker.logLevel — Log verbosity (debug, info, warn, error). Defaults to info.
worker.cleanup — Whether to clean up task Jobs after execution. Defaults to true.
worker.maxConcurrentTasks — Maximum concurrent tasks. Defaults to 0 (unlimited).
worker.idleOnComplete — Duration to keep the oz process alive after task completion.
worker.resources — Resource requests/limits for the worker Deployment. Defaults to 100m CPU and 128Mi memory.
worker.livenessProbe — Liveness probe for the worker Deployment. Defaults to an exec probe (kill -0 1). Override with a custom probe or set to null to disable.
worker.terminationGracePeriodSeconds — Grace period for worker Deployment shutdown. Defaults to 30.
worker.nodeSelector, worker.tolerations, worker.affinity — Scheduling constraints for the worker Deployment pod.

Kubernetes backend:

kubernetesBackend.namespace — Namespace for task Jobs. Defaults to the release namespace.
kubernetesBackend.defaultImage — Default Docker image for task pods when no Warp environment has been supplied. Leave empty (default) to fall back to ubuntu:22.04.
kubernetesBackend.imagePullPolicy — Image pull policy for task pods. Defaults to IfNotPresent.
kubernetesBackend.useImageVolumes — Use native Kubernetes image volumes instead of root init containers to materialize sidecars. Defaults to false.
kubernetesBackend.preflightImage — Image for the startup preflight Job. Set this if your cluster restricts allowed registries.
kubernetesBackend.preflightResources — CPU and memory requests and limits for preflight containers.
kubernetesBackend.sidecarImage — Internal-registry override for the Warp agent sidecar image.
kubernetesBackend.unschedulableTimeout — How long a pod may remain unschedulable before failing. Defaults to 30s.
kubernetesBackend.setupCommand — Shell command to run before each task.
kubernetesBackend.teardownCommand — Shell command to run after each task.
kubernetesBackend.extraLabels — Additional labels for task Jobs and Pods.
kubernetesBackend.extraAnnotations — Additional annotations for task Jobs and Pods.
kubernetesBackend.activeDeadlineSeconds — Maximum task Job lifetime. Defaults to eight hours.
kubernetesBackend.ttlSecondsAfterFinished — Retention period for failed Jobs and Jobs orphaned by worker disruption. Defaults to 24 hours when cleanup is enabled.
kubernetesBackend.workspaceSizeLimit — Size limit for workspace emptyDir volume.
kubernetesBackend.podTemplate — Raw PodSpec YAML for task Jobs (same as backend.kubernetes.pod_template in the config file).

API key Secret:

warp.apiKeySecret.create — Set to true to have the chart create a Secret from warp.apiKeySecret.value. Defaults to false (expects a pre-existing Secret).
warp.apiKeySecret.value — The API key value to store in the chart-managed Secret. Only used when warp.apiKeySecret.create is true.
warp.apiKeySecret.name — Name of the Secret containing WARP_API_KEY. Defaults to oz-agent-worker.
warp.apiKeySecret.key — Key within the Secret. Defaults to WARP_API_KEY.

See the self-hosted worker reference for the full config file schema.

Cluster selection

Cluster selection follows Kubernetes client config conventions:

Set backend.kubernetes.kubeconfig to use an explicit kubeconfig file.
If kubeconfig is omitted and the worker runs inside a Kubernetes pod, the worker uses in-cluster config automatically.
Otherwise, the worker falls back to the default kubeconfig loading rules and uses the current context.

namespace selects the namespace inside the chosen cluster. It defaults to default when omitted.

Pod template

The pod_template field accepts standard Kubernetes PodSpec YAML and is the declarative way to configure task pod scheduling, service accounts, image pull secrets, resources, and environment variables.

When using pod_template, define a container named task to customize the main task container directly. Otherwise, the worker appends its own task container to the PodSpec.

Use valueFrom.secretKeyRef to inject Kubernetes Secret values into task container environment variables:

pod_template:
  serviceAccountName: agent-task-sa
  imagePullSecrets:
    - name: my-registry-creds
  containers:
    - name: task
      resources:
        requests:
          cpu: "2"
          memory: 4Gi
        limits:
          memory: 8Gi
      env:
        - name: GITHUB_TOKEN
          valueFrom:
            secretKeyRef:
              name: my-k8s-secret
              key: github-token
  tolerations:
    - key: "dedicated"
      operator: "Equal"
      value: "agents"
      effect: "NoSchedule"

The worker Deployment’s ServiceAccount is separate from the task Job serviceAccountName you configure in pod_template. The Deployment ServiceAccount needs RBAC to manage Jobs and Pods. The task ServiceAccount (if any) controls what the agent process can access at runtime.

When a run specifies an instance shape, the worker sets the task container’s CPU and memory requests and limits from that shape. Those values override CPU and memory configured on the task container in pod_template.

Preflight check

On startup, the worker creates a short-lived preflight Job that uses the configured task PodSpec. The preflight confirms that the worker can create the Job and that the cluster can create the expected pod shape. It surfaces early failures from:

Insufficient worker RBAC permissions in the target namespace.
Cluster admission policies such as Pod Security Standards, OPA Gatekeeper, and Kyverno.
Sidecar-loading requirements, including native image-volume support when enabled.

If the preflight fails, the worker logs a diagnostic error and exits before accepting tasks. A successful preflight does not validate every task image, task Secret, setup command, or network dependency.

The preflight image defaults to busybox:1.36. If your cluster restricts allowed registries or images, set kubernetesBackend.preflightImage to an allowlisted image. When imagePullSecrets is configured in kubernetesBackend.podTemplate, those secrets apply to the preflight Job as well, so you can point kubernetesBackend.preflightImage at an image in your private registry.

Environment variables for Kubernetes tasks

There are two ways to pass environment variables to Kubernetes task containers:

pod_template (recommended for Kubernetes-native config) — Use standard Kubernetes env syntax in the task container, including valueFrom.secretKeyRef for Kubernetes Secrets.
-e / --env flags — Backend-agnostic runtime overrides that work across all managed backends.

When configuring the Kubernetes backend via YAML or Helm, declarative task-container env belongs in pod_template rather than a separate top-level list.

If your organization uses an external secrets manager (HashiCorp Vault, AWS Secrets Manager, GCP Secret Manager, etc.), you can inject secrets into task pods via the CSI Secrets Store Driver or a similar operator. Configure the required volumes, volumeMounts, and annotations in pod_template just as you would for any other Kubernetes workload. See your secrets provider’s documentation for details.

Setup and teardown commands

Use kubernetesBackend.setupCommand (Helm value) or backend.kubernetes.setup_command (config file) to run a shell command before each task. Use teardownCommand / teardown_command for cleanup after the task finishes. These run inside the task Pod and are useful for workspace bootstrapping or post-run reporting.

Protect active task pods from disruption

The worker Deployment and task Jobs have different disruption requirements. When Kubernetes terminates the worker pod normally, the worker leaves active task Jobs running. Replacing the worker pod does not intentionally stop their task pods.

The task pod holds the live agent process and an emptyDir workspace. Evicting that pod interrupts the run and loses pod-local state. Configure node lifecycle tooling to avoid voluntary disruption of active task pods.

For Karpenter, add its pod-level disruption annotation to every task Job through the Helm values:

kubernetesBackend:
  extraAnnotations:
    karpenter.sh/do-not-disrupt: "true"

Task pods exist only for the life of their Jobs. The annotation blocks Karpenter consolidation and, depending on your NodePool configuration, drift while a run is active. It does not prevent forceful expiration, interruption, node repair, or manual deletion. A configured NodePool terminationGracePeriod can also make a node eligible for drift and eventually terminate blocking pods. Review Karpenter’s disruption controls against your NodePool policy.

A PodDisruptionBudget only constrains tools that use the Kubernetes Eviction API. It does not preserve the task process or workspace after deletion or protect against involuntary node loss. If your platform requires a PDB, validate its selector and availability rule against concurrent task Jobs. Keep the disruption protection required by your node lifecycle tool.

For other autoscalers or node lifecycle controllers, use their equivalent protection for active task pods. Do not assume that a replacement pod can resume an interrupted run.

Plan capacity and scheduling

Task pods need capacity for their configured requests before the worker’s unschedulable timeout expires.

Set worker.maxConcurrentTasks to a finite value based on the task capacity your cluster can provide. The default, 0, does not cap concurrency.
Set task CPU and memory requests through a runner instance shape or the task container in kubernetesBackend.podTemplate. Leave headroom for init containers, node DaemonSets, and workload spikes.
Set kubernetesBackend.unschedulableTimeout longer than the slowest expected node provisioning time. The default is 30s; set it to 0s only when you want to disable the fail-fast check.
Use worker.nodeSelector, worker.tolerations, and worker.affinity for the worker Deployment. Use kubernetesBackend.podTemplate for task-pod selectors, tolerations, affinity, and topology constraints.
A toleration makes a pod eligible for a tainted node; it does not reserve capacity. Pair dedicated-node tolerations with matching selectors or affinity and autoscaler capacity.

Metrics

The Helm chart includes built-in support for exporting OpenTelemetry metrics from the worker. Enable metrics by setting metrics.enabled=true:

helm install oz-agent-worker ./charts/oz-agent-worker \
  --namespace warp-oz \
  --set worker.workerId=oz-k8s-worker \
  --set image.tag=VERSION \
  --set metrics.enabled=true

With the default metrics.exporter=prometheus, the chart creates a Service with Prometheus scrape annotations and exposes port 9464. For clusters using the Prometheus Operator, set metrics.podMonitor.create=true to create a PodMonitor.

To push metrics to an OTLP collector instead, set metrics.exporter=otlp and configure the endpoint via metrics.extraEnv.

See Monitoring for the full list of Helm values, the metric catalog, and sample PromQL queries.

Operational notes

Scaling — The chart always deploys a single replica for a given worker.workerId. To run multiple workers, deploy multiple Helm releases with distinct worker IDs rather than scaling a single release horizontally.
Security context — The Deployment defaults to a non-root security context (runAsUser: 10001) with allowPrivilegeEscalation: false and all capabilities dropped.
Liveness probe — The Deployment includes a default exec liveness probe (kill -0 1). Override worker.livenessProbe for a custom probe, or set it to null to disable.
In-cluster auth — The chart assumes the worker runs inside the target cluster and uses in-cluster Kubernetes auth by default.

Self-hosted worker reference — Full CLI flag and config file schema, including every Kubernetes backend field.
Self-hosting overview — Managed vs unmanaged and the backend decision guide.
Routing runs to this worker — How to send tasks to your connected worker from the CLI, schedules, integrations, the API, and the web UI.
Environments — Define the task image, repos, and setup commands.
Monitoring — OpenTelemetry metrics, including Helm chart metrics values.
Security and networking — RBAC, admission policies, and data boundaries.
Troubleshooting — Common Kubernetes-backend issues.