βΈοΈ Taking Agents from Local Prototypes to Enterprise Kubernetes Clusters
Running autonomous agents on local development environments is insufficient for enterprise workloads that require handling thousands of concurrent user goals, long-running multi-step reasoning tasks, and high-availability failover. Scaling Autonomous Agents on Kubernetes bridges this gap by containerizing agent microservices, orchestrating asynchronous job queues with Redis and Celery, utilizing Kubernetes Event-driven Autoscaling (KEDA) based on queue length, and managing persistent distributed states across clustered worker pods.
π End-to-End Kubernetes Agent Scaling & Asynchronous Queue Workflow Diagram
1. Client Task Request
Incoming user goal or API request submitted to ingress gateway.
2. Redis Task Queue
Asynchronous Celery broker buffers agent task payloads.
3. KEDA Autoscaler
Monitors Redis queue depth and triggers Kubernetes HPA.
4. Worker Pod Scaling
Spins up containerized agent worker pods dynamically.
5. Distributed Lock Sync
Redlock prevents concurrent execution clashes across pods.
6. Completed Result Store
Results written back to persistent database & returned to client.
from celery import Celery
app = Celery('agent_tasks', broker='redis://localhost:6379/0', backend='redis://localhost:6379/1')
@app.task(name="run_autonomous_audit")
def run_autonomous_audit(company_id: str):
print(f"[Celery Worker] Starting autonomous financial audit for company {company_id}...")
# Simulated multi-step agent reasoning loop
return {"company_id": company_id, "status": "Audit Completed Successfully", "findings": 2}
# Trigger task asynchronously from API web server
# result = run_autonomous_audit.delay("COMP-8810")
print("Celery task registered successfully for asynchronous Kubernetes worker execution.")
ScaledObject Kubernetes custom resource definition (CRD) to auto-scale customer support agent worker pods based on incoming Redis queue length.
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: agent-worker-scaler
namespace: production
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: agent-worker-deployment
minReplicaCount: 2
maxReplicaCount: 50
cooldownPeriod: 300
triggers:
- type: redis
metadata:
address: redis-master.production.svc.cluster.local:6379
listName: celery
listLength: "15" # Scale up if more than 15 tasks queue up
redis-py to prevent multiple Kubernetes worker pods from executing duplicate agent workflows for the same customer ID simultaneously.
import redis
import time
client = redis.Redis(host='localhost', port=6379)
def acquire_agent_lock(customer_id: str, lock_timeout_sec: int = 10) -> bool:
lock_key = f"lock:agent:{customer_id}"
acquired = client.set(lock_key, "locked", nx=True, ex=lock_timeout_sec)
return bool(acquired)
# Simulate pod attempting acquisition
is_locked = acquire_agent_lock("CUST-9912")
print("Lock Acquired by Worker Pod:", is_locked)
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: agent-pdb
namespace: healthcare-prod
spec:
minAvailable: 3
selector:
matchLabels:
app: triage-agent-worker
import json
class AgentCheckpointManager:
def __init__(self):
self.db_storage = {} # Simulated PostgreSQL store
def save_checkpoint(self, task_id: str, step_number: int, memory_state: dict):
self.db_storage[task_id] = {"step": step_number, "state": memory_state}
print(f"[Checkpoint] Saved task {task_id} at step {step_number}.")
def load_checkpoint(self, task_id: str) -> dict:
return self.db_storage.get(task_id, {"step": 0, "state": {}})
mgr = AgentCheckpointManager()
mgr.save_checkpoint("TASK-44", 3, {"last_tool": "tavily_search"})
print("Resumed State:", mgr.load_checkpoint("TASK-44"))
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: isolate-agent-workers
namespace: security-prod
spec:
podSelector:
matchLabels:
role: agent-worker
policyTypes:
- Ingress
- Egress
ingress:
- from:
- namespaceSelector:
matchLabels:
name: api-gateway
from celery import chord, current_app
# Mock celery task signatures for map-reduce discovery
def process_chunk(chunk_id: int):
return f"Processed chunk {chunk_id}"
def aggregate_discovery_reports(results: list):
return f"Aggregated {len(results)} document sections into final brief."
print("Celery chord map-reduce workflow configured for distributed legal discovery.")
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: tutor-hpa
namespace: edtech-prod
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: tutor-api-deployment
minReplicas: 2
maxReplicas: 30
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 75
def setup_rabbitmq_consumer():
# Simulated RabbitMQ connection binding
exchange = "iot_telemetry_exchange"
queue = "agent_processing_queue"
print(f"[RabbitMQ] Consumer bound to exchange '{exchange}' on queue '{queue}'.")
setup_rabbitmq_consumer()
apiVersion: v1
kind: Pod
metadata:
name: gpu-agent-worker
namespace: mlops-prod
spec:
tolerations:
- key: "nvidia.com/gpu"
operator: "Exists"
effect: "NoSchedule"
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: accelerator
operator: In
values:
- nvidia-a100
containers:
- name: agent-container
image: nikitainfotech/agent-worker:v2.6
resources:
limits:
nvidia.com/gpu: "1"