Auto-scaling done correctly reduces infrastructure costs by 40-70% while improving reliability during traffic spikes. Auto-scaling done incorrectly causes flapping (constant scale-up/down cycles), cold start latency that degrades user experience, and runaway costs from scaling triggers that never cool down. The difference between the two is configuration precision.
This guide covers the three Kubernetes scaling mechanisms — Horizontal Pod Autoscaler (HPA), Vertical Pod Autoscaler (VPA), and KEDA — with production-ready YAML, cooldown tuning, predictive scaling with KEDA cron, load test validation with k6, and cost impact modeling.
---
HPA vs VPA vs KEDA: Comparison Table
| Dimension | HPA | VPA | KEDA |
| Scaling direction | Horizontal (more pods) | Vertical (bigger pods) | Horizontal (event-driven) |
| Default metrics | CPU, memory | CPU, memory | Custom: queue depth, Prometheus, DB rows, cron |
| Custom metrics | Yes (via adapter) | No | Yes (50+ scalers natively) |
| Scale to zero | No (minimum 1 pod) | No | Yes |
| Restart required on scale | No | Yes (for now) | No |
| Best for | Stateless web services | Batch jobs, non-scalable apps | Event-driven workloads, queue consumers |
| Works alongside | VPA (different resources), KEDA | HPA (careful: CPU conflict) | HPA (disable CPU on same deployment) |
| Stabilization windows | Yes | Yes | Yes |
| Kubernetes version | 1.6+ | 1.9+ | 1.16+ |
| Production maturity | GA | Beta (in-place resize: Alpha) | GA |
When to use which:
- HPA: stateless web APIs, microservices scaling on request rate or CPU
- VPA: batch processing pods, ML inference pods, single-instance legacy apps
- KEDA: message queue consumers, event processors, scheduled workloads that should scale to zero
- HPA + KEDA: use KEDA to manage HPA (recommended pattern for complex metrics)
---
HPA with Custom Prometheus Metrics
CPU-based HPA reacts 30-60 seconds after traffic increases. Custom metrics from Prometheus (request queue depth, active connections, p95 latency) react in 5-15 seconds.
<h1>hpa-api-service.yaml</h1>
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: api-service-hpa
namespace: production
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: api-service
minReplicas: 2
maxReplicas: 20
metrics:
# Primary metric: requests per second per pod
- type: Pods
pods:
metric:
name: http_requests_per_second
target:
type: AverageValue
averageValue: "50" # Scale when any pod exceeds 50 req/s
# Secondary metric: p95 latency (scale on degradation)
- type: Object
object:
metric:
name: nginx_ingress_controller_request_duration_seconds_p95
describedObject:
apiVersion: networking.k8s.io/v1
kind: Ingress
name: api-service-ingress
target:
type: Value
value: "0.5" # Scale when p95 > 500ms
# Fallback metric: CPU (prevents runaway memory leaks)
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
behavior:
scaleUp:
stabilizationWindowSeconds: 30 # 30s before scaling up (prevents flapping)
policies:
- type: Percent
value: 100 # Double pods at most per period
periodSeconds: 60
- type: Pods
value: 4 # Or add 4 pods per period — whichever is larger
periodSeconds: 60
selectPolicy: Max
scaleDown:
stabilizationWindowSeconds: 300 # 5 min before scaling down (traffic may return)
policies:
- type: Percent
value: 25 # Remove at most 25% of pods per period
periodSeconds: 120
- type: Pods
value: 2 # Or remove 2 pods per period — whichever is smaller
periodSeconds: 120
selectPolicy: Min # Take the more conservative policy
---
Scaling Policies: Stabilization Windows and Cooldown Tuning
The most common auto-scaling bug is flapping: the system scales up, traffic drops slightly, the system scales down, traffic spikes again, repeat. Stabilization windows prevent this.
| Scenario | Scale-Up Window | Scale-Down Window | Rationale |
| Web API (bursty traffic) | 30s | 300s (5 min) | Quick scale-up; slow scale-down guards against lulls |
| Batch job processor | 60s | 600s (10 min) | Jobs take time; don't interrupt mid-batch |
| ML inference service | 120s | 180s | Model load is slow; keep warm |
| Queue consumer | 15s | 120s | Queue depth is precise; react fast both ways |
| Scheduled report generator | 0s (KEDA cron) | 60s | Pre-scale; immediate teardown acceptable |
Cooldown tuning process:
---
Predictive Scaling with KEDA Cron
If your traffic pattern is predictable (business hours, weekly cycles, known events), pre-scale before traffic arrives instead of reacting after it hits.
<h1>keda-scaledobject-cron.yaml</h1>
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: api-service-scheduled
namespace: production
spec:
scaleTargetRef:
name: api-service
minReplicaCount: 1
maxReplicaCount: 30
triggers:
# Pre-scale 15 minutes before business hours (UTC)
- type: cron
metadata:
timezone: America/New_York
start: 45 8 * * 1-5 # 8:45 AM weekdays
end: 00 18 * * 1-5 # 6:00 PM weekdays
desiredReplicas: "8" # Pre-warmed replica count
# Scale up for Monday morning traffic spike
- type: cron
metadata:
timezone: America/New_York
start: 45 8 * * 1 # Monday 8:45 AM only
end: 30 10 * * 1 # Until 10:30 AM
desiredReplicas: "15" # Higher for Monday spike
# Combine with Prometheus for reactive scaling on top of cron baseline
- type: prometheus
metadata:
serverAddress: http://prometheus.monitoring.svc:9090
metricName: http_requests_per_second
query: sum(rate(http_requests_total{service="api-service"}[2m]))
threshold: "100"
---
Load Test Validation with k6
Never deploy auto-scaling configuration without load testing. The k6 script below validates scale-out behavior under a ramp-up profile.
// k6-load-test.js
import http from 'k6/http';
import { sleep, check } from 'k6';
import { Rate, Trend } from 'k6/metrics';
const errorRate = new Rate('errors');
const responseTime = new Trend('response_time', true);
export const options = {
stages: [
{ duration: '2m', target: 20 }, // Ramp to 20 users
{ duration: '5m', target: 20 }, // Hold - observe baseline
{ duration: '2m', target: 100 }, // Spike to 100 users
{ duration: '5m', target: 100 }, // Hold — validate scale-out
{ duration: '2m', target: 200 }, // Peak: 200 users
{ duration: '5m', target: 200 }, // Hold at peak
{ duration: '5m', target: 0 }, // Ramp down — validate scale-in
],
thresholds: {
'http_req_duration': ['p(95)<500'], // 95% of requests under 500ms
'http_req_failed': ['rate<0.01'], // Less than 1% errors
'errors': ['rate<0.01'],
},
};
export default function () {
const res = http.get('https://api.your-app.com/health', {
headers: { 'Authorization': `Bearer ${__ENV.API_TOKEN}` },
});
check(res, {
'status is 200': (r) => r.status === 200,
'response time < 500ms': (r) => r.timings.duration < 500,
});
errorRate.add(res.status !== 200);
responseTime.add(res.timings.duration);
sleep(1);
}
// Run: k6 run --env API_TOKEN=your-token k6-load-test.js
// Watch scaling: kubectl get hpa api-service-hpa -w
What to verify during the load test:
- Pod count increases within 60 seconds of hitting the scale-up threshold
- p95 latency stays under 500ms during scale-out
- Zero 5xx errors during pod scale-out (readiness probes prevent premature traffic)
- Pod count decreases 5-10 minutes after traffic drops (stabilization window)
- No flapping: scaling events should be monotonic during each load phase
---
Cost Impact Modeling
Auto-scaling saves money only if your minimum replica count is set correctly and scale-in actually fires.
| Configuration | Off-Peak Cost | Peak Cost | Monthly Total | Savings vs Fixed |
| Fixed 10 replicas (no auto-scaling) | $400/day | $400/day | $12,000 | Baseline |
| HPA: min=2, max=10 | $80/day | $400/day | $4,800 | 60% savings |
| HPA + KEDA cron: min=1 off-peak | $40/day | $400/day | $3,200 | 73% savings |
| HPA + scale-to-zero (KEDA) | $0/day (inactive) | $400/day | $2,000* | 83% savings |
*Assumes 8h peak per day, cold start < 30s acceptable
Cost modeling formula:
Monthly savings = (fixed_replicas - avg_replicas) × instance_cost × hours_per_month
Example: (10 - 3.2 avg) × $0.05/hr × 720 hr = $244.80/month per service
---
Multi-Dimensional Scaling
For complex applications, combine multiple signals to make scaling decisions more accurate:
<h1>KEDA ScaledObject with multiple triggers — scales when ANY threshold is met</h1>
triggers:
- type: prometheus
metadata:
query: sum(rate(http_requests_total[2m]))
threshold: "500"
- type: prometheus
metadata:
query: histogram_quantile(0.95, rate(http_request_duration_seconds_bucket[5m]))
threshold: "0.4" # Scale when p95 > 400ms
- type: redis
metadata:
address: redis.production.svc:6379
listName: job-queue
listLength: "50" # Scale when queue exceeds 50 items
Arxentra's platform includes pre-configured HPA-style autoscaling for all services deployed through the service management dashboard. Scaling policies can be adjusted from the UI without modifying Kubernetes manifests directly. See platform features for the auto-scaling configuration panel, and check pricing to model cost savings at your expected traffic levels.
---
Auto-Scaling Checklist
- Set resource requests accurately — HPA calculates utilization against requests, not node capacity
- Configure readiness probes — prevents traffic before app is ready (critical during scale-out)
- Set PodDisruptionBudgets — prevents scale-in from taking down too many pods at once
- Test scale-to-zero behavior in staging before enabling in production
- Monitor HPA events: `kubectl describe hpa your-hpa` for scaling decision logs
- Set maxReplicas conservatively — prevent runaway costs from metric misconfiguration
- Load test every scaling configuration change before promoting to production
- Review actual vs predicted replica counts monthly and adjust thresholds