API response time is the foundation of application performance. Every 100ms of API latency adds perceptible delay to the user interface. This guide covers five optimization techniques that typically reduce p95 response times from 800ms to under 200ms: connection pooling, query optimization, response caching, payload compression, and rate limiting. Each technique includes production-ready configurations and before/after benchmarks.
Akamai's 2026 API performance study found that APIs with p95 response times under 200ms have 3.2x higher adoption rates than APIs exceeding 500ms. Developer satisfaction scores drop 18% for every additional 100ms of average latency. The median production API serves 73% of requests from cache when properly configured.
---
Connection Pooling
Every database query starts with a TCP connection. Without pooling, each request opens a new connection (50-100ms overhead) and closes it after the query. Connection pooling maintains a set of reusable connections, eliminating this overhead.
Connection Pool Configuration
<h1>FastAPI with asyncpg connection pool</h1>
import asyncpg
async def create_pool():
return await asyncpg.create_pool(
dsn="postgresql://user:[email protected]:5432/myapp",
min_size=5, # minimum idle connections
max_size=20, # maximum connections
max_inactive_connection_lifetime=300, # close idle after 5 min
command_timeout=10, # query timeout in seconds
)
<h1>Usage in route handler</h1>
async def get_services(user_id: int):
async with app.state.db_pool.acquire() as conn:
return await conn.fetch(
"SELECT id, name, status FROM services WHERE user_id = $1",
user_id
)
Pool Sizing Guidelines
| Concurrent Users | Min Pool | Max Pool | PostgreSQL max_connections |
| 1-50 | 3 | 10 | 50 |
| 50-200 | 5 | 20 | 100 |
| 200-1000 | 10 | 40 | 200 |
| 1000+ | Use PgBouncer | 100+ | 200+ (with PgBouncer) |
PgBouncer for High-Scale APIs
; pgbouncer.ini
[databases]
myapp = host=db.arxentra.cloud port=5432 dbname=myapp
[pgbouncer]
listen_addr = 127.0.0.1
listen_port = 6432
pool_mode = transaction ; release connection after each transaction
max_client_conn = 1000 ; accept up to 1000 app connections
default_pool_size = 25 ; 25 actual PostgreSQL connections
min_pool_size = 5
reserve_pool_size = 5
reserve_pool_timeout = 3
server_idle_timeout = 300
log_connections = 0
log_disconnections = 0
PgBouncer in transaction mode allows 1000 application connections to share 25 database connections. This reduces PostgreSQL memory usage by 95% while maintaining identical query performance. Check Services Overview for managed connection pooling options.
---
Query Optimization
Database queries account for 60-80% of API response time. Optimizing the top 10 slowest queries typically reduces average response time by 50%.
Finding Slow Queries
-- Enable pg_stat_statements (if not already enabled)
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;
-- Find top 10 time-consuming queries
SELECT
LEFT(query, 100) AS query_preview,
calls,
ROUND(mean_exec_time::numeric, 2) AS avg_ms,
ROUND(total_exec_time::numeric, 2) AS total_ms,
rows
FROM pg_stat_statements
ORDER BY total_exec_time DESC
LIMIT 10;
Common Query Optimization Patterns
| Problem | Solution | Typical Improvement |
| Sequential scan on large table | Add targeted index | 10-100x faster |
| N+1 query pattern | Use JOIN or batch query | 5-50x fewer queries |
| SELECT * on wide table | Select only needed columns | 2-5x less I/O |
| Missing WHERE clause filter | Add partition pruning | 10-50x faster |
| Unparameterized queries | Use prepared statements | 20-30% faster |
---
Response Caching Patterns
Not every API response needs to be computed fresh. Identify endpoints where stale data is acceptable and cache aggressively.
<h1>Tiered caching strategy</h1>
CACHE_TIERS = {
"dashboard_summary": 300, # 5 minutes - aggregate metrics
"service_list": 60, # 1 minute - service inventory
"user_profile": 600, # 10 minutes - rarely changes
"pricing_plans": 3600, # 1 hour - very stable
"system_health": 10, # 10 seconds - near real-time
}
<h1>Cache invalidation on write</h1>
async def update_service(service_id: int, data: dict):
await db.update_service(service_id, data)
# Invalidate related caches
await redis.delete(f"service:{service_id}")
await redis.delete(f"service_list:{data['user_id']}")
await redis.delete(f"dashboard:{data['user_id']}")
HTTP-Level Caching
<h1>FastAPI response caching with ETags</h1>
from fastapi import Response
import hashlib
@app.get("/api/services")
async def list_services(response: Response):
data = await get_cached_services()
etag = hashlib.md5(str(data).encode()).hexdigest()
response.headers["ETag"] = f'"{etag}"'
response.headers["Cache-Control"] = "private, max-age=60"
return data
---
Payload Compression and Optimization
- Enable Brotli compression (20-30% smaller than gzip)
- Use pagination for list endpoints (default page size: 25)
- Support sparse fieldsets (`?fields=id,name,status`)
- Remove null values from responses
- Use numeric enums instead of string enums for high-volume endpoints
<h1>Enable Brotli compression in nginx</h1>
brotli on;
brotli_types application/json text/plain application/javascript;
brotli_comp_level 6;
brotli_min_length 256;
---
Rate Limiting
Rate limiting protects API performance by preventing any single client from consuming excessive resources.
<h1>Redis-based sliding window rate limiter</h1>
import time
async def check_rate_limit(
redis_client, key: str, limit: int, window_seconds: int
) -> bool:
now = time.time()
pipe = redis_client.pipeline()
pipe.zremrangebyscore(key, 0, now - window_seconds)
pipe.zadd(key, {str(now): now})
pipe.zcard(key)
pipe.expire(key, window_seconds)
results = await pipe.execute()
return results[2] <= limit
<h1>Rate limit tiers</h1>
<h1>Free: 100 requests/minute</h1>
<h1>Starter: 500 requests/minute</h1>
<h1>Pro: 2000 requests/minute</h1>
| Plan | Requests/Minute | Burst Limit | Concurrent |
| Free | 100 | 20 | 5 |
| Starter | 500 | 50 | 20 |
| Pro | 2000 | 200 | 50 |
| Enterprise | Custom | Custom | Custom |
See Pricing for rate limit details by plan.
---
Performance Benchmarking
| Optimization | Before (p95) | After (p95) | Improvement |
| Connection pooling | 450ms | 180ms | 60% |
| Query optimization | 380ms | 95ms | 75% |
| Response caching | 250ms | 15ms | 94% (cache hit) |
| Payload compression | n/a | n/a | 65% size reduction |
| Rate limiting | unstable | stable | prevents degradation |
---
Frequently Asked Questions
What is a good p95 response time for a REST API?
Under 200ms for read endpoints and under 500ms for write endpoints. If your p95 exceeds 1 second, there is almost certainly an unoptimized database query or missing cache layer.
Should I cache at the application level or the CDN level?
Both. CDN caching works for public, identical-for-all-users responses. Application-level caching (Redis) works for user-specific data. Use CDN for asset-heavy responses and Redis for personalized API responses.
How do I handle cache invalidation without race conditions?
Use event-driven invalidation: when data changes, publish an event that deletes the relevant cache keys. Add a short TTL (60-300 seconds) as a safety net. Never rely solely on TTL-based expiration for data that users expect to update immediately.
What is the overhead of rate limiting on API performance?
A Redis-based sliding window rate limiter adds 0.5-2ms per request. This is negligible compared to the protection it provides against traffic spikes that could degrade performance for all users.
When should I use GraphQL instead of REST for performance?
GraphQL reduces over-fetching when clients need different subsets of data from the same endpoint. It adds complexity and makes caching harder. Use GraphQL when you have diverse clients (web, mobile, API consumers) with different data needs. Stick with REST when your client needs are uniform. See Documentation for API design patterns.