API gateways sit between clients and your services, handling cross-cutting concerns like authentication, rate limiting, and routing. Done well, they simplify your architecture. Done poorly, they become a bottleneck and single point of failure.
Here’s how to implement API gateways effectively.
What API Gateways Do
Core Responsibilities
┌────────────────────────────────────────────────────────────────┐
│ API Gateway │
├────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ Routing │ │ Auth │ │ Rate Limit │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ Transform │ │ Logging │ │ Caching │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
│ │
└────────────────────────────────────────────────────────────────┘
│ │ │
▼ ▼ ▼
┌───────────┐ ┌───────────┐ ┌───────────┐
│ Service A │ │ Service B │ │ Service C │
└───────────┘ └───────────┘ └───────────┘
Routing: Direct requests to appropriate services Authentication: Verify identity before forwarding Authorization: Check permissions Rate Limiting: Protect services from overload Request/Response Transformation: Adapt formats Logging/Monitoring: Centralized observability Caching: Reduce backend load
Patterns
Simple Reverse Proxy
Basic routing with minimal logic:
# nginx.conf
upstream orders_service {
server orders-service:8080;
}
upstream users_service {
server users-service:8080;
}
server {
listen 80;
location /api/orders {
proxy_pass http://orders_service;
}
location /api/users {
proxy_pass http://users_service;
}
}
When to use:
- Simple routing needs
- Services handle their own auth
- Minimal transformation required
Backend for Frontend (BFF)
Separate gateways for different clients:
┌──────────────────────────────────────────────────────────────┐
│ │
│ Mobile App ──► Mobile BFF ──┐ │
│ │ │
│ Web App ──► Web BFF ───┼──► Backend Services │
│ │ │
│ Third Party ──► Public API ──┘ │
│ │
└──────────────────────────────────────────────────────────────┘
// mobile-bff/routes/dashboard.ts
app.get('/dashboard', async (req, res) => {
// Aggregate data optimized for mobile
const [user, orders, notifications] = await Promise.all([
usersService.getUser(req.userId),
ordersService.getRecent(req.userId, limit: 5), // Limited for mobile
notificationsService.getUnread(req.userId)
]);
res.json({
user: { name: user.name, avatar: user.avatarThumb }, // Smaller image
recentOrders: orders.map(minimalOrderView),
unreadCount: notifications.length
});
});
When to use:
- Different clients have different needs
- Mobile needs aggregated/optimized responses
- Third-party API needs different rate limits/auth
Gateway Aggregation
Combine multiple service calls:
// Aggregate endpoint
app.get('/api/product/:id', async (req, res) => {
const productId = req.params.id;
const [product, inventory, reviews, pricing] = await Promise.all([
productService.get(productId),
inventoryService.getStock(productId),
reviewService.getSummary(productId),
pricingService.getPrice(productId, req.user?.tier)
]);
res.json({
...product,
inStock: inventory.quantity > 0,
stockLevel: inventory.quantity > 10 ? 'high' : 'low',
rating: reviews.averageRating,
reviewCount: reviews.count,
price: pricing.currentPrice,
originalPrice: pricing.listPrice
});
});
Gateway Offloading
Move cross-cutting concerns out of services:
# Kong gateway configuration
services:
- name: orders-service
url: http://orders-service:8080
routes:
- name: orders-route
paths:
- /api/orders
plugins:
- name: jwt
- name: rate-limiting
config:
minute: 100
hour: 1000
- name: correlation-id
- name: request-transformer
config:
add:
headers:
- X-Consumer-ID:$(consumer.id)
Services become simpler:
# orders-service (no auth, rate limiting, etc.)
@app.route('/api/orders')
def get_orders():
# Gateway already validated JWT
user_id = request.headers.get('X-Consumer-ID')
return orders_repo.get_by_user(user_id)
Implementation
Kong Gateway
Declarative configuration:
# kong.yml
_format_version: "2.1"
services:
- name: users-service
url: http://users:8080
routes:
- name: users-route
paths:
- /api/users
strip_path: false
plugins:
- name: jwt
config:
claims_to_verify:
- exp
- name: acl
config:
allow:
- users-read
- name: rate-limiting
config:
minute: 60
policy: redis
redis_host: redis
plugins:
- name: correlation-id
config:
header_name: X-Correlation-ID
generator: uuid
AWS API Gateway
Infrastructure as code:
# Terraform
resource "aws_api_gateway_rest_api" "main" {
name = "main-api"
}
resource "aws_api_gateway_resource" "orders" {
rest_api_id = aws_api_gateway_rest_api.main.id
parent_id = aws_api_gateway_rest_api.main.root_resource_id
path_part = "orders"
}
resource "aws_api_gateway_method" "orders_get" {
rest_api_id = aws_api_gateway_rest_api.main.id
resource_id = aws_api_gateway_resource.orders.id
http_method = "GET"
authorization = "COGNITO_USER_POOLS"
authorizer_id = aws_api_gateway_authorizer.cognito.id
}
resource "aws_api_gateway_integration" "orders_lambda" {
rest_api_id = aws_api_gateway_rest_api.main.id
resource_id = aws_api_gateway_resource.orders.id
http_method = aws_api_gateway_method.orders_get.http_method
integration_http_method = "POST"
type = "AWS_PROXY"
uri = aws_lambda_function.orders.invoke_arn
}
resource "aws_api_gateway_usage_plan" "standard" {
name = "standard"
throttle_settings {
rate_limit = 100
burst_limit = 200
}
quota_settings {
limit = 10000
period = "DAY"
}
}
Envoy Proxy
High-performance proxy:
# envoy.yaml
static_resources:
listeners:
- address:
socket_address:
address: 0.0.0.0
port_value: 8080
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
route_config:
virtual_hosts:
- name: backend
domains: ["*"]
routes:
- match:
prefix: "/api/orders"
route:
cluster: orders-service
- match:
prefix: "/api/users"
route:
cluster: users-service
http_filters:
- name: envoy.filters.http.jwt_authn
- name: envoy.filters.http.router
clusters:
- name: orders-service
type: STRICT_DNS
load_assignment:
cluster_name: orders-service
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: orders-service
port_value: 8080
Rate Limiting
Strategies
# Token bucket (common)
class TokenBucket:
def __init__(self, capacity, refill_rate):
self.capacity = capacity
self.tokens = capacity
self.refill_rate = refill_rate
self.last_refill = time.time()
def consume(self, tokens=1):
self._refill()
if self.tokens >= tokens:
self.tokens -= tokens
return True
return False
def _refill(self):
now = time.time()
elapsed = now - self.last_refill
self.tokens = min(self.capacity, self.tokens + elapsed * self.refill_rate)
self.last_refill = now
Implementation
# Redis-based rate limiting
import redis
def rate_limit(key, limit, window_seconds):
pipe = redis.pipeline()
now = time.time()
window_start = now - window_seconds
pipe.zremrangebyscore(key, 0, window_start)
pipe.zadd(key, {str(now): now})
pipe.zcard(key)
pipe.expire(key, window_seconds)
results = pipe.execute()
count = results[2]
return count <= limit
Response Headers
# Include rate limit info in response
response.headers['X-RateLimit-Limit'] = str(limit)
response.headers['X-RateLimit-Remaining'] = str(max(0, limit - count))
response.headers['X-RateLimit-Reset'] = str(int(reset_time))
if not allowed:
response.headers['Retry-After'] = str(int(reset_time - time.time()))
return response, 429
Security Considerations
Authentication at Gateway
# JWT validation
plugins:
- name: jwt
config:
secret_is_base64: false
claims_to_verify:
- exp
- iss
key_claim_name: kid
cookie_names: []
Request Validation
# JSON Schema validation
plugins:
- name: request-validator
config:
body_schema: |
{
"type": "object",
"properties": {
"email": { "type": "string", "format": "email" },
"name": { "type": "string", "minLength": 1 }
},
"required": ["email", "name"]
}
Header Sanitization
# Remove internal headers
INTERNAL_HEADERS = ['X-Internal-User', 'X-Debug-Token']
def sanitize_request(request):
for header in INTERNAL_HEADERS:
if header in request.headers:
del request.headers[header]
Anti-Patterns
Gateway as Business Logic
❌ Bad: Complex business logic in gateway
- Order validation
- Price calculation
- Inventory checks
✓ Good: Cross-cutting concerns only
- Authentication
- Rate limiting
- Routing
Single Gateway for Everything
❌ Bad: One gateway for all use cases
- Internal services
- External clients
- Admin APIs
✓ Good: Purpose-specific gateways
- Public API gateway (rate limited, strict auth)
- Internal gateway (service mesh)
- Admin gateway (different auth, audit logging)
Key Takeaways
- API gateways handle cross-cutting concerns: auth, rate limiting, routing
- Backend for Frontend pattern optimizes for different client needs
- Gateway aggregation reduces client round-trips
- Keep business logic out of the gateway; it’s for infrastructure concerns
- Rate limiting protects services; return meaningful headers to clients
- Validate and sanitize requests at the edge
- Consider separate gateways for different client types
- Kong, AWS API Gateway, and Envoy are solid choices with different trade-offs
- Monitor gateway performance; it’s on the critical path
- Gateway failures are highly visible; design for resilience
API gateways simplify your architecture when used appropriately. Keep them focused on infrastructure concerns, not business logic.