Load Balancing: Nginx & HAProxy Core Knowledge

Nginx vs HAProxy — When to Use Each

Dimension	Nginx	HAProxy
Primary role	Web server + reverse proxy + LB	Dedicated load balancer + proxy
Config complexity	Low-medium	Medium-high
HTTP modes	HTTP/1.1, HTTP/2	HTTP/1.1, HTTP/2 (enterprise), HTTP/3 (1.9+)
TCP/UDP LB	Nginx Plus or stream module	Native, very mature
Active health checks	Nginx Plus only (open-source: passive only)	Built-in, free
Stats/metrics UI	Third-party (nginx-lua, stub_status)	Built-in stats page
Sticky sessions	Nginx Plus (cookie) or ip_hash	Stick tables (any key), free
Connection reuse	Keepalive to upstream	Reuse connections, queue management
Dynamic reconfiguration	Nginx Plus (`upstream_conf` API)	Runtime API (HAProxy 2.0+)
Ecosystem / docs	Very mature, massive	Mature, industry standard for pure LB

Use Nginx when: you already use Nginx as your web server, you want a single tool for serving files + proxying + LB, or your team knows Nginx.

Use HAProxy when: you need advanced health checks, fine-grained ACL routing, TCP load balancing, or maximum LB performance and observability.

Nginx Upstream Configuration

Basic Upstream Block

# /etc/nginx/nginx.conf or included conf

http {
    # Shared memory zone for upstream state across workers
    # Required for proper load balancing with multiple workers
    upstream app_backend {
        zone app_zone 256k;         # Shared state (round_robin works without it too)

        # Balancing method (default is round_robin if nothing specified)
        # least_conn;               # Route to backend with fewest active connections
        # ip_hash;                  # Sticky: same client IP always → same backend
        # hash $request_uri consistent;  # Consistent hashing by URI (good for caching)
        # random two least_conn;    # Pick 2 random servers, send to less-loaded one

        server 10.0.1.10:3000 weight=3 max_fails=3 fail_timeout=30s;
        server 10.0.1.11:3000 weight=1 max_fails=3 fail_timeout=30s;
        server 10.0.1.12:3000 weight=1 max_fails=3 fail_timeout=30s;

        # Backup server — only used when all primaries are down
        server 10.0.1.20:3000 backup;

        # Permanently excluded (maintenance)
        # server 10.0.1.13:3000 down;

        # Keepalive connections to upstream (dramatically reduces TCP overhead)
        keepalive 64;               # Max idle keepalive connections per worker
        keepalive_requests 1000;    # Max requests per keepalive connection
        keepalive_timeout 60s;
    }

    server {
        listen 80;
        server_name api.example.com;

        # Logging with upstream info
        log_format upstream_log '$remote_addr - $upstream_addr [$time_local] '
                                 '"$request" $status $body_bytes_sent '
                                 'rt=$request_time urt=$upstream_response_time';
        access_log /var/log/nginx/api_access.log upstream_log;

        location / {
            proxy_pass         http://app_backend;
            proxy_http_version 1.1;                     # Required for keepalive
            proxy_set_header   Connection "";           # Required for keepalive
            proxy_set_header   Host              $host;
            proxy_set_header   X-Real-IP         $remote_addr;
            proxy_set_header   X-Forwarded-For   $proxy_add_x_forwarded_for;
            proxy_set_header   X-Forwarded-Proto $scheme;

            # Timeouts
            proxy_connect_timeout  5s;
            proxy_send_timeout     30s;
            proxy_read_timeout     30s;

            # Passive health check: try next upstream on errors
            proxy_next_upstream     error timeout http_502 http_503 http_504;
            proxy_next_upstream_tries 3;
            proxy_next_upstream_timeout 10s;

            # Buffering
            proxy_buffering    on;
            proxy_buffer_size  16k;
            proxy_buffers      8 16k;
        }

        # Health check endpoint for external monitors
        location /nginx-health {
            access_log off;
            return 200 "OK\n";
        }
    }
}

Active Health Check Workaround (Open-Source Nginx)

Nginx OSS only supports passive health checks. Simulate active checks with a small service or use the nginx_upstream_check_module (third-party).

# Install lua-nginx-module + lua-resty-upstream-healthcheck
# OR use OpenResty (Nginx + LuaJIT bundle)
# Simple approach: use a separate monitoring tool (Consul, HAProxy) alongside Nginx

SSL Termination at Nginx

server {
    listen 443 ssl http2;
    server_name api.example.com;

    # Certificate (from Let's Encrypt / Certbot)
    ssl_certificate     /etc/letsencrypt/live/api.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/api.example.com/privkey.pem;

    # Modern TLS settings
    ssl_protocols       TLSv1.2 TLSv1.3;
    ssl_ciphers         ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384;
    ssl_prefer_server_ciphers off;
    ssl_session_cache   shared:SSL:10m;
    ssl_session_timeout 1d;
    ssl_session_tickets off;

    # HSTS
    add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload" always;

    location / {
        proxy_pass http://app_backend;      # Plain HTTP to backend (internal network)
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

# HTTP → HTTPS redirect
server {
    listen 80;
    server_name api.example.com;
    return 301 https://$host$request_uri;
}

Stub Status (Metrics Endpoint)

server {
    listen 127.0.0.1:8080;   # Bind to localhost only
    location /nginx_status {
        stub_status;
        allow 127.0.0.1;
        deny all;
    }
}

HAProxy Configuration

Full HTTP Load Balancer Config

# /etc/haproxy/haproxy.cfg

global
    log         /dev/log local0 info
    log         /dev/log local0 notice notice
    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     50000               # Total concurrent connections
    user        haproxy
    group       haproxy
    daemon
    stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners

defaults
    log         global
    mode        http                # http | tcp
    option      httplog             # Structured HTTP log format
    option      dontlognull         # Don't log health checks
    option      forwardfor          # Add X-Forwarded-For header
    option      http-server-close   # Close server-side connection after each request
    option      redispatch          # Retry on different server if session fails
    timeout     connect  5s
    timeout     client   30s
    timeout     server   30s
    timeout     http-request 10s    # Max time to receive full HTTP request
    timeout     http-keep-alive 5s
    timeout     queue   1m          # Max wait in queue when all servers full
    timeout     tunnel  1h          # For WebSocket / long-lived connections
    retries     3

#──────────────────────────────────────
# Stats page
#──────────────────────────────────────
frontend stats
    bind *:8404
    stats enable
    stats uri /stats
    stats refresh 10s
    stats auth admin:strongpassword    # CHANGE THIS
    stats show-legends
    stats show-node
    # Restrict to internal IPs
    acl internal_nets src 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16
    tcp-request connection reject if !internal_nets

#──────────────────────────────────────
# HTTPS frontend (SSL termination)
#──────────────────────────────────────
frontend https_in
    bind *:443 ssl crt /etc/ssl/certs/example.com.pem  # Combined cert+key PEM
    bind *:80
    http-request redirect scheme https unless { ssl_fc }

    # Define ACLs for routing
    acl host_api   hdr(host) -i api.example.com
    acl host_app   hdr(host) -i app.example.com
    acl path_admin path_beg /admin

    # Security headers
    http-response set-header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
    http-response set-header X-Content-Type-Options    nosniff
    http-response set-header X-Frame-Options           DENY
    http-response del-header Server

    # ACL-based routing to backends
    use_backend api_servers  if host_api
    use_backend app_servers  if host_app !path_admin
    use_backend admin_server if host_app path_admin

    default_backend app_servers

#──────────────────────────────────────
# API backend
#──────────────────────────────────────
backend api_servers
    balance leastconn               # roundrobin | leastconn | source | uri | random

    # Active HTTP health checks
    option httpchk GET /health HTTP/1.1\r\nHost:\ api.example.com
    http-check expect status 200
    default-server inter 10s fastinter 2s downinter 5s rise 2 fall 3

    # Connection limits per server
    default-server maxconn 100 maxqueue 50

    # Keepalive to backends
    option http-server-close
    timeout connect 3s
    timeout server  15s

    server api1 10.0.1.10:3000 check weight 10
    server api2 10.0.1.11:3000 check weight 10
    server api3 10.0.1.12:3000 check weight 5   # Lower weight — less powerful
    server api_backup 10.0.1.20:3000 check backup

#──────────────────────────────────────
# App backend with sticky sessions
#──────────────────────────────────────
backend app_servers
    balance roundrobin
    option httpchk GET /health
    http-check expect status 200

    # Cookie-based sticky sessions
    cookie SERVERID insert indirect nocache httponly secure

    default-server inter 10s rise 2 fall 3
    server app1 10.0.1.30:8080 check cookie app1
    server app2 10.0.1.31:8080 check cookie app2
    server app3 10.0.1.32:8080 check cookie app3

#──────────────────────────────────────
# Admin backend — IP restricted
#──────────────────────────────────────
backend admin_server
    # IP whitelist using TCP-request (set in frontend ACL)
    option httpchk GET /admin/health
    server admin1 10.0.1.50:8080 check

#──────────────────────────────────────
# TCP mode example (e.g., PostgreSQL)
#──────────────────────────────────────
frontend postgres_in
    bind *:5432
    mode tcp
    default_backend postgres_servers

backend postgres_servers
    mode tcp
    balance leastconn
    option tcp-check
    server pg_primary 10.0.2.10:5432 check
    server pg_replica 10.0.2.11:5432 check backup

HAProxy Runtime API

# Enable in global section: stats socket /run/haproxy/admin.sock mode 660 level admin

# Show current server states
echo "show servers state" | socat stdio /run/haproxy/admin.sock

# Drain a server (stop sending new requests, finish existing)
echo "set server api_servers/api1 state drain" | socat stdio /run/haproxy/admin.sock

# Bring server back online
echo "set server api_servers/api1 state ready" | socat stdio /run/haproxy/admin.sock

# Change weight dynamically
echo "set server api_servers/api2 weight 20" | socat stdio /run/haproxy/admin.sock

# Show backend health
echo "show health" | socat stdio /run/haproxy/admin.sock

Error Pages

errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http

Anti-Patterns

Anti-Pattern	Problem	Solution
No health checks (Nginx passive only, never configured)	Dead backends receive traffic → client errors	Configure `proxy_next_upstream` in Nginx; use `option httpchk` in HAProxy
`ip_hash` with clients behind shared NAT / CDN	Uneven distribution — all clients from same office go to one server	Use `least_conn` or cookie-based stickiness instead of IP hash
No `proxy_http_version 1.1` + `Connection ""` with Nginx keepalive	Keepalive not actually enabled — new TCP connection per request	Always pair `proxy_http_version 1.1` with `proxy_set_header Connection ""`
Setting `timeout client 30s` for WebSocket connections	WebSocket connections dropped after 30 seconds idle	Use `timeout tunnel 1h` (HAProxy) or `proxy_read_timeout 0` (Nginx) for WS paths
Not logging `$upstream_addr` and `$upstream_response_time`	Can't diagnose which backend is slow	Add to Nginx log_format; use HAProxy `%b/%s` log variables
`maxconn` not tuned in HAProxy global	HAProxy queues or rejects connections under load	Set `maxconn` based on RAM: ~1 MB per 1000 connections; adjust per server too
No `proxy_buffering` tuning in Nginx	Slow clients cause upstream to wait	Keep `proxy_buffering on`; tune `proxy_buffers` for your response sizes
Nginx `upstream` without `zone` directive	Round-robin per-worker only, no true least_conn across workers	Always add `zone <name> 256k` to upstream block
HAProxy stats page exposed on public interface	Stats reveal server IPs, health, and allow admin actions	Bind stats to `127.0.0.1` or restrict with ACL `src 10.0.0.0/8`
TLS termination without `ssl_session_cache`	Full TLS handshake on every request → high CPU	Add `ssl_session_cache shared:SSL:10m` and `ssl_session_timeout 1d`

Troubleshooting

Symptom	Likely Cause	Fix
Uneven traffic distribution with `least_conn`	Single Nginx worker handles one backend; workers share if `zone` is set	Add `zone` directive to upstream block for shared state
Backend marked down immediately	Health check URL returns non-200 or times out	`curl http://10.0.1.10:3000/health` from load balancer host; adjust `rise`/`fall` thresholds
`502 Bad Gateway` on all requests	All backends down or proxy_pass pointing to wrong address	Check backend process; verify port; test `curl backend_ip:port` from LB
Session drops when scaling backends	No sticky sessions configured	Add `ip_hash` (Nginx) or `cookie` directive (HAProxy)
Keepalive not working (new TCP per request)	Missing `proxy_http_version 1.1` or `Connection ""` header	Add both headers; verify with `netstat -an
HAProxy shows "no server available"	All servers DOWN in health checks	`echo "show servers state"
Nginx returns 504 (gateway timeout)	Backend too slow; `proxy_read_timeout` too short	Increase `proxy_read_timeout`; investigate backend performance
SSL handshake errors	Cipher mismatch or TLS version too old	Check client TLS support; ensure `ssl_protocols TLSv1.2 TLSv1.3`
HAProxy rate higher than expected CPU	Too many health check connections	Increase `inter` interval: `inter 30s` for stable backends
X-Forwarded-For shows load balancer IP	`option forwardfor` not set (HAProxy) or `proxy_set_header X-Forwarded-For` missing (Nginx)	Add the respective directive; restart LB

Production Checklist

Nginx:

zone directive in all upstream blocks
proxy_http_version 1.1 + proxy_set_header Connection ""
proxy_next_upstream with appropriate error codes
X-Real-IP and X-Forwarded-For headers set
keepalive set on upstream block
$upstream_addr and $upstream_response_time in access log format
SSL session cache and modern TLS settings

HAProxy:

option httpchk on all backends with correct URL and Host header
inter, rise, fall tuned for your SLO
maxconn set globally and per server
Stats page on internal interface with auth
Runtime API socket configured
timeout tunnel set for WebSocket backends
Error files configured for all 4xx/5xx codes
Log format includes backend server name

load-balancer