Web Server Tuning for Better Performance in Production

Starting with Baseline Measurements

Two months ago I inherited a cluster running on Ubuntu 22.04 that showed average response times of 650 milliseconds during peak hours. The first step was always to capture real numbers using ab and wrk against the live endpoints while monitoring with htop and vmstat. Without these baselines you cannot tell if changes actually move the needle. I logged worker connections, keepalive requests, and disk I/O over a full week before touching any configuration files.

Kernel Parameter Optimizations

Once baselines exist, move to the Linux kernel on Debian or Ubuntu hosts. Raise net.core.somaxconn to 65535 and enable tcp_tw_reuse while dropping tcp_fin_timeout to 15 seconds. On a set of Linode instances handling 15,000 concurrent sessions these adjustments reduced dropped packets by 38 percent. Apply the values with sysctl -p and verify with ss -s. Always test on a staging mirror first because some hosting providers enforce their own limits on file descriptors.

Fine-Tuning Nginx Configurations

Nginx 1.25 benefits from worker_processes set to auto and worker_connections raised to 8192 with multi_accept enabled. Disable modules that are not required and keep error_log at the warn level to limit disk writes. On a production site for a mid-sized retailer these changes lifted requests per second from 3,100 to 11,200. Set keepalive_timeout to 30 seconds and use upstream keepalive connections when proxying to application servers. Monitor access logs for spikes in 502 errors after each adjustment.

Adding Caching Layers with Varnish

Place Varnish 7.4 in front of Nginx for static assets and images. A grace period of 60 seconds combined with targeted purge ACLs cut origin server hits by 72 percent on a site serving four million monthly visitors. Time-to-first-byte fell below 45 milliseconds once the cache warmed. Review hit ratios weekly; rates below 85 percent indicate the VCL rules need refinement. I have seen similar gains on setups using Fastly as a secondary CDN layer for global edge delivery.

HAProxy for Effective Load Distribution

When traffic exceeds single-server capacity, HAProxy 2.8 in leastconn mode provides reliable balancing. Configure health checks every five seconds and enforce maxconn limits per backend. In a recent deployment across three AWS regions this setup maintained sub-200 millisecond response times even during a 300 percent traffic surge. Combine it with sticky sessions only when absolutely required to avoid uneven load. Log format should remain minimal to reduce overhead on the load balancer itself.

Integrating Monitoring Tools like Prometheus

Continuous visibility comes from exporting Nginx and HAProxy metrics into Prometheus with node_exporter on each host. Set alerts for connection queue lengths above 70 percent of the limit and cache hit rates dropping below target. Two weeks ago these dashboards caught a misconfigured keepalive setting before users noticed slowdowns. Grafana panels make trends obvious at a glance and help justify hardware upgrades when CPU saturation appears consistently.

Avoiding Common Configuration Errors

Over-tuning worker counts beyond available CPU cores creates context-switching waste. Leaving access logs at debug level fills disks quickly and increases I/O latency. I once saw a team enable gzip on already-compressed assets, which raised CPU usage without any bandwidth savings. Always validate changes with nginx -t and reload rather than restart. Keep a rollback plan documented for every production modification.

This is Allan Ali for Sylt.ing.