研究日期：2026-08-10 來源：YellowDuck.be 轉載 ｜ Medium 原文（Stackademic）（Yusuf Seyitoğlu，2026-07-12）
一位 SRE 喺同一 EKS cluster（12 個 microservices、每分鐘 3.5 萬請求、峰值 6.1 萬）用六星期輪換實測 Nginx F5 NIC、Caddy 2.11、Traefik 3.7 三個 reverse proxy，結論：「冇最好嘅 proxy，只有啱你營運模式嗰個」——Traefik 因為 operator time（每條新 route 4 分鐘 vs Nginx 23 分鐘）同 ingress-nginx annotation 兼容層贏咗 EKS 主力位置，Nginx 保留喺大檔案 streaming CDN 路徑，Caddy 用嚟做個人 staging cluster（「我拒絕星期六維護 Certbot」）。
| 指標 | Nginx (F5 NIC) | Caddy 2.11 | Traefik 3.7 |
|---|---|---|---|
| 吞吐量 | ~46,000 req/s（最高） | ~41,000 req/s（-11%） | ~36,000 req/s（最低，但仍 3x 峰值餘量） |
| p99 proxy overhead | ~0.9ms | ~1.2ms | ~1.6ms（burst 下） |
| Idle 記憶體 | ~35 MB | ~28 MB（最低） | ~48 MB（最高） |
| 10k connections 記憶體 | ~120 MB | ~95 MB | ~135 MB |
| 新 route operator time | ~23-25 分鐘（YAML + annotation review） | ~8-9 分鐘（Caddyfile / JSON API） | ~4 分鐘（label，慣例建立後） |
| 每月成本（3 replicas 計） | ~$186 | ~$142 | ~$198 |
| Cert 失敗 | 0 | 1（DNS blip，自癒 11 分鐘） | 2（dns01 solver 配置錯） |
-
Traefik 贏喺 operator time 而唔係速度：每條新 service route，Nginx 要 ~25 分鐘 YAML surgery，Traefik 慣例建立後一個 label 搞掂（~4 分鐘）。客戶每月 route churn 慳 ~6 小時，contractor rate 計 ~$900/月——呢個數字先係 close the deal 嗰個。「Throughput vs Operator Time — Pick Your Poison」：你冇用到嗰 11% throughput，但你每日都要用 operator hour。
-
ingress-nginx EOL（2026 年 3 月）係成篇文嘅背景：客戶用咗四年 ingress-nginx，EOL 後要遷移。Traefik 嘅 ingress-nginx annotation 兼容層（rewrite-target、proxy-body-size 等）令遷移慳咗兩星期——呢個係「RKE2 預設 Traefik」之外，Traefik 最大嘅實戰優勢。
-
Incident 教訓：proxy 冇壞，config 冇測：Cutover 期間一個 malformed annotation（
proxy-read-timeout: "30"秒，export service 需要 300）令大型 CSV export 喺 30 秒死——502 Bad Gateway，error rate 0% → 8% 三分鐘。Support 賴「新 proxy」、Engineering 賴「export service 慢」——兩個都錯。真正問題係 config 冇喺測試環境測過大 timeout。 -
x-Forwarded-For 係隱形殺手：Corporat L4 LB 前面，Traefik 要設
forwardedHeaders.trustedIPs，否則 client IP 錯——mobile clients 入錯 rate-limit bucket 20 分鐘。作者：「That constraint eliminated more 'simple' configs than any benchmark did.」 -
kernel 先係最後兇手：Node 上
nf_conntracktable 92% 滿（long-lived downloads 未預留 connection tracking budget）——唔係 Traefik bug。「Half of 'proxy broke' is timeout misconfig」+ conntrack exhaustion，呢個先係 2 AM 真正要識嘅嘢。 -
三個 proxy 嘅「性格」：Nginx = 效能怪獸 + 十年 SRE 傷疤 + reload 失敗會保留舊 config（「annoying in CI, lifesaving in prod」）；Caddy = 自動 HTTPS 係真產品（兩行 config cert 出現）、HTTP/3 預設開、ECH 支援；Traefik = 動態發現係魔法（「until it isn't」——一個 mislabeled pod 攞咗 11 分鐘 public traffic）。
1. 呢篇係「benchmark 唔等於 production」嘅最佳示範 好多 proxy 比較文淨係跑 wrk 睇 req/s，但呢篇用六星期、真實 12 服務、真實流量形態（REST JSON + WebSocket + 15-40MB CSV export）、真實 TLS 要求（47 個 cert、Let's Encrypt rate limit 餘量）做測試。結果：速度排第三嘅 Traefik 贏咗，因為佢喺「你每日都要做嘅嘢」（開新 route、遷移 annotation）上面最慳時間。呢個係工程決策嘅成熟形態：benchmark 指標只係 input，operator model 先係決策框架。
2. Operator time 嘅 5.75x 差距係全篇最值錢嘅數字 Nginx 23 分鐘 vs Traefik 4 分鐘——唔係「Traefik 好用啲」咁簡單，而係長期成本結構：每次開新 service 都係一次 19 分鐘嘅差異，乘上每月 route churn，累積嘅就係個個月慳 $900。呢個視角（把「配置時間」當成 recurring cost 而唔係 one-time setup）先係 infra 決策入面最常被忽略嘅。
3. 「Proxy 唔製造 outage，佢揭露你冇測試嘅 config」 Incident 嘅 root cause 唔係 Traefik——係「一個 annotation 喺非 production 冇測過」。但真正嘅 insight 係 502 排查樹：由 status code 推斷層（502 = upstream 關閉或 timeout），到確認 upstream p99 係 42 秒，再到 kernel conntrack 92%——呢個先係「2 AM 你靠嘅係乜」。作者 link 咗自己嘅 production debugging library 做延伸閱讀（明顯嘅 self-promo，但內容本身有料）。
4. 揀 proxy 嘅決策樹（作者版）
- Nginx：超大流量專用 edge node、GB-scale 檔案 streaming、團隊有十年 Nginx 經驗/F5 合約、想喺 ingress-nginx EOL 後保持 vendor continuity
- Caddy：細 cluster 或 VM fleet、TLS 要唔使煩、開發速度 > 原始 throughput、HTTP/3 + ECH 重要、<10 個穩定 service
- Traefik：K8s-native + 頻繁 service churn、2026 遷移 off ingress-nginx（annotation 兼容係真）、RKE2/Rancher 環境、要 dynamic upstreams 唔想 reload 全世界
5. 誠實嘅缺點都講咗
- Traefik：idle 記憶體最高（~48MB，成本 $198/月 vs Caddy $142）、mislabeled pod 會自動攞 public traffic（「自動發現自動 misrouting」）、WebSocket 要正確 entrypoint middleware 否則 45 秒斷線
- Caddy：吞吐 -11%、K8s 入面 cert storage 要自己搞（唔係 zero config at scale）
- Nginx：operator time 最貴、YAML surgery、annotation archaeology
6. 對自架 infra 嘅實用建議 作者嘅「What You Should Do This Week」清單好實用：① 仲用緊 ingress-nginx 嘅人要 inventory annotations + 喺 non-prod 測 Traefik 兼容層；② 每個 502 都要 log X-Forwarded-For + upstream status；③ 懷疑 ingress 之前先 check nf_conntrack；④ 寫 rate-limit test 同時打兩個 IP path（catch corporate LB misconfig）；⑤ 用 Traefik 前寫好 label/schema 慣例文件。
- 雖然 Jena 家冇 K8s cluster，但決策框架（benchmark vs operator model、把配置時間當 recurring cost）對任何 infra/工具選擇都適用
- 「Proxy 揭露你冇測試嘅 config」——同我哋 research pipeline 嘅原則一致：工具唔會製造錯誤，佢揭露你冇驗證嘅假設
- 三款 proxy 性格對照（效能怪獸 / 自動化天才 / 動態魔法師）係好清晰嘅 mental model
- https://www.yellowduck.be/posts/nginx-vs-caddy-vs-traefik-in-2026-i-ran-all-three-under-real-production-load（轉載頁）
- https://blog.stackademic.com/nginx-vs-caddy-vs-traefik-in-2026-i-ran-all-three-under-real-production-load-64e412f4062b（Medium 原文，Stackademic，Yusuf Seyitoğlu）
點擊展開完整原文（Medium，Yusuf Seyitoğlu）
Nginx vs Caddy vs Traefik in 2026 — I Ran All Three Under Real Production Load
Same Kubernetes cluster, same microservices, same TLS requirements. One proxy broke during ingress-nginx migration week. Guess which.
Yusuf Seyitoğlu Follow 7 min read · Jul 12, 2026
I've terminated more TLS than I care to admit. Reverse proxies are the part of the stack everyone forgets until certificates expire, upstreams go silent, or — in March 2026 — your ingress controller hits end-of-life and compliance asks what your migration plan is.
A logistics client ran ingress-nginx on EKS for four years. Fine until it wasn't supported. They asked me to compare Nginx (F5 NIC), Caddy, and Traefik on the same cluster before picking a path forward. Same twelve microservices. Same cert issuer. Same load tests. I ran each as the edge for six weeks.
RKE2 v1.36 making Traefik the default ingress is not abstract news if you operate Rancher clusters. It is a migration ticket with a deadline.
The Test Bed: Production Shape, Controlled Variables
Cluster: EKS, 12 services, ~35,000 requests/minute average, ~61,000 peak during dispatch windows. Mix of REST JSON, WebSocket tracking, and one large CSV export (15–40 MB responses).
Each proxy setup:
- Automatic TLS (Let's Encrypt / ACME)
- HTTP/2 and HTTP/3 where supported
- Rate limiting on public endpoints
- Prometheus metrics scraped the same way
- Same pod resource requests: 2 CPU / 2Gi per ingress controller replica
I tracked: p50/p99 proxy latency, memory at idle and at 10k connections, config reload behavior, and operator time per new service route.
Six-week rotation schedule:
- Weeks 1–2: Nginx F5 NIC baseline, cert-manager ClusterIssuer validation, wrk suite calibration
- Weeks 3–4: Caddy on a parallel ingress class, HTTP/3 A/B on internal clients
- Weeks 5–6: Traefik with ingress-nginx annotation bridge, migration dry-run on two non-critical services
- Week 6 cutover: 60/40 Traefik/legacy split, incident on day three (documented below)
Corporate L4 load balancer in front: AWS NLB, three AZs, ~2.1 million connections/day at peak. Every proxy had to honor X-Forwarded-For correctly or rate limits and audit logs lied. That constraint eliminated more "simple" configs than any benchmark did.
Operator time log (median over six weeks, 14 route changes): Nginx 23 minutes, Caddy 9 minutes, Traefik 4 minutes once label conventions landed — 38 minutes on the first Traefik route before we wrote the schema doc.
Throughput vs Operator Time — Pick Your Poison
Nginx (F5 NIC ingress on Kubernetes):
- Proxy throughput: ~46,000 req/s in our wrk suite (2 KB JSON upstream) — highest of the three
- p99 proxy overhead: ~0.9ms at median load
- Idle memory: ~35 MB per controller pod; ~120 MB at 10k connections
- New service route: manual Ingress YAML + annotation review — ~25 minutes including PR
- Strength: raw performance, massive community knowledge, every senior SRE has scars here
Caddy 2.11 (standalone ingress on VMs + DaemonSet edge):
- Proxy throughput: ~41,000 req/s — ~11% behind Nginx on pure JSON proxying
- p99 overhead: ~1.2ms — TLS termination gap narrowed; automatic HTTPS is not a gimmick
- Idle memory: ~28 MB; ~95 MB at 10k connections
- New route: 12-line Caddyfile block or JSON API push — ~8 minutes for a standard service
- HTTP/3 on by default. ECH support mattered for a client with aggressive TLS inspection policies.
Traefik 3.7 (Kubernetes CRD + IngressRoute):
- Proxy throughput: ~36,000 req/s — lowest raw number, still 3x headroom over peak traffic
- p99 overhead: ~1.6ms under burst; token-bucket rate limiting smoother than Nginx leaky bucket for our spike pattern
- Idle memory: ~48 MB; ~135 MB at 10k connections
- New route: label on Deployment — often zero manual steps after conventions exist
- ingress-nginx annotation compatibility layer saved two weeks on migration — rewrite-target, proxy-body-size, most of what they already had
TLS, Reloads, and the Config Formats You'll Maintain at 2 AM
Nginx: Cert rotation via cert-manager worked. Config reload via nginx -s reload is battle-tested. When we fat-fingered a proxy_pass typo, Nginx refused reload and kept the old config — annoying in CI, lifesaving in prod.
Caddy: Automatic HTTPS is the real product. Two lines, cert appears, HTTP/3 works. JSON API reload with zero downtime felt like cheating on a homelab. In Kubernetes we still needed to wire cert storage — not hard, but not literally zero config at scale.
Traefik: Watching Docker/K8s events and building routes dynamically is magic until it isn't — a mislabeled pod got public traffic for eleven minutes before we caught it in access logs. Convention documentation became mandatory: label schema, entrypoint names, middleware chains.
WebSocket tracking for the fleet GPS service: all three handled it. Nginx needed explicit Upgrade headers. Caddy worked out of the box. Traefik needed the right entrypoint middleware — one wrong toggle and connections dropped every 45 seconds.
Certificate lifecycle numbers: 47 active certs across subdomains, Let's Encrypt rate limit headroom mattered during migration. cert-manager renewal failures: zero on Nginx, one transient DNS propagation blip on Caddy (self-healed in 11 minutes), two on Traefik when a misconfigured dns01 solver stalled a wildcard — caught by Prometheus alert certmanager_certificate_expiration_timestamp_seconds < 7 days.
Monthly cost for three ingress controller replicas (compute only): Nginx NIC ~$186, Caddy ~$142, Traefik ~$198 (higher idle RSS). None of these move the CFO's needle. Operator hours do — we estimated ~6 hours/month saved on route churn with Traefik versus Nginx YAML surgery, roughly $900/month at loaded contractor rates. That's the number that closed the deal.
The Migration Week Incident
Cutover Tuesday, 09:40 local — dispatch managers start their morning export ritual. Traefik at 60% traffic, ingress-nginx still handling legacy annotations on two services. A deployment rolled out with a malformed Ingress annotation — nginx.ingress.kubernetes.io/proxy-read-timeout set to "30" (seconds) on an export service that needed 300.
Nginx ingress honored the low timeout. Traefik's translated middleware did too — correctly, per config. Large CSV exports died at 30 seconds flat. Users saw 502 Bad Gateway. Error rate: 0% → 8% in three minutes.
Support blamed "the new proxy." Engineering blamed "the export service got slower." Both wrong.
I started with → Production Incident War Room — The Step-by-Step Response Playbook — the diagnosis tree for exactly this kind of "which layer is actually broken" triage, status codes included. 502 from the proxy: bad gateway, upstream closed or timed out. We extended the timeout annotation, confirmed upstream p99 was 42 seconds for huge exports, set 300s with a async export fallback on the roadmap.
Secondary issue: Traefik access logs went to stdout but client IP was wrong behind the corporate L4 load balancer until we set forwardedHeaders.trustedIPs. Mobile clients hit the wrong rate-limit bucket for 20 minutes before we noticed.
For the node-level debugging — journalctl, connection tracking, file descriptor limits — → Linux War Room: 12 Fix Patterns for Disk-Full, OOM Killer & SSH Lockouts at 3 AM has the diagnosis tree I use when the proxy looks guilty but the kernel is lying. Turned out nf_conntrack table was 92% full on two nodes from the export spike. Not a Traefik bug. A connection tracking budget we had never sized for long-lived downloads.
When I Would Choose Nginx
- You terminate massive traffic on dedicated edge nodes and need every req/s
- Large-file streaming (GB-scale) where Nginx's buffer tuning still wins
- Team has a decade of Nginx config and F5 support contract
- You are standardizing on F5 NIC post ingress-nginx EOL and want vendor continuity
When I Would Choose Caddy
- Small cluster or VM fleet — TLS just works with minimal config
- Developer velocity over raw throughput; HTTP/3 and ECH matter
- You hate maintaining Certbot sidecars and annotation archaeology
- <10 services, mostly stable routes, homelab-to-production pipeline
When I Would Choose Traefik
- Kubernetes-native stack with frequent service churn — labels beat YAML surgery
- Migrating off ingress-nginx in 2026 — annotation compatibility is real
- RKE2 / Rancher shops where Traefik is already the default path
- You want dynamic upstreams without reloading the world
Verdict
There is no "best" proxy — only the one that matches your operational model.
For this client's EKS fleet with weekly deploys and a migration deadline, Traefik won — not because it was fastest, but because operator time and ingress-nginx compatibility beat 11% throughput we weren't using. We kept Nginx on a dedicated edge VM for the CSV export CDN path where buffer tuning and raw streaming matter. Caddy runs my personal staging cluster because I refuse to maintain Certbot on a Saturday.
ingress-nginx EOL in March 2026 is not a reason to panic-migrate to the shiniest logo. It is a reason to test Traefik's annotation bridge on your real Ingress objects, measure operator time per route, and keep Nginx where raw throughput still pays rent.
Pick the proxy you can debug at 2 AM. Then read the HTTP status codes and kernel limits that make every proxy look broken.
We didn't stop at the bake-off. After Traefik won for the EKS fleet, I still migrated one low-risk internal service to Caddy on a VM edge — mostly to kill a Certbot timer that kept silently dying. The surprise wasn't automatic HTTPS. It was the reload model.
→ We Moved One Service From nginx to Caddy. Deleting the Certbot Timer Wasn't the Win — the Reload Model Was
What You Should Do This Week
- If you still run ingress-nginx, inventory annotations and test Traefik's compatibility layer on a non-prod namespace. March 2026 EOL is not a surprise anymore — it's a calendar event.
- Log X-Forwarded-For and upstream status on every 502. Half of "proxy broke" is timeout misconfig.
- Check nf_conntrack usage before blaming the ingress controller for connection drops on large downloads.
- Write a rate-limit test that hits the same endpoint from two IP paths — corporate LB misconfig shows up before customers complain.
- Document your label/schema conventions before Traefik auto-discovery becomes auto-misrouting.
Proxies do not create outages. They reveal the config you did not test. Make sure you know what's on the other side.
Your reverse proxy is the door everyone walks through. Make sure you know what's on the other side.
How fast can you tell a gateway timeout from a conntrack table exhaustion? That's the real skill — not memorizing status codes, but recognizing failure patterns before the postmortem.
That's exactly why I started building The Production Engineering Library.
Instead of isolated tutorials, it's a growing collection of practical books covering production debugging, incident response, Kubernetes, system design, distributed systems, backend architecture, performance engineering, databases, APIs, Linux, Docker, Git, and the engineering decisions that only become visible in production.
If you enjoy articles that compare real production trade-offs instead of repeating documentation, you'll probably find something valuable inside.
→ The Production Engineering Library