Skip to content

Instantly share code, notes, and snippets.

@rfay

rfay/traefik-tcp-sni-demo-summary.md

Created August 21, 2025 19:23
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save rfay/0ef61e07a44a0ce956da72511eb74f6f to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save rfay/0ef61e07a44a0ce956da72511eb74f6f to your computer and use it in GitHub Desktop.
Download ZIP
DDEV TCP Routing with SNI - Complete Technical Demo Summary
Raw
traefik-tcp-sni-demo-summary.md

DDEV Traefik TCP Routing with SNI - Complete Demonstration and Analysis

Overview

This documents a complete demonstration of DDEV's Traefik router routing TCP traffic using SNI (Server Name Indication) based routing. We successfully demonstrated the concept using a simple NetCat listener, then explored MySQL limitations.

Successful NetCat Demonstration

Configuration Files

1. NetCat Listener Setup

File: .ddev/config.nc.yaml

web_extra_daemons:
  - name: "nc-listener"
    command: "while true; do nc -l -k -p 8888 | tee -a /tmp/nc.out; done"
    directory: /var/www/html

hooks:
  post-start:
    - exec: "nohup tail -f /tmp/nc.out &"

dbimage_extra_packages: [netcat-traditional, inetutils-ping, telnet, sudo]

2. Traefik TCP Routing Configuration

File: .ddev/traefik/config/tcp-nc.yaml

tcp:
  routers:
    d11-nc-tcp-sni:
      entrypoints:
        - nc-tcp
      rule: "HostSNI(`d11.ddev.site`)"
      service: "d11-nc-service"
      tls: {}
        
  services:
    d11-nc-service:
      loadBalancer:
        servers:
          - address: "ddev-d11-web:8888"

3. Global Static Configuration (Required)

File: ~/.ddev/traefik/static_config.netcat.yaml

entryPoints:
  nc-tcp:
    address: ":8888"

4. Router Port Exposure (Required)

File: ~/.ddev/router-compose.netcat.yaml

services:
  ddev-router:
    ports:
      - "8888:8888"

5. Debug Logging (Optional)

File: ~/.ddev/traefik/static_config.loglevel.yaml

log:
  level: DEBUG
accessLog:
  filters:
    statusCodes: {}

Manual Testing Procedures

Step 1: Verify Infrastructure

# Check that port 8888 is exposed by router
docker port ddev-router
# Should show: 8888/tcp -> 0.0.0.0:8888

# Verify TCP router is configured
curl -s http://localhost:10999/api/tcp/routers | jq '.[] | select(.name | contains("nc"))'
# Should show: "rule": "HostSNI(`d11.ddev.site`)"

# Check nc listener is running in web container
ddev exec "ps aux | grep nc"
# Should show: nc -l -k -p 8888

Step 2: Test Container-to-Container Communication (Baseline)

# From inside router container, test direct connection to web container
docker exec ddev-router sh -c "echo 'Direct test message' | nc ddev-d11-web 8888"

# Check if message appeared in web container
ddev exec "cat /tmp/nc.out"
# Should show: Direct test message

Step 3: Test Host-to-Router TCP Routing with SNI

# Send message through Traefik with TLS and SNI
printf "Hello via SNI TCP routing!\nTimestamp: $(date)\n" | \
  openssl s_client -connect localhost:8888 -servername d11.ddev.site -quiet 2>/dev/null

# Check received message in container
ddev exec "cat /tmp/nc.out"
# Should show: Hello via SNI TCP routing!
#              Timestamp: [timestamp]

Step 4: Verify SNI Requirement

# Test WITHOUT SNI (should fail or route incorrectly)
echo "Test without SNI" | openssl s_client -connect localhost:8888 -quiet 2>/dev/null

# Test with WRONG SNI (should fail)
echo "Test wrong SNI" | openssl s_client -connect localhost:8888 -servername wrong.example.com -quiet 2>/dev/null

NetCat Test Results ✅

  • Port 8888 exposed: docker port ddev-router shows 8888/tcp -> 0.0.0.0:8888
  • TCP router configured: Traefik API shows d11-nc-tcp-sni@file with correct SNI rule
  • TLS handshake succeeds: OpenSSL shows certificate verification and successful connection
  • SNI routing verified: Messages only appear with correct -servername d11.ddev.site
  • End-to-end communication: Messages sent from host appear in container /tmp/nc.out

This proves DDEV's Traefik can successfully route TCP traffic with SNI-based routing!

MySQL Demonstration (Infrastructure Works, Protocol Limitations)

MySQL Configuration

1. MySQL TCP Routing Configuration

File: .ddev/traefik/config/tcp-mysql.yaml

tcp:
  routers:
    d11-mysql-tcp-sni:
      entrypoints:
        - mysql-tcp
      rule: "HostSNI(`d11.ddev.site`)"
      service: "d11-mysql-service"
      tls: {}
  services:
    d11-mysql-service:
      loadBalancer:
        servers:
          - address: "ddev-d11-db:3306"

2. Additional Global Configuration

File: ~/.ddev/traefik/static_config.mysql.yaml

entryPoints:
  mysql-tcp:
    address: ":3306"

File: ~/.ddev/router-compose.mysql.yaml

services:
  ddev-router:
    ports:
      - "3306:3306"

MySQL Testing Results

✅ Infrastructure Works Correctly

# Port properly exposed
docker port ddev-router
# Shows: 3306/tcp -> 0.0.0.0:3306

# TCP router configured correctly
curl -s http://localhost:10999/api/tcp/routers | jq '.[] | select(.name | contains("mysql"))'
# Shows: "rule": "HostSNI(`d11.ddev.site`)"

# TLS handshake succeeds, MariaDB greeting received
openssl s_client -connect localhost:3306 -servername d11.ddev.site -quiet
# Shows: Certificate verification successful + "11.8.3-MariaDB-ubu2404-log"

# Traefik logs confirm proper routing
docker logs ddev-router | grep mysql
# Shows: "Adding TLS route for HostSNI(`d11.ddev.site`)"
#         "Handling TCP connection address=ddev-d11-db:3306"

❌ MySQL Client Protocol Failure

# MariaDB 10.6.23 client
mysql -h localhost -P 3306 -u db -pdb --ssl -e "SELECT 1;"
# ERROR 2013 (HY000): Lost connection to server at 'handshake: reading initial communication packet', system error: 35

# MariaDB 12.0.2 client (newer version - same issue)
mysql -h localhost -P 3306 -u db -pdb --ssl -e "SELECT 1;"
# ERROR 2013 (HY000): Lost connection to server at 'handshake: reading initial communication packet', system error: 35

# Direct connection still works fine
mysql -h 127.0.0.1 -P 59795 -u db -pdb -e "SELECT 1;"
# SUCCESS: Returns "1"

Root Cause Analysis: MySQL Protocol Incompatibility

The Problem

The MySQL client failures are due to a well-documented incompatibility between MySQL/MariaDB protocol and TLS-terminating proxies like Traefik.

Supporting Evidence from Traefik Community

GitHub Issues:

  1. MySQL client cannot connect to database when using SNI routing with TLS #10505
  2. Error connecting to MySQL when TLS enabled on TCP router #8803

Community Forum Discussions:

  1. Multiple Mysql communication through TCP with TLS based on SNI
  2. TCP: Can't connect to mysql container behind and through traefik
  3. Error connect mysql via TCP by traefik
  4. MySQL/Mariadb behind Traefik Reverse Proxy

Stack Overflow Reports:

  1. Lost connection to MySQL server at 'reading initial communication packet'
  2. MySQL connection issues with TLS termination proxies

Technical Explanation

  1. MySQL Protocol Expectation: MySQL/MariaDB clients expect to negotiate TLS directly with the database server
  2. TLS Termination Disruption: When Traefik terminates TLS and forwards plain TCP, it breaks the MySQL protocol handshake sequence
  3. Handshake Failure: The client begins MySQL protocol negotiation but fails during the initial communication packet exchange

Why OpenSSL Works But MySQL CLI Doesn't

  • OpenSSL: Simply establishes TLS connection and displays raw server response
  • MySQL CLI: Attempts complete protocol negotiation (authentication, capabilities) which fails after TLS termination

Configuration Evolution

We tested multiple approaches:

1. TLS Passthrough (First Attempt)

tls:
  passthrough: true

Issue: Required MySQL server to handle TLS+SNI (not supported by default MariaDB container)

2. Plain TCP with HostSNI(*) (Second Attempt)

rule: "HostSNI(`*`)"

Issue: No SNI differentiation - defeats the purpose, only one project could use port 3306

3. TLS Termination (Final Working Infrastructure)

rule: "HostSNI(`d11.ddev.site`)"
tls: {}

Result: Infrastructure works perfectly, MySQL protocol incompatibility prevents client connections

Requirements for Working Solution

Based on research, a working SNI-based MySQL routing solution would require:

  1. TLS Passthrough instead of termination
  2. MySQL Server SNI Support (available in MySQL 8.1.0+)
  3. Container TLS Configuration (certificates, SSL settings)
  4. Client SNI Support (modern MySQL/MariaDB clients)

Potential Path Forward

A viable approach to make this work would be to:

  1. Enable SSL on the database container: Configure MariaDB/MySQL to handle TLS connections natively
  2. Use DDEV CA certificates: Configure the database container to use DDEV's mkcert CA certificates
  3. Switch to TLS passthrough: Change Traefik configuration from TLS termination to passthrough 
    tcp:
  routers:
    d11-mysql-tcp-sni:
      rule: "HostSNI(`d11.ddev.site`)"
      tls:
        passthrough: true
  4. Test with modern MySQL clients: Use MySQL 8.1+ or MariaDB 10.6+ clients that support SNI

This approach would maintain end-to-end TLS encryption while allowing Traefik to route based on SNI, potentially resolving the protocol compatibility issues we encountered with TLS termination.

Demonstration Value

This work proves that:

  1. DDEV's Traefik configuration is highly flexible - supports custom TCP routing with SNI
  2. The routing infrastructure works correctly - all TCP/TLS/SNI components function as expected
  3. MySQL protocol limitations are the blocker - not a DDEV or Traefik configuration issue

Files Created During Testing

Configuration Files

  • /Users/rfay/workspace/d11/.ddev/traefik/config/tcp-mysql.yaml
  • /Users/rfay/.ddev/traefik/static_config.mysql.yaml
  • /Users/rfay/.ddev/router-compose.mysql.yaml
  • /Users/rfay/.ddev/traefik/static_config.loglevel.yaml

Testing Commands Used

# Verify routing configuration
curl -s http://localhost:10999/api/tcp/routers | jq '.[] | select(.name | contains("d11"))'

# Test TLS connection
openssl s_client -connect d11.ddev.site:3306 -servername d11.ddev.site -quiet

# Test basic TCP connectivity  
nc -v d11.ddev.site 3306

# Test MySQL client (fails)
mysql -h d11.ddev.site -P 3306 -u db -pdb --ssl

# Verify direct connection still works
mysql -h 127.0.0.1 -P 59795 -u db -pdb -e "SELECT 1;"

# Monitor router logs
docker logs ddev-router | tail -20

Summary

This demonstration proves that DDEV's Traefik router can successfully route TCP traffic with SNI-based routing:

✅ What Works (Proven with NetCat)

  • Complete TCP routing chain: Host → Traefik (TLS+SNI) → Container (plain TCP)
  • SNI-based routing: Different hostnames can route to different services on same port
  • TLS termination: Traefik properly terminates TLS and forwards plain TCP
  • Infrastructure flexibility: DDEV's Traefik configuration supports complex routing scenarios

❌ What Has Limitations (MySQL Protocol)

  • MySQL client compatibility: Protocol negotiation fails with TLS termination
  • Well-documented issue: Multiple Traefik community reports confirm this limitation
  • Not a DDEV problem: The routing works; MySQL protocol doesn't handle TLS termination well

Key Insight

The NetCat success proves the routing infrastructure is solid. MySQL failures are due to protocol-specific limitations with TLS termination, not DDEV/Traefik configuration issues.

Why Manual Global Configuration is Required

How DDEV's Auto-Configuration Currently Works

DDEV automatically handles HTTP_EXPOSE and HTTPS_EXPOSE through this process:

  1. determineRouterPorts() (router.go:351) scans all projects for HTTP_EXPOSE and HTTPS_EXPOSE environment variables
  2. ProcessExposePorts() (router.go:454) extracts external port numbers from these variables
  3. Traefik static template (traefik_static_config_template.yaml:46) auto-generates HTTP entrypoints: 
    {{ range $port := .RouterPorts }}
http-{{$port}}:
  address: ":{{ $port }}"
{{ end }}

The Limitation: HTTP-Only Auto-Generation

DDEV only auto-generates HTTP entrypoints - the template creates http-{{$port}} entrypoints, not TCP entrypoints.

This works perfectly for HTTP services because:

  • All ports are assumed to carry HTTP traffic
  • HTTP routers reference entrypoints like http-8080, http-443
  • The system is designed around HTTP/HTTPS protocols

Why TCP Services Need Manual Configuration

Our TCP routing requires:

  • TCP entrypoints (mysql-tcp, nc-tcp) instead of HTTP entrypoints
  • Different naming convention than auto-generated http-{{port}}
  • No existing environment variable that DDEV recognizes for TCP services

Potential Enhancement: TCP_EXPOSE Support

DDEV could theoretically support automatic TCP routing by:

  1. Adding TCP_EXPOSE environment variable detection in router port scanning
  2. Extending the static template to generate both HTTP and TCP entrypoints: 
    # Current: HTTP only
{{ range $port := .RouterPorts }}
http-{{$port}}:
  address: ":{{ $port }}"
{{ end }}

# Potential: TCP support  
{{ range $port := .TCPRouterPorts }}
tcp-{{$port}}:
  address: ":{{ $port }}"
{{ end }}
  3. TCP routing auto-generation similar to existing HTTP routing in traefik.go

This analysis explains why manual global configuration is currently necessary - DDEV's auto-configuration system is HTTP-centric and lacks built-in TCP entrypoint and routing support.

Example: MQTT Server Implementation (Untested Proposal)

Here's how one could implement an MQTT server using the same TCP routing approach demonstrated with NetCat:

Docker Compose Configuration

File: .ddev/docker-compose.mqtt.yaml

services:
  mqtt:
    image: eclipse-mosquitto:2.0
    container_name: ddev-${DDEV_SITENAME}-mqtt
    networks:
      - default
    volumes:
      - "./mqtt/mosquitto.conf:/mosquitto/config/mosquitto.conf"
    environment:
      - VIRTUAL_HOST=mqtt.${DDEV_SITENAME}.${DDEV_TLD}
    labels:
      com.ddev.site-name: ${DDEV_SITENAME}
      com.ddev.approot: $DDEV_APPROOT

MQTT Configuration

File: .ddev/mqtt/mosquitto.conf

listener 1883 0.0.0.0
allow_anonymous true

Traefik TCP Routing

File: .ddev/traefik/config/tcp-mqtt.yaml

tcp:
  routers:
    ${DDEV_SITENAME}-mqtt-tcp-sni:
      entrypoints:
        - mqtt-tcp
      rule: "HostSNI(`${DDEV_SITENAME}.ddev.site`)"
      service: "${DDEV_SITENAME}-mqtt-service"
      tls: {}
        
  services:
    ${DDEV_SITENAME}-mqtt-service:
      loadBalancer:
        servers:
          - address: "ddev-${DDEV_SITENAME}-mqtt:1883"

Global Static Configuration

File: ~/.ddev/traefik/static_config.mqtt.yaml

entryPoints:
  mqtt-tcp:
    address: ":1883"

Router Port Exposure

File: ~/.ddev/router-compose.mqtt.yaml

services:
  ddev-router:
    ports:
      - "1883:1883"

Testing MQTT over SNI

# Install MQTT client
brew install mosquitto

# Publish message through Traefik TCP routing with SNI
mosquitto_pub -h myproject.ddev.site -p 1883 -t "test/topic" -m "Hello MQTT via SNI!"

# Subscribe to messages
mosquitto_sub -h myproject.ddev.site -p 1883 -t "test/topic"

Expected Behavior

  • Client connects with TLS and SNI (myproject.ddev.site) to port 1883
  • Traefik terminates TLS and forwards plain TCP to MQTT container
  • MQTT broker receives standard MQTT protocol traffic
  • Multiple projects could each have their own MQTT broker on the same port, routed by SNI

This MQTT example demonstrates how the TCP routing pattern can be applied to any TCP-based service, providing SNI-based multi-tenancy through DDEV's Traefik router.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment