How a Hotel WiFi Taught Me That Happy Eyeballs Can't See Everything

Or: The Empty Error String That Led Me Down a Rabbit Hole of Broken Networks

TL;DR: httpx AsyncClient was failing with empty error messages. After hours of debugging, I discovered a hotel WiFi firewall that accepts IPv6 TCP connections but kills them during TLS handshake - an edge case so specific it defeats Happy Eyeballs and every other protection mechanism.

It started with the most frustrating error message I've ever seen:

{"detail":"An error occurred while requesting TikAPI: "}

That's not a typo. The error message was literally an empty string. It's like calling 911 and having someone whisper "Help" and then hang up.

"That's weird," my colleague told me over Slack. "Let me check on my machine."

Five minutes later: "Works fine for me!"

What followed was a journey through broken networks, false assumptions, and the discovery of an edge case so specific that it defeats every protection mechanism modern networking has to offer. This is that story.

Act I: The Scene of the Crime

Our TikAPI MCP server was supposed to be simple. Make request to TikAPI, return data. What could go wrong?

# The offending code
try:
    response = await client.get("https://api.tikapi.io/...")
except httpx.RequestError as e:
    raise TikAPIError(f"An error occurred while requesting TikAPI: {e}")

That {e} was supposed to tell us what went wrong. Instead, it gave us the digital equivalent of a shoulder shrug.

First debugging instinct: bypass all the fancy code and use curl:

# Test the server
$ curl -X POST http://127.0.0.1:8000/resources/read \
    -H "Content-Type: application/json" \
    -d '{"uri": "/videos/7003402629929913605"}'

{"detail":"An error occurred while requesting TikAPI: "}  # 😭

# Test TikAPI directly
$ curl "https://api.tikapi.io/public/video?id=7003402629929913605" \
    -H "X-API-KEY: YOUR_TIKAPI_KEY_HERE"

{"itemInfo":{"itemStruct":{"id":"7003402629929913605"...}}}  # 🎉

So TikAPI works fine, but our Python server can't reach it. Time to dig deeper.

Act II: The HTTP Client Lineup

I asked Claude Code to help debug this systematically. It immediately wrote a script to test every major Python HTTP client, and the results were baffling:

📊 HTTP Client Test Results:
========================================
urllib          ✅ SUCCESS
requests        ✅ SUCCESS  
httpx_sync      ✅ SUCCESS
httpx_async     ❌ FAILED - ConnectError: 
aiohttp         ✅ SUCCESS

Only httpx AsyncClient failed! The same library works in sync mode but fails in async? This is like having a car that only breaks down when you use cruise control.

Act III: Happy Eyeballs Should Save Us... But It Doesn't

Claude Code hypothesized that httpx might be missing Happy Eyeballs support. Happy Eyeballs (RFC 8305) is supposed to try both IPv4 and IPv6 in parallel and use whichever works.

It wrote a test script to verify:

# Test 1: Default behavior - ❌ Fails with empty error
# Test 2: Force IPv4 with local_address="0.0.0.0" - ✅ Works!

The results confirmed Happy Eyeballs EXISTS but doesn't help:

=== Test 1: Default (Happy Eyeballs enabled) ===
❌ Failed: ConnectError: (empty)

=== Test 2: IPv4 forced ===
✅ Success! Status: 404

So Happy Eyeballs is present but failing. Claude Code then added socket monitoring to see why:

# Monitor what's actually happening
class MonitoredSocket(socket.socket):
    def connect(self, address):
        print(f"🔌 Connecting to: {address}")
        print(f"   Address family: {self.family.name}")
        return super().connect(address)

socket.socket = MonitoredSocket

The output revealed the crucial detail:

httpx.AsyncClient:
🔌 Connecting to: ('2606:4700:3034::ac43:cbb7', 443, 0, 0)
   Address family: AF_INET6 (IPv6)
❌ ConnectError (empty message)

It's using IPv6! But curl works... let me test:

$ curl -4 https://api.tikapi.io/public/check  # Force IPv4
✅ {"status": "success"}

$ curl -6 https://api.tikapi.io/public/check  # Force IPv6
curl: (35) Recv failure: Connection reset by peer

Act IV: Blaming TikAPI... Until The Pattern Emerged

At this point I was convinced: TikAPI is broken! They publish IPv6 addresses in DNS but don't properly support IPv6. How incompetent!

But Claude Code suggested testing multiple services systematically. It wrote a script that would test both TCP and TLS separately:

Claude Code's systematic TCP vs TLS testing across multiple services revealed something shocking:

Testing: TikAPI (api.tikapi.io:443)
IPv6: 2606:4700:3037::6815:2525
  ✅ TCP: Connected
  ❌ TLS: ConnectionResetError

Testing: Google (www.google.com:443)
IPv6: 2607:f8b0:4005:803::2004
  ✅ TCP: Connected
  ❌ TLS: ConnectionResetError

Testing: Cloudflare (cloudflare.com:443)
IPv6: 2606:4700::6810:84e5
  ✅ TCP: Connected
  ❌ TLS: ConnectionResetError

Testing: GitHub (api.github.com:443)
IPv6: ::ffff:140.82.116.5
  ✅ TCP: Connected
  ✅ TLS: Handshake successful

Testing: OpenAI (api.openai.com:443)
IPv6: ::ffff:172.66.0.243
  ✅ TCP: Connected
  ✅ TLS: Handshake successful

Wait. Google and Cloudflare too? That can't be right. Unless...

Notice the pattern? GitHub and OpenAI work fine - but they're using IPv6-mapped IPv4 addresses (::ffff:x.x.x.x) - essentially IPv4 wrapped in IPv6 format. Only "real" IPv6 addresses fail!

Act V: The Hotel WiFi Plot Twist

Later that evening, back at home, I ran the same test:

📋 SUMMARY TABLE
======================================================================
Service         IPv4 Status          IPv6 Status
----------------------------------------------------------------------
TikAPI          ✅ Works             ✅ Works (real IPv6)
Google          ✅ Works             ✅ Works (real IPv6)
Cloudflare      ✅ Works             ✅ Works (real IPv6)
GitHub          ✅ Works             ✅ Works (IPv6-mapped IPv4)
OpenAI          ✅ Works             ✅ Works (IPv6-mapped IPv4)

Everything works at home. The realization hit me like a ton of bricks:

It wasn't TikAPI. It was the hotel WiFi all along.

Act VI: Why This Defeats Everything

Let me show you exactly why this specific failure mode is so insidious:

The hotel's firewall/router is broken in the most specific way possible:

It allows IPv6 TCP connections through ✅
Happy Eyeballs sees this and thinks "Great! IPv6 works!"
It commits to the IPv6 connection
The firewall kills the connection when it sees TLS ClientHello ❌
Too late! Happy Eyeballs already made its choice

This defeats EVERY protection mechanism:

Happy Eyeballs: "TCP worked, my job is done!"
Connection retry: "We had a successful connection, no need to retry"
Dual-stack fallback: "We already committed to IPv6"
Application layer: "WTF just happened??"

What We've Learned

Error Messages Are Sacred: That empty string cost hours. Always preserve the original error context.
Happy Eyeballs Can't See Everything: It only handles TCP failures, not TLS failures. This edge case defeats it completely.
The Most Dangerous Bugs Accept Your Connection Then Kill It: These are the hardest to debug and defend against.
AI Assistants Debug Differently: Where I might test 2-3 cases manually, Claude Code wrote scripts to test systematically. The pattern only emerged from testing 5+ services - something I probably wouldn't have done manually.

The Moral of the Story

Was it worth spending hours to discover that a hotel WiFi firewall was configured wrong? As engineers, we tell ourselves it's about "understanding the root cause." But really, sometimes we just can't help ourselves. We see a mystery and we have to solve it.

And sometimes, that obsessive curiosity reveals edge cases so specific that they defeat every protection mechanism we've built into modern networking. The most sophisticated algorithms can't save you from a firewall that says "yes" to your connection and then immediately stabs it in the back.

Appendix: Debugging Scripts

The actual scripts Claude Code wrote during this investigation (in order): • Sync vs Async Comparison • Alternative Clients Test • HTTP Client Detective • Happy Eyeballs Tester • Socket Spy • Network Capture Analysis • DNS Resolution Deep Dive • DNS Environment Scanner • IPv6 Pattern Test

odysseus0/gist_blog_post.md