Skip to content

Instantly share code, notes, and snippets.

@odysseus0

odysseus0/The Empty HTTPX Error String That Cost Me 3 Hours.md

Created June 20, 2025 02:00
Show Gist options
  • Save odysseus0/c7ba9cee4e45f6896181674aaeabf7a7 to your computer and use it in GitHub Desktop.
Save odysseus0/c7ba9cee4e45f6896181674aaeabf7a7 to your computer and use it in GitHub Desktop.
Download ZIP
The Empty HTTPX Error String That Cost Me 3 Hours (Spoiler: It Was DNS) - A debugging journey through Python's network stack
Raw
The Empty HTTPX Error String That Cost Me 3 Hours.md

The Empty HTTPX Error String That Cost Me 3 Hours (Spoiler: It Was DNS)

Or: How I Learned to Stop Worrying and Blame DNS

TL;DR: httpx AsyncClient was failing with empty error messages while every other HTTP client worked. After 3 hours debugging through Python's entire network stack, I discovered it was trying IPv6 (TikAPI's DNS returns IPv6 but refuses IPv6 connections). The fix? One line: httpx.AsyncHTTPTransport(local_address="0.0.0.0"). Yes, it was DNS.

It started with the most frustrating error message I've ever seen:

{"detail":"An error occurred while requesting TikAPI: "}

That's not a typo. The error message was literally an empty string. It's like calling 911 and having someone whisper "Help" and then hang up.

"That's weird," she told me over Slack. "Let me check on my machine."

Five minutes later, she sent back the six words that haunt every developer's dreams:

"That's strange, it works on my machine!"

But on mine? Same empty error.

What followed was a three-hour journey through five layers of the network stack that made me question everything I thought I knew about debugging. This is that story.

Act I: The Scene of the Crime

Our TikAPI MCP server was supposed to be simple. Make request to TikAPI, return data. What could go wrong?

# The offending code
try:
    response = await client.get("https://api.tikapi.io/...")
except httpx.RequestError as e:
    raise TikAPIError(f"An error occurred while requesting TikAPI: {e}")

That {e} was supposed to tell us what went wrong. Instead, it gave us the digital equivalent of a shoulder shrug.

First debugging instinct: bypass all the fancy code and use good old curl:

# Test the server
$ curl -X POST http://127.0.0.1:8000/resources/read \
    -H "Content-Type: application/json" \
    -d '{"uri": "/videos/7003402629929913605"}'

{"detail":"An error occurred while requesting TikAPI: "}  # 😭

# Test TikAPI directly
$ curl "https://api.tikapi.io/public/video?id=7003402629929913605" \
    -H "X-API-KEY: [YOUR_API_KEY]"

{"itemInfo":{"itemStruct":{"id":"7003402629929913605"...}}}  # 🎉

So TikAPI works fine, but our Python server can't reach it. The plot thickens.

Act II: The Lineup

Time to interrogate every HTTP library in Python. (Spoiler: This is where I found the solution, but refused to accept it.)

# The Great HTTP Client Shootout of 2025
# Testing urllib, httpx sync/async against TikAPI

async def test_all_clients():
    url = "https://api.tikapi.io/public/check"
    
    # Test each client and capture results
    results = {
        'urllib': test_urllib(url),           # Standard library
        'httpx_sync': test_httpx_sync(url),   # Sync mode
        'httpx_async': await test_httpx_async(url)  # Our problem child
    }
    
    return results

Running this revealed something interesting (I tested without an API key to see which clients even reach the server):

📊 HTTP Client Test Results:
========================================
urllib          ✅ WORKS!
httpx_sync      ✅ WORKS!
httpx_async     ❌ Failed: ConnectError:

Wait, what? The same library works in sync mode but fails in async mode? This is like having a car that only breaks down when you use cruise control.

I couldn't let it go. I had to know WHY httpx async was failing. What followed was a 3-hour rabbit hole, driven purely by engineering curiosity.

Act III: Going Deeper Than We Should (Because We Can't Help Ourselves)

At this point, I did what any reasonable developer would do: I started monkey-patching Python's socket library to spy on network calls.

# socket_spy.py - Monkey-patch socket.connect to see what's really happening
class MonitoredSocket(socket.socket):
    def connect(self, address):
        print(f"🔌 Connecting to: {address}")
        print(f"   Address family: {self.family.name}")
        return super().connect(address)

# Replace the real socket with our spy
socket.socket = MonitoredSocket

The results were illuminating:

httpx.Client (sync):
🔌 Connecting to: ('104.21.37.37', 443)
   Address family: AF_INET (IPv4)
✅ Success!

httpx.AsyncClient:
🔌 Connecting to: ('2606:4700:3034::ac43:cbb7', 443, 0, 0)
   Address family: AF_INET6 (IPv6)
❌ ConnectError (empty message)

The async client was using IPv6! But why? And more importantly, why was it failing?

Quick test:

$ curl -4 https://api.tikapi.io/public/check  # Force IPv4
✅ {"status": "success"}

$ curl -6 https://api.tikapi.io/public/check  # Force IPv6
curl: (35) Recv failure: Connection reset by peer

Act IV: The Plot Twist

I had found the smoking gun: TikAPI's DNS returns IPv6 addresses and accepts TCP connections, but then fails during the TLS handshake with "Connection reset by peer". It's like inviting someone in, shaking hands, and then slamming the door mid-handshake.

But wait... it worked on my colleague's machine. Same office, same network, same code. How?

Time for environmental forensics:

# dns_detective.py - What does your computer see?
def check_dns_resolution(hostname):
    addresses = socket.getaddrinfo(hostname, 443, socket.AF_UNSPEC)
    
    ipv4_addrs = [addr[4][0] for addr in addresses if addr[0] == socket.AF_INET]
    ipv6_addrs = [addr[4][0] for addr in addresses if addr[0] == socket.AF_INET6]
    
    print(f"🔍 DNS Resolution for {hostname}:")
    print(f"IPv4 addresses: {list(set(ipv4_addrs))}")
    print(f"IPv6 addresses: {list(set(ipv6_addrs))}")

My machine:

🔍 DNS Resolution for api.tikapi.io:
IPv4 addresses: ['104.21.37.37', '172.67.203.183']
IPv6 addresses: ['2606:4700:3037::6815:2525', '2606:4700:3034::ac43:cbb7']

Her machine:

🔍 DNS Resolution for api.tikapi.io:
IPv4 addresses: ['104.21.37.37', '172.67.203.183']
IPv6 addresses: []  # 🤯

Same network. Different DNS results.

It turns out her machine wasn't getting IPv6 addresses from DNS at all! No IPv6 addresses = no IPv6 connection attempts = no problem.

Act V: The Grand Unifying Theory

Here's what was happening in this perfect storm:

┌──────────────────┐
│   Your Code      │ "Please connect to api.tikapi.io"
└────────┬─────────┘
         │
┌────────▼─────────┐
│       DNS        │ "Here's IPv4 and IPv6 addresses!"
└────────┬─────────┘     (but not on everyone's machine)
         │
┌────────▼─────────┐
│ httpx AsyncClient│ "I'll try IPv6 first!"
└────────┬─────────┘     (sync client chooses differently)
         │
┌────────▼─────────┐
│     TikAPI       │ "IPv6? Connection reset!"
└────────┬─────────┘     (despite DNS returning IPv6)
         │
┌────────▼─────────┐
│ httpx Exception  │ ConnectError('')
└────────┬─────────┘     (empty string representation)
         │
┌────────▼─────────┐
│   Your Error     │ "An error occurred: "
└──────────────────┘     (helpful as a chocolate teapot)

The Fix

After three hours of debugging across the entire network stack, diving through socket operations, DNS resolution, and IPv6 connectivity issues, I finally understood the problem:

  • TikAPI's DNS returns IPv6 addresses and accepts TCP connections, but the TLS handshake fails
  • httpx AsyncClient prefers IPv6 when available
  • httpx sync client and other libraries make different choices
  • My colleague's DNS doesn't return IPv6 addresses at all

The elegant fix? Force httpx to use IPv4:

# Before (broken) - httpx tries IPv6 first
client = httpx.AsyncClient()

# After (works) - explicitly bind to IPv4
transport = httpx.AsyncHTTPTransport(local_address="0.0.0.0")
client = httpx.AsyncClient(transport=transport)

That's it. One line to configure the transport. By binding to 0.0.0.0 (the IPv4 "any" address), we force httpx to use IPv4 for outgoing connections. It's surgical, explicit, and solves exactly the problem we identified.

Why This Works

When you set local_address="0.0.0.0", you're telling httpx to bind the client socket to the IPv4 "any" address before connecting. This has a subtle but important effect: a socket bound to an IPv4 address can only connect to IPv4 destinations. It's like putting on IPv4-only glasses – suddenly all those IPv6 addresses returned by DNS become invisible to your client.

The beauty of this solution:

  • It's a documented httpx feature, not a hack
  • It only affects this specific client instance
  • It clearly communicates intent: "use IPv4 for this client"
  • When TikAPI eventually fixes their IPv6 support, removing one line reverts to default behavior

What We've Learned

  1. Error Messages Are Sacred: That empty string cost hours. Always preserve the original error context. Future you will thank present you.

  2. IPv6: The Future That's Still Complicated: It's 2025 and we're still dealing with services that publish IPv6 DNS records but don't support IPv6 connections. The IPv4/IPv6 transition is the tech equivalent of "we'll switch to metric eventually."

  3. DNS Is Never Simple: Two machines, same network, different DNS settings = different results.

  4. "Works on My Machine" Is Real: And sometimes it's due to something as obscure as DNS not returning AAAA records.

  5. Abstractions Leak: Each layer (httpx → httpcore → asyncio → socket) lost a bit of error context until we were left with nothing.

  6. Happy Eyeballs Missing: httpx doesn't implement Happy Eyeballs (RFC 6555), which would try IPv4 and IPv6 in parallel and use whichever connects first.

The Moral of the Story

I spent 3 hours debugging a problem that had a one-line fix. But here's the thing: understanding why something breaks is often more valuable than just knowing how to fix it.

The journey taught me:

  • Why httpx AsyncClient fails with empty errors (IPv6 TLS handshake failures)
  • How to spy on Python's socket operations
  • Why "works on my machine" happens (different DNS configurations)
  • That TikAPI's DNS returns IPv6 addresses but the service doesn't properly support it
  • How a simple local_address="0.0.0.0" elegantly forces IPv4

Was it worth 3 hours to find a one-line fix? As engineers, we tell ourselves it's about "understanding the root cause." But really, sometimes we just can't help ourselves. We see a mystery and we have to solve it. And sometimes, that obsessive curiosity leads us to elegant solutions we wouldn't have found otherwise.

And yes, it's always DNS.

P.S. - Want to reproduce this investigation yourself? I've polished the debugging tools I built during this adventure into standalone scripts:

  • HTTP Client Detective - Test all Python HTTP clients against any endpoint. This is how I discovered httpx AsyncClient was the only one failing.
  • Socket Spy - Monkey-patch Python's socket to reveal IPv4 vs IPv6 connection attempts in real-time.
  • DNS Environment Scanner - Compare DNS resolution and connectivity across machines. Shows exactly why it "works on my machine."

All scripts use UV's inline dependencies, so just uv run script.py and you're debugging!

Update: Turns out we simply had different default DNS server settings on our machines. My colleague's DNS doesn't return AAAA records for certain domains. Mystery solved. Of course it was DNS configuration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment