Skip to content

Instantly share code, notes, and snippets.

@p-sherratt

p-sherratt/go-dns-resolver.md

Created September 20, 2024 18:09
Show Gist options
  • Save p-sherratt/e326953e12cf2e6c74cb88484985e480 to your computer and use it in GitHub Desktop.
Save p-sherratt/e326953e12cf2e6c74cb88484985e480 to your computer and use it in GitHub Desktop.
Download ZIP
Understanding Go's DNS Resolver Selection
Raw
go-dns-resolver.md

Understanding Go's DNS Resolver Selection

Courtesy of ChatGPT o1-preview

This document provides a comprehensive guide to how Go's net package selects a DNS resolver. It includes a flowchart illustrating the decision-making process, a table summarizing environment variables that influence resolver choice and behavior, and example scenarios demonstrating practical applications.

Table of Contents

  1. Introduction
  2. DNS Resolver Selection Flowchart
  3. Environment Variables Affecting DNS Resolver Choice
  4. Example Scenarios
  5. Implications for Developers
  6. Conclusion

Introduction

Go's net package provides a robust set of networking utilities, including DNS resolution functions. Depending on various factors like build tags, environment variables, and system configurations, Go decides whether to use:

  • The pure Go DNS resolver
  • The cgo-based resolver, which relies on the system's libc resolver
  • Platform-specific APIs (e.g., Windows API)

Understanding this selection process is crucial for developers who need consistent and predictable DNS resolution behavior across different environments and platforms.

DNS Resolver Selection Flowchart

Below is a flowchart that illustrates the decision-making process Go uses to select the DNS resolver.

flowchart TD
    Start[Start] --> GODEBUGCheck{Is GODEBUG netdns set?}
    GODEBUGCheck -- "netdns=go" --> PureGo[Use pure Go resolver]
    GODEBUGCheck -- "netdns=cgo" --> UseCgo[Use cgo resolver]
    GODEBUGCheck -- "Not set or netdns=auto" --> NetcgoTag{Is 'netcgo' build tag specified?}
    NetcgoTag -- Yes --> UseCgo
    NetcgoTag -- No --> NetgoTag{Is 'netgo' build tag specified?}
    NetgoTag -- Yes --> PureGo
    NetgoTag -- No --> IsCgo{Is cgo enabled?}
    IsCgo -- No --> PureGo
    IsCgo -- Yes --> Platform{Platform?}
    Platform -- Windows --> WindowsAPI[Use Windows API for DNS resolution]
    Platform -- "Unix-like (Linux, macOS, etc.)" --> NSSwitchCheck{Is /etc/nsswitch.conf hosts line simple?}
    Platform -- "Other platforms (e.g., Android, iOS)" --> PlatformSpecific[Use platform-specific resolver]
    NSSwitchCheck -- Yes --> PureGo
    NSSwitchCheck -- No --> UseCgo
Loading

Flowchart Explanation

  1. Start: The decision process begins.

  2. GODEBUG Environment Variable Check:

    • netdns=go: Forces the use of the pure Go resolver.
    • netdns=cgo: Forces the use of the cgo resolver.
    • Not Set or netdns=auto: Proceeds to the next check.

  3. Build Tags Check:

    • netcgo Build Tag:
      • Yes: Use cgo resolver.
      • No: Check for netgo build tag.
    • netgo Build Tag:
      • Yes: Use pure Go resolver.
      • No: Proceed to cgo availability check.

  4. cgo Availability Check:

    • cgo Enabled:
      • Yes: Proceed to platform check.
      • No: Use pure Go resolver.

  5. Platform Check:

    • Windows: Use Windows API for DNS resolution.
    • Unix-like Systems: Examine /etc/nsswitch.conf.
    • Other Platforms: Use platform-specific resolver.

  6. /etc/nsswitch.conf Examination (Unix-like Systems):

    • Simple hosts Line (e.g., hosts: files dns): Use pure Go resolver.
    • Complex hosts Line: Use cgo resolver.

Environment Variables Affecting DNS Resolver Choice

Environment variables can influence both the choice of DNS resolver and its behavior. The primary environment variable is GODEBUG, but others affect the cgo resolver's behavior.

Table of Environment Variables

Environment Variable Possible Values Effect Notes
GODEBUG netdns=go Forces the use of the pure Go DNS resolver. Overrides build tags and cgo availability.
netdns=cgo Forces the use of the cgo-based DNS resolver. Bypasses automatic detection and build configurations.
netdns=auto (default) Lets Go decide the resolver based on system configuration. Follows the standard decision-making process.
RES_OPTIONS Resolver options (e.g., ndots:2 timeout:3 attempts:2) Modifies the behavior of the libc resolver when using cgo. Only affects the cgo resolver; ignored by the pure Go resolver.
LOCALDOMAIN Space-separated domain names (e.g., example.com sub.example.com) Overrides the domain search list for the libc resolver. Relevant when using the cgo resolver; ignored by the pure Go resolver.
SEARCH Space-separated domain names (e.g., example.com sub.example.com) Sets the domain search list for the libc resolver. May override /etc/resolv.conf settings; ignored by the pure Go resolver.
HOSTALIASES Path to a host aliases file (e.g., /path/to/aliases) Specifies hostname alias mappings for the libc resolver. Used by the cgo resolver; the pure Go resolver does not use this file.

Detailed Explanation of Variables

GODEBUG

  • Purpose: Controls the DNS resolver choice in Go's net package.
  • Usage:
    • To force the pure Go resolver: 
      export GODEBUG=netdns=go
    • To force the cgo resolver: 
      export GODEBUG=netdns=cgo
  • Notes:
    • Takes precedence over build tags and cgo availability.
    • It's the first condition checked in the resolver decision process.

RES_OPTIONS

  • Purpose: Specifies options for the libc resolver when the cgo resolver is in use.
  • Example: 
    export RES_OPTIONS="ndots:1 timeout:2 attempts:2"
  • Notes:
    • Only affects the cgo resolver.
    • Ignored by the pure Go resolver.

LOCALDOMAIN

  • Purpose: Overrides the search domains specified in /etc/resolv.conf for the libc resolver.
  • Example: 
    export LOCALDOMAIN="example.com sub.example.com"
  • Notes:
    • Relevant only when using the cgo resolver.
    • Ignored by the pure Go resolver.

SEARCH

  • Purpose: Sets the domain search list for the libc resolver.
  • Example: 
    export SEARCH="example.com sub.example.com"
  • Notes:
    • Similar to LOCALDOMAIN.
    • Ignored by the pure Go resolver.

HOSTALIASES

  • Purpose: Specifies a file containing hostname alias mappings for the libc resolver.
  • Example: 
    export HOSTALIASES="/path/to/host_aliases"
  • Notes:
    • The file should contain lines in the format: alias real_hostname.
    • Only used by the cgo resolver.

Example Scenarios

These scenarios illustrate how to apply the knowledge of Go's DNS resolver selection in practical situations.

Scenario 1: Forcing the Pure Go Resolver

Situation: You have a Go application that requires consistent DNS resolution behavior across different environments, some of which have complex nsswitch.conf configurations.

Solution:

  • Set the GODEBUG environment variable to force the pure Go resolver: 
    export GODEBUG=netdns=go
  • Alternatively, in code, set PreferGo in a custom net.Resolver: 
    resolver := &net.Resolver{
    PreferGo: true,
}

Outcome: The application uses the pure Go resolver, ensuring consistent DNS resolution regardless of system configurations.

Scenario 2: Customizing cgo Resolver Behavior

Situation: Your application uses the cgo resolver but needs to adjust DNS query timeouts and retry attempts due to slow DNS responses.

Solution:

  • Set the RES_OPTIONS environment variable: 
    export RES_OPTIONS="timeout:5 attempts:3"

Outcome: The cgo resolver waits up to 5 seconds for a DNS response and retries queries up to 3 times, improving reliability in environments with slow DNS servers.

Scenario 3: Using Host Aliases

Situation: You need to map a custom hostname to a specific server without modifying the /etc/hosts file.

Solution:

  1. Create a host aliases file (e.g., /path/to/host_aliases): 
    myservice myservice.internal.example.com
  2. Set the HOSTALIASES environment variable: 
    export HOSTALIASES="/path/to/host_aliases"

Outcome: The cgo resolver resolves myservice to myservice.internal.example.com, allowing your application to use the alias without changing system-wide configurations.

Implications for Developers

Control Over DNS Resolution

  • Forcing Resolver Choice:

    • Use GODEBUG=netdns=go or GODEBUG=netdns=cgo to control which resolver Go uses.
    • Set PreferGo in a custom net.Resolver for programmatic control.

  • Consistent Behavior:

    • For applications requiring consistent DNS behavior, especially across diverse environments, forcing the pure Go resolver can be beneficial.

Performance Considerations

  • Pure Go Resolver:

    • May offer better performance due to being implemented in Go and avoiding cgo overhead.
    • Might not respect all system-specific configurations.

  • cgo Resolver:

    • Leverages system's libc resolver, respecting all system configurations.
    • May introduce overhead due to cgo calls.

Debugging and Testing

  • Diagnosing Issues:

    • Temporarily setting GODEBUG=netdns=go can help identify if DNS issues are related to the cgo resolver.

  • Environment Awareness:

    • Be aware of environment variables and system configurations that might affect DNS resolution during development and testing.

Security Considerations

  • Untrusted Environments:
    • Ensure that applications do not unintentionally use environment variables that could compromise security.
    • Consider sanitizing or explicitly setting resolver options in code.

Conclusion

Understanding how Go's net package selects and uses DNS resolvers is essential for developing robust networked applications. By leveraging environment variables and build configurations, developers can control DNS resolution behavior, ensuring consistency, performance, and adherence to system policies.

Whether you're troubleshooting DNS issues, optimizing performance, or ensuring consistent behavior across platforms, the knowledge of Go's DNS resolver selection process is a valuable tool in your development toolkit.

References:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment