Interfaces and satisfaction checks in Go packages

interfaces-and-tests.md

Interfaces and Satisfaction Checks in Go Packages

When a Go package defines a concrete type to implement some interface, it is common to ask the compiler to verify that your type satisfies the desired interface, e.g.,

package mything

import "some/other/pkg"

// Concrete implements pkg.Interface using a pellucid ammonite in
// gel concentrate to subvert the dominant paradigmatic ouevre of
// modern artistic practice.
type Concrete struct { /* … */ }

// Satisfaction check.
var _ pkg.Interface = (*Concrete)(nil)

This is a good and useful trick, but I argue it is better to put such satisfaction checks into test files, rather than the production code for the package.

Rationale

These are the main reasons satisfaction checks should be packaged with the tests, rather than the production code:

• Dependencies have cost: Building a package that depends on mything requires downloading and building the transitive closure of all its dependencies. For small packages the compile time is small, and the module cache reduces the cost of repeating it, but it still requires locating and possibly fetching packages, verifying checksums, updating caches, and loading their type information. If the only reason mything imports some/other/pkg is to evaluate a no-op satisfaction check, those costs have no direct benefit to the user of mything. More importantly, even if the user cares about it…
• The check is redundant: If the caller uses a Concrete value to satisfy pkg.Interface, the compiler will do a satisfaction check on their code anyway, regardless of what mything does, so now we've done it twice. Conversely, if they are using Concrete in some other capacity, we have now done a satisfaction check that was not needed by anyone. This leads to the view that…
• Interface satisfaction is a test: Whether Concrete satisfies pkg.Interface is not a question about how it is implemented, but about its shape and behaviour. If we deleted a satisfaction check from the library, it would have no effect (in either direction) on its correctness, but it's still an important claim we want to verify. That's exactly what tests are for.

Hiding the Goods

The desire to import packages solely for type checks is sometimes exacerbated by a couple of other package design issues. Programmers accustomed to other languages often take additional steps to hide their implementations, by defining constructors in terms of an interface rather than the implementation. For example:

package mything

// A concrete satisfies pkg.Interface by quietly burying it in the sand.
type concrete struct { /* … */ }

// NewInterface returns a pkg.Interface from the given arguments.
// You cannot do anything useful with the concrete type of this value,
// so don't even try, muahahahaha.
func NewInterface(args Args) pkg.Interface { return concrete{bits: args} }

In a way, this is just a more explicit variation of the no-op satisfaction check from the first example above. It forces the compiler to verify that the concrete type satisfies the interface (else NewInterface will not type-check), and also prevents the caller from constructing a concrete value in any unapproved manner.

In my view, these latter steps are a mistake, and should be avoided in idiomatic Go code. In the discussion that follows, I will argue that packages should return concrete types rather than interfaces, and that those concrete types should almost always be exported.

Claim 1: Constructors should return concrete types

When a constructor returns an interface, it says two things to the reader:

1. Explicit: The value I'm returning to you definitely implements pkg.Interface, and
2. Implicit: The type of that value has no other interesting properties you would care about.

To understand why this distinction matters, let's consider the three parties to an API transaction in software: The provider, the user, and the consumer. The provider is the API whose constructor produces the value (e.g., mything). The consumer is the API that receives the value and does some useful work with it (e.g., pkg), and the user is the program that connects the two together.

• For the consumer, the explicit property (1) is what matters. For example, io.ReadAll does not care what its argument is, as long as it satisfies the io.Reader interface.
• The user, however, generally does care: They want a value that reads data from a particular source, say, and want to perform activities besides reading from it.

If the provider returns only the interface, it serves the consumer but not the user. A good example of the contrast is the *strings.Reader type in the standard library: It satisfies the io.Reader interface, but also provides methods to report the size (Len), move the reader to a different position (Seek), read and un-read runes (ReadRune, UnreadRune), and so on.

If strings.NewReader returned io.Reader, the user couldn't use these methods, without doing an explicit type assertion, e.g.,

package badstrings
// A hypothetical constructor…
func NewReader(s string) io.Reader { return &Reader{ /* … */ } }

package user

import "badstrings"

// The user:
r := badstrings.NewReader(s)
r.(*badstrings.Reader).Seek(5, io.SeekStart)
data, err := io.ReadAll(r)
// …

Besides being ugly, this is basically pointless: If badstrings.NewReader had returned the concrete type, the same code would have worked without the type assertion, and easier to read.

Claim 2: Concrete types should be exported

The situation gets worse when the package doesn't export the concrete type:

package evilstrings

func NewReader(s string) io.Reader { return &reader{ /* … */ } }

package user

import "evilstrings"

r := evilstrings.NewReader(s)
// well, what do I do now?

In this case, the user can't even write the type assertion, because the type is unexported.

This example may seem ridiculous, but this pattern is regrettably common in Go libraries found in the wild. With the standard library there's no real solution, but in third-party packages this often leads to weird expansions of the interface type, since that's the only escape hatch. Maybe the interface starts out like this:

package fizzy

type Widget interface {
  Widge(string) error
}

But then it turns out that some of our widgets need to flush buffers before we're done with them:

package magic

import "fizzy"

type bufferedWidget struct {
  buf *bufio.Writer
  // …
}

func (w bufferedWidget) Widge(s string) error {
  _, err := w.buf.Write(s)
  return err
}

func (w bufferedWidget) Close() error {
  return w.buf.Flush()
}

type NewWidget(wc io.Writer) fizzy.Widget {
  return &bufferedWidget{buf: bufio.NewWriter(wc)}
}

Since the caller can't get at the Close method directly, you often see the base interface expand:

package fizzy

type Widget interface {
  Widge(string) error
  Close() error
}

Now all the implementations of fizzy.Widget need to implement Close, even if they don't need it. A good litmus test for this problem is if you see n different implementations of an interface where all but one have no-op implementations of certain methods, and the one "special" interface has real behaviour for them. Another red flag is if you see embedded "helper" types that provide "default" implementations of interface methods, which would otherwise have to be replicated to each type.

To sum up: Return exported, concrete types, rather than interfaces.

As a side note, returning concrete types also makes it easier to write tests. To test an unexported type, you either have to put the tests in the same package as the library code, or test only the surface visible through the interface. Having tests in a separate lib_test package is good documentation for idiomatic use, but it doesn't work if the types under test are unexported.

Exceptional Cases

Naturally, there are a couple of valid exceptions:

1. A factory function that chooses an implementation based on its arguments usually has to return an interface (e.g., net.Dial returns different implementations of net.Conn depending on the address it has been asked to dial).
2. An adapter function that provides new behaviour to an existing interface (e.g., io.MultiReader which concatenates readers, avoids a level of indirection by special-casing a single argument whose concrete type is arbitrary).

As a rule of thumb, however, don't do this unless you have a specific reason like one of these.