Mock Client Fixture Abstraction

fixture_abstraction.md

This doc proposes an abstraction to set fixture expectations for mock clients during a test run.

Overview

The goal is to treat fixtures management as first class citizen and maintain clean fixture semantics. Fixtures that belong to a specific client should reside in a centralized place associated with that client. It should be recognized that the mock client and its affinitivie fixtures together can become one functional unit. With this realization, the action of setting input/output expectations for a mock client method becomes implementation details and can be abstracted away by defining a method with the same expectation semantic on the combined functional unit. For example, instead of writing

import store "import/path/to/store/fixture"

// ad-hoc fixtures
userHeader := map[string]string{
	"x-client-version": "1.1.2",
}
getUserRequest := &gen.GetUserRequest{
	Name: "John",
	Age:  42,
}
notFoundResponse := &gen.GetUserException{
	Message: "user not found",
}
mockUserClient.EXPECT().GetUser(gomock.Any, userHeader, getUserRequest).Return(notFoundResponse, nil, nil)

// maybe central but unmanaged fixtures
mockStoreClient.EXPECT().Update(gomock.Any, store.Header, store.UpdateRequest).Return(store.UpdateOkResponse, nil, nil)

One would write

mockUserClient.ExpectGetUser().NotFound()
mockStoreClient.ExpectkUpdate().Ok()

where method NotFound and Ok implements the actions of setting fixture expectations for the GetUser and Update calls.

Doing so avoids ad-hoc fixture definitions and littering at test site, therefore keeps the test code clean and readable. More importantly, it enforces a managed single source of truth for fixtures, i.e., when a client's behavior changes, updating the corresponding fixture is sufficient for the goal of updating call tests impacted by that change, eliminating the need to find and update fixtures in each individual tests.

Implementation

To implement such abstraction, we need to takes care of a couple of things:

• define struct types for fixtures associated with a client
• augment mock client with abstract expectation semantics
• wire the augmented mock client into test service generation.

All these work is taken care of by Zanzibar itself, i.e., fixture types and augmented mock clients will be generated, except providing concrete fixtures. Note the scope of this proposal does not include how a fixture is created, whether it is hand written or generated through some sort of auto-gen pipeline is not Zanzibar's concern. In this sense, an importPath needs to be configured in client-config.json to indicate fixture import path.

Struct Type Definition for Grouped Fixtures

The entity relationship model for a client is:

• a client has one or more methods
• a method has zero or more arguments and returns.

The associated fixture type has a similiar but slightly different entity relationship model:

• a client fixture has one or more method fixtures
• a method fixture has one or many scenario fixtures
• a method scenario fixture has zero or more arguments and returns.

Consider a user client of following interface:

// code generated by zanzibar

type Client interface {
	GetUser(ctx context.Context, headers map[string]string, req *gen.GetUserRequest) (*gen.GetUserResponse, map[string]string, error)
	GetCount(ctx context.Context, headers map[string]string) (*gen.GetCountResponse, map[string]string, error)
}

The associated fixture type is:

// code generated by zanzibar

type ClientFixture struct {
	GetUser  *GetUserScenarios
}

type GetUserScenarios struct {
	NotFound *GetUserFixture `scenario:"notFound"`
}

type GetUserFixture struct {
	Arg0 context.Context
	Arg1 map[string]string
	Arg2 *gen.GetUserRequest
	
	// Arg{n}Any indicates the nth argument could be a gomock.Any
	Arg0Any bool
	Arg1Any bool
	Arg2Any bool

	Ret0 *gen.GetUserResponse
	Ret1 map[string]string
	Ret2 error
}

With above type definition, a concrete user client fixture could be:

var getUserFixtures = &GetUserScenarios{
	NotFound: &GetUserFixture{
		Arg0Any: true,
		Arg1Any: true,
		Arg2: &gen.GetUserRequest{
			Name: "John",
			Age: 42,
		},
		
		Ret0: &gen.GetUserResponse{
			Message: "user not found",
		},
	},
}

var Fixture = &ClientFixture{
	GetUser: getUserFixtures
}

Note the "a method fixture has one or many scenario fixtures" relationship is also implemented with a struct. Each field corresponds to a scenario, the tag of the struct field is the string literal configured in the client-config.json file.

Augment Mock Client

The augmented mock client must implement the same interfaces as the client and must have access to its fixtures, naturally

// code generated by zanzibar

type MockClientWithFixture struct {
	*MockClient
	fixture *ClientFixture
}

where the embedded MockClient is generated by mockgen.

However, this will also expose the EXPECT method on the MockClientWithFixture, which allows setting expectations manually at test site. Therefore it needs to be redefined to shadow the same method on the underlying MockClient:

// code generated by zanzibar

func (m *MockClientWithFixture) EXPECT() {
	panic("should not call EXPECT directly.")
}

For each method that is mocked, a struct type is necessary to contain the mock scenarios for that method.

// code generated by zanzibar

type getUserMock struct {
	scenarios  *GetUserScenarios
	mockClient *MockClient
}

func (m *MockClientWithFixture) ExpectGetUser() *getUserMock {
	return &getUserMock{
		scenarios:  m.fixture.GetUser,
		mockClient: m.MockClient,
	}
}

For each mock scenario, Zanzibar needs to generate a static function, which sets the given expectations as in the fixture when it is invoked.

// code generated by zanzibar

func (s *getUserMock) NotFound() {
	f := s.scenarios.NotFound

	var arg0, arg1, arg2 interface{}
	arg0 = f.Arg0
	if f.Arg0Any {
		arg0 = gomock.Any()
	}
	arg1 = f.Arg1
	if f.Arg1Any {
		arg1 = gomock.Any()
	}
	arg2 = f.Arg2
	if f.Arg2Any {
		arg2 = gomock.Any()
	}

	s.mockClient.EXPECT().GetUser(arg0, arg1, arg2).Return(f.Ret0, f.Ret1, f.Ret2)
}

In order to generate such static methods, Zanzibar needs information about the intended mock scenarios at build time. Such information can be provided via client-config.json. For Example:

{
  "name": "user",
  "type": "http",
  "config": {
    "thriftFile": "clients/user/user.thrift",
    "exposedMethods": {
      "GetUser": "User::getUser",
      "GetCount": "User::getCount"
    },
    "fixture": {
      "importPath": "github.com/uber/zanzibar/examples/example-gateway/clients/user/fixture",
      "scenarios": {
        "GetUser": [
          "notFound"
        ]
      }
    }
  }
}

This configuration will be source of truth for generating scenario static method as well as creating concrete fixtures.

Wire Augmented Mock Client

The fixture struct types and augmented mock client require information of the client methods' argument and return types, therefore should be generated along the client generation. The tricky part is to not break the test service if the dependent clients have yet not provided fixture configurations, i.e., allow a transition phase for existing clients. In order to do so, Zanzibar should fall back to the existing mock client approach. Moving forward, it can be enforced that a new client config should always have a fixture field, if we desire so.