If you have several tools that are very closely related, you can make them easier to use, discover, and distribute by combining them into a single tool (and a single executable binary artifact).
If you’ve got a tool that’s sufficiently complex, you can reduce its complexity by making a set of subcommands. This is also useful for sharing stuff—global flags, help text, configuration, storage mechanisms.The above guidance doesn’t help you to decide when something is unrelated and should be separated, BTW.
It is worth comparing this advice to the Command Pattern from the “Gang of Four” “Design Patterns” book, where you encapsulate an an action or a request as an object that can be parameterized. Usually, in the Command Design Pattern the invoker doesn’t know anything about the implementation details of the command or it’s receiver, it just knows the command interface and its only responsibility is to invoke the command and optionally do some bookkeeping of what commands are possible and/or valid. There are a couple examples of this design pattern in the Go standard library.https://christiangiacomi.com/posts/alternative-command-pattern-go/
https://rolandsdev.blog/posts/the-command-pattern-in-go/ I made the districts-jobs monorepo be structured into individual tools (admin-reports, alerter, distutil, roster, etc.) that have a single main.go with commands that can be divided into subcommands. This allows us to have only a few binary artifacts to distribute (or install for a developer tool). If we are making an app that will listen to pubsub messages, we can also add to that a command (or subcommand) that will allow us to send a suitable message for troubleshooting purposes. This keeps the related functionality distributed together.
The khanx application was deliberately not structured using this command pattern in the beginning. It used a simple flag and if/else conditional to determine whether it should behave as a web application or whether to act as a GitHub action. The command pattern is great when you have more than two things combined together. It can accommodate fractal complexity through subcommands and subsubcommands and so on. The Command pattern seemed like complete overkill for a single if/else, and since my hope was we could entice Product Growth to clone khanx if it didn’t look too weird to them, I deliberately wanted to keep it simple and familiar (but not look like webapp) if we could.
Q: Why is having a single executable binary artifact better than multiple executable binary artifacts.
Use case one: User distribution of CLI tool binary artifacts is easier
Before we had the devadmin ui for districts, we all used the cli distutil built from districts-jobs. Every developer on the team used this single swiss army knife binary to manually do most everything. Listing districts, creating districts, creating admins, deduping users, editing any district field or status, looking up a UDI (with unencrypted fields). The purely frontend developers did not want to keep track of which obscure backend code had changed and then trace back to the various main.go entry points so they could decide which needed to be rebuilt. Otherwise, they risked unintentionally writing using old data models and breaking stuff in production! Having a single binary meant they could just rebuild that one before they did anything destructive (e.g. in their morning) and be pretty confident they were ok.We decided to split out the more dangerous stuff like yearend (that would downgrade and permanently delete district data) in case someone ran the wrong command on the wrong district. We also decided to split the listdistricts tool since we wanted to make it so you could fuzzy match any jumble of strings and that didn’t work with a hierarchy of commands and subcommands.Use Case Two: Continuous Deployment and Delivery in a Distributed Environment
Building a single roster binary means we compile that into a single binary artifact (docker image) corresponding to a single Git commit SHA1. For all roster related tools, we do not need to wait to test, rebuild, and transmit multiple artifacts, nor do we need to store multiple artifacts. This leads to a substantial improvements in development speed, computational resources, and efficiency. This means, a single physical Kubernetes worker node that is running one roster tool (like the PubSub puller) is more likely to be running another roster tool (like a single roster job) and not need to copy the same binary artifact over the network and to waste more in temporary physical binary storage. When we need to rollback to a prior version, we do not need to retain as many GB of prior docker images, because all the roster related ones are consolidated into just the roster tool for each deployment Git commit SHA1.Similarly, we get an additional iron clad guarantee that any roster related tool will not need as much end-to-end contractual testing between any another roster related tool. A roster tool that sends PubSub messages using one version of protobuf might become incompatible with the receiver if the version of protobuf of the tools changes in an incompatible way. Since they are built and and deployed together, and are using the same libraries, they cannot possible diverge in this way, so there’s no need to build and end-to-end test verification to detect this problem.
So like I mentioned, the command pattern is a design pattern that encapsulates an action or request as an object that can be parameterized. And it’s commonly associated with terms like receiver, command, invoker and client.
Usually, the invoker doesn’t know anything about the implementation details of the command or receiver, it just knows the command interface and its only responsibility is to invoke the command and optionally do bookkeeping of commands.
The command is the object that knows about the receiver and its responsibility is to execute methods on the receiver.
Go’s exec package is an example of this pattern:
package main
import (
"os/exec"
)
func main() {
cmd := exec.Command("sleep", "1")
err := cmd.Run()
}This is a good example of the command pattern itself. However, this is so generic that it is not helpful for making a nice command pattern encapsulation of a specific executable command (like when you want to use the git executable to do something like find the commit of the current working directory).So I made this as a way of manufacturing command pattern encapsulations of those specific things:
https://github.com/Khan/districts-jobs/tree/main/pkg/shell Then when I wanted to encapsulate something specific I can churn them out like here:
https://github.com/Khan/districts-jobs/blob/main/pkg/mages/git.go The implementation details of that could be shelling out to use the local git executable (as it is), or it could instead be using a pure Go implementation of git like go-git and the caller doesn’t know or need to know.
The command pattern in an HTTP request is like more like in the net/http package:
package main
import (
"http/net"
)
func main() {
c := &http.Client{}
req, err := http.NewRequest("GET", "http://example.com", nil)
res, err := c.Do(req)
}Although it’s sort of confusing which is the invoker and receiver there.There’s a good article about why it is good to take advantage of this pattern for requests though:
https://www.0value.com/let-the-doer-do-it
The Slack client in districts-jobs is all about that.
In Go, usually an HTTP client is:
type doer interface { Do(req *http.Request) (*http.Response, error) }
Go’s HTTP situation is that the `http.RoundTripper` interface is explicitly designed to provide customization of how an HTTP request executes, as that’s exactly what it’s there for.If you take a look at the definition of RoundTripper and compare it to the `doer` interface above, I think you’ll see that the similarity is striking:
type RoundTripper interface {
RoundTrip(*Request) (*Response, error)
}(See: https://golang.org/pkg/net/http/#RoundTripper )It’s exactly the same down to every type!For instance:
type tracingRoundTripper struct {
http.RoundTripper
tr opentracing.Tracer
}
func (tr *tracingRoundTripper) RoundTrip(req *http.Request) (*http.Response, error) {
req, ht := nethttp.TraceRequest(tr.tr, req)
defer ht.Finish()
return tr.RoundTripper.RoundTrip(req)
}