17twenty / Nick's Golang Reading List

golang-reading-list.md

Getting Started with Go

• The Go Tour - Entirely online, no need for any setup/install
• Setup your Go path and environment

Go Reading List

The following are a list of resources I find myself referring to:

• Dave Cheney's Getting Started Guide
• Structure and Style Guide
• Writing Effective Go Code
• How the hell slices and memory management work in Go
• Go Strings vs Bytes
• Sharing Memory without mutexes
• Using Make New and Maps
• Using Interfaces Successfully
• Defer Panic and Recover - You see this a lot in other code
• 50 Shades of Go - Gotchas and Stuff
• Ten Useful Go Techniques
• Table Driven Testing

These ones are more focused on cross-compiling and that sort of thing:

• Go's Runtime Environment
• Working with Go on ARM
• Cross Compiling Go Code in Weird Configurations
• Using C with Go - the article I originally based the above magic on
• Cost and Complexity of CGo

Installing Project dependencies

$ cd $GOPATH/src/proj
$ go get ./...

Project dependencies will be created under their own directories (namespaces) in $GOPATH/src

You should be able to test it works by running their respective testsuites:

$ go test -v proj
ok  	proj	0.012s

The flags for the Go tools everyone should know.

$ go build -x

-x lists all the commands go build invokes.

If you are curious about the Go toolchain, or using a cross-C compiler and wondering about flags passed to the external compiler, or suspicious about a linker bug; use -x to see all the invocations.

    $ go build -x
    WORK=/var/folders/00/1b8h8000h01000cxqpysvccm005d21/T/go-build600909754
    mkdir -p $WORK/hello/perf/_obj/
    mkdir -p $WORK/hello/perf/_obj/exe/
    cd /Users/jbd/src/hello/perf
    /Users/jbd/go/pkg/tool/darwin_amd64/compile -o $WORK/hello/perf.a -trimpath $WORK -p main -complete -buildid bbf8e880e7dd4114f42a7f57717f9ea5cc1dd18d -D _/Users/jbd/src/hello/perf -I $WORK -pack ./perf.go
    cd .
    /Users/jbd/go/pkg/tool/darwin_amd64/link -o $WORK/hello/perf/_obj/exe/a.out -L $WORK -extld=clang -buildmode=exe -buildid=bbf8e880e7dd4114f42a7f57717f9ea5cc1dd18d $WORK/hello/perf.a
    mv $WORK/hello/perf/_obj/exe/a.out perf

$ go build -gcflags

Used to pass flags to the Go compiler. go tool compile -help lists all the flags that can be passed to the compiler.

For example, to disable compiler optimizations and inlining, you can use the following the gcflags.

    $ go build -gcflags="-N -I"

$ go test -v

It provides chatty output for the testing. It prints the test name, its status (failed or passed), how much it took to run the test, any logs from the test case, etc.

go test without the -v flag is highly quiet, I always use it with -v turned on. Sample output:

    $ go test -v context
    === RUN   TestBackground
    --- PASS: TestBackground (0.00s)
    === RUN   TestTODO
    --- PASS: TestTODO (0.00s)
    === RUN   TestWithCancel
    --- PASS: TestWithCancel (0.10s)
    === RUN   TestParentFinishesChild
    --- PASS: TestParentFinishesChild (0.00s)
    === RUN   TestChildFinishesFirst
    --- PASS: TestChildFinishesFirst (0.00s)
    === RUN   TestDeadline
    --- PASS: TestDeadline (0.16s)
    === RUN   TestTimeout
    --- PASS: TestTimeout (0.16s)
    === RUN   TestCanceledTimeout
    --- PASS: TestCanceledTimeout (0.10s)
    ...
    PASS
    ok  	context	2.426s

$ go test -race

Go’s race detector is available from the Go tools via -race. go test also supports this flag and reports races. Use this flag during development to detect the races.

$ go test -run

You can filter tests to run by regex and the -run flag. The following command will only test examples.

$ go test -run=Example

$ go test -coverprofile

You can output a cover profile as you are testing a package, then use go tool to visualize them on a browser.

$ go test -coverprofile=c.out && go tool cover -html=c.out

The command above will create a coverage profile and open the results page in the browser.

$ go test -exec

It is a lesser known feature in Go that you can intercept the tools with another program by using the -exec flag. This flag allows you to delegate some work to an external program from the Go tool.

A commonly required scenario for this flag is when you need more than just executing the tests on the host machine. The Android builder for Go, uses -exec to push the test binaries to an Android device by using adb and collects the results. Android exec program can be used as a reference.

$ go get -u

If you run go-get for a package that is already in your GOPATH, go-get is not going to update the package to its latest version. -u forces the tool to sync with the latest version of the repo.

If you are a library author, you might like to write your installation instructions with a -u flag, e.g. the way golint does.

    $ go get -u github.com/golang/lint/golint

$ go get -d

If you just want to clone a repo to your GOPATH and skip the building and installation phase, use -d. It downloads the package and stops before trying to build or install it.

I often use it as a replacement for git clone for repos with vanity URLs, because it clones the repo to its proper GOPATH. For example,

    $ go get -d golang.org/x/oauth2/...

will clone the package to $GOPATH/src/golang.org/x/ouath2. Given golang.org/x/oauth2 is a vanity URL, go-getting the repo is useful rather than trying to figure out where the actual repo is (go.googlesource.com/oauth2).

$ go get -t

If your package has additional dependencies for tests, -t will allow you to download them during go-get. If you don’t pass -t, go get will only download the dependencies for your non-test code.

$ go list -f

Allows you to list Go packages with a custom format. It is highly useful for writing bash scripts.

The following command will print the dependencies of the runtime package:

    go list -f '{{.Deps}}' runtime
    [runtime/internal/atomic runtime/internal/sys unsafe]

More formatting ideas can be found at Dave Cheney’s article on go list.

Debugging What You Deploy (Go 1.12+)

https://blog.golang.org/debugging-what-you-deploy

High Performance / Tuning Your Golang

High Performance Go Workshop

Cross Compilation with CGo and Golang

Useful background: http://dave.cheney.net/2012/09/08/an-introduction-to-cross-compilation-with-go

• errcheck Install with go get github.com/kisielk/errcheck - Ensures you have no unchecked errors
• Generics - You can't stop people trying to implement the feature that Go doesn't have (nor want!)
  • https://github.com/ncw/gotemplate
  • https://github.com/clipperhouse/gen
  • https://github.com/cheekybits/genny
• go generate Looks for directives in Go code and can generate more code in turn
• go tool cover Generates code-coverage metrics
• go vet Provided linter for Go code
  • Useless assignments
  • Common mistakes when using sync/atomic
  • Invalid +build tags
  • Using composite literals without keyed fields
  • Passing locks by value
  • Comparing functions with nil
  • Using wrong printf format specifiers
  • Closing over loop variables the wrong way
  • Struct tags that do not follow the canonical format
  • Unreachable code
  • Misuse of unsafe.Pointer
  • Mistakes involving boolean operators
• gofmt Format your code to the only acceptable way
• godef - Install with go get https://github.com/17twenty/rog-go/exp/cmd/godef
• godepgraph Install with go get golang.org/x/tools/cmd/gorename - Generates dependency graphs (in Graphviz format)
• goimports Install with go get golang.org/x/tools/cmd/goimports - autofix those horrible import errors automagically
• golint Install with go get github.com/golang/lint/golint - not a linter, actually a style violation checker
• gorename Install with go get golang.org/x/tools/cmd/gorename - powerful type aware idenifier renamer
• goreturns Install with go get golang.org/x/tools/cmd/goimports - fills in incomplete return statements with zero values.
• impl Given an interface, generates method stubs for implementing that interface.
• oracle Install with go get golang.org/x/tools/cmd/oracle - oracle is a source analysis tool capable of answering a questions about Go programs including:
  • What is the type of this expression? What are its methods?
  • What’s the value of this constant expression?
  • Where is the definition of this identifier?
  • What are the exported members of this imported package?
  • What are the free variables of the selected block of code?
  • What interfaces does this type satisfy?
  • Which concrete types implement this interface?
  • What are the possible concrete types of this interface value?
  • What are the possible callees of this dynamic call?
  • What are the possible callers of this function?
  • What objects might this pointer point to?
  • Where are the corresponding sends/receives of this channel receive/send?
  • Which statements could update this field/local/global/map/array/etc?
  • Which functions might be called indirectly from this one?
• stringer Install with go get golang.org/x/tools/cmd/stringer
• jsonenums - A tool to automate the creation of methods that satisfy the json.Marshaler and json.Unmarshaler interfaces.

Accept Interfaces, Return Structs - this is used to great effect in the SQL Interface
Correctly getting goroutines to finish - incidentally, Nathan's blog is awesome reading for Go stuff
Avoiding Reflection
Working around type system of Go with unsafe
JSON and GO - Not the official article, this is better
Debugging With Delve
Intel Developers Guide to Debugging Performance Issues
Using Interfaces in Go
Dockerizing Golang Apps
How Remote Import Paths Work
Handling Signals like SIGUSR1
Best Practice as of 2016
Go Gotchas
Awesome Go Patterns
Using Contexts
Using Label Breaks in Go
Build a serverless website from scratch using S3, API Gateway, AWS Lambda, Go and Terraform
Go Anti Patterns / Idiomatic Golang -- Definitely read up on tiny package syndrome / micropackages!
Using S3 with Golang
Writing a Server Sent Events server in Go

Find find things that used to exist in a repo and then got removed. Very useful when working with turd codebases.

git grep <your term> $(git rev-list --all)

Also, find when a file was introduced:

git log --diff-filter=A -- *<filename>*

Update your entire GOPATH

go get -v -d -u -f .../

Profiling In Go

• PProf in Go and how to use it
• Introducing Profile - Simple One-Line Profiling
• Profiling with Golang - HowTo Use Profile

func withLockContext(fn func()) {
    mu.Lock
    defer mu.Unlock()

    fn()
}
....
withLockContext(func() {
    // Locky related stuff
})

How To Close Channels in Golang Elegantly

package main

import "fmt"

type T int

func IsClosed(ch <-chan T) bool {
	select {
	case <-ch:
		return true
	default:
	}
	
	return false
}

func main() {
	c := make(chan T)
	fmt.Println(IsClosed(c)) // false
	close(c)
	fmt.Println(IsClosed(c)) // true
}

See also, the Channel Axioms guide

Building totally static binaries:

$ go build -a --ldflags '-extldflags "-static"' -tags netgo -installsuffix netgo  .

Embedding Build Time Variables

Quite a useful technique for reproducible builds or stuff like that - let's say you have this source file saved as hello.go:

package main

import "fmt"

var (
    who = "World"
)

func main() {  
    fmt.Println("Hello,", who)
}

Then you can use go run (or other build commands like go build or go install) with the -ldflags option to modify the value of the who variable:

$ go run hello.go
Hello, World.  
$ go run -ldflags="-X main.who=Nick" hello.go
Hello, Nick

The format is importpath.name string, so you can set the value of any string in your program, not just in main.

https://dave.cheney.net/2014/10/17/functional-options-for-friendly-apis

Modules and Golang

• So you used Dep and want to migrate to modules
• The Official Mod Tutorial
• Just Tell Me How To Use Modules - a quickstart

Using the DAO / Repo pattern for Golang - https://adodd.net/post/go-ddd-repository-pattern/

Log

https://play.golang.org/p/8jtyDzQZZ3h ... but ideally use uber/zip or similar.

Testing Code Effectively with Go

0. Test what matters
1. Test the right level
2. Mock but dont be silly
3. func fields make testing easier (see Safety culture article on better mocking)
4. test failures point to the problems.

Better mocking
https://medium.com/safetycultureengineering/flexible-mocking-for-testing-in-go-f952869e34f5

How to test / The art of testing
https://www.youtube.com/watch?v=EOpj9aZ8Kfo
Slides

Good tests dont mock databases
https://qvault.io/clean-code/writing-good-unit-tests-dont-mock-database-connections/

What not to use
https://www.youtube.com/watch?v=5DVV36uqQ4E