Let's hack with Go!

In this Gist, I'll show you how to get a simple HTTP/JSON server working with the Go programming language.

Link to this guide: goo.gl/cDvRKj.

Thanks to Evan for critiquing this guide!

Intro

Go is an open source programming language made by Google and used by many developers around the world. It aims to address common backend development headaches while promoting a simple and friendly user interface. If you've used C or Python before, Go will feel kind of familiar.

In this guided tour, I will show you how to set up a simple HTTP server that can be used to send JSON (a simple data encoding format) to a frontend application.

First, you should install Go:

You can install Go through your platform's package manager (e.g. brew, yum, etc).

Otherwise, download the archive from the Golang website.

Next, follow the install instructions.

Here are some alternative directions that might be more user-friendly.

Getting started

Now, let's test that everything works correctly.

$ go version # this should produce output

Let's write a simple "hello world" program:

$ echo $GOPATH # this should print something
$ mkdir -p $GOPATH/src/github.com/<github_username>/cunyhack
$ cd $_
$ $EDITOR hello.go

Write the following (comments optional):

// "package main" denotes that this file (package) is the one we intend to execute.
package main

// Imports go after the package declaration.
// They can be grouped in parenthesis like so:
import (
	"fmt"
)
// Package fmt is for formatting and printing strings and other things. Quite useful!

// Much like in C or Java, we denote the first thing to execute via a `main` function.
func main() {
	fmt.Printf("hello world!\n")
}

Finally, let's run it!

$ go run hello.go # this is one way to run Go code
# this is the other, preferred way
$ go build hello.go # this creates an executable called `hello`
$ ./hello

Installing our dependency

We'll use the gorilla toolkit and its mux library, in particular.

To install the library, run:

$ go get -u github.com/gorilla/mux

go get pulls the source code from the given repo (in this case, gorilla/mux on GitHub) and saves it to your $GOPATH's source tree. That way, any other project can use it, too.

Similar to importing fmt, we can import mux by writing import "github.com/gorilla/mux".

Milestone 1: A `hello world` server.

In Go, the model of HTTP servers is that they have a set of paths on which they accept requests, which are handled by written responses. The gorilla mux helps us solve this problem quite simply.

Therefore, at the heart of our simple server, we need to write "hello world" to a response object. Luckily, it is easy in gorilla:

r := mux.NewRouter()
r.HandleFunc("/", func(res http.ResponseWriter, req *http.Request) {
	res.Write([]byte("hello world!\n"))
})

If you squint, you can see how res.Write is giving us similar behavior to fmt.Print.

Now, we wrap everything in the standard packaging and write it to server.go:

package main

// We need `net/http` to serve our HTTP handler.
import (
	"net/http"

	"github.com/gorilla/mux"
)

func main() {
	// The mux router will handle all of our requests and reponses using matching logic.
	r := mux.NewRouter()

	// When someone makes a request to the root path "/", respond with "hello world!\n".
	r.HandleFunc("/", func(res http.ResponseWriter, req *http.Request) {
		res.Write([]byte("hello world!\n"))
	})

	// Once your code hits this line, the program will appear to hang.
	// It is just happily serving requests!
	http.ListenAndServe(":5678", r) // We serve on port 5678.
}

Now, we can run go build server.go

Finally, we can test that our server works in the terminal:

$ ./server & # run the server in the background
$ curl localhost:5678/ # notice the port!
# you should see `hello world!`
$ pkill server # kill the server

Milestone 2: Using multiple handlers.

A server that can only handle the "/" route is kind of boring.

Let's add some more routes.

First, let's create a subrouter. I know a passable amount of Spanish, so let's start there:

func main() {
	r := mux.NewRouter()

	spanishRouter := r.PathPrefix("/spanish").Subrouter()

	// Router setup below.

Before we start adding handlers for our Spanish router, let's add a helper function that takes the string we want to respond with and returns a function we can pass to HandleFunc. We can write almost exactly what we had for the first exercise.

func respondWithString(s string) func(http.ResponseWriter, *http.Request) {
	return func(w http.ResponseWriter, r *http.Request) {
		w.Write([]byte(s))
	}
}

Now, let's finish our Spanish work:

// Router setup..
spanishRouter.HandleFunc("/hello", respondWithString("hola\n"))
spanishRouter.HandleFunc("/bye", respondWithString("adios\n"))
spanishRouter.HandleFunc("/", respondWithString("raiz\n"))

// Refactor the previous implementation to use our new function.
r.HandleFunc("/", respondWithString("hello world!\n")

Again, let's test our implementation:

$ go build server.go
$ ./server &
$ curl localhost:5678/spanish/ # should see 'raiz'
$ curl localhost:5678/spanish/bye # should see 'adios'
$ pkill server

Milestone 3: Using URL parameters.

So we have several handlers set up now! We have a pretty decent server, but all of our responses are static. What if our client wants to pass some parameters to our routes?

For this milestone, let's add the feature that the user can pass a boolean-like parameter in the URL that allows them to uppercase the response they get back. We'll name this variable up.

If you've ever seen a URL have something like example.com/?foo=bar&baz=quux, these are parameters at work!

We just need to update respondWithString and import the strings package from the standard library:

import (
	"net/http"
	"strings"

	"github.com/gorilla/mux"
)

// ...

func respondWithString(s string) func(http.ResponseWriter, *http.Request) {
	return func(w http.ResponseWriter, r *http.Request) {
		message := s
		
		// Functions can return multiple things in Go. Here's what going on below:
		// Look for a value in the query parameters map using the key "up".
		// `upVals` is the list we get back.
		// `ok` is a boolean indicating whether the key was present in the map.
		upVals, ok := r.URL.Query()["up"]
		
		if ok && len(upVals) == 1 && upVals[0] == "true" {
			message = strings.ToUpper(s)
		}

		w.Write([]byte(message))
	}
}

Now, any of our handlers using respondWithString can use this request variable! Let's test.

$ go build server.go
$ ./server &
$ curl localhost:5678/spanish/bye # should see 'adios'
$ curl localhost:5678/spanish/bye?up=true # should see 'ADIOS'
$ pkill server

Milestone 4: Using `mux.Vars` for dynamic routes.

Up to this point, we defined all of our routes statically. Sure, parameters help provide additional data to our server, but what if we want to have routes defined dynamically? Dynamics routes provide more readable and semantic paths, as well as the ability to "automatically" create new paths based on external factors. For example, if we were building a shopping site, we could have our server allow new routes just by having new items in our database.

We can accomplish dynamic routing through request variables!

Calling mux.Vars(r) gives us a map of request variable name to request variable value.

(The user can also pass data via the request body. Try this as a follow-up activity!)

Let's make a dynamic handler that takes any word and returns a Mocking SpongeBob verison of the word to the client.

First, add the handler with the dynamic route.

func main() {
  // Main and Spanish router above...

  spongebobRouter := r.PathPrefix("/spongebob").Subrouter()

  spongebobRouter.HandleFunc("/{word}", respondWithString(""))

  // http.ListenAndServe call below...
}

Notice that we put curly braces around the route /{word}. This tells the spongebobRouter that if we see a full route like /spongebob/<something>, it should handle it.

Furthermore, the curly braces show that we capture whatever comes after the /spongebob/. By using mux.Vars, we can access the captured string.

Now, update respondWithString:

func respondWithString(s string) func(http.ResponseWriter, *http.Request) {
	return func(w http.ResponseWriter, r *http.Request) {
		var message string // What is written to the client.

		if word, ok := mux.Vars(r)["word"]; ok {
			message = spongebob(word) + "\n"
		} else if okParam(r.URL.Query(), "up", "true") {
			message = strings.ToUpper(s)
		} else {
			message = s
		}

		w.Write([]byte(message))
	}
}

Finally, let's implement the spongebob function. I also added the okParam helper function to reduce param handling noise. Be sure to add any standard library imports.

func spongebob(s string) string {
	var cpy []rune
	for _, sr := range []rune(s) {
		var cr rune
		if rand.Float32() > 0.5 {
			cr = unicode.ToUpper(sr)
		} else {
			cr = unicode.ToLower(sr)
		}

		cpy = append(cpy, cr)
	}
	return string(cpy)
}

func okParam(values map[string][]string, key, wantValue string) bool {
	valueList, ok := values[key]
	return ok && len(valueList) == 1 && valueList[0] == wantValue
}

After all of those edits, your server.go should look like this:

package main

import (
	"math/rand"
	"net/http"
	"strings"
	"unicode"

	"github.com/gorilla/mux"
)

func main() {
	r := mux.NewRouter()

	spanishRouter := r.PathPrefix("/spanish").Subrouter()

	spanishRouter.HandleFunc("/hello", respondWithString("hola\n"))
	spanishRouter.HandleFunc("/bye", respondWithString("adios\n"))
	spanishRouter.HandleFunc("/", respondWithString("raiz\n"))

	spongebobRouter := r.PathPrefix("/spongebob").Subrouter()
	spongebobRouter.HandleFunc("/{word}", respondWithString(""))

	r.HandleFunc("/", respondWithString("hello world!\n"))

	http.ListenAndServe(":5678", r)
}

func respondWithString(s string) func(http.ResponseWriter, *http.Request) {
	return func(w http.ResponseWriter, r *http.Request) {
		var message string // What is written to the client.

		if word, ok := mux.Vars(r)["word"]; ok {
			message = spongebob(word) + "\n"
		} else if okParam(r.URL.Query(), "up", "true") {
			message = strings.ToUpper(s)
		} else {
			message = s
		}

		w.Write([]byte(message))
	}
}

func spongebob(s string) string {
	var cpy []rune
	for _, sr := range []rune(s) {
		var cr rune
		if rand.Float32() > 0.5 {
			cr = unicode.ToUpper(sr)
		} else {
			cr = unicode.ToLower(sr)
		}

		cpy = append(cpy, cr)
	}
	return string(cpy)
}

func okParam(values map[string][]string, key, wantValue string) bool {
	valueList, ok := values[key]
	return ok && len(valueList) == 1 && valueList[0] == wantValue
}

Let's test:

# build and launch as usual...
$ curl localhost:5678/spongebob/krabby_patty
$ pkill server

Milestone 4: Using JSON.

So far, we've been able to send and receive data pretty easily with out server. However, it's rare that API servers (like ours) send data in simple text format. Usually, you want to wrap your data in some structure, especially if you want to respond to clients with more complex data.

For this workshop, we'll use JSON, which should sound familiar if you've used JavaScript before. JSON is nice because it is a format that many programming languages understand, as well as being human-readable (easy to debug ;)).

If you are interested in using more advanced encoding formats, look into Protocol Buffers, which is what we use at Google.

We'll keep things simple and respond to clients with a simple structure of a string message and a number representing the current Unix time.

First, let's define our struct. Feel free to put this where ever you like in server.go, but I recommend putting it near where it is (or will be) used. Notice that we add struct tags using backtick strings (`, not ').

type payload struct {
	Message string `json:"message"`
	Num     int64  `json:"num"`
}

It's important to note that even though payload is unexported, we need to export its fields for the Go JSON library.

We also annotate the fields with struct tags to show how the data should be marshalled. For example, even though the field is named Message, in JSON, it should say message.

Next, we'll add our JSON route:

jsonRouter := r.PathPrefix("/json").Subrouter()
jsonRouter.HandleFunc("/{message}", respondWithJSON(""))

And then the route's handler:

func respondWithJSON(s string) func(http.ResponseWriter, *http.Request) {
  return func(w http.ResponseWriter, r *http.Request) {
    // Let clients know that we're sending JSON content and not just simple text.
    w.Header().Set("Content-Type", "application/json")
    message, _ := mux.Vars(r)["message"]
    // Respond with a payload of whatever message we got and whatever the current unix time is.
    p := payload{Message: message, Num: time.Now().Unix()}
    w.WriteHeader(http.StatusOK)
    json.NewEncoder(w).Encode(p)
  }
}

After adding any necessary imports from the standard library, your code should look something like this:

package main

import (
	"encoding/json"
	"math/rand"
	"net/http"
	"strings"
	"time"
	"unicode"

	"github.com/gorilla/mux"
)

func main() {
	r := mux.NewRouter()

	spanishRouter := r.PathPrefix("/spanish").Subrouter()

	spanishRouter.HandleFunc("/hello", respondWithString("hola\n"))
	spanishRouter.HandleFunc("/bye", respondWithString("adios\n"))
	spanishRouter.HandleFunc("/", respondWithString("raiz\n"))

	spongebobRouter := r.PathPrefix("/spongebob").Subrouter()
	spongebobRouter.HandleFunc("/{word}", respondWithString(""))

	jsonRouter := r.PathPrefix("/json").Subrouter()
	jsonRouter.HandleFunc("/{message}", respondWithJSON(""))

	r.HandleFunc("/", respondWithString("hello world!\n"))

	http.ListenAndServe(":5678", r)
}

func respondWithString(s string) func(http.ResponseWriter, *http.Request) {
	return func(w http.ResponseWriter, r *http.Request) {
		var message string // What is written to the client.

		if word, ok := mux.Vars(r)["word"]; ok {
			message = spongebob(word) + "\n"
		} else if okParam(r.URL.Query(), "up", "true") {
			message = strings.ToUpper(s)
		} else {
			message = s
		}

		w.Write([]byte(message))
	}
}

type payload struct {
	Message string `json:"message"`
	Num     int64  `json:"num"`
}

func respondWithJSON(s string) func(http.ResponseWriter, *http.Request) {
	return func(w http.ResponseWriter, r *http.Request) {
		w.Header().Set("Content-Type", "application/json")
		message, _ := mux.Vars(r)["message"]
		p := payload{Message: message, Num: time.Now().Unix()}
		w.WriteHeader(http.StatusOK)
		json.NewEncoder(w).Encode(p)
	}
}

func spongebob(s string) string {
	var cpy []rune
	for _, sr := range []rune(s) {
		var cr rune
		if rand.Float32() > 0.5 {
			cr = unicode.ToUpper(sr)
		} else {
			cr = unicode.ToLower(sr)
		}

		cpy = append(cpy, cr)
	}
	return string(cpy)
}

func okParam(values map[string][]string, key, wantValue string) bool {
	valueList, ok := values[key]
	return ok && len(valueList) == 1 && valueList[0] == wantValue
}

Let's test our JSON route:

$ go build server.go
$ ./server &
$ curl localhost:5678/json/it_works
# You should see the payload with the `message` and `num` fields.
# If you want to have a better visualization of the data, install `jq`:
$ curl localhost:5678/json/it_works | jq
# Or using Python:
$ curl localhost:5678/json/it%20works | python -m json.tool
$ pkill server

Milestone 5: Communicating with a frontend.

So now we have a nice little API server that responds to certain requests with JSON. Besides other backend servers (or curl), our server can communicate with a web frontend!

First, here is a basic HTML+JavaScript file that calls our server with the route /json/live_test and expects a JSON response, using the fetch API. It will print the JSON that it got to the web page, or it will show the not-so-descriptive error it got while trying to fetch.

Since the focus of this workshop is not frontend development, feel free to copy and paste the content below into a file called fetch_test.html.

<!DOCTYPE html>
<meta charset="utf-8">
<title>Fetch Test</title>
<body>
  <h1>Welcome to the Fetch Test!</h1>
  <h2>Result</h2>
  <pre id='result'>Nothing yet...</pre>
  <h2>Error</h2>
  <pre id='error'>Nothing yet...</pre>
  <script>
    // Run the following when all the HTML is interactive.
    self.onload = () => {
      console.log('loaded!');
      // Fetch the JSON from our server.
      fetch('http://localhost:5678/json/live_test').then(response => {
        // Return the JSON embedded in the server's response.
        // https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch#Response_objects.
        return response.json();
      }).catch(err => {
        // Otherwise, something bad happened when fetching!
        console.error('Error: ', err);
        document.getElementById('error').textContent = err.toString();
      }).then(json => {
        // Once we get our final JSON result, put it in the result <pre>.
        console.log('JSON result: ', json);
        document.getElementById('result').textContent = JSON.stringify(json);
      });
    };
    console.log('present!');
  </script>
</body>

In your browser (I tested with Chrome), you can then navigate to file://<path-to>/fetch_test.html to see the HTML. You will most definitely not be able to get a valid response from your server (especially if it isn't running). Your problem will be caused by CORS, which is a security check to stop clients from pulling data from unknown servers. Let's get back to Go and fix that! Thankfully, there's not too much left to get this working.

First, we need to install the gorilla/handlers package that gives us CORS support out of the box:

$ go get -u github.com/gorilla/handlers

Then, we change the http.ListenAndServe line in main to be the following:

http.ListenAndServe(":5678", handlers.CORS()(r))

Now, start up your server, and refresh the fetch_test.html page in your browser. You should now have a valid and successful response!

Just in case you're curious, your server.go code should look like this:

package main

import (
	"encoding/json"
	"math/rand"
	"net/http"
	"strings"
	"time"
	"unicode"

	"github.com/gorilla/handlers"
	"github.com/gorilla/mux"
)

func main() {
	r := mux.NewRouter()

	spanishRouter := r.PathPrefix("/spanish").Subrouter()

	spanishRouter.HandleFunc("/hello", respondWithString("hola\n"))
	spanishRouter.HandleFunc("/bye", respondWithString("adios\n"))
	spanishRouter.HandleFunc("/", respondWithString("raiz\n"))

	spongebobRouter := r.PathPrefix("/spongebob").Subrouter()
	spongebobRouter.HandleFunc("/{word}", respondWithString(""))

	jsonRouter := r.PathPrefix("/json").Subrouter()
	jsonRouter.HandleFunc("/{message}", respondWithJSON("got your message"))

	r.HandleFunc("/", respondWithString("hello world!\n"))

	http.ListenAndServe(":5678", handlers.CORS()(r))
}

func respondWithString(s string) func(http.ResponseWriter, *http.Request) {
	return func(w http.ResponseWriter, r *http.Request) {
		var message string // What is written to the client.

		if word, ok := mux.Vars(r)["word"]; ok {
			message = spongebob(word) + "\n"
		} else if okParam(r.URL.Query(), "up", "true") {
			message = strings.ToUpper(s)
		} else {
			message = s
		}

		w.Write([]byte(message))
	}
}

type payload struct {
	Message string `json:"message"`
	Num     int64  `json:"num"`
}

func respondWithJSON(s string) func(http.ResponseWriter, *http.Request) {
	return func(w http.ResponseWriter, r *http.Request) {
		w.Header().Set("Content-Type", "application/json")
		message, _ := mux.Vars(r)["message"]
		p := payload{Message: message, Num: time.Now().Unix()}
		w.WriteHeader(http.StatusOK)
		json.NewEncoder(w).Encode(p)
	}
}

func spongebob(s string) string {
	var cpy []rune
	for _, sr := range []rune(s) {
		var cr rune
		if rand.Float32() > 0.5 {
			cr = unicode.ToUpper(sr)
		} else {
			cr = unicode.ToLower(sr)
		}

		cpy = append(cpy, cr)
	}
	return string(cpy)
}

func okParam(values map[string][]string, key, wantValue string) bool {
	valueList, ok := values[key]
	return ok && len(valueList) == 1 && valueList[0] == wantValue
}

Conclusion

There you have it -- a simple HTTP server that can communicate with web frontends using JSON. Now, you should be dangerous enough to start adding your own cool features to your server. Good luck and enjoy Go!

Ideas for new features:

Add and play around with more parameters and dynamic routes.
Add a classic 404 page.
Pass the port as a flag.
Add rate limiting to your server.
Read data from and write data to text files or a database.
Pass data in the request body.
Try switching JSON for ProtoBufs.
Serve a static file: https://github.com/gorilla/mux#static-files.
Pair your server with your partners frontend.
Add more complex data and structures to your program and its JSON.
Refactor and write tests.

LOZORD/go_talk.md

Let's hack with Go!

Intro

Getting started

Installing our dependency

Milestone 1: A `hello world` server.

Milestone 2: Using multiple handlers.

Milestone 3: Using URL parameters.

Milestone 4: Using `mux.Vars` for dynamic routes.

Milestone 4: Using JSON.

Milestone 5: Communicating with a frontend.

Conclusion

Ideas for new features:

WowSuchRicky commented Nov 12, 2017

Uh oh!

ekivolowitz commented Nov 17, 2017

Uh oh!

LOZORD/go_talk.md

Let's hack with Go!

Intro

Getting started

Installing our dependency

Milestone 1: A hello world server.

Milestone 2: Using multiple handlers.

Milestone 3: Using URL parameters.

Milestone 4: Using mux.Vars for dynamic routes.

Milestone 4: Using JSON.

Milestone 5: Communicating with a frontend.

Conclusion

Ideas for new features:

WowSuchRicky commented Nov 12, 2017

Uh oh!

ekivolowitz commented Nov 17, 2017

Uh oh!

Milestone 1: A `hello world` server.

Milestone 4: Using `mux.Vars` for dynamic routes.