Clojure Club Recap: Loading JSON

loading_json_recap.md

Loading some JSON from the intertubes

To make libraries available, add them to the :dependencies vector in your project.clj. Remember that you can create a new project folder with lein new [<template-name>] <project-name>. Here are the dependencies used for this JSON loading adventure:

:dependencies [[org.clojure/clojure "1.7.0"]
               [clj-http "2.0.0"]
               [cheshire "5.5.0"]
               [camel-snake-kebab "0.3.2"]]

The rest of this stuff should just be entered in the REPL.

(require '[clj-http.client :as http])
(require '[cheshire.core :as cheshire])
(require '[camel-snake-kebab.core :refer [->kebab-case-keyword]])

(def data-url "https://gist.githubusercontent.com/amacdougall/749749b9e5837f6de6b4/raw/a2fb3795756b0ab7d0901da489a1f08faf818113/json_dataset.json")
(def data (atom nil))

Now to actually load the JSON. Each of the following expressions has the same effect:

This is the most straightforward, but also harder to read. It is equivalent to just doing a set of nested function calls in JS or Ruby, such as foo(bar(baz(subject))).

(reset! data (cheshire/parse-string (:body (http/get data-url)) ->kebab-case-keyword))

The -> macro is used for chain-like behavior. For more detail on exactly how it works, check the beginning of Chapter 8 of The Joy of Clojure. For basic documentation, check out https://clojuredocs.org/clojure.core/-%3E It is equivalent to doing a chained or "fluent" method call in JS or Ruby, such as foo().bar().baz().

(reset! data (-> data-url
                 (http/get)
                 (:body)
                 (cheshire/parse-string ->kebab-case-keyword)))

Note how each of these loading calls gives ->kebab-case-keyword to cheshire/parse-string as a post-load key munging function.

The let form lets you define bindings (like short-lived constants), which are valid only within the let. You can define a binding in terms of a previous one, like this. This is equivalent to defining vars one by one in JS or Ruby.

(let [response (http/get data-url)
      body (response :body)
      new-value (cheshire/parse-string body ->kebab-case-keyword)]
  (reset! data new-value))

In all three cases, the data atom now has an idiomatic Clojure representation of the JSON data. Access it by dereferencing the atom with (deref data) or @data. Note that this would still work without giving Cheshire the ->kebab-case-keyword function, but the resulting data structure would use strings as keys instead of Clojure keywords, which would be more awkward to work with. (If you're dealing with a huge JSON, you might want the efficiency of skipping the key conversion step.)

Exercises, if you're into that:

This is a preview of next session. If you do it ahead of time, you can help people next week. If everyone does it ahead of time, we can do something else.

Use conj.io for quick function reference. If you know you want to do something to a map or vector, look up the data type instead.

1. Find unique eye colors. Look up distinct, map, etc. Remember that you can use a keyword as a map function: (map :name @data).
2. Filter list by eye color. You probably need to use an anonymous function.
3. Make a standalone filter function with defn and then use it in the above.
4. Filter using a partially applied filter function; look up partial.
5. Find distinct tags -- you probably want to use flatten; ->> might also be helpful, but let works too.
6. Get only users with a certain tag, using techniques shown above.