Using existing namespaces from cljs.js

cljs-js-analysis-cache-usage.md

Let's say you want to use the new cljs.js bootstrapped compiler to eval code in your project. Using cljs.js/eval looks like this:

(ns foo.try-eval
  (:require [cljs.js :as cljs]))

(let [c-state (cljs/empty-state)]
  (cljs/eval c-state                       ;compiler state
             `(+ 1 1)`                     ;form to eval
             {:eval cljs/js-eval           ;options
              :context :expr} 
             (fn [{:keys [value error]}]   ;callback
               (if error (prn "error" error)
                  (prn "value is:" value)))))

Beautifully simple, and two things to note:

1. The first thing we do is create a new, empty compiler environment with (cljs/empty-state). This compiler state comes pre-loaded with 'cljs.core, so you can run all the normal functions you'd expect, but it does not know anything about the rest of your project.
2. By default, this code will be evaluated in the namespace cljs.user.

Now let's say you have some helper methods you want to use, and you've put them in the namespace eval-helpers:

(ns 'eval-helpers
  (:require [foo.http :refer [GET]]
            ...))

You want to eval the form:

(GET " https://hacker-news.firebaseio.com/v0/item/8863.json?print=pretty)

If you just swap out the form in the first example, this won't work, for two reasons. First, the code is evaluated in the namespace cljs.user, not foo.eval-helpers. This is easy enough to fix: we can include :ns 'foo.eval-helpers in the compiler options, and it will attempt to eval from that namespace.

But that won't work, because of our second problem: the compiler state doesn't know anything about the rest of your project. It hasn't any idea what to do with 'foo.eval-helpers. c-state is still empty.

Solving this problem will involve learning a bit about a relatively new feature in the ClojureScript. When the experimental :cache-analysis compiler option is true (which is the default for REPLs and :optimizations :none), building a ClojureScript project writes a slew of *.cache.edn files to your output folder, alongside source and compiled js files. Given a project called foo with :output-dir "resources/public/js/compiled/out", a tiny subset of out might be

out
└── cljs
    ├── analyzer.cljc
    ├── analyzer.cljc.cache.edn
    ├── analyzer.cljc.js
    ├── analyzer.cljc.js.map
    ├── compiler.cljc
    ├── compiler.cljc.cache.edn
    ├── compiler.js
    └── ...
└── foo
    ├── core.cljs
    ├── core.cljs.cache.edn
    ├── ...
    ├── eval_helpers.cljs
    ├── eval_helpers.cljs.cache.edn
    └── ...

These *.cache.edn files contain the output of running the ClojureScript analyzer over a source file, and are saved to speed up re-compile times as you work on a project. They populate the compiler state to enable validation, optimization, and static access to var info - they tell the compiler what a pre-compiled JS file "means". By caching them, we prevent the compiler from having to do all this work for your entire project every time you make a small change to a single file.

But we can also use them when we want to eval code using cljs.js, and want the compiler to become aware of existing namespaces in a project. To do this we need to follow a couple of steps:

1. Decide which namespaces you need the compiler to be aware of (don't do this except where necessary)
2. Copy the *.cache.edn files for those namespaces to a public location. While developing, you can access them from your output folder, but they are wiped out when you run lein clean and are obviously not created when using :optimizations :simple. You need to make sure these files are updated whenever the source changes. I put mine in resources/public/js/caches, and mirror the same internal directory structure as my output folder.
3. From your app, before you try to eval from an existing namespace, fetch the the appropriate *.cache.edn file and load it into your compiler state using cljs.js/load-analysis-cache!

I am currently using the following load-cache function:

(defn load-cache
 ([cstate s] (load-cache cstate s {}))
 ([cstate s opts]
  (let [ext (or (:ext opts) :cljs)]
    (go
      (let [path (str "js/caches/" (clojure.string/replace (name s) "." "/") "." (name ext) ".cache.edn")
            cache-edn (<! (GET path))
            cache (read-string cache-edn)]
        (cljs.js/load-analysis-cache! cstate s cache)
        cache)))))

A couple of notes:

• in my ns I require [cljs.reader :refer [read-string]], and I am using a very simple GET function
• David Nolen's example with cljs.core uses transit to encode the edn file. I imagine this is better; for development it's handy to read edn directly.
• I'm sure there are improvements that can be made to this process, & would welcome feedback.

Much of my understanding of cljs.js came from helpful conversations with @mfikes and @dnolen on Slack - thanks!