eval.md

Practical Considerations of Intercepting Clojure evaluation

One important extension that Gradually Typed Clojure will require is hooking into the standard Clojure evaluation process. There are at least two motivations for this:

1. As discussed in previous articles, it is often necessary to insert extra checks in code depending on how it is used.
2. Typed Clojure can uncover type-based optimisations, such as identifying a missing type hint, and we want to automatically apply this.

The existing prototype has been a useful experiment which has revealed some of the subtleties of Clojure evaluation.

The basic approach is to use nREPL middleware to type check REPL interactions.

Handlers

The first step in creating middleware is to provide nREPL with a wrapper function via mid/set-descriptor!

(ns clojure.core.typed.repl
  (:require [clojure.tools.nrepl.middleware :as mid]
            [clojure.tools.nrepl.transport :as transport]

(defn wrap-clj-repl [ ... ] ... )

(mid/set-descriptor! #'wrap-clj-repl
  {:requires #{"clone"}
   :expects #{"eval"}
   :handles {}})

I don't know what the :requires, :expects and :handles options do yet, but it tells nREPL when to fire your middleware. If nREPL has a message it needs to "eval", then it will send the packet to your function because we have "eval" in our :expects set.

The type of wrap-clj-repl is

(defalias nREPLPacket '{:op String ... })
(defalias Handler [nREPLPacket -> Any])

(defn wrap-clj-repl [handler :- Handler] :- Handler
   (fn [msg]
     ...))

The output of a handler function is based on side effects, hence the Any return type. The nREPLPacket alias is a stub; it really looks something like (U '{:op (Val "eval") ... } '{:op (Val "interrupt)} ...).

nREPL Packets

It would be useful at this point to demonstrate what an nREPL packet looks like.

First we need to add this to our project.clj to inform nREPL we are providing it middleware to use as early as possible.

  :repl-options {:repl-middleware [clojure.core.typed.repl/wrap-clj-repl]}

Let's define our handler function like this.

(defn wrap-clj-repl [handler :- Handler] :- Handler
   (fn [msg]
      (prn "Log msg")
      (prn msg)
      (prn "End Log")
      (handler msg)))

This first prints the packet then delegates the processing of our message to the handler nREPL provides us.

user=> 42
"Log msg"
{:code "42",
 :id "87e87d0a-1093-4000-b691-eec5d2c9d7b3",
 :op "eval",
 :session
 #<Atom@1265cf2: 
   {#<Var@a4f040: true> true,
    #<Var@3fe922: true> true,
    #<Var@4f5403: 1024> 1024,
    #<Var@65db85: nil> nil,
    #<Var@8fff06: 10> 10,
    #<Var@16cfb07: {}> {},
    #<Var@e0cdc9: 1056> 1056,
    #<Var@888fc9: 
      #object[java.io.OutputStreamWriter 0x109958a "java.io.OutputStreamWriter@109958a"]>
    #object[java.io.PrintWriter 0xad782e "java.io.PrintWriter@ad782e"],
    #<Var@9003aa: 1> 1,
    #<Var@1d4894c: nil> nil,
    ...}>,
 :transport
 #object[clojure.tools.nrepl.transport.FnTransport 0x1615682 "clojure.tools.nrepl.transport.FnTransport@1615682"]}
"End Log"
42

We can roughly type an "eval" packet like so:

(defalias VarMappings  (Map (Var1 Any) Any))
(defalias EvalPacket
   '{:op (Val "eval")
     :code Str
     :id Str
     :session (Atom1 VarMappings)
    :transport transport/FnTransport})

When I first demonstrated this to Sam Tobin-Hochstadt, we were both puzzled as to why :code was a String rather than just a normal Clojure form. Surely this form as already passed the reader, and we have redundantly converted it back to a string.

user=> (pr-str (read-string "42"))
"Log msg"
{:code "(pr-str (read-string \"42\"))" ...}
"End Log"
"42"

I suspect this is because nREPL intends to send these forms over the wire to clients. I also discovered the rich information that can be encoded in string with *print-dup* on, so you're really not losing any syntactic information.

user=> (binding [*print-dup* true] (pr-str '(let [^String x 1] x)))
"^#=(clojure.lang.PersistentArrayMap/create {:line #=(java.lang.Integer. \"1\"), :column #=(java.lang.Integer. \"38\")}) (let [^String x 1] x)"

Handling "eval" Packets

The next step is figuring out whether our handler should actually touch the packet at all. This is achieved with a simple test the packet's :op.

To demonstrate, let's change every "eval" interaction of 41 to 42.

(defn wrap-clj-repl [handler :- Handler] :- Handler
  (fn [{:keys [op code] :as msg}]
    (if (= "eval" op)
      (handler (assoc msg :code (if (= "41" code) 
                                  (-> code read-string inc pr-str)
                                  code)))
      (handler msg))))

user=> 24
24
user=> 41
42

Neat!

Next: Type checking

There are a bunch of other considerations if we want to type check a form:

• when do we type check?
• what happens if there's a type error?
• do we evaluate the output of type checking?
• what if other middleware garbles the :code in our "eval" packet to where it is pointless to type check?
• where do we want the type-checking middleware to sit in relation to other middlewares?

We will address these in the next article.

To demonstrate, let's change every "eval" interaction of 42 to 24.

...but then you demonstrate how to inc the input :-)

Fixed :)