The following was sent to the JACL e-mail announcement list on 2021-02-26.
Hello everyone, and a belated happy 2021 to you! I hope your year has been off to a good start.
It's been almost a year since JACL became public, and nearly as long since any other announcement or release, so I thought to share three major related educational and design developments. The project is very much alive, just mostly in my head :-)
First, while I haven't made significant tangible progress on the JACL compiler or runtime since last year, I have continued to research and study implementation techniques. I have prototyped several schemes for efficient and JavaScript-friendly multiple-value returns, consuming relevant literature (mostly from the Scheme community) along the way. I have also studied and experimented with CLOS, and to speculate about how best to support a maximum of CLOS functionality without compromising JACL's calculated relationship with the JavaScript platform. One interesting study along the way has been that of ISLISP, and of its CLOS subset. I liked what I saw of ISLISP so much that I briefly considered standardizing JACL on ISLISP instead of Common Lisp. Alas, I discovered ISLISP doesn't specify a package system, which I regard as a critical feature. A Common Lisp subset, JACL will remain.
Second, I have begun to flirt with the ideas of compiling ECL FASL (.fasc) to JavaScript, or implementing an ECL bytecode interpreter in JavaScript, or some combination of the two. Of the major extant CL implementations, I suspect ECL is the most amenable to build on, for the purposes of JavaScript delivery. I'm attracted to the ECL route because it would significantly reduce the scope of the JACL project by leaning heavily on ECL's runtime, the relevant C parts of which should port to JavaScript easily. It would also limit design work to that of efficient bytecode interpretation or compilation; most other aspects of implementation would be rote. Because so much of ECL is written in C, self-hosting is an unlikely mid-term prospect; however, some degree of runtime availability of a complete Lisp environment may be possible, at least in development, by compiling ECL's C bits to WASM via Emscripten or similar. Longer term, alignment with ECL facilitates future, speculative alignment with SICL, to the extent that SICL-via-ECL seems likely, based on the active maintainership of ECL and the high velocity of the SICL project. Such a union would be another means to self-host.
Finally, and maybe most importantly, I've made significant progress on a once-vague conception of how to best build UIs and manage data reactively in the browser. Having a good Common Lisp in the browser will help me ultimately make UIs most comfortably in this way that I envision, but isn't requisite.
UI maintenance is something like a data-flow problem, in the sense that a UI program primarily defines and manages relationships between UI components such as buttons or text areas. The choice of component combinators by the toolkit designer dominates the flavor of the resulting development experience.
JavaScript's native data model is weak (compared to Common Lisp), and the DOM adds stateful UI representation objects, but only the most basic means of composing them. Consequently, contemporary browser UI toolkits generally either introduce their own stateful representation objects/conventions/combinators (MVC is one example), or they facilitate mapping JavaScript data to DOM structures through the addition of an immediate-mode layer (React) in front of the underlying stateful DOM elements. React allocates, deallocates, and manipulates the stateful DOM objects "behind the scenes", effectively presenting an immediate-mode interface to a retained-mode display device (the browser).
One fruitful but heavy approach is to impose value semantics on all data at the app level, and manage relationships between UI components in terms of values. This approach is fruitful because it facilitates functional programming, but heavy because it requires significant additional runtime in the form of (nestable) persistent data structures. It also obviates the need for the React layer, because values and their difference map directly to stateful DOM objects, and facilitate their management. The Hoplon framework I helped with, based on ClojureScript, works this way. Common Lisp never seemed amenable to the Hoplon approach because it doesn't promote general structural equality/value semantic (the definition of such things are left to the user).
In the course of writing many Hoplon apps I became keenly aware of the fact that most of the transformations I performed on values shuttled between UI components were relational, in the sense that most constituent values were tabular, and that most combination operations were relational. This experience planted the seed of the idea that the ideal data model for tying UI components together was reactive and relational, but I didn't have a concrete idea about how to do it any more efficiently than persistent data structures already (relatively inefficiently) did.
As of last week, I do. I revisited some old Javelin code (the reactive value propagation library part of Hoplon) and I was able to cleanly separate traversal of the "cell" dependency graph from the value semantic of graph components. This allowed me to see how I could store mutable sets in the graph, and efficiently propagate only their difference. This suggests strongly the viability of a UI toolkit paradigm that forwards a relational data model (and change propagation based on set difference) that lays nicely atop JavaScript's data model, and doesn't necessitate the generality (or runtime baggage) of persistent collections. I have to study it more, but I believe this approach bears some resemblance to McSherry's "Differential Dataflow". I think there's also a lot of literature from the SQL side in this area around the topic of "incremental view maintenance".
The discovery of this possibility is a great relief, because until I encountered it I wasn't absolutely certain the presence of CL would improve the JavaScript app development experience enough to overcome the underlying, dominating problem of UI component maintenance. In this new and promising approach, I hope to have found a way to make UIs efficiently, without much runtime, in a way that's REPL friendly, and that can be extended and improved eventually in JACL (through macros and generic functions).
Whew, that was a lot longer of an update than I expected, but if you got his far, you probably don't mind :-)
Thanks for reading, and thanks for your support of the JACL project! Till next time -
Alan
Thank you for sharing, nice read!