When is Clojure "the right tool for the job"?

clojure-right-tool.md

My answer to: https://www.reddit.com/r/Clojure/comments/pcwypb/us_engineers_love_to_say_the_right_tool_for_the/ which asked to know when and at what is Clojure "the right tool for the job"?

My take is that in general, the right tool for the job actually doesn't matter that much when it comes to programming language.

There are only a few cases where the options of tools that can do a sufficiently good job at the task become limited.

That's why they are called: General-purpose programming languages, because they can be used generally for most use cases without issues.

Let's look at some of the dimensions that make a difference and what I think of Clojure for them:

1. Compute performance

Clojure has access to three runtimes: JVM, JS and CLR. There are also some niche runtimes like Erlang and Dart, but I'd consider them too immature right now to be considered.

When it comes to compute, Clojure will be generally as fast or slightly slower than its runtime host. That means you're looking at Java-like compute performance, which is pretty damn good.

This will be sufficient most of the time, but in some circumstances it won't cut it. There are very few, but the ones I can think of are: real-time high performance game engines, graphic rendering, audio processing, and things like that. You probably won't want to implement those in Clojure.

That said, even for those use cases, you need to separate making use of high performance things to write programs, versus implementing it. For example, Clojure can easily be used to write a program that uses high performance 3D renderers, audio processors, and the likes, but it will do it through interop with an implementation written in a language with faster compute performance like Fortran, C, C++, Rust, D, Assembly, OpenCL, CUDA, etc.

2. Memory constraints

Keeping in mind the various runtimes again, Clojure will tend to use equal or more memory than its given runtime. So think Java/JS level memory use plus an extra 0%-20% more on top.

Again, this will be sufficient for most use cases, since memory is pretty cheap nowadays, and most machines have at least 8 GB of RAM. But if you need to run in memory constrained environments, it won't cut it. I'm talking things under 512MB I'd say, like a Raspberry Pie.

Now with the JS runtime, memory use will be less, and you might get away with things as low as 128MB I think.

And finally, with the GraalVM native compilation runtime (aka SubstrateVM) you might get it down even more, but still I'd say in general for memory constrained use cases, Clojure is not an ideal choice. Better use C, Go, Pascal, Ada, Rust, Lisp, Forth, etc.

3. Concurrency support

Concurrency is the ability to do multiple things at the same time from the user perspective. They need not be actually happening at the same time, but progress is made on many things at ounce. It means the user can download a file and scroll down in a way they don't have to wait for the download to finish so they can start seeing the page scrolling down.

Clojure is really good for this. It was designed with concurrency in mind, so there are many ways to tackle it. Some of them will vary based on the choice of runtime though: JVM, JS, CLR, etc. In general, it can make use of both OS thread scheduled concurrency as well as lightweight application thread concurrency. It allows safe memory sharing between the concurrent tasks as well. It provides nice ways to handle back-pressure and communication between concurrent tasks too. And there's a number of libraries providing many different style for handling concurrency, e.g., claypoole, missionary and promesa.

4. Horizontal scale

Horizontal scale is the ability to distribute an application across many machines in order to scale the number of tasks or data it can process beyond what could be done on one machine.

Clojure is very good here as well. All three runtimes excel at this: JVM, NodeJS and CLR, as they all offer robust servers that can handle the communication.

I'd say Clojure shines here because it tends to model application as data, and data is very easy to exchange between machines. It also favour pure computation, which is also easier to distribute. It also has really good serialization support, from EDN to Transit to Fressian to Nippy.

And the hosts it runs on support a large number of database drivers. They also offer some really good distributed data processing frameworks like Spark, Storm, Flink, Hadoop, etc. that Clojure can leverage.

5. Vertical scale

That's another one Clojure is very good at. Vertical scale is the ability to make use of more and more powerful machines. Clojure on JVM and CLR has full support for using every single core available on a machine. It can also leverage the GPU if needed. It can make use of crazy large RAM, like using all the memory on a 24TB RAM EC2 instance for example.

It can also nicely run on ARM64 CPUs, not just x86/64, so you can make use of those machines as well.

6. Ability to leverage existing libraries and frameworks (ideally for free)

Clojure excels here. It can leverage all Java libs with ease, as well as Clojure ones, plus those in Scala or Groovy (albeit slightly more annoying). It can use .Net libraries (when on the .Net runtime). It can use all JavaScript libraries as well (when running on JS runtime).

And if that wasn't enough, it turns out it can also use most Python libraries too, and R libraries as well!!

And there's a Clojure on Erlang that can use Erlang and Elixir libs, and a Dart one coming too, but those I'd say are still a bit experimental.

Finally, Clojure can really easily use C libraries as well!

7. Target platforms

Clojure is good here too, I mean, we already know it runs on many hosts. So everywhere Java, C# and JS runs so can it. That means browsers, Unity, x86/64, ARM64, Linux, Windows, MacOS, etc.

That said, there are some cases where Clojure doesn't cut it here. Any kind of native iOS and Android development, Clojure won't cut it. You can use Clojure for iOS and Android development, but you'd do it the same way it's done in JavaScript, with things like React Native and the like.

8. Correctness

People will disagree here, but Clojure appears to be one of the best for program correctness based on the little scientific studies that were done on the topic, ranking equal to Haskell it seems, in the top 3. This is mostly due to its immutability, its functional style and insistence on pure code, spec and generative testing, as well as the REPL providing quick testing and feedback throughout development.

There are still some places though where it won't cut it, and that's when instead of being focused on overall program correctness, you are focused on known guaranteed to be correct properties. That means static proofs that the code is correct given some property/constraint. You'd want Haskell, Agda, Coq, TLA+, Idris, etc. for that.

9. Startup time

This is probably the most grey aspect when it comes to Clojure. In a lot of ways, it does really bad here. But it really depends. Clojure JVM in its standard form is pretty slow to start, much slower than Java even. But then Clojure on JS runtime starts sufficiently fast for most use case, though still slightly slower than the equivalent JS would. But then Clojure JVM using a native compilation with GraalVM will start crazy fast, equal or faster than Python fast. Similarly there is a scripted Clojure called Babashka, which will start super fast, I'd say Ruby/Python fast.

So it's a bit all over the place, and will depend a lot on which variant of Clojure you're gonna go for.

10. Ergonomics

This is a personal thing, because it depends how you personally think about problems and go about programming tasks. For me, Clojure has amazing ergonomics, because the live interactive programming style really resonate with how I go about coding. The data oriented, data driven, functional first style similarly just feels so compact, simple and letting me fit all my problems beautifully, that I find Clojure has amazing ergonomics.

I don't want to go and say this is Clojure's biggest strength, but I think it kind of is for the people who love Clojure. For example, Scala, Haskell, Scheme, Racket, Erlang, Elixir, Lisp, and company all have a lot of the same pros/cons of Clojure that I talk about in the other categories, but they have a very different ergonomic feel. So I'd say it's Clojure's ergonomics that become the differentiator for why people choose Clojure over them.

Think like: Arcade Fight Stick VS GamePad. It'll be hard to say one is better than the other, but it'll be clear that some people will be better with one than with the other.

So I think it's good to try Clojure for its ergonomics, see if once you learn to use them, you end up becoming a better programmer due to them or not. It did for me, but I know it doesn't for everyone. I think just as this might be why people who love Clojure love Clojure, it is also probably why people who don't like Clojure don't like it.

11. Productivity

I will say Clojure is a very productive language. People will argue that their language is just as productive, and part of productivity is how fast you personally are at using it, which comes with experience and familiarity with it, so fair enough. But in any case, it is clear Clojure is definitely in the category of highly productive languages. Here are some reasons why:

• It tends to require a lot less code to implement a similar thing than in other popular languages, and is known to be very compact.
• It has access to a gazillion existing libraries and frameworks, so you have so much you can reuse for any problem you'd be trying to tackle, saving you the time to implement that stuff yourself.
• The way it's designed, it makes it really easy to write highly modular code that can be glued in lots of different ways. So over time, you don't need to refactor as often, and you can reuse more and more things by simply recombining existing things into new use cases.
• The live interactive development environment for me is a huge productivity boost. Say I'm trying to integrate with a remote API with kind of crappy doc, being able to just try the call in real-time to figure out the right way to call it is a huge productivity boost. Or if you are making a front-end, being able to see as you change the code the effect to the UI is a huge time saver. Or when I implement a complicated algorithm, being able to walk through it step by step and keep testing it in real-time helps me a ton to better understand how to implement it.
• Macros are another other productivity booster. All the tedious boilerplate code you need to write over and over can just be automated and eliminated away by a macro.
• Personally, I find the Lisp syntax is a productivity boost as well. Being able to navigate and edit the code structurally, while hard to truly measure, I feel at least saves me time.
• Clojure has really good IDEs and command shells. Yes people struggle with learning how to set them up and use them at first, but once you do know how, they are really awesome. Color coded, pretty printed, inline doc, great linting, good refactorings, find usage, show source, list functions, list variables, good auto-complete, etc. It might not be as feature complete as say IntelliJ for Java, but it's really close to it, and unlike that, you kind of get the full IDE experience everywhere. You get it at the command line, in your preferred editor, in a notebook, on your phone, etc. Where as with Java, if you don't use one of its big IDEs like IntelliJ or Eclipse, you get pretty much nothing.
• Clojure compiles pretty much instantenously. It supports incremental compilation, which probably helps with that. So no waiting around for things to compile just to run your tests, or try out your code, is a productivity boost.
• The tooling is really good. Beyond IDE, Clojure has great dependency managers and build tools. Similar to IDEs, people struggle to learn how to use them at first, but again, once you know, they are really good. Being able to quickly pull down dependencies, have multiple dependencies installed that don't clash for various projects, orchestrate builds like compilation, running tests, checking for test coverage, auto-formatting, getting idiom recommendations, getting linting, etc.
• Again with the tooling, there are some tools in Clojure I havn't really seen as much elsewhere. Things like Reveal, Rebel and Cider inspector, are really good to let you inspect your running program in nice visual ways. Things like Notespace and Marginalia are great for litterate programming, data-science and all that. Depending on what you do, they can help boost producitivity as well.
• No breaking changes. That might be surprising to people, but being able to just upgrade to a new Clojure version, or update to newer versions of dependencies, and nothing breaks, that careful attention to backwards compatibility, that's a huge time saver.
• Last aspect of productivity for me would be the community around Clojure, it's just full of great people. Not only do they provide awesome libraries that often help with being productive, like say core.match or core.logic. The support they provide when you need it, I mean, it's like real-time immediate access to some super senior devs, just go in the Slack, ask a question, and get a Clojure maintainer or expert answer you in under 2 min.

Yup, productivity is a BIG one. After ergonomics, this is probably the biggest factor that goes into why I personally choose Clojure as my main language whenever it can cut it, which is most of the time, as I've been expaining in this very post ;)

12. Real-time guarantees

This is the property that you can predict the time for something to start and complete. Imagine knowing that no matter what else is currently executing, you want to be sure that if you press on the break, the break will activate 10ms from the time you pressed it, guaranteed!

Well, here I'd say it depends. There is a spectrum when it comes to real-time, what is the worst case and average case you can tolerate? For critical code, where someone's life is on the line, I'd say Clojure won't cut it. For financial trading, I'd say it can cut it if you design carefully for that.

This will also kind of depend on the choice of host and its settings. Some GCs on the JVM have certain guarantees for real-time, and that will hold for Clojure.

In this space, for Clojure, the best I know of is Ferret, a small dialect of Clojure that targets real-time use cases. But it's a one man show, and I'd say not fully mature.

So if you have pretty strict needs for real-time guarantees, I'd say Clojure won't cut it.

13. Application bundling/delivery

How the program you wrote gets bundled and delivered to end-users or to your host is an important aspect as well. Clojure is actually pretty good here too, but it's another place where people will struggle to learn how at first.

This is also very dependent on what Clojure host you are using. If you use a JS host for example, Clojure will be able to do pretty advanced code minification, tree shaking, and such things to reduce its bundle size. And it can be packaged however a JS app can be, so a single .js file, or broke into multiple files, or as a NPM package, etc.

If you use a JVM host, well it can be made into a single self contained executable that bundles a pruned down JVM. Think Portable Apps. Or it can be bundled as an Uberjar + a startup script to be installed on a machine, but a JVM must already exist. Or it can be compiled to a statically (or dynamically) linked binary using GraalVM (though you need to use a subset of Clojure in your code, so this limits how much dynamism you can use slightly). Or it can be packaged as a Jar lib, published to a Maven repo and pulled down with any Maven supported dependency manager (Maven, Lein, tools.deps, Gradle, etc.). And it can also be delivered as a source based library, so published to a Git repo, and pulled down from git for use. There are some more ways beyond all that, like a WAR file for some server containers, and all that too, I won't list them all. All in all, there's a lot of options here.

You can also deliver Clojure as executable scripts. Babashka really shines for this, since you won't even need a JVM installed, a single binary and you can run Clojure scripts. People have used this to have Clojure running on cheap shared hosting for example, just like PHP, and is a good option for CI pipelines, or on remote hosts. You can also use Clojure to make NodeJS scripts, or Clojure itself to make standard Clojure scripts, which can even pull down dependencies defined in the script itself.