about_a_mess.md

Too long for a tweet...so, I tweeted this:

"One trend I'm observing is a lot of rubyists being fed up with messy ruby ecosystem are getting excited or even switch to clojure."

People asked me to comment further, so here it goes. First of all I said I'm seeing a trend, which is a fact. I know people who are learning Clojure (and other languages like some people pointed out) hoping to switch from Ruby or they even already left Ruby. Those people got tired of something I called "a messy ecosystem". Maybe not the best selection of words, I dunno. I meant that some people, myself included, find ruby ecosystem - which consists of multiple VMs, practically one web framework and a gazillion of libraries from which huge amount are only semi-working - to be a mess where it's hard to pick up a solid stack to solve bigger problems.

I really don't have time (which is a shame) to come up with some specific details. I could maybe only quickly describe what we're dealing with now at gitorious.org which is being upgraded to rails 3 and ruby 1.9. Guess what, MAJOR issues with encoding. Suddenly we need to find all the places where some string has to be force-encoded to utf8. It used to just work™. Now it doesn't. What gives? I dunno, it's a mess.

It's probably not the best example I could come up with but it is our current struggle. I'm pretty sure a lot of people are having all sorts of problems because of the messy tools/libraries/frameworks that they decided to use.

That's why I'm excited about Rubinius (X or not, doesn't matter) because I hope to see a rock solid ruby vm that doesn't have the problems we've been facing when using MRI. I also hope to see a more reliable "stdlib". I also hope to see more small libraries that are used to build rock solid tech-stacks that is not called Rails.

I would love to write more but I gotta go :)

We can continue in comments if you want.

I think the stdlib as it stands is mostly useless and a bit of an anachronism. It's a mish-mash of oddities and a handful of useful things, but almost everything of true value nowadays comes into a project as a gem via Bundler.

That said, I don't think the Ruby ecosystem itself is particularly messy, especially if you compare it to Perl (5, 6, 11, a ton of implementations), Python (2 vs 3 issues, worse than 1.8/1.9 ever was) or PHP (5 vs 6, package management sucks, 4 and 5 both widely deployed, 6 is a mess). And I think MRI is a fine implementation in its current state, although having other options like JRuby and Rubinius is something to be proud of and happy with (especially JRuby, IMHO).

I think the example is a bad one. The encoding-border is a really hard one, but most libraries have actually crossed it. Database connectors are now all handling encoding properly and I haven't seen an encoding error in my non-legacy projects for... months?

And thats actually a good thing. The Ruby community, in general, has crossed a border that a lot of communities are stuck with. Take for example Python 2->3, a switch that will probably stretch out for the next few years. The move from Ruby 1.8 to 1.9 was also a switch from a version that originated in the pre-Rails world where Ruby was an obscure footnote to something that was built with some professional mind behind it.

The issue of half-working libraries: thats always a problem. Try to find a command-line parser in Java. The field is littered with half-finished and broken libraries. Yet, a lot of people use Java - for the good working libraries. This issue comes with age and there is no technological fix for that.

The amount of runtimes can be seen as a good or a bad thing: Ruby is one of the rare languages that can be run on and off the JVM with a reasonable amount of porting work. Cross-VM testing is an accepted fact in the Ruby world, so most libraries do actually allow that. Heck, I can use Ruby on iOS and Android. Not only theoretically. Thats a luxury we ignore all to often.

But all these things are messy and hard topics. Also, some decisions went wrong and can probably never be mitigated.

So a lot of advanced programmers want to try something new. I know why my last 100 commits were on personal Rust projects. There is not a lot there. I can build without caring about how the ecosystem works. I don't have to integrate with the interface of 12 different templating libraries, because there are 0. Lots of achievement and new approaches without having to integrate. Lots of new approaches. This is also good thing!

But it certainly doesn't mean that Ruby is dead, because it has a very wide ecosystem of things that actually work and are missing in lots of other communities. There is no sequel, no sinatra, no Rack, stuff that - as broken as it sometimes is - isn't as bad as it is while I am ranting about it.

I do agree that the stdlib is problematic, but most are. A language has to come with a standard lib and upon success, those libs will most likely be superseded.

@peterc I'm not following php/perl/python but I did hear about python 2 => 3 migration to be a never-ending story. It is true that ruby people are moving much faster and it's an incredibly awesome thing. Regarding MRI, it's decent, it works for many but we all know that at the same time it's a major problem for others (I've heard horror stories from github for example). And stdlib, yeah I think you're right, it's an anachronism that's why I'm excited about gemified stdlib thanks to rubinius. It's a good move I think.

@skade well, issues with encoding are still a major problem in many projects so I'm not sure if it's such a bad example but yeah, there are probably better ones. Ruby on JVM, Ruby on mobile OSes, yup, that's absolutely fantastic. Can't deny that.

What I really fear, is that Ruby becomes "Conservative". If it fears to move we will keep Perl-like syntax and add stupid prefixes after our strings. New features will feel odd or inefficient.

I really want to see some active development toward Parallelism, a better GC and some deep new features with powerfulness, which have Clojure.

@solnic I would like to know any of those that started out with 1.9 first. Encoding problems usually bang really loud immediately or you are dealing with messy data, in which 1.8 was the wonderful land of ignorance. Combined with data stores that ignore encoding settings and treat strings as byte arrays as well, this has created a very toxic environment if you want to handle encoding correctly.

Considering the horror stories from Github: quite frankly, given the number of Ruby deployments and the draw in developers they get, I would have expected to see a new GC being developed by them. On the other hand, they are the company that switched back from JRuby to MRI and send people around on conferences talking about how they don't believe in the JVM and use MRI instead (and yes, they are sometimes the same people, compare EuRuKo 2012 and eurucamp 2013).

@skade we're dealing with messy data in Gitorious, that's exactly the case. And 1.8 wasn't giving a shit about it, so there's that.

The mess can also be observed by the amount of people failing to setup simple Rails apps on their servers. During last week I've seen 2 cases of a Linux admin (a decent ones) trying to do that and they both failed. The app was reasonable simple with almost no moving parts. But they were struggling with bundler, missing libraries/headers (for native extensions), unicorn/passenger etc. I don't have problems with setting up any Rails app because I'm in it for 7 years now. But it amazes me that the skilled administrator who has installed and configured hundreds/thousands of apps in his live struggles with running Rails in 2013.

Re python2 -> python3. It's not the same situation as ruby 1.8 -> 1.9. Python3 is a separate language. At least they announced it as a "new, non-backwards compatible language". While the 90% of the syntax is the same the stdlib changed in a major way. And the fact the migration takes so long is probably because they are not in a rush to migrate (python2 isn't going anywhere yet afaik).

@solnic I agree with Rails getting more and more unwieldy. It requires a lot of deep knowledge to push and turn the right buttos and knobs and implicitely relies on a lot of things nowadays.

On python2->3. Let's frame it another way: Ruby has been a surprisingly stable language, conceptually. Any concept (and almost all syntax) you learned in Ruby 1.8 is still valid today in 2.0. So I wouldn't even accept that Ruby 1.8 and 1.9 are incompatible in all regards.