Skip to content

Instantly share code, notes, and snippets.

@tsloughter
Last active November 4, 2016 01:59
Show Gist options
  • Select an option

  • Save tsloughter/4d18c474f009dd3a3eeb094f8933e90b to your computer and use it in GitHub Desktop.

Select an option

Save tsloughter/4d18c474f009dd3a3eeb094f8933e90b to your computer and use it in GitHub Desktop.
Erlang/OTP libraries/modules replaced by third party projects

I'm looking to collect information on why certain functionality found in Erlang/OTP is usually handled instead by a third party library in people's projects.

This could be bugs, missing functionality, poor interface, performance, etc.

Examples off the top of my head that need expanding on are:

  • httpc
  • httpd
  • http_uri
  • reltool

Please comment on this gist with libs and reasons, even if you aren't sure they are accurate or still accurate today.

@okeuday

okeuday commented Sep 7, 2016

Copy link
Copy Markdown

The 3 functions: unicode:to_lower/1 unicode:to_upper/1 unicode:to_title/1 are very basic functionality and are missing from Erlang/OTP. They should be like the implementation in https://github.com/rambocoder/unistring but automatically generated from unicode data to make it maintainable.

@elbrujohalcon

Copy link
Copy Markdown

Is cowboy considered a replacement for httpd? If it is, the main thing I like from cowboy is cowboy_rest. I build REST-like APIs all the time and cowboy is simple to use and very reliable for that purpose.

@fenollp

fenollp commented Sep 7, 2016

Copy link
Copy Markdown

A gofmt for Erlang, not a solution but a beginning (and links) here

@zugolosian

Copy link
Copy Markdown

We use gproc all over the show because of pretty much all the features here:
https://github.com/uwiger/gproc#introduction

@IgorKarymov

Copy link
Copy Markdown

Looks like gproc like functionality should be part of OTP (or maybe vm?).

The biggest part of the projects have some kind of process worker pools.

Ordered data structures like ordsets etc. I have a few projects with add hook handwritten "reverse" modules aka rordsets.

cowboy > httpd
httpc usable from time to time, but not for any hard work.

@bullno1

bullno1 commented Sep 8, 2016

Copy link
Copy Markdown

Roll your own protocol for moderately big messages instead of disterl.

@vans163

vans163 commented Sep 8, 2016

Copy link
Copy Markdown

http decode is overall annoying in erlang. The decode_packet parser returns a mix of atoms and strings. So now that http2 is all lowercase, you need to normalize all your headers if you want to share code. Also decode_packet does not take things into consideration like max size. A much simpler decode_packet for active receive should be created. Also decode_packet could perhaps start to cover http2 and hpack, with the end user being responsible for managing the streams.

Something like accept(Socket, [{packet, {http, 1024, 8192}}, {active, once}]) perhaps.   1024 = max prefix_size, 8192 max size of headers.
It procs:

{http, {error, header_header_size_exceeded}
{http, {error, header_sze_exceeded}
{http, {...Type...}, {...Headers...}, BodySize}

I have a bunch of things I include often because OTP does not have that functionality, but that is beyond the scope of the question.

@jlouis, How do you get around the limitations of RPC module for scaling?

@tsloughter

Copy link
Copy Markdown
Author

@vans163 missing functionality could be within the scope. The unicode example from @okeuday is a good one. Stuff that really should be in OTP but is missing.

@aerosol

aerosol commented Sep 8, 2016

Copy link
Copy Markdown

eldap support is poor, the app isn't even proper OTP

@waisbrot

Copy link
Copy Markdown

JSON's prevalent enough that it feels a surprising not to have a JSON-parsing library built in (like how xmerl is). I use jsx because compiling jiffy hasn't always been a smooth experience.

@jlouis

jlouis commented Sep 11, 2016

Copy link
Copy Markdown

Unicode is a good example. And we should look at Go for the solution:

Just providing to_upper and to_lower is 99% of the time going to be a mistake. It is a complex area where you need to take care about the details.

@vans163 (RPC) Most applications use the rpc module sparingly in which case it is fine. The problem occurs as soon as you use it for every cross-call you have in the system (which you don't have to). You can build your own more efficient variant, or you can avoid even using the module in the first place. The problem is the gotcha: people see rpc and go "I know RPC! I'll use it!". But if you have a pid() in Erlang you can send it a message, even if it is on a another host.

As for @waisbrot's comment: xmerl should never have been part of the stdlib. Likewise, jsx shall not be part of the stdlib. I'd much rather have a small set of "blessed", "maintained" packages on hex.pm which we use for these kinds of things. Once these things goes into a stdlib, they are set in stone and will move very slowly. The world around them changes, and then you have the same problem as always. jsx is an excellent example because it makes the trade-off of being written in Erlang, and thus about 10 times slower than jiffy which uses a NIF. But using a NIF has its own set of complexities, so not everyone wants that library. You must be able to pick among several poisons.

It also goes for my view on library design: include the things in the stdlib which are impossible to implement as a library (ETS, ...), or are so tightly coupled to the ERTS system that you can't evolve them separately on their own lifecycle.

@IngelaAndin

Copy link
Copy Markdown

When it comes to HTTP it has never been a top priority to Ericsson. The httpc absolutely still suffers from the problems I described regarding eldap. E.i something half done was contributed and we have to try making it better. We do have some plans for further improvements but alas I can not make any promises, as we also have a problem prioritizing it.

When it comes to httpd our vision is to try to simplify it and make it a small simplistic embedded web server. We do not have the people to make it otherwise. Also there are Ericsson projects that use it this way and are happy that it solves their needs.

@seriyps

seriyps commented Sep 12, 2016

Copy link
Copy Markdown

Some better calendar module would be great. Current calendar have only basic primitives, but some more high-order library often required to perform date arithmetics, timezone calculations, formatting and parsing. (And I guess everybody hates to type this datetime_to_gregorian_seconds, gregorian_seconds_to_datetime each time).

@IngelaAndin

Copy link
Copy Markdown

Humm... I got this dupplicated so I thought I removed one copy but it seems it went away altogether! So here we go again!
@aerosol
This is a symptom of that the OTP team have not had a good enough process for including contributions. I think it is just lately that PR-process has started to work good enough.
In the past it was quite common that prototypes that work well for some use case where included without proper tests, documentation or code reviews. We do have it in our backlog to
make eldap into a proper OTP application, it just has not been prioritized high enough yet. PR are welcome.

@seriyps

seriyps commented Sep 13, 2016

Copy link
Copy Markdown

I also guess that not everybody are happy with the way how we do configuration. sys.config files and application:get_env sometimes don't fit our needs: config files should be written in erlang syntax; sometimes configs should have some other logical structure, not just <application>.<key>.

@psyeugenic

Copy link
Copy Markdown

Yes, calendar module is a disaster in the user experience sense. It should really be replaced.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment