Why C/C++ package managers fail for now ?

Notes on C&C++ package managers.md

Motivation

As for now (mid 2016), there doesn't seem to be a C/C++ package manager that stands out of the crowd. To understand the reasons behind this failure, I will try to list in this README most C/C++ package managers, highlight differences between them, then list critics that are made about them.

Note: this README is merely a gathering of personnal notes, doesn't intend to be a reference in any way.

A standard proposal : http://open-std.org/JTC1/SC22/WG21/docs/papers/2016/p0235r0.pdf

Package managers

CPM

• repository : [https://github.com/iauns/cpm]
• website : [http://www.cpm.rocks/]
• current status : dead. Last commit from Feb. 17 2015

Conan

• repository : [https://github.com/conan-io/conan]
• website : [https://conan.io/]
• current status : in development. Website uncomplete (profile page for instance). Core however seems functionnal

For being in the process of trying it, a few remarks:

• Sure, it uses python. I personnaly like python, so no big deal for me.
• However, I dislike the fact that you can write conanfile.py and conanfile.txt. It is not unified, although they might serve different purposes.

Biicode

• dead

clib

• repository : https://github.com/clibs/clib

Critics

• (Conan) Not using C/C++ as the language for developping the package manager itself
• (Conan) Lacking support for cross-compilation ?
• (Conan) Copies all files from a local module into local Conan store, which can be an issue with large libs
• (Conan) Possible to overwrite an existing version
• (Conan) Doesn't seem to support fully CI flows !

Requirements & Challenges

Dependency hell

See http://dominictarr.com/post/25516279897/why-you-should-never-write-a-package-manager

Dependency hell is for instance when a package A depends on B1.0 and C, but C depends on B2.0. In this case, putting all B module in the search PATH will generate a conflict. Need to resolve version of B that satisfies both.

Note: Doesn't seem to be a cause of failure of adoption. For instance, python doesn't have such mechanism implemented yet

Support for large packages

Large packages, ideally should not need to be downloaded for each-new projet but only each time a different version of the package is required by a project. And, in the same time, several projects might need different version of the package, with possibly different binaries (debug, release, build config in general...) To solve this, Conan uses a global install location with a different sha for each binary, to differentiate them using build configuration (debug, release, 32, 64, compiler...).

Cross-compilation

It seems the package manager needs to be thought from the ground up for cross-compilation. Arguments ? Key points for handling this issue ?

Lack of generic build process

There is not a single way to build a C++ project, but many, very diverses.

C/C++ offer many different build tools (unlike framework/languages like nodeJS, python, etc, where there is always a single build/execution program).

In C++, there is no convention either on project folder hierarchy. Depending on your build tools (Makefile, Ninja, CMAKE, etc) and project type (native app, kernel, embedded real-time firmware) there can be better and different ways to structure a project.

Also, there is no consensus either on build tools. This could mean that a package manager must choose and enforce a specific build tool and build flow, which of course never makes everyone happy.

How to solve this ?

Hosting package elsewhere than central repository

Transitive dependency handling

https://www.reddit.com/r/cpp/comments/42wme6/modern_build_systems/d1hkyni

Personnal questions

Embedded librairies

How to structure low-level embedded (bare-metal, real-time) librairies ?

• If the library is implemented at some level with hardware abstraction, that calls hardware specific routines, how should they be structured ?
• Should the hardware-specific routines be packages on their own, making it easy to port the library for anyone on a new platorm ?
• Or part of the core library itself ?

Open questions

From predatorian3

what's a good strategy for dependency management so your contributors, or maintainers can also use the same libraries that you do?

Resources

• evanjones.ca/build-c.html

@KelsonBall

Buckaroo is very much alive, with v2 released quite recently.

Some differences between Buckaroo and the alternatives:

• Modern Build System. Other systems are built around CMake (or in practice 90% CMake), whereas Buckaroo uses one modern packaging format. This is because CMake (even "modern" CMake) does not scale to a large ecosystem of dependencies, since it is not hermetically sealed and does not have a sane scripting language. Note that there are migration + wrapping tools from CMake to Buckaroo.
• Full decentralization. Buckaroo discovers dependencies directly from Git and HTTP. Note that other solutions are cook-book driven, and so are not practically decentralized, even if they claim to be.
• Reproducible Installations. Buckaroo installs packages with information only from the lock-file, so you get the exact same code, every time.

Basically, if you want an experience like Cargo / NPM / Godep then Buckaroo is the closest to that. Other solutions are less opinionated, but you lose out on simplicity.