Monorepo or mutiple repositories?

to-monorepo-or-not-to-monorepo.md

With our current architecture we're facing a crossroad, where we need to talk about how could be the best approach to handling all the git repositories and all the problems that we're having right now and the problems to come.

Starting point

On one hand, this is the picture right now:

Official modules on decidim/decidim

• decidim-accountability
• decidim-admin
• decidim-api
• decidim-assemblies
• decidim-budgets
• decidim-comments
• decidim-core
• decidim-dev
• decidim-meetings
• decidim-pages
• decidim-participatory_processes
• decidim-proposals
• decidim-surveys
• decidim-system
• decidim-verifications

Official modules outside decidim/decidim

• decidim-module-consultations
• decidim-module-sortitions
• decidim-initiatives
• decidim-results
• decidim-module-blogs
• decidim-module-mentions

Non official modules

• Census (DIBA)
• Crowdfundings (podemos-info)
• DataViz (Barcelona)
• Debates (Barcelona)
• Members (FundAction)
• Pol.is (OSP)
• Surveys (Hospitalet)
• User Export (OSP)
• Verification Census API (DIBA)
• Verification Census API (Barcelona)
• Votings (podemos-info)

Two possible roads

Monorepo

This is mostly how we're working right now. The only difference between what belongs on decidim/decidim right now are not technical issues but mostly administrative reasons (which company has made that development). This would involve moving all the Official modules outside decidim/decidim to Official modules on decidim/decidim.

Pros

This is simpler, specially because it allows the APIs to change on any given moment and make that change on a single PR, and all the CI and translations on the same place. Also simpler to grep.

A couple big companies that had this model are Google and Facebook. More info: https://danluu.com/monorepo/

Cons

This doesn't scale that well (lots of LOC for every module, lots of times to CI to finish).

Multiple repos

Most of the modules of Official modules on decidim/decidim should go outside, leaving a leaner core module:

• decidim-admin
• decidim-api
• decidim-core
• decidim-dev
• decidim-pages
• decidim-system

All the other modules would be on separate repositories.

Pros

Allows to make the APIs between modules more mature, allow better scaling.

Cons

This have some pain points, regarding:

• Translations: we should create a new crowdin project with their integration
• CI: configure again. Maybe with dummy apps and default configs it's a little less pain
• Docs?
• Issues

Having somekind of generators or a skeleton on making a new gem would reduce these pain points.

The only difference between what belongs on decidim/decidim right now are not technical issues but mostly administrative reasons (which company has made that development)

@andreslucena I don't think that's right. The criteria for a module to be inside decidim/decidim is if it should be a default feature or not, I think.

Also, decidim-results was deprecated by decidim-accountability.

This doesn't scale that well (lots of LOC for every module)

This is written under Monorepo -> Cons. If the modules were split and taken into a different repo, not only would the LOC stay, but we'd have to keep less configuration, so actually the number of files would increase.

For the Multiple repos section, why would you leave decidim-pages as a core module? They are features, not static pages like T&C.

For multiple repos, a CON would be that if you need anything new you need to first do a PR to the core repo, wait until it gets accepted, then keep working on your original repo. This makes the development time longer.

Edit: comment made for the Revision 1 of this gist.

I'd like to add a "con" to using a monorepo: We'd be coupling the release of all the modules. This means it won't be possible to get to release a version of a module without releasing a new version for the whole framework. It's something we've been doing lately (especially with decidim-initiatives) and has proven useful.

As you might know, I've always been a proponent of the multirepo solution. However, the rspec core team seems to regret their decision, arguing some of the cons stated here. Haven't read it all since I just found it, but I will since I think it will be interesting for us and it might make me change my current opinion: rspec/rspec-core#2509.

Well, I read the whole thing and except for the initial posts which include motivations to migrate to a monorepo solution, the rest is a migration path discussion, which doesn't really apply to us since we are currently a monorepo.

@deivid-rodriguez that's a very interesting read indeed

AFAIS, if we go on decoupling almost every module, we should have lots of documentation/procedure of how to start a new module (with CI/Crowdin/etc).

@deivid-rodriguez that's the input that I wanted to hear, I'll keep reading. Thanks! If any knows other projects with a similar architecture it'd be nice to have it :)