This document gives an overview of the issues I see in the current composer implementation in TYPO3. First of all I'll write it in the form of user stories and afterwards write an actual text.
As a developer I want to:
- ☑ composer init and require typo3/cms
- ☑ install TYPO3 extensions via composer
- ☐ want the dependencies of the required extensions to be resolved recursively
As an admin/integrator I want to:
- ☐ install extensions via the extension manager (also in composer mode)
As an admin/integrator I don't want to:
- ☑ dump the autoload information on the command line (in non composer mode, fixed in 8.x via install tool)
- ☐ dump the autoload information on the command line (in composer mode)
As a developer I want:
- ☐ To have a fixed satis that doesn't drop the extra part with the "typo3/class-alias-loader" information
As Alexander Schnitzler I want:
- ☐ Extensions to be namespaced (folder structure wise, like in the regular vendor dir) [typo3conf/ext/vendor/extension]
- ☐ The TER to only act as a GUI/Proxy for available extensions.
- ☐ The extension manager to be a GUI for composer itself (being operable only if possible)
Let's begin with the most crucial part of this operation. I do believe that composer is the package management the TYPO3 environment needs and which should be embraced completely to reduce the functionality of the TER to a bare minimum. Why is that so? Because I do believe, that TYPO3 does not only want to have the very good working autoloading of composer but also the rest of that beautiful composer package being the package management itself and the possibility to decentralize the packages. Composer can replace the dependency resolver completely (one thing less to take care of) and it can even resolve dependencies recursively. As more and more code is packaged into small chunks of useful libraries, you don't want x numbers of extension to ship the same libraries in a Resources/Private/PHP folder. It's a proper intermediate solution but in the end you don't want to have foreign code in your own repository. Besides that, composer enables you to fetch your code from a lot of different sources. It's even possible to mix public and private repositories and still get the dependencies right.
All this needs a GUI for people who are not familiar with the command line. If you strip down the extension manager to a bare minimum, you have the functionality of a composer require statement. As packages are installed by default if physically loaded, one can simply drop the step of having packages with installed/uninstalled states. What is that good for anyway? Either you need phpmyadmin or you don't. With composer you can even just install packages in dev mode require-dev and keep the live version untouched. Something that is not possible with the current approach. One can drop the PackageStates.php and all ext_emconf.php files (just needed for the dependency resolver and displaying the title in ext manager anyway). Once you operate your installation in development mode (finally a use case for the contexts), extension manager operates with the composer --dev flag. As there is no pollution of any PackageStates.php file, there is no risk to deploy crap to the live server.
So, what about the people that use the extension manager on live systems? Well, screw them. Why do we even invent our own deployment solution when we accept the thinking that one needs to be able to install packages on a live system? That's insane. We are living and working in 2017, you do not work on a live server to test something. So, what's the problem with composer consuming so much RAM and CPU time anyway? Never mind, it takes place locally and/or on your build server. Even if you deploy via ftp, you install/uninstall packages before that. This means, that people need to cope with that fact and if they really need to install packages on their live servers, they should go ask their clients for more money for their own convenience.
What about the namespaces? We adjust the core to search for extensions one folder deeper than usual. That way we can namespace our extensions folder wise and do not have any conflicts with similar extension names. For the developer this gives a lot more clarity which packages are installed. But the real benefit is, that people do not need to register extension keys and fight about it. You want to publish your own gridelements? Fine, go ahead! And suddenly the TER is a gui like packagist itself. It can hold meta information about packages but is not responsible for the availability of packages themselves. This frees the resources of the server team and the t3org team can update typo3.org more regularly without the TER bound to it.
On top of that, we can drop the satis, installed on composer.typo3.org or make it real proxy. I hate installing typo3-ter/foo extensions because it is the reason for most issues I have with TYPO3 and composer. It's due to the fact, that the composer.json of satis is different from the actual one of the package, leading to nasty brainfuck situations. Example: compatibility6 has a "typo3/class-alias-loader" section which is the key feature – no, it's the reason – why that extension exists. You want to register class aliases but that information is missing in the composer.json of satis because the way satis builds these extensions is dependent on our very own algorithm defined in https://github.com/TYPO3/CmsComposerPackageGenerator which noone wants to maintain. Everyone simply wants to use the original, the pure composer integration. Once you face any issue with composer.typo3.org and you are not very much into that whole composer topic you start believing that composer is shit.
Do we really want to live with the status quo like we do with Extbase? It's becoming too many construction sites in just one cms that seem to never get finished. I don't think that this split, which is still done in many areas, is a benefit to TYPO3. One cannot always get everyone aboard but leaving a few behind can make many more join.
Helmut, I do not focus on the negative. I wrote a wishlist that's built on top of what we already have. I do emphasize the issues and my view on the solutions. Your interpretation, that mentioning the issues means blaming the status quo and not valuing the achievements, does harm the discussion because it drags it down to a level of emotion and personal feelings. That's not what all this is about.
Actually it is something to worry about. Have a look at Extbase. Years ago we had the same situation. Everybody was happy in the first place and we felt like we really did a big step. Yes, we made people think about DDD but the technical implementation is still horrible. For years, people wish for a proper solution but at the same time we always talk about legacy, backwards compatibility and such. There is no lack of manpower to fix this, it's a lack of vision and communication to get enough people aboard.
It's the same with composer. It might be that your console solves a lot of issues but I don't want to rely on a 3rd-party-package to have a working composer setup. (btw. it's the same with gridelements, I don't want to be forced to rely on it to have grids). It's not a matter of trust, it's a matter of visibility and clarity. Imagine your are new to TYPO3 and you try setting up TYPO3 with composer the first time. At that point you don't know that there is a console extension and you get a horrible experience. That's the reason why a lot of people do think badly about the project in whole. They have a bad experience because of the lack of knowledge and guidance. But at this point I disagree that just helping people will make everything alright. At some point there need to be decisions in favor of or against a solution and it needs to be part of the core. We cannot call TYPO3 an enterprise cms when you need dozens of extensions to work like other projects do out of the box.
Maybe you do not see it but have a look at the facebook groups etc. There are the people that do not use stack overflow for questions. Who do not write questions in english. Who come up with 1998 solutions in 2017 and don't realize how bad it is. Who have issues with Extbase and Composer because they don't know the best practices and get annoyed by TYPO3.
I do believe that once you come up with a vision (yes, that's the hard part) and have a majority aboard, you will find enough motivated people to join. Of course we lack developers because noone knows if it is worth creating a patch in the first place. It works for those that are deeply connected to the core team but you don't see many big, sophisticated patchsets out of the blue.
However, your solution always has been and still is, Fork, Publish, Merge. That's something I don't believe in any more. Not for the TYPO3 project. It can happen too easily that you invest hours of work for nothing. Been there, done that, will never do it again. I can help to form a vision, to be part of the process of evaluating ideas technically, but I will not go ahead as a single person, being the one that does all the work in the first place.
Open source can be about single persons, dedicating huge parts of their live to a project and thus shaping it heavily or it can be about a lot of people having a vision and working together on the best solution. I'm in for the latter.