Directory based automatic service creation

MR !6669 implementation notes:

• symfony/config is brought in as a dependency so we have the same behaviour for globbing as Symfony. We're planning to bring in the dep for other issues like 📌 Support resource key in services.yml Active so should be fine here.
• The changes here are very much inspired by the resource parts of Symfony's version of YamlFileLoader, at the tail of \Symfony\Component\DependencyInjection\Loader\YamlFileLoader::parseDefinition and the method it calls: \Symfony\Component\DependencyInjection\Loader\FileLoader::registerClasses.
• Though 'module' / 'modules' are used in this MR, there is nothing specific to modules in particular, compared to other extension types.
• Terminology such as 'ModuleDiscovery' is used, though I'm happy to hear better names.
• Compared to resource:
  • paths is relative to module directories.
  • No namespace override is permitted.
  • Multiple paths allowed.

This sounds like a really nice feature. If we're going to bring in symfony/config we should just do ✨ Use native Symfony YamlLoader + Config Needs work and then extend Symfony's YamlFileLoader to add our own features, if we can - but there was quite a lot of resistance to adding that dependency over there, for one because it starts to imply that we use other features of Symfony's configuration system, but Drupal does things entirely differently. Is there any way we can do this or 📌 Support resource key in services.yml Active without adding the dependency?

> A module may choose to opt in any directory, including `src/` itself.

Can we make this automatic, rather than opt-in?

The opt-in code is just going to be repeated boilerplate.

I'd suggest we make it opinionated about where service classes are located, and allow modules to override if they want to. E.g.

- services in src/Services
- event subscribers in src/EventSubscriber

Agreed with #6, and curious about the responses to #6 😊

There is also an interesting choice to make here regarding public vs private services. Drupal services are still public by default; Symfony switched to private by default some time ago. And when you use resource in Symfony:

Wait, does this mean that every class in src/ is registered as a service? Even model classes? Actually, no. As long as you keep your imported services as private, all classes in src/ that are not explicitly used as services are automatically removed from the final container. In reality, the import means that all classes are "available to be used as services" without needing to be manually configured.

#6 + #8, @longwave @wimleers

We've since discussed in Slack, the symfony/config is undesirable as it conflicts with Drupal config.

And we've discussed that globs are not necessary, so that negates the only use of symfony/config code.

I've updated the issue summary with these decisions, and I'll update the associated MR removing use of symfony/config, tests and supports for glob, and re-implement our own directory traversal utility.

If we do decide to do globs in a future issue, I don't think what we're doing will conflict.

A module may choose to opt in any directory, including `src/` itself.

Can we make this automatic, rather than opt-in?
The opt-in code is just going to be repeated boilerplate.

I'd really like to do this, but I think we're going to get into trouble. One thing that comes to mind is plugins: Drupal has plugins which include references to non existent classes as a part of of constructors, and that plainly breaks autoloader. Migrate plugins immediately come to mind. In fact during the development of class-attribute autodiscovery for Hux, if I didnt limit to a precise subdirectory (instead of src/*), then service compilation/autoloader would just blow up on missing classes.

I think the proposal here isnt incompatible with straight up `src/*` as you're suggesting, since both this proposal and `resource` act more as templates. Whereas if we do `src/*` on a framework level, we'd be setting global defaults for all services, such as `public: false, autowire: true, autoconfigure: true`. This proposal is still useful as things like default tags and public: true would be used.

As @longwave linked the quote from https://symfony.com/doc/current/service_container.html#importing-many-se... we can certainly do it if the automatic services are private by default.

My suggestion is that we can do this as a separate issue. Since this proposal is complimentary, and less likely to cause fundamental PHP and other conflicts.

I'd suggest we make it opinionated about where service classes are located, and allow modules to override if they want to. E.g.
services in src/Services
event subscribers in src/EventSubscriber

I dont see why we need to add these constraints. I'd rather put a class anywhere and it just works. Directory/namespaces then become convention rather than constraints, just liek Symfony.

I've removed the dependency on symfony/config, and the associated glob support it provides that we agreed on.

The now-removed \Symfony\Component\Config\Resource\GlobResource also used Finder, so I've copied relevant non-glob code and moved some path normalization/relative-to-absolute pieces up to our YamlFileLoader.

Green and back to needs review.

Created a change record.

The Needs Review Queue Bot → tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide → to find step-by-step guides for working with issues.

> I dont see why we need to add these constraints. I'd rather put a class anywhere and it just works. Directory/namespaces then become convention rather than constraints, just like Symfony.

I wasn't suggesting constraints, rather, defaults:

- if you don't specify in a mymodule.services.yml file, services are looked for in src/Services, and event subscribers in src/EventSubscriber
- if you do specify in a mymodule.services.yml file, you can override that.

Bur more generally, I don't see what is so wrong about constraints. They enforce consistency which is good for DX. And we already have constraints -- entity types have to be in src/Entity, plugins in src/Plugin/SUBFOLDER, and so on. Meanwhile, form classes are ALL OVER THE PLACE, and it is MUCH nicer working with plugins where you find the same structure everywhere you go, than working with forms where you never know where you'll find form classes (HELLO node module, with some in src/ and some in src/Form).

Furthermore, scanning for everything is just inefficient - it degrades performance and Drupal's efforts to be more sustainable.

event subscribers in src/EventSubscriber

I'm sure it should not be so strict as 📌 Support #[AsEventListener] attribute on classes and methods Active

Moreover the restriction only applies to plugin discovery but it could have common reusable code

PS: curios how discovery could be optimized with composer's classmap

PS: curios how discovery could be optimized with composer's classmap

+1, but until Drupal has a module installer feature, it cannot be an option, can be?

I'm always in favour of making working with services easier, especially if it brings us closer to the way Symfony works and reduce WTFs that Symfony (or Laravel) developers may have coming to Drupal.

I do want to voice my opposition for the proposal where other modules can determine what namespaces are made available as services.

Allow a module to define directories within *all* enabled modules that will be used for discovery.

As someone working in projects where we have 300 modules enabled on average this sounds like a potential nightmare where any module can suddenly start exposing classes from any of the other modules. This creates scenario's where my module's behaviour can no longer be evaluated only looking at my own module but must be specifically evaluated within the context of every other possible Drupal module.

The example is to make classes of a specific type available (e.g. CacheTagInvalidator) but whether I expose all the services in my module as cache tag invalidator really should be up to me as module author, not to another module.

Symfony provides us with Service Tags which can provide exactly the desired behaviour. The module that provides the CacheTagInvalidatorInterface can configure the service container to automatically tag services with that interface as cache tag invalidator. Any service that would want all cache tag invalidators could use a Tagged Iterator to collect all those services.

Similarly within my module I may have many more services/components than I want in src/Services. For example if I have sub-functionalities in my module that I want to break up I might have src/EventManagement/{Controller/*.php,ScheduleAvailabilityManager.php} and src/EventRegistration/{Controller/*.php,PaymendHandler.php}. We should be careful not to make assumptions about how module authors may want to lay out their modules, especially as we evolve the capabilities of the service container.

Great point about performance of discovery, probably it needs something generalized like file cache as except services there's plugins and help topics and twog components