Add Symfony Messenger support for async messsages (emails as queues)

I'd appreciate feedback on the Q's asked in the issue summary.

Development can resume after I get back here re stability of Messenger code. Though technically could be merged as messenger is an optional dependency.

Prelim review requested per above.

Thanks @dpi. I didn't yet do a review, however I have some quick remarks.

1) I think it's a good time to make a 2.x release soon, and it can require D10.1. This issue potentially makes some significant changes so it makes sense to add it in a major branch.

2) I propose to hold off committing this until the matching symfony code is released and definitely not going to change. However we can develop, review and test it before that.

3) I plan to move the transports to a separate project mailer_transport in 2.x, allowing use in multiple places. It should also match the development in Core 📌 [PP-1] Add symfony mailer transports to DIC Postponed , although the current Core MR might need adjusting to work with this module.

1) I think it's a good time to make a 2.x release soon, and it can require D10.1.

Perfect. 2.x with new deps sounds good.

I propose to hold off committing this until the matching symfony code

Absolutely, I'll update as I progress.

I plan to move the transports to a separate project mailer_transport

Ah!

Then I might suggest borrowing and utilising a bunch of the mailer classes I added in this MR for this kind of decoupling.

One other thought - if this issue will make widespread changes then perhaps we need to split it into more issues.

We already have 📌 Consider marking some classes/functions @internal and find more convenient ways of dependency injection Active .

I think you can introduce the services on their own.

That would effectively be the majority of the functional code in this MR: Services.yml, Connections, TransportFactoryFactory, TransportsFactory

The other half would be split out: the Mailer class change, and two services: mailer.messenger.message_handler + mailer.mailer

50% of the code for this MR is tests for messenger. So it would be slimmed down further.

Created an issue for the services pieces 🐛 Add services/container features from upstream Symfony Mailer component Active . The code will continue to be duplicated in this issue as it is a requirement.

The Symfony Messenger and Doctrine transport → code has been pushed publicly. You should be able to review it locally now.

Gitlab CI doesnt have a 10.1 runner, so its not going to work with that for now.

Keep in mind there are two core patches required. The patching stuff works great on Gitlab (see Gitlab ymls' for SM), I'm not sure if it works with DrupalCI. You might be interested in borrowing matrix support from SM in order to test multiple core/php/database-engines.

Back to needs review w/ @AdamPS

^Yeah DCI gets stuck on allow plugins. Just wait for a D10.1 + Gitlab setup.

OK I'm still trying to get my head around this. I'll have some high-level comments and questions, probably in some cases from my lack of understanding.

Mostly I'm confused by why you want to change all these things. Aha maybe I can guess: you want to use DI instead of fixed Symfony\Component\Mailer\Mailer. In fact perhaps this is exactly what the title of the issue should be?? Any maybe it's scheduled for 2.x anyway, as described below? So this issue may come to nothing😃???

1) Changing the transport factory code should be separate from this issue. It would be part of the discussion for creating the new " Mailer Transport → " module. Would you be interested in working together on it?

Please first of all explain your goal (or maybe I already guessed above).

We already thought about this code quite hard in previous issues, and we rejected using the Symfony library Transport and Transports classes:

• They seem to rely on a static array of transports known from configuration, which prevents setting the transport DSN programmatically. The new Connections class will not actually work for the same reason.
• The use of an "X-Transport" seems clumsy and unnecessary in our case.

Perhaps we could create our own equivalent of Transports?? We can extend the symfony Email class with one that includes a getTransportDsn() method, and call that in the TransportInterface::send().

2) I feel the tests for this module should not depend on drupal/sm. Also probably StateTransport does not belong in this module, and this module should not use patches in composer.json. Instead why not put these tests in the SM module - which I believe contains the code that is actually being tested here.

It looks like the main problem is that this module creates the Symfony\Component\Mailer\Mailer object with MessageBusInterface = null. We should instead allow injecting a bus.

@dpi - please tell me if this meets your requirements.

In terms of the rest, lets keep them as separate issues

• likely some refactoring of transports will occur, following 📌 [PP-1] Add symfony mailer transports to DIC Postponed
• we can do auto-wiring in 📌 Consider marking some classes/functions @internal and find more convenient ways of dependency injection Active

Unless someone gets to this before me, I'll be revisiting in the near future. Right now I'm doing this voluntarily, without sponsors or motivation.

I think I still want to include a service to access Symfony\Component\Mailer\Mailer instead of burying it in src/Mailer.php. So the container has the responsibility of putting together the object and messenger integration. I'm not particularly fond of an approach where we'd manually checking if messenger is available and inject that.

As for autowiring and all that, sure that can be removed as it's noisy.

Some of the factory and connection things are necessary to allow Symfony\Component\Mailer\Mailer to bridge with how this project puts things together. Its not something the core issue ( 📌 [PP-1] Add symfony mailer transports to DIC Postponed ) will be addressing either, as the core implementation doesn't have legacy from this project, particularly the plugin systems.

On Mailer as a service, its quite advantageous to have it available, as you can do things like this without dealing with the legacy mail pipeline.

$email = (new \Symfony\Component\Mime\Email())
  ->to('jane@example.com')
  ->from('john@example.com')
  ->subject('Hello world!')
  ->text('Some sample text.')
  ->html('<p>some <strong>sample</strong> text.</p>');
/** @var \Symfony\Component\Mailer\MailerInterface $mailer */
$mailer = \Drupal::service(\Symfony\Component\Mailer\MailerInterface::class);
$mailer->send($email);

> I'll be revisiting in the near future.
Great

> Right now I'm doing this voluntarily, without sponsors
Same here for this whole module😃

> I think I still want to include a service to access Symfony\Component\Mailer\Mailer
The difficulty is that Mailer requires injection of a single fixed Transport at creation time. This module uses a different transport for each mail. There are some ways to workaround/change that however they are a bit messy. I feel that we shouldn't make any change to that area in this module until Core has decided what it will do.

> Its not something the core issue ( #3379794: Add symfony mailer transports to Dependency Injection Container ) will be addressing either
The core issue contains a big discussion (still in progress) of whether to require a single fixed transport. Maybe you have noticed that since you wrote this sentence, as I see that you have commented on the other issue😃.

> On Mailer as a service, its quite advantageous to have it available
The code in your example is very very wrong if using this module😃. This module provides a service with Drupal\symfony_mailer\MailerInterface, and a new Email class that should be created using a factory class.

I've also created a Symfony Messenger Lite → edition over at ✨ Add Symfony Messenger support for async messsages (emails as queues) Needs review .

I believe the best way is to fix DSM-L and DSM together - having 100s of lines copied between the two modules doesn't really make sense. I created a module to do exactly that: https://www.drupal.org/project/mailer_transport → . I don't really have time at the moment - would you like to be a maintainer??

Once 📌 [PP-1] Add symfony mailer transports to DIC Postponed has been committed then that gives the direction we need to follow.

I think thats a question for @zengenuity, and perhaps @znerol.

Selfishly, my motivation is to bring as many people to the Symfony Messenger party. My passion is less in the Mailer systems.

having 100s of lines copied between the two modules doesn't really make sense

For now, duplication should be fine til we come up with some long term solutions. In the least, this module and SML are unlikely/shouldn't be used together.

I propose that this issue belongs in this module.

In 📌 Split to use a new Mailer Transport module Active we would copy in lots of existing code.

The other part is the code that you already started to write for a mailer transport service. It should be adapted to match 📌 [PP-1] Add symfony mailer transports to DIC Postponed . It would multiplex to the actual transports based on a custom header "mailer transport dsn". It needs to be a DSN rather than a name, because we allow hooks to set the transport to an arbitrary DSN, rather than limiting it to a fixed set of configured named transports.