Support core's batch API

Sure, supporting the batch API sounds very attractive. I'm just worried that the batch API only works in a fairly limited context and not across the board. Should this be implemented as a special route in the eca_endpoint sub-module? There we can ensure the proper context and let it work with multiple http requests in a row that make sure that none of the individual requests runs into a timeout.

I'm just worried that the batch API only works in a fairly limited context and not across the board.

Sorry I currently don't understand this, could you maybe provide an example?

There we can ensure the proper context and let it work with multiple http requests in a row that make sure that none of the individual requests runs into a timeout.

Coincidentally, I recently experimented a bit with the endpoint as I had a timeout issue on a large operation on a magnitude of records.
At the end of an endpoint request, I used the Redirect action to redirect back to the same endpoint, continuing the next chunk of records. Surprisingly that already worked. Just wondering when a browser may complain about too many redirects...
So I implemented another experimental action that renders the contents of an (ECA) endpoint lazily via Javascript. A bit similar to HTMX lazy loading feature. And that works for a "poor man's batch" quite well, it even can output arbitrary status information and it even supports "parallel" processing since I can put in multiple lazy loading items on a page at the same time. If interested I could provide the action, however it needs a small Javascript file to be included in order to work.

Let me try and explain my worry about the multiple HTTP requests in more detail: when we build an action which itself creates a batch and then processes the batch with batch_process(), this will get Drupal to redirect to the batch processing route that completely takes over the processing of junks and that causes a new HTTP request for each junk. In the end, that process will return to the result summary page, unless a destination is being provided.

If such a batch is created by ECA with a form submit event, then everything should be fine as Drupal form handler will process such batches automatically as part of the form submission.

But what happens, if we create a batch in different events where e.g. Drupal is about to save an entity. This may interrupt the current HTTP request and the expected entity save will not complete. Note, I'm using this just as an example, there could probably be many other events where such an interruption might be equally problematic.

But maybe not? I'm actually not sure, it often felt a bit insecure to process batches in the middle of something else, but that's talking from past experience and not related to ECA.

Interesting that you describe your manual approach to redirect back to the same endpoint to process junks of data until all junks have completed. That's certainly not something for inexperienced users, and if batch API could resolve that, sounds great.

And you mention Htmx, which is interesting too. As we already have full support for all of Drupal's Ajax commands, I'm also planning to provide full Htmx support. I had a meeting with @fathershawn, who maintains the Html module → , about this and he's in full support of this idea. We just hold back a bit on this idea until we have a better understanding if this will actually go into core, or remain in contrib for some time.

But what happens, if we create a batch in different events where e.g. Drupal is about to save an entity. This may interrupt the current HTTP request and the expected entity save will not complete. Note, I'm using this just as an example, there could probably be many other events where such an interruption might be equally problematic.

Same argument would go for core's existing "action_goto_action" that performs a redirect. But what actually happens there is that the page contents of the current route are still being fully processed, but then at the end of the request a redirect response will be set to be returned (by adding an event listener on the fly).

The same mechanism can be applied to perform the redirect to the batch UI route, i.e. everything is still being processed in the backend, and at the end of the request the redirect response will be returned. One negative side effect that is also happening on the redirect action is that whenever you render a message on the current page, it won't be shown to the user because it got processed before being redirected.

That's actually a great idea. And yes, I've been annoyed by that caveat with messages before as well, it's unfortunate, but maybe should be seen as an edge case that currently can't be handled as expected.