Return htmx responses as SimplePageVariant

Opened an MR that uses a default simple page with a header control for using a block page variant.

Can we make use of a main content renderer like we do for dialogs?

Maybe. That doesn't necessarily look simpler to me but I don't know the internals as well as @larowlan. The work in 📌 Process attachments (CSS/JS) for HTMX responses and add drupal asset libraries Active is based on returning an HtmlResponse object, so would we be creating a simpler version of \Drupal\Core\Render\MainContent\HtmlRenderer? I would think we would still need an event subscriber to divert the request to the alternate renderer.

Another advantage of the page variants is that SimplePageVariant contains system messages as well as main content. That would allow the calling element to use hx-select-oob, or for us to add an hx-swap-oob to the response, and display error messages as well as the selected content from the response.

All tests green, including new unit test for this subscriber

System messages in the AJAX system are rendered via MessageCommand, and the same infrastructure is used by BigPipe since 📌 Use MessagesCommand in BigPipe to remove special casing of the messages placeholder Fixed . I'm not sure how that would compare to #8 in terms of final implementation, although presumably we might need to support an HTMX version of MessageCommand for bc anyway?

For the idea in #6 we would need HTMX to send ?_wrapper_format=htmx in the query string of all requests. This looks like it is doable via the htmx:configRequest event. Then, we add a service similar to HtmlRenderer but which can do less work - it wouldn't need to care about display variants at all.

Rendering the system messages in the response would mean that we wouldn't need a command to define the messages. The core systems of defining and displaying messages is available.

We would need to implement in a later issue either

1. Always select and display system messages as additional inserted content
2. Provide an operation for selecting the messages from the response and displaying them to be used with appropriate

@longwave Okay, that's two experienced maintainers recommending a renderer over page variants. Thanks for weighing in!

We don't need the query string as HTMX adds the header HX-Request: true to all requests. I'll start to work on that approach as an alternative. I think we do still want title though so that the HTMX feature of swapping the title is available if appropriate but we turn it off by default.

We do need the query string, as that's how MainContentViewSubscriber decides which instance of MainContentRendererInterface is needed to render the page; without the query string it will fall back to HtmlRenderer.

+1 to #14 it will use seriously less code selecting renderer earlier

btw about messages ...hope it covered with drupal.message since 8.7 →

btw that's exactly a place to extend because attachments also should be delivered and merged, and not sure about "ajax_page_state"

and maybe a middlegroud is to add middleware to append query string when header provided

RE: #16 - yes we have that covered in 📌 Process attachments (CSS/JS) for HTMX responses and add drupal asset libraries Active

Unsure how this interacts with caching as well - if the same route is usable via both standard HTTP and HTMX then we want the cached responses to differ.

If it's a GET request (which it should be for HTMX when possible), then the internal page cache won't differentiate so it could get corrupted, and same problem with varnish/CDNs, so we would have to enforce that the HTMX route is only used for HTMX and throw a bad request exception or something when the header is missing.

That makes me think we should just set the query string on the client side, even if we also have the header set.

#19 reminded me of 🐛 Enable header-based proactive content negotiation with optimizations and opt-outs available. Needs work where Drupal does not use the Accept header for format negotiation, primarily because of external caching issues.

#2364011: [meta] External caches mix up response formats on URLs where content negotiation is in use → was I think the definitive issue for Drupal 8 where accept negotiation got canned. I haven't re-read that issue recently but remember the whole area being extremely painful at the time. So yes, on the basis we would want at least some HTMX responses to be cacheable in edge caches, let's use a query string from the client.

I'm so glad to be working with collaborators who have a historical perspective on the code! This is an enhancement not a requirement that will give a bit of performance boost. I'm going to postpone it and move it down the task list before BigPipe.

In the module, this simplified HTML response is a route option and scanning the linked history that may be a good thing. Let's think about routes we might build to serve requests from HTMX and pick up this work in a while. Ideas that I have for routes are:

• the one I have in the module /htmx/{entityType}/{entity}/{viewMode}
• one to render placeholders as the main content

I don't think the routing concern is necessarily a big pipe conversion blocker. Big pipe just uses render placeholders that return an AjaxResponse object, there's no routing involved.

It might not be, but if we refactor BigPipe to use HTMX we should return HTML - probably with the placeholder as the main content, and I wondered if that might intersect with this use case??

Don't think this needs postponing

That's true @nod_

I would return to my proposal in #23 that we bring the route option over from the contrib module.

For the _wrapper_format query string solution I have a MR up at https://git.drupalcode.org/project/drupal/-/merge_requests/12900

The Needs Review Queue Bot → tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide → to find step-by-step guides for working with issues.

This is really good @nod_! Setting to "needs work" for a test for the new renderer. I'm also recommending that we hold FormBuilder changes for 📌 Support dynamic forms using HTMX Active

What if we moved these lines and the associated ::processAssetLibraries method from HtmlResponseAttachmentsProcessor into an asset processor service?

     $ajax_page_state = $this->requestStack->getCurrentRequest()->get('ajax_page_state');
      $assets->setAlreadyLoadedLibraries(isset($ajax_page_state) ? explode(',', $ajax_page_state['libraries']) : []);
      $variables = $this->processAssetLibraries($assets, $attachment_placeholders);

We now have two situations where we would like to get just the asset markup: here and in BigPipe.

As a side note, we can also prevent direct access to this renderer by checking for the HX-Request header.

I'm seeing double escaping in the output from our test page. I have a refactoring idea that could also benefit our big pipe needs. I'll post some code soon.

fathershawn → closed merge request !12032

The use of a template wrapper inspired me to make the markup group the assets that we are looking for. I'm also thinking that with a bit more work on asset processing we can use the same template approach in the refactored bigPipe.

I left off template wrapping the content so that our approach stays consistent with HTMX documentation (https://htmx.org/docs/#troublesome-tables) . We also can document the need for parsable HTML. For direct select/swap a developer can wrap the content that they really want in the needed structure and use an appropriate CSS selector to get the elements that they want.

I also offer this regular expression: https://regex101.com/r/WMznxt/3 that leverages the template markup I'm offering if we don't want to depend on html parsing to get the asset markup.

I'm happy to write a test for the renderer when we come to consensus on the output.

Here's the output from the templated assets version for our test page response

<!doctype html>
    <html>
    <head>
        <title>Ajax Content</title>
    </head>
    <body>
        <div class="dialog-off-canvas-main-canvas" data-off-canvas-main-canvas>
            <div class="layout-container">
                <header role="banner">
                </header>
                <main role="main">
                    <a id="main-content" tabindex="-1"></a>
                    <div class="layout-content">
                        <div data-drupal-messages-fallback class="hidden"></div>
                        <h1>Ajax Content</h1>
                        <div class="ajax-content">Initial Content</div>
                    </div>
                </main>
            </div>
        </div>
        <template data-drupal-htmx-assets>
            <link rel="stylesheet" media="all" href="/core/modules/system/tests/modules/test_htmx/css/style.css?t13305" />
            <script type="application/json" data-drupal-selector="drupal-settings-json">{"path":{"baseUrl":"\/","pathPrefix":"","currentPath":"htmx-test-attachments\/replace","currentPathIsAdmin":false,"isFront":false,"currentLanguage":"en","currentQuery":{"_wrapper_format":"drupal_htmx","ajax_page_state":{"theme":"stark","theme_token":"null","libraries":"core\/drupal.htmx,system\/base"},"replace":""}},"pluralDelimiter":"\u0003","ajaxPageState":{"libraries":"eJxLzi9K1U8pKi1IzNHLKMmt0CmuLC5JzdVPSixO1SlJLS6JB4nqJxYXp5YUAwCThhG5"},"user":{"uid":0,"permissionsHash":"063738de691cc2450c1e8bd886e26fe4403ae290bb2b7031ca3342ee6b15822c"}}</script>
            <script src="/core/assets/vendor/once/once.min.js?v=1.0.1"></script>
            <script src="/core/modules/system/tests/modules/test_htmx/js/behavior.js?v=11.3-dev"></script>
        </template>
    </body>
</html>

we don't need the page wrapper, the ajax framework doesn't have it and this shouldn't either, offcanvas, skiplink are not relevant for an ajax response

Thinking about #44 brought a tickle in the back of my mind to full thought. I think we've gotten down a wrong path on this issue again. I started thinking about the use case where what the developer needed was to replace the entire body tag as it would be rendered by the active theme. Then I started thinking about refreshing a dynamic block. Which leads me to these parameters:

1. We launch in core with the routes that core provides.
2. Developers need to be able to request and select any markup.

The ajax api was able to return partials via the callback system. We have the tools to add routes that render entities in specified view modes, render blocks, even render individual fields from entities. We are there yet and even then those will be routes.

I wondering if the route option is the best way to craft a minimal response? Or should we build the wrapper format such that it is not expected but available to be added in individual cases?

fathershawn → changed the visibility of the branch 3522597-wrapper-format-template-assets to hidden.

I've extended the work from @nod_ in MR12900 to include the route option and to make the wrapper format available but not enforced. Created a test for the renderer based on these options which is now in MR13115.

All tests passing.