[Plan] Gradually replace Drupal's AJAX system with HTMX

Created on 27 November 2023, 12 months ago
Updated 22 June 2024, 5 months ago

What is the problem to solve?

Drupal's AJAX system has served us well for over a decade. For over 15 years if you count its predecessors in CTools and AHAH. At the time of its creation, we had to create a custom system, because there weren't existing libraries with the functionality that we needed. It was fun to be at the bleeding edge of integrating AJAX into a server-rendered framework like Drupal.

But now HTMX exists. It's a well designed, well documented, well maintained library that's gaining a lot of traction, and I think it supports pretty much everything that Drupal needs from an AJAX system, including the ability for the response to trigger additional actions, which we can use to implement any of our current custom commands that don't have an already implemented corollary in HTMX. In the long run, I think we'd be well served by moving from our custom AJAX system to a library that an increasing number of developers are gaining familiarity with.

However, I don't think it would be easy to preserve Drupal's current AJAX API and change it to use HTMX under the hood. Instead, I think it would be better to convert usages of AJAX (for example, in Field UI, Layout Builder, Views UI, etc.) to use an HTMX-based API.

This would mean maintaining both Drupal's current AJAX API and a new HTMX API until contrib/custom modules have had sufficient time (possibly several major Drupal versions) to convert their ajax usages to htmx.

Who is this for?

Developers

Result: what will be the outcome?

(Describe the result: how will this improve things for the stated audience(s)?)

How can we know the desired result is achieved

(Usability test, a metric to track, feedback)

🌱 Plan
Status

Active

Version

11.0 🔥

Component
Ajax 

Last updated about 1 hour ago

Created by

🇺🇸United States effulgentsia

Live updates comments and jobs are added and updated live.
  • Needs framework manager review

    It is used to alert the framework manager core committer(s) that an issue significantly impacts (or has the potential to impact) multiple subsystems or represents a significant change or addition in architecture or public APIs, and their signoff is needed (see the governance policy draft for more information). If an issue significantly impacts only one subsystem, use Needs subsystem maintainer review instead, and make sure the issue component is set to the correct subsystem.

Sign in to follow issues

Comments & Activities

  • Issue created by @effulgentsia
  • 🇺🇸United States cosmicdreams Minneapolis/St. Paul

    2 birds, 1 stone? How about the Off Canvas Dialog?

  • 🇬🇧United Kingdom catch

    There's already some work done on 📌 Add a native dialog element to deprecate the jQuery UI dialog Needs work on replacing dialog, although not moving especially quickly.

    On HTMX this is new to me, it seems like given it allows adding arbitrary extra things we could probably replicate everything. An annoying thing to replicate will be ajax_page_state and library handling, but that's perhaps a decent test case along with a relatively simple otherwise AJAX callback like replace.

  • I hadn't heard of HTMX, but after looking at the README and some examples, I think it's really cool. It would probably make writing code for AJAX forms a lot easier.

  • 🇺🇸United States cosmicdreams Minneapolis/St. Paul

    The aspect of what makes HTMX interesting in OffCanvas dialog (and perhaps we should talk about the "toolbox" part of the layout builder experience) is that dynamic behaviors are all executed on the server. In Drupal, and other systems that were built in it's era, priority was given to server controlled logic. HTMX allows server driven system to have more dyanamic behaviors, as long as responses to dynamic requests are provided in HTML (instead of JSON).

    For Drupal to fulfill that requirement, we can provide a solution as simple as a Controller that returns a renderable (like every example of a custom block has ever showed) or something as advanced as BipPipe's placeholder handling.

    I'll take a shot at this idea this weekend and see what I can hack up.

  • Is the licensing compatible?

  • 🇬🇧United Kingdom catch

    It's BSD licensed which afaik is fine: https://github.com/bigskysoftware/htmx/blob/master/LICENSE

    Can't remember off the top of my head whether we've shipped BSD licensed js with core before.

  • I understand it's the holidays, but just chiming in that I'm very interested to see @cosmicdreams's demos of this.

    I've been experimenting with HTMX on a project, and it works great for giving the site a SPA-like feel. The problem I've run into is one typical of AJAX loaded content; That is, Javascript won't fire on subsequent page loads because the content was not a part of the DOM. HTMX's onLoad() function (https://htmx.org/docs/#3rd-party) is the remedy for this, and I can make it work if I write vanilla JS, but I do not know how to integrate it with Drupal behaviors.

    This works. It fires on every "page" loaded via HTMX:

    (function (Drupal, once, htmx, drupalSettings) {
    
      'use strict';
    
      htmx.onLoad(function(content) {
        var items = document.querySelectorAll('.js-listItem');
        console.log(items);
      });
    
    })(Drupal, once, htmx, drupalSettings);

    These do not:

    (function (Drupal, once, htmx, drupalSettings) {
    
      'use strict';
    
      Drupal.behaviors.testHTMX = {
        attach: (context, settings) => {
          var items = context.querySelector('.js-listItem');
          console.log(items);
        }
      };
    
    })(Drupal, once, htmx, drupalSettings);
    (function (Drupal, once, htmx, drupalSettings) {
    
      'use strict';
    
      Drupal.behaviors.testHTMX = {
        attach: (context, settings) => {
          htmx.onLoad(function(content) {
            var items = document.querySelectorAll('.js-listItem');
            console.log(items);
          });
        }
      };
    
    })(Drupal, once, htmx, drupalSettings);

    Let me know if there is any assistance I can provide in moving this forward. I'm no JS expert, but I am happy to assist where I can.

  • In the third example “content” is undefined. Did you men “context”?

    “Context” is sometimes the full DOM and sometimes part of the DOM, if there was an Ajax update. Usually you need to use it within the implementation.

  • You are correct. I did do it that way originally, but in my copy/pasting to get it into the post, I did not change it back to context.

  • 🇪🇨Ecuador jwilson3

    This works. It fires on every "page" loaded via HTMX:
    [...]
    These do not:
    [...]

    Behaviors are typically "reattached" to a new DOM context after an ajax operation runs, therefore, the reason the later examples do not work for you is because Drupal.behavior.testHTMX.attach is not called after an HTMX operation, only after a Drupal AJAX operation.

    Interoperability will be one problem to surmount if we have to maintain both systems as a hybrid solution possibly through several versions of Drupal (as mentioned in issue summary).

  • 🇩🇰Denmark ressa Copenhagen

    HTMX looks incredible, a real game changer. ThePrimeagen just released Creator of HTMX Talks HTMX and also a HTMX & Go course (Preview).

    Theo's Truth About HTMX describes the concept well, and it would be fantastic with a simple demo of a HTMX-based Drupal.

  • 🇦🇺Australia larowlan 🇦🇺🏝.au GMT+10

    If I'm honest, I'm kind of wary/weary of replacing something that's worked for 17 years with 'the new shiny frontend thing'

    Can we get a use-case/list of pain points with the current setup to justify this change, which would be disruptive for many custom/contrib modules as well as core.

  • I agree with @larowlan. I don't know of any site that uses HTMX, as it's really new. Would this require a complete rewrite of large amounts of custom/contrib code?

    If HTMX sweeps the web and it becomes well-used across the web or as popular as jQuery, then go for it. Otherwise, it could become another sizeable barrier to getting started with Drupal or to upgrading to the next version.

  • 🇫🇷France andypost

    Moreover jQ4 said they improved CSP and ajax requests https://blog.jquery.com/2024/02/06/jquery-4-0-0-beta/

  • 🇺🇸United States fathershawn New York

    However, I'm very intrigued by HTMX, which I only encountered because you opened this issue. I'm working my way through the documentation still but so far I'm very pleased with the way that it intentionally works with the hypermedia architecture of the web.

    I don't know enough to form an opinion yet, but I'm grateful for being asked to think about and explore a new idea. It may or may not be a replacement for jQuery powered AJAX but we also been discussing removing jQuery for some time.

  • 🇬🇧United Kingdom catch

    Can we get a use-case/list of pain points with the current setup

    It's pretty rare that I use the current AJAX system in depth on client projects, partly chance, partly because I'm happy if other people do that while I work on other stuff. The most recent time I did anything with it was in 📌 Use MessagesCommand in BigPipe to remove special casing of the messages placeholder Fixed .

    I will say from inheriting various sites/reviewing contrib over the years that I have seen some horrible custom js that is trying to re-implement things that should have been done in AJAX commands (like closing a dialog on success, for a recent example). And I don't personally find the AJAX system enjoyable or intuitive to work with, I think it can be hard for people who aren't already familiar with it to learn what should live where and how.

    For me a test of HTMX would be whether we're able to reduce some of the js in core modules that is interacting with the JavaScript js API - views_ui, field_ui, history, toolbar, editor, media_library, bigpipe, claro etc. (grep -rl core/drupal.ajax). Or if not, whether those examples become nicer.

    The things that ajax commands need to do is probably(?) not going to change that much, but if we're returning HTML from AJAX callbacks, or HTML + extras, they might just end up as new render elements instead of their own API, which would consolidate things a bit and be one less thing to learn.

  • 🇫🇷France fgm Paris, France

    HTMX is about 4 years old, so not really new, but awareness seems to have exploded in the last 6 months.

    I happened to have considered creating a module for it, considering how it works. The basic idea would be to either implement new elements or alter existing elements when the module is active, with the matching properties as alternatives to the current AJAX ones. It would not prevent existing attributes from working, so could be created entirely in contrib initially, the main limitation being that a given context (dom subtree) should probably not combine both traditional Drupal Ajax and HTMX.

  • 🇬🇧United Kingdom AaronMcHale Edinburgh, Scotland

    Can we get a use-case/list of pain points with the current setup

    I hit up against this issue back in 2018 and it's still a problem today, required a hacky workaround 🐛 Ajax is broken in forms built within an Ajax callback Active .

    Still a problem 6 years later. (Also was 2018 really 6 years ago?! Oh my...)

    I think that illustrates why we may benefit from replacing our custom AJAX solution with a solution well supported by the wider community.

  • 🇩🇪Germany Anybody Porta Westfalica

    Thanks for all the feedback here. I think all makes sense. Introducing HTMX in Drupal is not a simple task and will require many resources.
    So starting in contrib, making concepts and keep an eye on the general adoption of HTMX is surely the right way to go. Especially we should have a look at DX improvements. I agree that the current AJAX system in Drupal works, but for me it always has been and still is kind of (wonderful) magic in many parts. I agree with @catch in #17 and saw many developers struggling with it. It's not really straight forward, but that doesn't mean it's not good!

    So I think we're at a point, where things work well, but it doesn't mean it couldn't get better. Especially for developers new to Drupal it might be a simplification and unification, similar to what happened with Symfony in Drupal 8 in contrast to the custom codebase in Drupal 7, that also worked great, but was a good choice to use a standard library to make the switch to Drupal easier.

    So there are many good reasons for and against HTMX and the decision has to be made wisely. Trying it out in contrib is a good next step and perhaps someone would like to research the general pro's and con's and feedback for HTMX?

  • 🇺🇸United States cosmicdreams Minneapolis/St. Paul

    I agree with the contrib approach. HTMX needs to be able to demonstrate that in cases where it can impact pieces within core that it can be a "drop ship" replacement of those pieces.

    If I could have a magic wand and get everything I want, Core wouldn't use any 3rd party javascript libraries (and still be awesome). Some days it feels like that reality is achievable but others it seems foolish.

    That said, HTMX's value appears to be in interactivity. So when thinking about the intersection of core and HTMX the areas that have a lot of user interactivity, places like Views, Content Administration, and Recent log messages. Other parts that feel way too ambitious like Layout Manager and impacting ajax.js itself are things that may one day be on the table, but yea, whew sounds like a lot of dramatic change.

    And it should be said that HTMX isn't the "end of the line" / final form of the ideas that it advances. I'm concerned that if we would invest a ton of effort adopting this library in core that we would have to redo / undo some of that work in the near term.

    Going forward

    Where should we meet to collaborate on ideas?

  • 🇫🇷France nod_ Lille

    Very much +1 to #17

  • 🇦🇺Australia larowlan 🇦🇺🏝.au GMT+10

    Interesting that in this PHP UK session he points out HTMX is something we've already had in Drupal https://youtu.be/Yb4mDfwOUXE

  • 🇺🇸United States effulgentsia

    Yes, there's not much conceptually new in HTMX that we didn't have in Drupal 6's AHAH which then became Drupal 7's AJAX. But the HTMX community did a really nice job in creating a more refined API and documenting it well.

  • 🇺🇸United States fathershawn New York

    It seems to me that what is new in HTMX is expressing and documenting the actions in the HTML. By expanding the hypermedia controls using attributes and including all the HTTP verbs in those controls it completes HTTP/HTML to be a fully expressive hypermedia system.

    I've done some creative things in custom projects in the past with our AJAX system, but much of that was not evident by looking at the markup. I haven't built anything yet with HTMX, having just encountered it via this issue, but I'm really looking forward to taking it for a spin!

  • 🇫🇷France fgm Paris, France

    For anyone curious about the Drupal parts in the talk above, the segment mentioning Drupal (and WP) starts at 21:00.

  • 🇺🇸United States fathershawn New York

    I've stepped up to the HTMX module's request for a new maintainer and will be developing ideas for Drupal integration there. Collaborators encouraged 😀

  • 🇬🇧United Kingdom joachim

    One limitation of HTMX is that it's geared towards a simple swap: one element makes a request, and the server returns a single element which is replaced.

    Our AJAX API is more powerful than that: it allows any number of AJAX commands to be returned as a response.

    HTMX does have https://htmx.org/attributes/hx-swap-oob/, and an extension https://htmx.org/extensions/multi-swap/ but these don't seem as elegant as what we have.

  • 🇺🇸United States fathershawn New York

    Regular swaps in HTMX can replace or append with additional control on how and where that is executed.

    Out of band swaps add additional payload and they are not limited in number. Multiple out of band swaps could insert a variety of tags in the same request.

    When paired with javascript event listeners, those payloads can be processed. For example, using the out of band swap currently to add json in a script tag that describes additional CSS/JS assets needed by the inserted content, then loading those assets after the json is inserted.

    It is pushing me to think differently. It's like making stew with a new set of spices.

  • 🇺🇸United States rlnorthcutt Austin, TX

    I'm really excited to see this conversation, and I wanted to add a few things:

    Age of HTMX
    While HTMX itself was released in 2020, it is actually a rebuild of an earlier library called IntercoolerJs. So, the library itself is just about 4 years old, but the original version is 11 years old. This is not a "new shiny thing". It is actually an "old" shiny thing that has only recently been widely discovered.

    HTMX _is_ Ajax
    "The core of htmx is a set of attributes that allow you to issue AJAX requests directly from HTML" (from the docs). So, this is really just a more standardized and *easier* way to do Ajax. Think about how powerful it will be for developers to do much of their frontend Ajax work in Twig files. Pretty enticing.

    At the same time, the complexity of the current Ajax system (while powerful) means that it is not seeing the usage or improvements people expect in a modern framework. I can click a button to get Ajax filtering of a view (cool!), but I still get that same old little spinner in a potentially random place... and trying to improve or adjust that behavior is a non-starter for most backend devs. HTMX provides a cleaner and more standard way of dealing with the Ajax issue.

    Everything old is new again
    The reason that HTMX is getting so much attention these days is similar to the increased focus we have seen on static sites. Speed, performance, and the recognition that things have gotten a bit out of hand for for building content focused websites. In this case, we see that the push for React/Angular/Vue as a full stack framework has led to ever more complex systems to build, tree shake, compile, partially render, and hydrate HTML code. This means that many, many applications are way over-engineered. So, there is a recognition that simpler approaches are just as good and often better. Highly reactive apps need a large JS framework, but most apps only need basic interactivity.

    The core argument *for* HTMX goes like this - you make a call to a server to find data, run business logic, and then format it as JSON which is returned to the client. Then the client has to do more work to render that JSON at HTML and put it on the page. So, why not just return the HTML from the server directly? Its faster, its cache able, and it reduces the load (both time and JS assets) in the browser.

    HTMX is a perfect fit for Drupal
    Drupal is incredibly good at letting us generate HTML, and the templating system means we can generate it down to the field level. So, with very minor changes, we should be able to allow Drupal core to easily return HTML fragments of any entity template. Add in display modes and views, and suddenly you have the ability to create an entire HTMX backend with little to no code. Thats insanely powerful, and can give the project a shot in the arm. Instead of building a custom backend in python, laravel, or go... just use Drupal.

    Then, add in the incredibly powerful caching system, and now we are in completely new territory. Drupal's caching system is one of the "best kept secrets" of the project, and by adopting some minor enhancements (like HTML fragments), we can make it even more visible and accessible. I can see how an HTMX + Drupal approach could invigorate the project and bring it more attention and a whole new type of modern developer.

    HTMX is a perfect fit for Drupalistas
    The average Drupalista is either a backend dev (php) or a front end themer (twig, css, js). Most "modern" app builders using React/Vue/Svelte don't want to work with Drupal - they want to use their framework. We have seen how much resistance there is to using Drupal as a purely decoupled content API - most devs who want that will go for Contentful or Strapi. Drupal can act as a content API, but it compeltely ignores the rendering engine... which is a shame.

    HTMX is largely in line with this approach and is appealing to backend developers. It simplifies the front-end experience and gives us more
    "spa-like" functionality with much less effort.

    I think that this suggestion is about more than just making Ajax easier, adopting new tech, or even growing the community. I believe that this combination has the potential to reinvigorate the project by amplifying Drupal's core strengths and making the framework more relevant in the modern view.

  • 🇫🇷France pdureau Paris

    Thanks Ron for this detailed and exciting note.

  • 🇺🇸United States brianperry

    Want to highlight this from @rlnorthcutt's thoughtful post.

    > Drupal is incredibly good at letting us generate HTML, and the templating system means we can generate it down to the field level. So, with very minor changes, we should be able to allow Drupal core to easily return HTML fragments of any entity template. Add in display modes and views, and suddenly you have the ability to create an entire HTMX backend with little to no code. Thats insanely powerful, and can give the project a shot in the arm. Instead of building a custom backend in python, laravel, or go... just use Drupal.

    Had some similar comments in the HTMX issue queue 🌱 1.1 Goals for HTMX Active . A relatively small amount of functionality could make Drupal the ideal HTMX backend.

    Also worth noting, Astro recently released Page Partials as a feature targeted at this use case.

  • 🇩🇪Germany Anybody Porta Westfalica

    Thanks for the comments! Whao!

    Well, how to proceed?

    I think we all agree, this isn't a low hanging fruit, but might be a very tasty, juicy one high on the tree. So we should think about:

    • Raise Dries & Acquia's attention - I think he might be interested in the concept, and it might be very helpful to have his opinion and network
    • Discuss an initiative for this - it's far too large for single persons
    • Make a plan which Drupal version this might target. 12.x? 13.x?
    • Then we need a plan and proof of concept? ...
    • ?
  • 🇬🇧United Kingdom catch

    We don't target core major versions for things as such, if there was a working HTMX patch with a bc layer (however it might work), that could land in any Drupal core minor release. The only thing then is deciding that's specific to major versions, is deciding what version to deprecate the AJAX API for.

    The main people who need to be convinced here are Drupal core framework managers (tagging for that). I'm also going to move this to the core issue queue, the ideas queue somewhat works for new user-facing features but it doesn't really for developer-facing things since people just don't follow it enough.

    The hardest thing here (going by every other issue to do with the AJAX system recently) is going to be the backwards compatibility layer.

    For PHP, without having reviewed everything properly, it feels like we could make the current PHP AJAX API work with HTMX - i.e. #ajax on forms and AJAX commands don't specify the format that things are returned in, so the same inputs could be put into an HTMX response potentially, then we can change things with deprecations from there, but it wouldn't need every PHP implementation to be updated at the same time that way or maintaining two completely different ways to do things.

    On the other hand, adding bc layers for even one jQuery UI library that we're trying to replace (autocomplete, dialog) can take months of work and get stuck, and usage of the JavaScript API directly is less frequent than the PHP one, although e.g. Views uses it directly in core.

    So I wonder if we could have AJAX vs. HTMX modes which are mutually exclusive on a site. That would mean modules with their own JavaScript interacting with the AJAX API would need to re-implement the functionality so it works in HTMX mode and support both for a while, then sites could switch from one to the other over time, and we could eventually change the default for new sites. This would mean no 'bc layer' as such for the JavaScript AJAX API, it would just co-exist for a while and then be removed.

  • 🇺🇸United States fathershawn New York

    Then we need a plan and proof of concept? ...

    We are working on some proofs of concept in the HTMX contrib module Brian linked in #33. The first thing was to get assets from inserted content loading using the ajax page state approach we have now. The approach I took there may be a good pattern for some of the other components of the AJAX system in core:

    1. data is returned in JSON appended to the body
    2. An event listener processes the data and makes the change.
    3. The additional assets use case responds on HTMX load, but HTMX responses can return custom events so actions in our AJAX system beyond content replacement could be paired with an event and a listener.

    Since we often innovate and learn in contrib, collaborators warmly encouraged over there!

    That would mean modules with their own JavaScript interacting with the AJAX API would need to re-implement the functionality so it works in HTMX mode and support both for a while, then sites could switch from one to the other over time, and we could eventually change the default for new sites. This would mean no 'bc layer' as such for the JavaScript AJAX API, it would just co-exist for a while and then be removed.

    If we approached this in a similar manner to how we are dealing with the change from Annotations to Attributes for plugins, a module wouldn't have to support both but we would need to do so in core for a time. This would allow modules to change anything custom that they have done as needed without a bc layer.

  • 🇫🇷France nod_ Lille

    As one of the people tagged in #35 my response in #17 was pretty short, longer version: I'm all for it. I don't think the other want to keep the existing code either. Next step would be to open an issue with a PoC, (this issue is more for meta-talk, i'd rather have the discussion of implementation in it's own non-plan issue). Iterating in contrib is a good idea too, FYI I personally won't have time to be involved outside of core for the next few months.

    For the PoC what I have in mind:

    • One or two example of a replacement in core (at least)
    • A list of the various ajax commands we have in core and their feasibility (with a very rough complexity estimate easy/hard) with htmx
    • Test runs don't need to be green, although it' be nice if a couple of them were passing.
    • can we use this to handle ajax form submit? if yes, how complex does it look? would be great to drop our jquery formsubmit fork
    • A big thing here is to improve DX, so a couple of examples of how it could make it easier to introduce some ajax stuff in contrib/projects would be good.
    • rough dependency evaluation https://www.drupal.org/about/core/policies/core-dependency-policies/depe... (being a great meme poster is not enough :p)
    • BC strategy (support existing calls, or a new codepath altogether, etc.) I'm not too keen on two separate mods. I feel it'll make adoption much, much slower than spending time on the BC layer even if it takes a long time. (we can always put restrictions on what the BC layer can do) if we have 2 code paths, we'd need a contrib module to bridge the two and let contrib maintainers test with htmx easily without changing their code

    Feel free to open an issue already, you don't need to have all the above to get started and send code, focus on the fun parts for now :). It's just a list of the things we'll need at minimum on the way to a core commit, it'll probably be split up in several issues once it's mostly ready to make review/commit easier. I checked the code for the contrib module, some stuff could be simplified on the js side (you don't need to attach the eventlistener to the body in a behavior for example) all in all, looks promising.

    I'm expecting a lot of discussion in that issue so please have in mind it'll be a marathon, not a sprint. Ajax support in core is ancient and changing it might be challenging, but it's worth it and I'll be here to help along the way.

  • 🇺🇸United States fathershawn New York

    Thanks for the support and clear guidance @nod_ :)

    I opened 📌 [POC] Implementing some components of the Ajax system using HTMX Active

  • 🇺🇸United States fathershawn New York
  • 🇬🇧United Kingdom hugronaphor

    Glad to hear there is a consensus that existing approaches to handle ajax are outdated and not DX friendly.
    This is the exact reason why a while ago when I needed to build an SPA-like app, I needed something more mature and modern.

    I did look at that time at htmx but it felt to me that it's not the right solution to use especially that I have found out about Livewire. As a result I have ported it to Drupal as Wire.

    The main benefit is that it's architecture has been tested and iterated for years already by Laravel community and it can handle any task mentioned here including form submissions, reacting/emitting events, web-socket, ... in fact Symfony ecosystem got their own Livewire inspired Live Components

    To be clear, I'm not saying that I want my Wire in core instead of playing with htmx but I think we should look at the scope broader and consider something in between Wire and Symfony's Live components. Huge benefit on top of being able to create dynamic interfaces with ease, Laravel and Symfony developers would have a familiar tool in Drupal's core.

  • 🇫🇷France nod_ Lille

    Thanks! That's an interesting project. Haven't had time to dig into the code but happy to see alternative takes being explored.

    I'm personally for htmx because the philosophy is aligned with Drupal and what we try to do. And I like that a lot of things live in the markup instead of hidden away In a js object somewhere.

  • 🇦🇺Australia sime Melbourne

    My first encounter with htmx was watching theprimeagen review it through non-drupal lens. To me it felt like the DX is better than Drupal core and very much made me think about how comparitively hard Drupal ajax is.

  • 🇫🇷France fgm Paris, France

    Before doing a lot with that, there's an issue with HTMX which I would not know how to solve for now : HTMX basically makes the assumption of a stable <head> section across pages, except for the title: when elements are swapped by a hx-boost, possibly inheriting if from <body hx-boost="true">, all <head> content in responses is dropped except for the <title> element.

    However, Drupal pages have long been dependent on a strong relationship between <head> and <body> content, with different JS/CSS bundles from page to page, for example.

    Now, you can indeed push changes to <head> using multiple response top-level element with the hx-swap-oob attribute, but that needs an ID on each modified target, which needs to pre-exist on the source page, which seems like a lot of potential trouble.

    Even without using <body hx-boost="true">, partial replacement responses are often likely to require <head> changes.

    I think this is something which needs to be solved earlier than later for a PoC, to outline the patterns that can be used to work around that discrepancy between the HTML model and the Drupal model.

  • 🇬🇧United Kingdom catch

    However, Drupal pages have long been dependent on a strong relationship between and content, with different JS/CSS bundles from page to page, for example.

    So this is true, but for AJAX requests we already deliver assets separately outside the head markup and it's also the case for big pipe placeholders (which would be another thing to try a conversion of fairly early on since that relies entirely on the AJAX system with a small amount of custom js)..

    I think the existing concepts of ajax_page_state ought to work - i.e. we track the libraries that have already been sent, and when new libraries are sent, they go through the asset rendering system and are sent as a new aggregate - both script and stylesheet link tags are allowed in the body of the document so they can be sent with the HTML (or injected via js wherever). Drupal settings and HTML head links (favicon, feeds etc.) are a bit different though so definitely need to figure out those.

  • 🇫🇷France fgm Paris, France

    big pipe placeholders (which would be another thing to try a conversion of fairly early on

    Indeed, that reminds me of another potential difficulty, which is the use of the hx-trigger="load" or "revealed" or "intersect" relying on the default browsers events or the intersection API, which is likely to interfere with our use of behaviours to support multiple loads per context like document or other node e.g. with BigPipe.

  • 🇺🇸United States fathershawn New York

    Some more thoughts on the questions raised in #43 🌱 [Plan] Gradually replace Drupal's AJAX system with HTMX Active , #44 🌱 [Plan] Gradually replace Drupal's AJAX system with HTMX Active , and #45 🌱 [Plan] Gradually replace Drupal's AJAX system with HTMX Active above:

    However, Drupal pages have long been dependent on a strong relationship between <head> and <body> content, with different JS/CSS bundles from page to page, for example.

    I think the existing concepts of ajax_page_state ought to work

    I think I do have this working with ajax_page_state in my contrib implemenation. HTMX includes a header, HX-Request with every request it makes. I have a response subscriber that only processes the response if this header was included on the request. On HTMX requests, the response is processed using logic adapted from our current AjaxResponseAttachmentsProcessor. The resulting asset data is attached to the response as JSON.

    Now, you can indeed push changes to <head> using multiple response top-level element with the hx-swap-oob attribute, but that needs an ID on each modified target, which needs to pre-exist on the source page, which seems like a lot of potential trouble.

    Perhaps HTMX has evolved it's capabilities as all my exposure is recent. The <script> tag containing the JSON asset data is wrapped with <div hx-swap-oob="beforeend:body"></div> as part of the response processing. The value on hx-swap-oob defines the swap strategy and target that is used for the inner HTML of the <div>. The result is a script tag appended to the end of the body. HTMX fires an event, htmx:oobAfterSwap, after it has processed and inserted our asset JSON. Currently I have a listener for this event that uses the loadjs library we already use to process and attach the assets.

    which is likely to interfere with our use of behaviours

    HTMX fires another event, htmx:load after each response has finished processing. I'm currently using an event listener on this event to call our Drupal.attachBehaviors(). All of this is verified in a test.

    I have less experience with the JS side of our core code and @nod_ said the JS here could be improved 🌱 [Plan] Gradually replace Drupal's AJAX system with HTMX Active so please offer improvements either in 📌 [POC] Implementing some components of the Ajax system using HTMX Active or in the HTMX module issue queue .

    A related issue will be history support. If we enable back/forward navigation with the hx-push-url we probably need to make it so that the cacheability headers, among others, be updated for updated pages, because the result will often not match the source. Think, for example, of a search page filled with results: the results will at least carry cache tags for the results but the original page will not.

    If we did enable history support, what I think is important is that the url that is updated in the browser should be able to recreate the current display. Using the example of a search page, we commonly add query parameters to such urls containing the search terms. I don't believe that we are currently updating anything in the header on ajax responses, other than header JS included in the differential assets payload. If we find that we do want to add fully harmonizing the head section, we could explore adapting the HTMX head-support extension for our specific needs.

  • 🇦🇺Australia sime Melbourne
  • 🇺🇸United States kevinquillen

    What is the easiest or smallest POC to proof out here?

  • 🇺🇸United States fathershawn New York

    @kevinquillen At the request of @nod_ we are discussing that in 📌 [POC] Implementing some components of the Ajax system using HTMX Active

  • 🇺🇸United States fathershawn New York

    I've had an inquiry about the appropriate markup to use and have moved that to a child issue: 🌱 [policy, no patch] Choose a markup strategy for HTMX POC Fixed Please add your guidance there!

  • 🇫🇷France nod_ Lille

    done

Production build 0.71.5 2024