Create a mirror for external library dependencies for composer support

Thanks @pameeela for introducing this.

Happy to help out with this, if maintainers agree to proceed with this. Here is what we've done in other places like Fullcallendar → or Klaro → :

• Create a general project on d.o as a mirror of the remote library
• Add a composer.json to that repository, set the name to drupal/NAMEOFLIB and set the type to drupal-library, also add the installer-name in the extra section to the name of the directory into which this library needs to be installed
• Push that code and create a release, ideally with the same version number as the referencing upstream library

The packaging system from d.o will publish that project to Packagist and you can now add drupal/NAMEOFLIB to require section of the composer.json of this project. From now on, the library will be installed automatically without the user having to do anything manually.

Of course, this needs to be done for each external library and yes, there will be some overhead to keep the mirrors up-to-date. From experience, I can tell, this doesn't happen too often and should be manageable.

Note: We are not asking the maintainers to set this all up, just seeking buy-in for this approach. We can coordinate creation of the mirror projects and are not expecting you to take on the job of keeping them updated!

But I'm also updating the tag considering the last-minute nature of this request and the fact that it is not trivial.

This problem and solution for external libraries makes sense to me.

What will be the namespace for these JavaScript projects?

Key in mind there is https://git.drupalcode.org/project/webform/-/blob/6.3.x/composer.librari...

Could we move this file to something webform_libraries/composer.json and include it via a path?

The below example is entirely untested.

{
    "repositories": [
        {
            "type": "path",
            "url": "webform_libraries/composer.json"
        }
    ],
    "require": {
        "webform_libraries": "*"
    }
}

The namespace for such libraries will be drupal/NAMEOFLIB.

After that change, I think that the extra webform libraries composer.json file won't be required anymore.

I think this makes it more urgent that Webform sub-modules that depend on external libraries be moved into separate projects. That way, the library would only get installed if the site wants to use it. The sites I work on only use a very few external libraries that we have specifically chosen to add to our Composer files.

@liam morland do you mean sub-modules or separate modules? For the former, this won't make any difference, because only the main module comes with a composer.json, and that requires everything that itself and its packaged sub-modules potentially need. Only if some modules were moved into stand-alone separate modules, the list of required libraries would be reduced.

I am suggesting that some modules that are currently sub-modules of the Webform project should be moved into their own projects. If you choose to install the module, its Composer file will bring in the corresponding library.

Thanks for the clarification, so that would be separate modules then. You're correct, that would work. But I doubt this could be done in the short timeframe that's available in light of the Drupal CMS requirement.

Perhaps we should separate them out for Webform 6.4.x.

I'm concerned about adding all those libraries to the Webform Composer file. This would mean that all Webform users would suddenly get libraries installed that they are not using. Could you have the Drupal CMS Composer file require the libraries? Once the Webform sub-modules are moved into being top-level modules in their own projects, then the libraries would get installed from each of those.

I tested my suggestion in #5 and using the a path repository won't work as expected.

I don't think we need to require the new webform external library projects in webform/composer.json but Drupal CMS could require them.

The webform external library projects would be another (recommended) way to install external libraries.

We are going to run into namespacing issues unless we add a prefix or suffix. For example, there is already a https://www.drupal.org/project/select2 → and https://www.drupal.org/project/codemirror → project.

We could suffix the namespace, below are some example.

https://www.drupal.org/project/select2_library →
https://www.drupal.org/project/select2_lib →
https://www.drupal.org/project/select2_ext →

I think select2_library would be good, pretty clear. Does Drupal have a pattern for this?

There is no established pattern AFAIKT, we've used the suffix *_js in other places before.

Adding the dependencies to Drupal CMS' composer.json does work, too. I just wonder why other users of webform shouldn't benefit from this improvement as well. The fact that a dozen JS libraries get downloaded and stored in the libraries folder, doesn't sound too bad. At least compared to the gain factor in convenience.

*_js is perfect because it is the shortest and easiest-to-understand suffix.

Adding the dependencies to Drupal CMS' composer.json does work, too. I just wonder why other users of webform shouldn't benefit from this improvement as well. The fact that a dozen JS libraries get downloaded and stored in the libraries folder, doesn't sound too bad. At least compared to the gain factor in convenience.

The added benefit is that the site gets notified of a new version or security issue. The one downside is files in /libraries are publically available, and there have been test *.html pages in J /libraries with XSS issues.

there have been test *.html pages in J /libraries with XSS issues.

That's true, but we could easily clean-up the mirrored libraries and remove those docs and example directories.

Also, another benefit is that webform will then be installable with project-browser without the user having to bother about anything.

Even if the test pages are gone, I would much rather not have any JS library available on the server unless I am using it.

Drupal CMS, we should create and require the *_js external library projects.

We can also rework the webform module to recommend these external library projects.

At a later point, we can decide to require these external library projects via the webform module's composer.json.

Does this seem like a reasonable way forward?

Yes, that makes sense. I don't think we should require any external library unless Webform cannot operate without it.

Still, I wonder how users who just work in the UI and install the webform module with project browser would ever get that up and running. It sounds like this isn't possible right now, or am I missing something.

As for requiring the libraries by the recipe in Drupal CMS, yes, that possible. The original intent of this issue has been to make this a webform improvement in general. But if that's not what the maintainers want, then @pameeela needs to make the call how Drupal CMS wants to deal with that.

As it currently is, there is no way to install the libraries from Project Browser; it would be CDN-only.

Probably @phenaproxima's call really: #3472226-12: Handle webform external libraries →

At this point it's really not about what's ideal but what we can do for the release. I'll update that issue to highlight the conversation here so we can decide on an initial path.

We could also create a webform_libraries project with a composer.js file that requires all the external libraries. The Webform Libraries project could be used via Project Browser.

If the Drupal community agrees that external libraries should be mirrored as projects (w/ *_js suffix ) on Drupal.org, I will be 100% committed to adopting this solution for the Webform module.

Well that seems like a really sensible idea?

That could work, it should be a recipe rather than yet another module, though.

That could work, it should be a recipe rather than yet another module, though.

I am not used to thinking in terms of recipes but that it is perfect.

To help move us along, I updated the UPDATE_LIBRARIES.md and exported libraries

@see https://git.drupalcode.org/project/webform/-/blob/6.3.x/docs/UPDATE-LIBR...
@see https://git.drupalcode.org/sandbox/jrockowitz-2941983/-/tree/6.3.x

Below are all the libraries

• algolia.places
• choices
• codemirror
• jquery.chosen
• jquery.geocomplete
• jquery.hotkeys
• jquery.icheck
• jquery.image-picker
• jquery.inputmask
• jquery.intl-tel-input
• jquery.rateit
• jquery.select2
• jquery.textcounter
• jquery.timepicker
• jquery.toggles
• popperjs
• progress-tracker
• signature_pad
• svg-pan-zoom
• tabby
• tippyjs

How would we use composer.json to ensure that the project with *_Js is placed in the correct directory?

It is also worth noting that the progress-tracke only contains CSS, so I suggested using an *_library project extension. Using *_js is also okay.

Lastly, we also need to account for 📌 Deprecate the use of drgullin/icheck Active