Create a separate rsync shim Composer library for Composer Stager

Created on 19 January 2024, 10 months ago

Problem/Motivation

tl;dr: We want to consider creating an rsync shim for Composer Stager instead of concurrently maintaining a costly PHP-based fallback.

Composer Stager has a thin wrapper around rsync, which it uses for syncing directories for Automatic Updates. It's feature complete and reliable. Unfortunately, we can't count on rsync being available on all web hosts, so we've spent enormous effort creating a PHP-based fallback option. It is the most complex part of our code by itself, and it requires much supporting functionality that would be otherwise unnecessary. It is still not feature complete, and we continue to discover new issues and requirements (e.g.). In short, it makes the whole project large and complex, it increases cost and risk to maintainers and users, and there are almost certainly more issues we don't yet know of. Therefore, we've begun to wonder whether there is a better approach.

Proposed resolution

Bundle platform-specific rsync executables via a shim (a separate Packagist package) that we can count on it always being available. Obviously, there are pros and cons:

Pros

Enormous reduction in code, including internal complexity and API surface, and possibly an entire Symfony component dependency (Filesystem).
We wouldn't have to maintain two parallel implementations of the same functionality and worry that they weren't functionally equivalent.
We would control the version of rsync, so even on hosts that already have it installed, we wouldn't have to worry about it being different from what we've tested with.
Single Responsibility Principle: We would separate dissimilar concerns, enabling us to develop, test, and release them at the pace that made sense for each one, respectively--especially beneficial for security releases that only affected one of them.
The rsync library would have a wider potential audience, enabling us to get more of the benefits of Open Source, including more in-the-wild testing--especially from Windows users, whom we have fewer of in the Drupal community.
We already have experience now with maintaining a separate library off Drupal.org.
Tests would run meaningfully faster because they wouldn't have to be run once for each implementation like they are now.
Runtime performance would be improved--by about 50% on Linux for the file syncing operation (from ~8 seconds to ~4) and possibly as much as 80% on Windows (from ~24 seconds to ~4).
Memory usage would drop by about 50% (~20mg to ~10mb).
The Drupal Automatic Updates module should be able to eliminate some fallback functionality as well.

Cons

We would have to maintain another library.
It could multiply dependency management synchronization and overhead. (Good design decisions could conceivably minimize or avoid this.)
We already have a largely working solution and know many (though not all) of its problems, whereas a new solution would have more unknown unknowns.
We would have to solve a different kind of problem than we've solved before (though it's not without precedent in general, c.f.).
Its wider audience, despite its benefits, could mean added community overhead.