- π³πΏNew Zealand quietone New Zealand
As a start for #5. The wiki page is updated with a warning that tarballs are being deprecated.
What about combining 1 and 2 and move both
- πΊπΈUnited States drumm NY, US
Tarball downloads are currently featured prominently on https://www.drupal.org/download β
- π³π΄Norway hansfn
Also the evaluator guide β needs to rewritten since it uses the tar-ball for quick setup.
- π³πΏNew Zealand quietone New Zealand
@hansfn, Thanks!
I've update the IS with a new step in the remaining tasks where links can be added to any other pages that needed to be changed.
- πΊπΈUnited States dww
Maybe I'm missing something, but
composer
doesn't magically remove the need for tarballs. It's just a different (mostly better) way to install tarballs for you. But that's what a "dist" installation is. Composer has to get your code from somewhere. Either it's "prefer source" (Git checkout) or "prefer dist" (a "tarball").- Yes, we can hide them from the UI of release nodes. π Remove download of tar files Postponed Not sure why that's postponed, seems it could proceed any time.
- Yes, we can deprecate the parts of Update Manager that let you install "directly from tarball" via the UI once we have project browser. I believe that's π Remove adding an extension via a URL Postponed
- Yes, we can deprecate the parts of Update Manager that update your currently installed codebase once we have autoupdates. Is that #3201968: Augment then Replace current Update Manager URL download based updates with Staged-Composer workflow β ? The summary here lists #3417136 twice. 3201968 has its own ancestry via autoupdates, not sure we should re-parent it.
But none of that means we can "Deprecate tarballs", because all of it still depends on a dist release artifact under the hood. E.g. if you run
composer require drupal/slim_select
, you'll eventually see:- Installing drupal/slim_select (1.0.0-alpha4): Extracting archive
"Extracting archive" means
tar -zxf slim_select-1.0.0-alpha4.tar.gz
in this case. πWe can't actually do #3417136 or #3201968 (or whatever issue we should use for point 2) until we have a stable replacement. #3419966 could happen anytime.
- π¬π§United Kingdom catch
@dww this is only about the core packaged tarballs like https://ftp.drupal.org/files/projects/drupal-10.2.3.tar.gz that include /vendor code and are generated by the packaging script. drupal/core, drupal/core-recommended etc. would still be in .tar.gz format but they're only pulled via composer and not exposed for direct download and install. Updated the title to make this a bit more explicit.
- πΊπΈUnited States drumm NY, US
This in the issue summary hasn't been true
Tarball installations are completely incompatible with modules that declare their own Composer dependencies. Furthermore, switching from a tarball installation to a Composer installation on an existing site is a difficult process for many users.
The tarballs are made with composer create-project legacy-project.
It is somewhat good to have tarballs as a test in packaging, to make sure composer create-project works.
- π¬π§United Kingdom longwave UK
@drumm I think that statement means: if you download core and Commerce as tarballs, Commerce won't work because it has its own Composer dependencies that never get merged into the top-level
vendor
directory. And if you do have a working tarball setup, to switch to Composer you need to manually add each of your dependencies to composer.json, there is no automatic way of doing this.We do have build tests in core to ensure that create-project works, but obviously they can't test a released version until the version is actually released :)
- πΊπΈUnited States drumm NY, US
β ~/Downloads tar xf tar.gz β ~/Downloads cd drupal-10.2.5 β ~/Downloads/drupal-10.2.5 composer require 'drupal/commerce:^2.38' ./composer.json has been updated Running composer update drupal/commerce Loading composer repositories with package information Updating dependencies Your requirements could not be resolved to an installable set of packages. Problem 1 - Root composer.json requires drupal/commerce ^2.38 -> satisfiable by drupal/commerce[2.38.0]. - drupal/commerce 2.38.0 requires drupal/inline_entity_form ^1.0@RC || ^3.0@RC -> found drupal/inline_entity_form[dev-1.x, dev-3.x, 1.0.0-alpha1, ..., 1.x-dev (alias of dev-1.x), 3.0.0-rc15, ..., 3.x-dev (alias of dev-3.x)] but it does not match your minimum-stability. Installation failed, reverting ./composer.json and ./composer.lock to their original content. β ~/Downloads/drupal-10.2.5 composer require drupal/inline_entity_form ^1.0@RC ./composer.json has been updated Running composer update drupal/inline_entity_form Loading composer repositories with package information Updating dependencies Lock file operations: 1 install, 0 updates, 0 removals - Locking drupal/inline_entity_form (1.0.0-rc17) Writing lock file Installing dependencies from lock file (including require-dev) Package operations: 1 install, 0 updates, 0 removals - Downloading drupal/inline_entity_form (1.0.0-rc17) - Installing drupal/inline_entity_form (1.0.0-rc17): Extracting archive Generating autoload files Hardening vendor directory with .htaccess and web.config files. 42 packages you are using are looking for funding. Use the `composer fund` command to find out more! Cleaning installed packages. No security vulnerability advisories found. β ~/Downloads/drupal-10.2.5 composer require 'drupal/commerce:^2.38' ./composer.json has been updated Running composer update drupal/commerce Loading composer repositories with package information Updating dependencies Lock file operations: 10 installs, 0 updates, 0 removals - Locking commerceguys/addressing (v2.2.0) - Locking commerceguys/intl (v2.0.5) - Locking doctrine/collections (2.2.1) - Locking drupal/address (2.0.1) - Locking drupal/commerce (2.38.0) - Locking drupal/entity (1.4.0) - Locking drupal/entity_reference_revisions (1.11.0) - Locking drupal/profile (1.10.0) - Locking drupal/state_machine (1.11.0) - Locking drupal/token (1.13.0) Writing lock file Installing dependencies from lock file (including require-dev) Package operations: 10 installs, 0 updates, 0 removals - Downloading doctrine/collections (2.2.1) - Downloading commerceguys/addressing (v2.2.0) - Downloading drupal/token (1.13.0) - Downloading drupal/state_machine (1.11.0) - Downloading drupal/entity (1.4.0) - Downloading drupal/profile (1.10.0) - Downloading drupal/address (2.0.1) - Downloading commerceguys/intl (v2.0.5) - Downloading drupal/commerce (2.38.0) 0/9 [>---------------------------] 0% 5/9 [===============>------------] 55% 6/9 [==================>---------] 66% 8/9 [========================>---] 88% 9/9 [============================] 100% - Installing doctrine/collections (2.2.1): Extracting archive - Installing commerceguys/addressing (v2.2.0): Extracting archive - Installing drupal/token (1.13.0): Extracting archive - Installing drupal/state_machine (1.11.0): Extracting archive - Installing drupal/entity (1.4.0): Extracting archive - Installing drupal/profile (1.10.0): Extracting archive - Installing drupal/entity_reference_revisions (1.11.0): Extracting archive - Installing drupal/address (2.0.1): Extracting archive - Installing commerceguys/intl (v2.0.5): Extracting archive - Installing drupal/commerce (2.38.0): Extracting archive 0/10 [>---------------------------] 0% 7/10 [===================>--------] 70% 9/10 [=========================>--] 90% 10/10 [============================] 100% Generating autoload files Hardening vendor directory with .htaccess and web.config files. 43 packages you are using are looking for funding. Use the `composer fund` command to find out more! Cleaning installed packages. No security vulnerability advisories found.
- πΊπΈUnited States drumm NY, US
Sorry, I missed this part:
and Commerce as tarballs
Tarballs are indeed not good for contrib, and https://www.drupal.org/project/commerce/releases/8.x-2.38 β says to use Composer over downloads.
Core tarballs are composer-compatible, you get the same thing as
composer create-project drupal/legacy-project
- π¬π§United Kingdom longwave UK
I'm not sure why you would do #26 - if you have Composer then you can run create-project yourself already? But given that example maybe core tarballs aren't really a problem, it's contrib ones that can be problematic. Maybe we can start on the remaining tasks like removing the Updater UI, and start removing more references to tarballs from d.o without waiting for Automatic Updates?
Is there any way of telling from access logs how many people still download tarballs manually compared to via Composer? Unsure also how we will achieve point 3 of notifying existing tarball users that they should switch to Composer.
- πΊπΈUnited States drumm NY, US
Yes, we should go ahead and continue discouraging using contrib tarballs directly. They are still totally used by Composer internally of course.
We prominently feature core tarballs on https://www.drupal.org/download β and various other places mentioned in this issue. Those need auditing & updating before we think about removing core tarballs. Otherwise, I'm mostly mentioning that core tarballs do work with composer, and the issue summary needs updating.
Is there any way of telling from access logs how many people still download tarballs manually compared to via Composer?
We could probably get the access logs from S3, filter to tar/zip release downloads, and see how many have Composer as a user agent. That would be good numbers for contrib, since Composer and manual downloads are the same URL. We would not get a comparison for core, since the downloads via Composer donβt hit our servers, weβd only see the volume of non-composer downloads.
Unsure also how we will achieve point 3 of notifying existing tarball users that they should switch to Composer.
Could core check for the codebase mismatching composer's installed.json? I think automatic updates already has a check for that.
- π¬π§United Kingdom NaheemSays
Can an alternative installer be provided instead that simply (potentially bundles and) runs a composer command to download and install drupal?
- πΈπ°Slovakia poker10
Would it help to remove this before D7 EOL, if we still need to keep all the code for 7.x tarballs?
I do not know the usage stats, but if these are relatively high, would it make sense to remove these tarballs before Project Browser / Auto Updates?
- First commit to issue fork.
- πΊπΈUnited States drumm NY, US
D7 core actually has the same packaging as contrib. Only core D8+ is special, and will remain special for subtree splits, which are part of packaging.
What we do get to simplify around D7 EOL is special cases for displaying tarballs. At some point https://www.drupal.org/project/webform/releases/7.x-3.31 β could look the same as https://www.drupal.org/project/webform/releases/6.2.2 β , probably not for a few months after D7 EOL since there will inevitably be stragglers. And D7 packages.drupal.org support will have to remain until usage is sufficiently low.
- π³πΏNew Zealand quietone New Zealand
Tried to add clarification to the first item in the issue summary. Rearranged the remaining task to be more like the order in which the items can be done.
Do we have a date when sites using tarballs will no longer be available and site must switch to composer?
- π¬π§United Kingdom catch
@quietone
Do we have a date when sites using tarballs will no longer be available and site must switch to composer?
No as I understand it, this issue is for deciding when that date would be amongst other details of deprecating tarballs.
What I think might make sense is to pick a date that is a few months in the future and announce 'Drupal core tarball installs will no longer be supported after x date' so people have plenty of warning, and then we can figure out all the steps between now and then. It wouldn't be a deadline, but it would mean we don't end up stuck here because we didn't do an announcement in time.
- π¬π§United Kingdom longwave UK
Given we have a convenient future date of 5 January 2025 for Drupal 7 EOL should we announce the same date for EOL of tarballs as well?
- π¬π§United Kingdom catch
That works for me - even if d.o doesn't actually drop support for a few months afterwards it still makes sense to announce as the policy change.
- πΈπ°Slovakia poker10
I agree with setting the date to 5.1.2025 to match the D7 EOL.
- πΊπΈUnited States dww
π Remove adding an extension via a URL Postponed was listed for both point 3 and 4. I replaced the mention in point 3 with #3201968: Augment then Replace current Update Manager URL download based updates with Staged-Composer workflow β which I think is closer. Remember, the current "Update Manager" has 2 main aspects:
- Update your currently installed stuff (point 3) -> to be replaced with Auto Updates
- Install new stuff (point 4) -> to be replaced with Project BrowserThey really need to be handled separately, so let's not conflate them and use the same issue to track both.
Meanwhile, I still think point #6 (the d.o UI) could happen any time, I don't think that needs to be blocked on everything else.
All that said, I think we need to be careful how we word this effort and the related communications. As I wrote at #22, we're still going to have tarballs, even "packaged" ones. We're just telling people they have to use composer to install them. Maybe I'm hung up on pedantic semantics, but I believe the title here is misleading and confusing.
- π³πΏNew Zealand quietone New Zealand
Updated the Issue Summary and title to reflect the recent comments.
As for the wording, should we start a google doc for that?
For #6 It doesn't seem right to remove the ability to download until the deprecation date. In the meantime a warning could be added.
- πΊπΈUnited States dww
Thanks for addressing my concerns. I believe @quietone's version was closer to reality. π This issue started as (paraphrase) "Release managers hate tarballs of core that include vendor", but the title from #41 is going in the wrong direction in a few ways:
- The Update Manager doesn't do anything with core, tarballs or otherwise. π
- There are many other things to deal with if we want to stop letting folks directly install from tarball than just Update Manager (d.o changes, communications, etc). These are spelled out in the summary, but the title from #41 only covers 2 of 6 points.
Therefore, restoring the title from #40 (until something better comes along).
Thanks again,
-Derek - πΊπΈUnited States dww
whoops, just notice the grammar bug, sorry for the noise
- πΊπΈUnited States dww
p.s. I just opened π RFC for Ludwig maintainers: Core wants to require composer for everything Active to alert the Ludwig maintainers about this issue, since I figured those folks might care and want to chime in...
- π¬π§United Kingdom catch
Arggh I thought I was on a different issue entirely.
- πΊπΈUnited States drumm NY, US
I thought this issue was originally about Drupal core. Remove tar/zip download links from https://www.drupal.org/download β , https://www.drupal.org/project/drupal/releases/10.2.5 β , and elsewhere, focusing on only composer for installing Drupa. Then dismantle as much of the supporting infrastructure as we want.
Some of the reasons in the issue summary are not true, and others are new, like π Remove drupal/legacy-project as a starting point for Drupal 10 Postponed .
For contrib, packaging and tarballs remain, download links may be more-hidden or removed from release pages, and π Remove adding an extension via a URL Postponed moves forward.
- πΊπΈUnited States cmlara
A possibly better title might be "Only support Core installs managed by composer" so that we get away from the argument about the existence of tarballs in the backed.
- πΈπ°Slovakia poker10
After retitling this and reading #46 it seems to me that this is not dependent on Auto Updates and Project Browser (as I have though previously), as we are talking mostly about core installs. So if I understand it correctly, after we remove these download links, it would not be possible to install Drupal for anyone without the knowledge of composer (if they do not know about the github repository)? So all users currently doing download - extract - install will be cut out, if they do not understand composer? Then I think it would be great to have a look at the stats mentioned in #28/#29. Because this change could increase the level of knowledge for needed for even installing the basic Drupal installation for newcommers (mostly for those without deeper technical knowledge). It is good to take this factor into account as well.
- πΊπΈUnited States cmlara
Considering sites will have already designed and implemented their D10 deployment processes, should this be reserved for a major release onward so that site owners can implement this at the same time they typically plan to refactor their deployments? Possibly change this to "In the release of D11 onward"
Positive:
- Moves the first implementation of this forward
- Allows sites a more delineated changeover that they can choose to adopt when ready (either in the D10 release cycle before upgrading to D11, or when they choose to upgrade to D11)
- Helps encourage sites upgrading from D7 to implement the the new workflow.
Negative:
D10 is now slated to survive until D12 release in 2026 meaning full removal would be considerably longer possibly decreasing the overall gain to Release Managers.Re Statsticis:
Would those be biased towards composer due to CI runs? Since they do not have core releases do they tell us much about the actual ecosystem?