Manually test TUF-enabled Composer projects

Tested on a small personal project. Results:

• Packages: 15
• TUF metadata: 2.9M

Tested on a second small side-project:

Packages: 19
TUF metadata: 2.8M

I'm planning to test on a bunch of different projects. To make this more efficient, I put together a couple simple scripts that just run the steps outlined in the issue summary.

Report from a medium-sized application:

Packages: 53
TUF metadata: 3.6M

When running this composer tuf:protect https://packagist-signed.drupalcode.org I got the following:

Authenticity of packages from https://packagist-signed.drupalcode.org are not verified by TUF.

[red] There are no commands defined in the "tuf" namespace. [/red]

Composer [tuf:protect https://packagist-signed.drupalcode.org] failed, composer command failed: exit status 1. stderr=

I edited the composer.json file manually to have:

        "drupal-core": {
            "type": "composer",
            "url": "https://packagist-signed.drupalcode.org",
            "tuf": true
        },

And then I could continue.

Report:
- Packages: 32
- TUF metadata: 3.0M

I re-did the whole thing from scratch on the above project and I did not get that small error, so all the steps worked directly.

FYI, you may occasionally see an error like:

The 'bin_c50-c53' contents does not match hash 'sha256' specified in the 'snapshot' metadata.

Sometimes this is transitory, and will fix itself in a matter of seconds. Re-running `composer update` will likely succeed. If not, but the 'bin_c50-c53' part changes, wait a minute or two and try again. This is a symptom of temporary inconsistency during metadata updates on the server-side. This is being addressed upstream in rugged/rugged#208: Ensure metadata consistency.

On the other hand, this kind of error can also persist longer. If re-running `composer update` show no change in the 'bin_c50-c53' part of the error message, this likely indicates that the TUF server has "stalled" during a metadata update. This will generally require intervention by a TUF server administrator. This is being addressed upstream in rugged/rugged#204: Add a monitor task to clean up stuck targets.

We expect this to stabilize as we address the upstream issues. However, if you do encounter such an error, it is worthwhile to report it here, in this ticket.

I did the process and it didn't work initially. As @fjgarlin said in #8, only after I did the process from start a second time it did work. Maybe we should have first executed the code to download the initial root metadata for the repos and only then try to add the signed repository for Drupal core and enable TUF protection for it? I mean, it doesn't seem like an issue of the Packaging component but more like an incorrect order of the commands as described in the issue summary here.

Packages: 77
TUF metadata: 4.5M

I also had problems with installation and now get out of memory errors when running composer up in both ddev and on my MacOS host.
PHP Fatal error: Allowed memory size of 1610612736 bytes exhausted (tried to allocate 20480 bytes) in /var/www/html/vendor/php-tuf/php-tuf/src/CanonicalJsonTrait.php on line 46

FWIW, here is my data on one client's installation after several attempts:
Packages: 118
TUF metadata: 5.3M

Hit an immediate hard failure - opened PR https://github.com/php-tuf/composer-integration/pull/126
With applying that, get:

https://packagist-signed.drupalcode.org could not be fully loaded (Maximum allowed download size reached. Downloaded 1165080 of allowed 1165080 bytes), package information was loaded from the local cache and may be out of date

(this continues after many runs)

Packages: 81
TUF metadata: 5.4mb

Failing with "The 'bin_d9c-d9f' contents does not match hash 'sha256' specified in the 'snapshot' metadata." - for a while.

I haven't done the steps yet but #10 suggests this needs work one way or the other.

On the other hand, this kind of error can also persist longer. If re-running `composer update` show no change in the 'bin_c50-c53' part of the error message, this likely indicates that the TUF server has "stalled" during a metadata update. This will generally require intervention by a TUF server administrator. This is being addressed upstream in rugged/rugged#204: Add a monitor task to clean up stuck targets.

https://gitlab.com/drupal-infrastructure/package-signing/rugged-metrics now alerts us when targets are stuck, but it does require manual intervention to resolve. Once https://gitlab.com/rugged/rugged/-/issues/204 is deployed, this will be more automatic.

Most of the issues we have seen are actually directories stuck in the building phase, before Rugged is involved. There is also some chance of disruption on Tuesdays during the regular AWS maintenance window for the managed queue.

PHP Fatal error: Allowed memory size of 1610612736 bytes exhausted (tried to allocate 20480 bytes) in /var/www/html/vendor/php-tuf/php-tuf/src/CanonicalJsonTrait.php on line 46

1.6MB is not a lot, especially when Drupal will require 64MB. Regardless, the php-tuf client will need to be better at detecting constrained memory situations, and keeping memory usage low. We do want to be judicious with memory use, since the hope is to expand TUF signing to the wider PHP ecosystem.

https://packagist-signed.drupalcode.org could not be fully loaded (Maximum allowed download size reached. Downloaded 1165080 of allowed 1165080 bytes), package information was loaded from the local cache and may be out of date

I think the Composer plugin client will need some error handling improved. This suggests something is amiss server-side, good chance cached at the CDN, but I don’t know what it might be. Running with -vvv might help point to which URL might be the issue.

Failing with "The 'bin_d9c-d9f' contents does not match hash 'sha256' specified in the 'snapshot' metadata." - for a while.

This looks like it is resolved now:

$ curl --silent https://packagist-signed.drupalcode.org/metadata/snapshot.json | grep -A2 bin_d9c-d9f
   "bin_d9c-d9f.json": {
    "hashes": {
     "sha256": "bf9500067989a47152416e9dde6b7c059286657d080f2fd526949e462d5c0172",
$ curl --silent https://packagist-signed.drupalcode.org/metadata/bin_d9c-d9f.json | sha256sum
bf9500067989a47152416e9dde6b7c059286657d080f2fd526949e462d5c0172  -

Right now we are permanently caching at the CDN and purging when changes happen. I’ve seen caching be enough of an issue that I think I will put a 5 or 10 minute lifetime on the cache at the CDN. So issues like this should self-heal after a few minutes.

Opened a few issues to track what I mentioned in my last comment:

• Improve resilience of uploading building directories https://gitlab.com/drupal-infrastructure/package-signing/packagist-signe... this is an operational improvement. When a building directory is stuck, rugged continues as usual
• Improve handling of low-memory situations https://github.com/php-tuf/composer-integration/issues/127
• Improve error messages to show which HTTP request is the source of errors https://github.com/php-tuf/composer-integration/issues/128
• The fallback TTL of 1 hour was being used for TUF metadata at the CDN. It is now reduced to 10 minutes

I moved “To provide the root of trust for TUF verifications, download the initial root metadata for these two repos:” up in the issue summary, which should make the setup steps run more smoothly. While there are things to improve, we do need more real-world testing to see if the improvements have been effective, so setting back to needs review.

I also got the same result as #7 and @fjgarlin's notes fixed it.
I had to increase my php memory limit to run composer -vv update

Report from my medium D11 site:

- Packages: 31
- TUF metadata: 3.9M

Same here as the previous tests, some memory issues, plus the order of commands.
My results:

- Packages: 89
- TUF metadata: 6.6M

Putting my edits to address #7 back in the issue summary.

skyejohnson & kyriazo - what was your PHP memory limit that you hit when you started, and how much did you raise it to succeed?

@drumm I just set it to -1 tbh

Using the time utility (make sure its the binary one not the bash built-in) would also probably be useful to log memory usage.

I put this into a contrib module lab (in no way representative of an actual deployment, just 10 packages of drupal/core using 3.0 mb in the tuf folder to develop a single contrib module)

With the tuf plugin:

$ /usr/bin//time composer -vv update
22.15user 3.12system 0:50.02elapsed 50%CPU (0avgtext+0avgdata 975440maxresident)k
88inputs+297600outputs (0major+270631minor)pagefaults 0swaps

Without the tuff plugin:

$ /usr/bin//time composer -vv update
10.19user 3.09system 0:20.90elapsed 63%CPU (0avgtext+0avgdata 280376maxresident)k
0inputs+285856outputs (0major+90243minor)pagefaults 0swaps

Max resident memory consumption increase, 695mb, a 3.4x increase..
(Jut to be clear #14 mentions 1.6Mb, the OOM report is actually 1.6Gb)

COMPOSER_MEMORY_LIMIT=960m - succeed with tuff (succeeds by less than 64mb)
COMPOSER_MEMORY_LIMIT=256m - succeed without tuf (by less than 10mb)

I misread the conversion here:

PHP Fatal error: Allowed memory size of 1610612736 bytes exhausted (tried to allocate 20480 bytes) in /var/www/html/vendor/php-tuf/php-tuf/src/CanonicalJsonTrait.php on line 46

1.6MB is not a lot, especially when Drupal will require 64MB. Regardless, the php-tuf client will need to be better at detecting constrained memory situations, and keeping memory usage low. We do want to be judicious with memory use, since the hope is to expand TUF signing to the wider PHP ecosystem.

This is actually 1.6GB, which is already quite a high memory limit. Along with #21, it does look like the client seems to have a memory usage problem. So https://github.com/php-tuf/composer-integration/issues/127 is looking like the biggest blocker to getting this closer to core.

Neil brought this to my attention, so I've been taking a quick look at this as part of DrupalCon contribution day in Singapore.

I've used a fresh clone of drupal_cms for testing with a fresh install and initialization of DDEV, on Apple silicon.

As specified in the description, here are the following desired metrics:

fubarhouse@drupal-cms-dev-web:/var/www/html$ composer show | grep drupal/ | wc -l
Authenticity of packages from https://repo.packagist.org are not verified by TUF.
104

fubarhouse@drupal-cms-dev-web:/var/www/html$ du -sh vendor/composer/tuf
7.7M	vendor/composer/tuf

I also ran into this one.... once - in a series of tests, however this seems inconsistent as I only got it once.

In TrustedAuthorityTrait.php line 50:

  [Tuf\Exception\MetadataException]
  The 'bin_c50-c53' contents does not match hash 'sha256' specified in the 'snapshot' metadata.

I have posted a couple of graphs in Drupal Slack if anybody is interested - they show the memory usage at ~800MB without the running process and at its peak it was using 7.1GB, however it did not run out of memory or show an error. A link to these graphs is below:

• https://drupal.slack.com/archives/C7QJNEY3E/p1733886574903149?thread_ts=...

We expect this to stabilize as we address the upstream issues. However, if you do encounter such an error, it is worthwhile to report it here, in this ticket.

I set up composer as described, and I'm running composer update -vvv manually.
I've been getting the same "Maximum allowed download size reached. Downloaded 1024 of allowed 1024 bytes" on these two for the past hour:
https://packages.drupal.org/8/metadata/bin_628-62b.json
https://packages.drupal.org/8/metadata/bin_04c-04f.json

On repeated runs, when one succeeds, the other fails.

Can you include a bit more context, ideally the full console output? What I was understanding in https://github.com/php-tuf/composer-integration/issues/128 was that the specific files weren’t mentioned in the error message. HTTP requests happen in parallel, so I don’t think the line above the error message is necessarily the cause, which would also explain why it appears to be switching.

Piped output to log:
COMPOSER_MEMORY_LIMIT=-1 /usr/bin/php83 /usr/local/bin/composer28 -vvv update --no-ansi &> ../temp/composer.log

I am prototyping with a very basic install.

I just tried this using the steps in the issue summary, and immediately got this error during composer update:

  - Downloading drupal/core-project-message (11.x-dev 656efa0)
    Failed to download drupal/core-project-message from dist: Maximum allowed download size reached. Content-length header indicates 10666 bytes. Allowed 10658 bytes
    Now trying to download from source


.... [ binary output ] ....


In CurlDownloader.php line 374:
                                                              
  [TypeError]                                                 
  rewind(): supplied resource is not a valid stream resource  
                                                             

Then I manually remove drupal/core-project-message from composer.json, and tried composer update again, got this:

 1/7 [====>-----------------------]  14%    Failed to download drupal/core from dist: Maximum allowed download size reached. Content-length header indicates 26347482 bytes. Allowed 25427652 bytes
    Now trying to download from source

Are the steps in the issue summary outdated, or this is a regression somewhere?

With a completely fresh install via composer create-project I was able to get the composer update to run. Maybe it's because I was on 11.x-dev?

Once I was able to run composer update, I did some profiling with xhprof, and opened https://github.com/php-tuf/php-tuf/issues/385 with initial findings.

MR up at https://github.com/php-tuf/php-tuf/pull/386 which appears to be a 50% improvement for both CPU and memory.

  - Downloading drupal/core-project-message (11.x-dev 656efa0)
    Failed to download drupal/core-project-message from dist: Maximum allowed download size reached. Content-length header indicates 10666 bytes. Allowed 10658 bytes

This is a server-side issue. https://packagist-signed.drupalcode.org/metadata/bin_094-097.json says

   "drupal/core-project-message/11.9999999.9999999.9999999-dev": {
    "hashes": {
     "sha256": "78163df67a5207b9b0ba0ede7ff2652396f5c66ee9d6ce5f137db24d4242bd6e"
    },
    "length": 10657
   },

But https://packagist-signed.drupalcode.org/dist/drupal/core-project-message... is a different file. It was last updated Feb 3, which is around when I was shoring things up, so it's hard to say if it was a systematic issue or debris from something that was fixed. I'll look at getting that re-signed.

TUF is signing a target named drupal/core-project-message/11.9999999.9999999.9999999-dev and presumably it has a previous version of that file. For packagist-signed, dev zipballs do change location with every commit, the commit hash is in the filename. I wonder if there’s some way to use this for better error messaging or working around issues.

Second MR:
https://github.com/php-tuf/php-tuf/pull/387

If I combine the two of these, memory usage in xhprof is more or less equivalent to vanilla composer.

  - Downloading drupal/core-project-message (11.x-dev 656efa0)
    Failed to download drupal/core-project-message from dist: Maximum allowed download size reached. Content-length header indicates 10666 bytes. Allowed 10658 bytes

This is now fixed. I had put in logic that only signed each zip file once. The dev version zip archives have the Git commit ID in their name and change when a new commit is added. However, the target name drupal/core-project-message/11.9999999.9999999.9999999-dev isn’t changing and what is actually checked. The fix was https://gitlab.com/drupal-infrastructure/package-signing/packagist-signe... and https://gitlab.com/drupal-infrastructure/package-signing/packagist-signe...

Ok I went through this with a mature Drupal project on 10.4.6

Highlights:
ddev composer show | grep drupal/ | wc -l
Authenticity of packages from https://asset-packagist.org are not verified by TUF.
Authenticity of packages from https://repo.packagist.org are not verified by TUF.
68

du -sh vendor/composer/tuf
6.5M vendor/composer/tuf

High level impression, it took forever, it was almost 30 minutes I think for the update step and there was so much output it went past scrollback.
> pre-file-download: Tuf\ComposerIntegration\Plugin->preFileDownload

Steps taken:
ddev composer config allow-plugins.php-tuf/composer-integration true
No output
ddev composer require php-tuf/composer-integration:dev-main --dev
Abridged output:

./composer.json has been updated
Running composer update php-tuf/composer-integration
Gathering patches for root package.
Loading composer repositories with package information
Updating dependencies
Lock file operations: 4 installs, 0 updates, 0 removals
  - Locking paragonie/random_compat (v9.99.100)
  - Locking paragonie/sodium_compat (v1.21.1)
  - Locking php-tuf/composer-integration (dev-main 05a6d3e)
  - Locking php-tuf/php-tuf (0.1.6)
Writing lock file

And

Authenticity of packages from https://packages.drupal.org/8 are not verified by TUF.
Authenticity of packages from https://asset-packagist.org are not verified by TUF.
Authenticity of packages from https://repo.packagist.org are not verified by TUF.

Next I ran the mkdir and curl commands, nothing remarkable

ddev composer tuf:protect https://packages.drupal.org/8
Authenticity of packages from https://packages.drupal.org/8 are not verified by TUF.
Authenticity of packages from https://asset-packagist.org are not verified by TUF.
Authenticity of packages from https://repo.packagist.org are not verified by TUF.
https://packages.drupal.org/8 is now protected by TUF.

ddev composer config repositories.drupal-core composer https://packagist-signed.drupalcode.org
Authenticity of packages from https://asset-packagist.org are not verified by TUF.
Authenticity of packages from https://repo.packagist.org are not verified by TUF.

ddev composer tuf:protect https://packagist-signed.drupalcode.org
Authenticity of packages from https://packagist-signed.drupalcode.org are not verified by TUF.
Authenticity of packages from https://asset-packagist.org are not verified by TUF.
Authenticity of packages from https://repo.packagist.org are not verified by TUF.
https://packagist-signed.drupalcode.org is now protected by TUF.

Packages: 114

When I ran composer -vv update I saw

PHP Fatal error:  Allowed memory size of 1610612736 bytes exhausted (tried to allocate 4096 bytes) in /vendor/php-tuf/php-tuf/src/CanonicalJsonTrait.php on line 46

Check https://getcomposer.org/doc/articles/troubleshooting.md#memory-limit-errors for more info on how to handle out of memory errors.%

There have been significant memory usage improvements which need review linked from https://github.com/php-tuf/composer-integration/issues/127. Those could help the runtime as well.