Record total css and js file size in performance tests

You would think this would be easy, but it's not that easy. First of all the performance log doesn't provide the filesize in the log details, only the number of bytes returned from the response, if the file is cached by the browser, this can be the result of a HEAD request for 304 not modified etc.

Then I switched to trying the actual contents of the files, and got different results between gitlab and local. Started building conspiracy theories about filesystems then realised that they actually are different because we replace the base_path in CSS aggregates.

After going through all that, I'm now getting green results both locally (give or take chromedriver crashes which are unrelated) and gitlab.

Another case to add a test for: 📌 Use placeholdering for more elements to optimize asset serving Needs work

Parsing the URLs from the response isn't ideal here, because it won't work for 📌 Use placeholdering for more elements to optimize asset serving Needs work which needs a request for two pages. I was originally getting them from the chrome network log, so will try to go back to that.

Main MR is green.

Now three draft on this MRs, showing how the test coverage here identifies improvements in related issues. If this gets committed we'll need to adjust the test coverage on those issues to show the improvement, if one of them gets committed first, will need to make the same changes here.

What's a good way to test this one? Or should I just review all the MRs?

Can't really test it (well you could run the tests locally, but I've been doing that working on it, so it's not necessary), but looking at the draft MRs and noting the differences in the assertion counts due to the various changes, shows it's doing what it's supposed to.

Thanks @catch could the issue summary be updated to show what MR is the intended merge one and purpose/what is being shown in the others.

@smustgrave the issue summary already says:

This issue also includes three draft MRs that combine the test coverage here with different issues to improve aggregation performance or re-organise libraries, it successfully picks up the improvement in each of those issues.

The three draft MRs are marked as drafts, the MR for commit is not marked as draft. Do you have concrete improvements to this you can suggest?

Added this to the issue summary just to make it more clear

MR to be merged 7213

MR 7312 = contains 📌 Use placeholdering for more elements to optimize asset serving Needs work and shows stylebytes go down by almost 2000
MR 7313 = contains 📌 Only send libraries with aggregate URLs that have the aggregate type included Needs review and shows no change that I can see, guess that can be discussed in that ticket
MR 7231 = contains 📌 Move system/base component CSS to respective libraries where they exist Fixed and show stylebytes go down in a few spots.

Marking this RTBC and other tickets can have more discussion.

Thanks that is indeed clearer!

MR 7313 = contains #3437839: Only send libraries with aggregate URLs that have the aggregate type included and shows no change that I can see, guess that can be discussed in that ticket

There's no difference for CSS, but a lot for JavaScript, if you compare https://git.drupalcode.org/project/drupal/-/merge_requests/7213/diffs and https://git.drupalcode.org/project/drupal/-/merge_requests/7313/diffs side by side just look at the script bytes assertions. e.g. testFrontAndRecipesPagesAuthenticated() you can see 264076 reduced to 132038.

alexpott → changed the visibility of the branch 3436527-with-3437499 to hidden.

alexpott → changed the visibility of the branch 3436527-with-3437839 to hidden.

alexpott → changed the visibility of the branch 3436527-with-system-base to hidden.

Committed and pushed b9be0d83b4 to 11.x and 3b7800ea54 to 10.3.x. Thanks!

Seeing consistent failures in HEAD, committing this as a hotfix on the basis it might be a conflict with another MR.

diff --git a/core/profiles/demo_umami/tests/src/FunctionalJavascript/OpenTelemetryAuthenticatedPerformanceTest.php b/core/profiles/demo_umami/tests/src/FunctionalJavascript/OpenTelemetryAuthenticatedPerformanceTest.php
index 8dfbb67747..9b4686ebef 100644
--- a/core/profiles/demo_umami/tests/src/FunctionalJavascript/OpenTelemetryAuthenticatedPerformanceTest.php
+++ b/core/profiles/demo_umami/tests/src/FunctionalJavascript/OpenTelemetryAuthenticatedPerformanceTest.php
@@ -38,7 +38,7 @@ public function testFrontPageAuthenticatedWarmCache(): void {
     $this->assertSame(2, $performance_data->getStylesheetCount());
     $this->assertSame(47552, $performance_data->getStylesheetBytes());
     $this->assertSame(1, $performance_data->getScriptCount());
-    $this->assertSame(132038, $performance_data->getScriptBytes());
+    $this->assertSame(132083, $performance_data->getScriptBytes());
 
     $expected_queries = [
       'SELECT "session" FROM "sessions" WHERE "sid" = "SESSION_ID" LIMIT 0, 1',

If it turns out to be random and we get fails the other direction, we might need to revert :(

One more:

diff --git a/core/profiles/demo_umami/tests/src/FunctionalJavascript/AssetAggregationAcrossPagesTest.php b/core/profiles/demo_umami/tests/src/FunctionalJavascript/AssetAggregationAcrossPagesTest.php
index 02acb5cfec..1e56dcdd3c 100644
--- a/core/profiles/demo_umami/tests/src/FunctionalJavascript/AssetAggregationAcrossPagesTest.php
+++ b/core/profiles/demo_umami/tests/src/FunctionalJavascript/AssetAggregationAcrossPagesTest.php
@@ -41,7 +41,7 @@ public function testFrontAndRecipesPagesAuthenticated() {
     $this->assertSame(4, $performance_data->getStylesheetCount());
     $this->assertSame(94355, $performance_data->getStylesheetBytes());
     $this->assertSame(2, $performance_data->getScriptCount());
-    $this->assertSame(264076, $performance_data->getScriptBytes());
+    $this->assertSame(264166, $performance_data->getScriptBytes());
   }

Haven't backported these to 10.3.x, let 11.x stabilise first.

Looks like this also needed a 10.3.x backport branch. Made two more hotfixes. If this doesn't resolve things, going to roll back, at least on 10.3.x

diff --git a/core/profiles/demo_umami/tests/src/FunctionalJavascript/AssetAggregationAcrossPagesTest.php b/core/profiles/demo_umami/tests/src/FunctionalJavascript/AssetAggregationAcrossPagesTest.php
index 1e56dcdd3c..836a574a1a 100644
--- a/core/profiles/demo_umami/tests/src/FunctionalJavascript/AssetAggregationAcrossPagesTest.php
+++ b/core/profiles/demo_umami/tests/src/FunctionalJavascript/AssetAggregationAcrossPagesTest.php
@@ -41,7 +41,7 @@ public function testFrontAndRecipesPagesAuthenticated() {
     $this->assertSame(4, $performance_data->getStylesheetCount());
     $this->assertSame(94355, $performance_data->getStylesheetBytes());
     $this->assertSame(2, $performance_data->getScriptCount());
-    $this->assertSame(264166, $performance_data->getScriptBytes());
+    $this->assertSame(264702, $performance_data->getScriptBytes());
   }
 
   /**
diff --git a/core/profiles/demo_umami/tests/src/FunctionalJavascript/OpenTelemetryAuthenticatedPerformanceTest.php b/core/profiles/demo_umami/tests/src/FunctionalJavascript/OpenTelemetryAuthenticatedPerformanceTest.php
index 9b4686ebef..442e11b3e3 100644
--- a/core/profiles/demo_umami/tests/src/FunctionalJavascript/OpenTelemetryAuthenticatedPerformanceTest.php
+++ b/core/profiles/demo_umami/tests/src/FunctionalJavascript/OpenTelemetryAuthenticatedPerformanceTest.php
@@ -38,7 +38,7 @@ public function testFrontPageAuthenticatedWarmCache(): void {
     $this->assertSame(2, $performance_data->getStylesheetCount());
     $this->assertSame(47552, $performance_data->getStylesheetBytes());
     $this->assertSame(1, $performance_data->getScriptCount());
-    $this->assertSame(132083, $performance_data->getScriptBytes());
+    $this->assertSame(132351, $performance_data->getScriptBytes());
 
     $expected_queries = [
       'SELECT "session" FROM "sessions" WHERE "sid" = "SESSION_ID" LIMIT 0, 1',

OK I think we're back to green but will keep an eye on it.

Very nice!

Automatically closed - issue fixed for 2 weeks with no activity.