Automated report on core config validatability

This is what it looked like a few hours ago.

That only worked going back to August, for July I had to do some cache clearing, and for Jan 1, I had to do crazy composer work-arounds 😬

But it’s working! Still chasing one more bug…

Turns out that this is a reproducible bug in Drush 12:

$ vendor/bin/drush updatedb --yes
 -------- ----------- --------------- ---------------------------------------- 
  Module   Update ID   Type            Description                             
 -------- ----------- --------------- ---------------------------------------- 
  system   10101       hook_update_n   10101 - Change the {batch} table [bid]  
                                       field to serial.                        
 -------- ----------- --------------- ---------------------------------------- 


 // Do you wish to run the specified pending updates?: yes.                                                             

>  [error]  TypeError: ArrayObject::__construct(): Argument #1 ($array) must be of type array, bool given in ArrayObject->__construct() (line 15 of /Users/wim.leers/core/vendor/consolidation/output-formatters/src/StructuredData/AbstractListData.php) #0 /Users/wim.leers/core/vendor/consolidation/output-formatters/src/StructuredData/AbstractListData.php(15): ArrayObject->__construct(false)
> #1 /Users/wim.leers/core/vendor/consolidation/output-formatters/src/StructuredData/UnstructuredListData.php(21): Consolidation\OutputFormatters\StructuredData\AbstractListData->__construct(false)
> #2 /Users/wim.leers/core/vendor/drush/drush/src/Commands/core/UpdateDBCommands.php(137): Consolidation\OutputFormatters\StructuredData\UnstructuredListData->__construct(false)
> #3 [internal function]: Drush\Commands\core\UpdateDBCommands->process('10', Array)
> #4 /Users/wim.leers/core/vendor/consolidation/annotated-command/src/CommandProcessor.php(276): call_user_func_array(Array, Array)
> #5 /Users/wim.leers/core/vendor/consolidation/annotated-command/src/CommandProcessor.php(212): Consolidation\AnnotatedCommand\CommandProcessor->runCommandCallback(Array, Object(Consolidation\AnnotatedCommand\CommandData))
> #6 /Users/wim.leers/core/vendor/consolidation/annotated-command/src/CommandProcessor.php(176): Consolidation\AnnotatedCommand\CommandProcessor->validateRunAndAlter(Array, Array, Object(Consolidation\AnnotatedCommand\CommandData))
> #7 /Users/wim.leers/core/vendor/consolidation/annotated-command/src/AnnotatedCommand.php(391): Consolidation\AnnotatedCommand\CommandProcessor->process(Object(Symfony\Component\Console\Output\ConsoleOutput), Array, Array, Object(Consolidation\AnnotatedCommand\CommandData))
> #8 /Users/wim.leers/core/vendor/symfony/console/Command/Command.php(312): Consolidation\AnnotatedCommand\AnnotatedCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
> #9 /Users/wim.leers/core/vendor/symfony/console/Application.php(1040): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
> #10 /Users/wim.leers/core/vendor/symfony/console/Application.php(314): Symfony\Component\Console\Application->doRunCommand(Object(Consolidation\AnnotatedCommand\AnnotatedCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
> #11 /Users/wim.leers/core/vendor/symfony/console/Application.php(168): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
> #12 /Users/wim.leers/core/vendor/drush/drush/src/Runtime/Runtime.php(110): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
> #13 /Users/wim.leers/core/vendor/drush/drush/src/Runtime/Runtime.php(40): Drush\Runtime\Runtime->doRun(Array, Object(Symfony\Component\Console\Output\ConsoleOutput))
> #14 /Users/wim.leers/core/vendor/drush/drush/drush.php(139): Drush\Runtime\Runtime->run(Array)
> #15 /Users/wim.leers/core/vendor/drush/drush/drush(4): require('/Users/wim.leer...')
> #16 /Users/wim.leers/core/vendor/bin/drush(119): include('/Users/wim.leer...')
> #17 {main}. 
> TypeError: ArrayObject::__construct(): Argument #1 ($array) must be of type array, bool given in /Users/wim.leers/core/vendor/consolidation/output-formatters/src/StructuredData/AbstractListData.php on line 15 #0 /Users/wim.leers/core/vendor/consolidation/output-formatters/src/StructuredData/AbstractListData.php(15): ArrayObject->__construct(false)
> #1 /Users/wim.leers/core/vendor/consolidation/output-formatters/src/StructuredData/UnstructuredListData.php(21): Consolidation\OutputFormatters\StructuredData\AbstractListData->__construct(false)
> #2 /Users/wim.leers/core/vendor/drush/drush/src/Commands/core/UpdateDBCommands.php(137): Consolidation\OutputFormatters\StructuredData\UnstructuredListData->__construct(false)
> #3 [internal function]: Drush\Commands\core\UpdateDBCommands->process('10', Array)
> #4 /Users/wim.leers/core/vendor/consolidation/annotated-command/src/CommandProcessor.php(276): call_user_func_array(Array, Array)
> #5 /Users/wim.leers/core/vendor/consolidation/annotated-command/src/CommandProcessor.php(212): Consolidation\AnnotatedCommand\CommandProcessor->runCommandCallback(Array, Object(Consolidation\AnnotatedCommand\CommandData))
> #6 /Users/wim.leers/core/vendor/consolidation/annotated-command/src/CommandProcessor.php(176): Consolidation\AnnotatedCommand\CommandProcessor->validateRunAndAlter(Array, Array, Object(Consolidation\AnnotatedCommand\CommandData))
> #7 /Users/wim.leers/core/vendor/consolidation/annotated-command/src/AnnotatedCommand.php(391): Consolidation\AnnotatedCommand\CommandProcessor->process(Object(Symfony\Component\Console\Output\ConsoleOutput), Array, Array, Object(Consolidation\AnnotatedCommand\CommandData))
> #8 /Users/wim.leers/core/vendor/symfony/console/Command/Command.php(312): Consolidation\AnnotatedCommand\AnnotatedCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
> #9 /Users/wim.leers/core/vendor/symfony/console/Application.php(1040): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
> #10 /Users/wim.leers/core/vendor/symfony/console/Application.php(314): Symfony\Component\Console\Application->doRunCommand(Object(Consolidation\AnnotatedCommand\AnnotatedCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
> #11 /Users/wim.leers/core/vendor/symfony/console/Application.php(168): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
> #12 /Users/wim.leers/core/vendor/drush/drush/src/Runtime/Runtime.php(110): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
> #13 /Users/wim.leers/core/vendor/drush/drush/src/Runtime/Runtime.php(40): Drush\Runtime\Runtime->doRun(Array, Object(Symfony\Component\Console\Output\ConsoleOutput))
> #14 /Users/wim.leers/core/vendor/drush/drush/drush.php(139): Drush\Runtime\Runtime->run(Array)
> #15 /Users/wim.leers/core/vendor/drush/drush/drush(4): require('/Users/wim.leer...')
> #16 /Users/wim.leers/core/vendor/bin/drush(119): include('/Users/wim.leer...')
> #17 {main}
>  [warning] Drush command terminated abnormally.

… happens 100% of the time when running drush updatedb --yes against core commit fca42b9d59da506d9c9aac1c3be018dc04205d67.

Root cause appears to be:

    public function process(string $batch_id, $options = ['format' => 'json']): UnstructuredListData
    {
        $result = drush_batch_command($batch_id);
        return new UnstructuredListData($result);
    }

… where $result is FALSE, UnstructuredListData's processing chokes on it.

Perhaps because drush_batch_command() is itself using the batch system? 🤔

For now, ignoring this, because no precise data for a few commits is not a big deal. The trend should still be clear, and my script SHOULD be robust enough to handle this.

To debug that, I added this to \Consolidation\SiteProcess\ProcessBase::getOutputAsJson():

$msg .= "\n" . print_r(debug_backtrace(DEBUG_BACKTRACE_IGNORE_ARGS, 20), TRUE). "\n";

(just before throwing the exception).

This is what that should look like:

🤓

You are adding a gitlab CI yml here :) It would also be important to port the regular test to gitlab CI. I think it would be easier to rely on the standard templates like I did in upgrade_rector recently for easier contributor involvement, rather than making it custom. For the custom job I agree custom config is needed :) Do you think the GitLab CI config added here could still accommodate porting the drupalci tests over without much rework?

FYI this is the matrix I have there, although with Drupal 9 going EOL in a few weeks, we may not need to test that anymore? https://git.drupalcode.org/project/upgrade_rector/-/blob/2.x/.gitlab-ci....

Porting the existing test suite to GitLab CI is an independent task/concern. They can keep running just fine the way they are for now. I'm sure I'll get to that some time in the next few months though!

GitLab CI adoption by this project now is a fact: 📌 Adopt GitLab CI Fixed .

I'll need to modify this MR.

Wim Leers → credited diqidoq → .

Per @diqidoq in Slack:

#3 seem to be gone and faded away like in miracles after updating Drupal core to 10.2 beta ...

— https://drupal.slack.com/archives/C1BMUQ9U6/p1701264422604679

Picking this back up where I left it off. I want to make the data much easier to consume too, and thanks to #3384688: Automatically publish a GitLab pages site → (and https://docs.gitlab.com/ee/user/project/pages/getting_started/pages_from...) that should become feasible 🤓

Merged in upstream, which means this now no longer conflicts with 📌 Adopt GitLab CI Fixed , but extends it :)

Well 🤣

…
xhprof
xhprof support => enabled
…

 [success] Successfully enabled: config_inspector
real	0m 3.97s
user	0m 2.41s

4 seconds for installing a single module. Yeah. That's gonna make things mind-bogglingly slow 😬

Now down to ~8 seconds/commit. With a 1-hour time limit and a few minutes for setup, that should mean roughly 55 minutes gives us 55*60/8 = ~410 days that can be analyzed. That's good enough for now! (It took 49 minutes to process the 355 days of 2023, including all overhead.)

And … the countless "permission denied" problems are now also gone! 👍

That means that next week, I'll be able to continue this for real 😄

Just looked at the latest output, super amazed that we are already at +50%. This is already very cool to see this output. Looking forward to see this in graphical form.

There are a lot of type: labels in Drupal core 🤣

Note that this does not mean 53% of all core config is validatable. Only 53% of core config property paths is validatable. We'll eventually need to clarify that, but right now the key goal is to get some visualization going on…

It is MUCH faster now! 14 minutes for analyzing all of 2023.

Next up: using R to generate charts.

This now is generated automatically:

🥳

A specific job called pages in the configuration file makes GitLab aware that you’re deploying a GitLab Pages website.

— https://docs.gitlab.com/ee/user/project/pages/getting_started_part_one.h...

Now starting 1 month earlier (Dec 1, 2022) and actually publishing to https://project.pages.drupalcode.org/config_inspector/ 👍

Automatically closed - issue fixed for 2 weeks with no activity.