[SPIKE] Prove that it's possible to apply block settings update paths to stored XB component trees

Can we use 📌 Introduce unit test coverage for both ComponentSource plugins (Block + SDC) Active to use xb_test_block as blocks to update?

Plan articulated → at 🌱 [META] Production-ready ComponentSource plugins Active .

1. We need to write an update path + update path test that does the exact same thing for a search block stored in an XB component tree.

Are we thinking here that we need block implementations to provide separate code paths to update XB blocks? If so, what happens if a contrib author decides not to do that? Or, who takes responsibility for core block updates, if core does not depend on XB?

On the other hand, I am not sure if this will work, but I wonder if we can do this without contrib maintainers needing to do anything at all! Block updates are generally written like this:

function search_post_update_block_with_empty_page_id(&$sandbox = []) {
  $config_entity_updater = \Drupal::classResolver(ConfigEntityUpdater::class);
  $config_entity_updater->update($sandbox, 'block', function (BlockInterface $block): bool {
...

Can we decorate ConfigEntityUpdater so as well as updating config entities, we additionally reconstruct config entities for blocks in XB content, and allow the closure to make changes to them in the same way? The closure is not responsible for saving the config entity - so because these won't be real config entities, we never need to directly save them; we can just extract the settings (or whatever else might have changed?) and put them back into the XB storage.

Will discuss #8 + #9 tomorrow with @longwave :)

I think that this is an existing core bug, because we should already be doing the same thing for layout builder and navigation modules, but we don't.

Fixing it requires adding a new API to core, don't see another way, I opened 🐛 Block plugins need to be able to update their settings in multiple different storage implementations Active which doesn't have a nice API but I think probably does have the places that API would need to be implemented and roughly what it would need to do.

I'm not postponing this issue on that one yet, but I think it probably should be.

The sentence that got me to that point was this one from the parent issue:

However … it can't actually work because we won't know which of those post-update hooks are associated with which block plugins! In theory, we could run all those block-targeting post-update hooks, but then we'd need to track for every single block-sourced component instance which update hooks have already been executed (equivalent to how core does the system-wide UpdateRegistry). That doesn't scale, just like it doesn't scale to update potentially millions of (revision) rows.

This is the missing piece that the new BlockPluginUpdater API would address.

Given block updates would be writing config, can we instead listen to \Drupal\Core\Config\ConfigEvents::SAVE and detect if it occurs during an update/post update (assuming that's possible) and if so propagate those changes to component trees we can identify as referencing blocks via 📌 Calculate field and component dependencies on save and store them in an easy to retrieve format Active

I don't think it's possible to reliably detect whether the save happens during an update/post-update except perhaps via a backtrace and function name guessing (e.g. does post_update appear in the stack trace).

Assuming that could be detected though, it would need to compare the before and after of the block entity, determine what the change is, and then apply that to the update - it wouldn't be possible to run the same code that the update itself does, only to reverse-engineer the change from the config entity original.

Some block post updates won't result in any config saves at all, because there's no affected blocks on the site, but the plugins being targeted could be used in XB templates - then there's nothing to listen to. The search block would be an example where it might be in an XB template but not used in a block region.

Additionally, this approach wouldn't be able to take into account xb templates that are shipped as config with modules or recipes, which equally need to apply the same changes so that they're correct after import/recipe apply.

Thanks to 📌 Calculate field and component dependencies on save and store them in an easy to retrieve format Active + 📌 [PP-1] Evaluate storing XB field type's "deps_*" columns in separate table Active , it's now trivial to find all the content entity revisions in which a certain block plugin is used.

But what's still missing is fundamental core support for generically updating the inputs expected by plugins that may be used in multiple contexts, and not just e.g. Block config entities. The core issue for that is 🐛 Block plugins need to be able to update their settings in multiple different storage implementations Active .

Thanks, @catch, for opening that, and stating it as starkly as you did: — I don't either. 😬

Self-assigned in #10, forgot to link here to the write-up I posted at #3520484-22: [META] Production-ready ComponentSource plugins → as promised. Unassigning.

Once 📌 [PP-1] Consider not storing the ComponentTreeStructure data type as a JSON blob Postponed is in, I think we should be able to create a working PoC here that does the following in a test:

1. Create a new input_schema_change_poc block plugin in the xb_test_block module which has e.g. ['foo' => 'bar'] as its default configuration, and has this as its config schema:
   

   block.settings.input_schema_change_poc:
     type: block_settings
     mapping:
       foo:
         type: string
         constraints:
           Choice: [bar, baz]
     constraints:
       FullyValidatable: ~
   

2. Creates a component tree using that
3. Installs a xb_test_block_simulate_input_schema_change module that:
   • overrides which class is used for that block plugin, and changes the default configuration to ['foo' => 'bar', 'change' => 'is scary'], plus changes its render logic in ::build() so we can easily assert the difference
   • alters the config schema to be:
     

     block.settings.input_schema_change_poc:
       type: block_settings
       mapping:
         foo:
           type: string
           constraints:
             Choice: [bar, baz]
         change:
           type: string
           constraints:
             Choice: ['is scary', 'is necessary']
       constraints:
         FullyValidatable: ~
     

4. Make the test verify that the existing component instance renders the message provided by RenderSafeComponentContainer
5. Then write manual logic because 🐛 Block plugins need to be able to update their settings in multiple different storage implementations Active does not exist yet that:
   • uses \Drupal\experience_builder\Audit\ComponentAudit to find all instances, in both config and content
   • update all instances of block.xb_test_block_simulate_input_schema_change in all component trees across both content and config
   • verify all those entities are again/still valid
   • verify that all those component trees render correctly again 👍

👆 That would allow us to be as confident as we can that we'll be able to do 📌 Component Source plugins: support for schema changes of explicit inputs Postponed in the refactored server-side data model 👍

Just did a video call with @isholgueras to unblock him on next steps and clarify the unclear bits in my outline at #15 (🙈 Sorry!).

Updated issue title + summary to convey the intended scope.

Explain relation to SDC and code components.

Fix HTML 🙈

Reviewed and … nice progress here! 👍😄

Steps 4 and 5 in the plan I articulated in #15 are not yet done.

Step 4 is not yet done due to a bug in the test coverage, once that's fixed, that'll force #4 😇

FYI: 📌 Version component prop definitions for SDC and Code components Active just landed, and ideally the test coverage for #15.5 would live in ComponentInputsEvolutionTest. That already contains testStorablePropShapeChanges(), this MR would add testInputSchemaChanges(). 🙏

Late to the party but #15 sounds good to me.

@larowlan in #26: thanks, much appreciated! 🙏

I kicked off a deep review and pushed a few small fixes/improvements which revealed test weaknesses, @f.mazeikis will continue 👍

P.S.: The 3 new test methods added to BlockComponentTest should be moved as I described in #24. 🙏 This observation underlines that.

f.mazeikis → made their first commit to this issue’s fork.

Between @isholgueras addressing some of your feedback and me moving tests out of BlockComponentTest into ComponentInputsEvolutionTest I think this is ready for another review.
I have also taken liberty to rename test methods and expand their doc blocks, in attempts to make this PoC easier to read.

Per @larowlan's review remarks 1 and 2, let's land 📌 Ensure predictable config export order of config-defined component trees Active first.

📌 Ensure predictable config export order of config-defined component trees Active is in :)

Actually … I'm pretty sure that @larowlan meant to link to 🐛 Deterministic Component version hash should depend not only on source-specific settings, but also slots + explicit input schema Active instead of 📌 Ensure predictable config export order of config-defined component trees Active 😅

I found some pretty serious flaws in the test coverage: 1, 2, 3, 4 and most importantly: this was indeed asserting various edge cases/expectations wrt updates to components in the block ComponentSource, but was not yet testing the very purpose of this issue as described in #15.5:proving that a future automated update path will be possible once #3521221 in core adds the necessary infra.

To be fair: this issue/MR is very abstract and is about ensuring something will be possible, which makes it much harder to test.

Will push this over the finish line. Already addressed ~half of my concerns, other half to follow.

See the remaining open MR threads — they provide pointers and context for the most crucial test coverage changes/enhancements.

The plan I outlined in #15 is now fully implemented.

Next steps are:

1. upstream infrastructure: 🐛 Block plugins need to be able to update their settings in multiple different storage implementations Active
2. expand the ComponentAudit service in 📌 Component Source plugins: support for schema changes of explicit inputs Postponed
3. add component revision support to the ComponentAudit service in 📌 Component Source plugins: support for schema changes of explicit inputs Postponed and/or 📌 Revisit storage of dependencies in separate table now we have separate deltas per component Active

This apparently introduced 2 test failures on all DBs except SQLite: https://git.drupalcode.org/project/experience_builder/-/pipelines/519184 — JsComponentTest::testFallback() and SingleDirectoryComponentTest::testFallback().

This makes it pass again:

diff --git a/src/Plugin/Field/FieldType/ComponentTreeItem.php b/src/Plugin/Field/FieldType/ComponentTreeItem.php
index 93df89230..547a5b3b0 100644
--- a/src/Plugin/Field/FieldType/ComponentTreeItem.php
+++ b/src/Plugin/Field/FieldType/ComponentTreeItem.php
@@ -393,7 +393,6 @@ class ComponentTreeItem extends FieldItemBase {
     }
     $pairs = [
       ['component_id', 'component'],
-      ['component_version', 'component'],
       ['parent_uuid', 'parent_item'],
     ];
     foreach ($pairs as $pair) {

… but that's a necessary change. I bet @larowlan will figure this out in no time 🤓

New MR https://git.drupalcode.org/project/experience_builder/-/merge_requests/1125 has the fix.

The SQL fails were 'data too long' which (see the screenshot above) was because when component was updated it was forcing the component version to match.

The intent of the change added in this issue was to permit setting a new component version and having ::getComponent load the correct new version.

I've rewritten it to work as intended and it fixes the tests. Luckily we put a 16 char limit on that column else we wouldn't have caught this.

Clearly I misunderstood how “pairs” worked 😅

Thanks!

Automatically closed - issue fixed for 2 weeks with no activity.