- Issue created by @wim leers
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Proposed to @longwave to close #3468269: #3468269-23: `ComponentTreeStructure` data type: simplify the stored structure β .
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Moved a significant chunk out of here and into π± [META] Production-ready ComponentSource plugins Active .
- π¬π§United Kingdom catch
There is a @todo in ComponentTreeItem linking to π Cannot delete a field which uses JSON type Active so I think that probably needs to be included here?
- π¬π§United Kingdom catch
Opened π [PP-1] Use JSON schema type for sqlite and remove text workaround Active .
- π¦πΊAustralia larowlan π¦πΊπ.au GMT+10
Additional things I think we need for a 1.0
* plugabble/flexible ComponentTreeLoader so that LB/PG can provide a translation layer for legacy data
* it would be good if default values were stored as an array instead of JSON for the sake of git diff on exported config - same reasons as β¨ Create a schema for "allowed_html" which provides a better config diff Needs work - the current json is going to be very painful for content template git conflicts - π¬π§United Kingdom catch
I think the last point is being tackled in π Stop storing JSON blobs in XB config entities: improve DX Active .
- πͺπΈSpain penyaskito Seville π, Spain πͺπΈ, UTC+2 πͺπΊ
Created
- π [PP-2] Add paging to Component "Audit" operation content listing Postponed
- π [PP-2] Support translations tracking on Component "Audit" operation content listing when using asymmetric translations Postponed
- π [PP-2] Consider translations field usage when uninstalling the module providing a field Postponed
The third one might be a data integrity issue.
- π¬π§United Kingdom longwave UK
Track explicit input schema version:
hash
(the normalized) explicit input schema, assignversion
(sequential integer), storeversion: {hash, provider_version}
mapping in Component config entity.I don't think anything with automated sequential integers is going to work because we have to handle partial deployments of config between sites. Let's say you export a Section from your site that uses a component that is on version 2 of the schema. The SDCs and component config entities might already exist on the destination site, but if it was installed later than the site it was exported from, the same component might only be on version 1 of its schema.
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
π [later phase] Support matching `{type: array, β¦}` prop shapes Postponed is in.
- πͺπΈSpain penyaskito Seville π, Spain πͺπΈ, UTC+2 πͺπΊ
π π Calculate field and component dependencies on save and store them in an easy to retrieve format Active landed!
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Added π Stop storing JSON blobs in XB config entities: improve DX Active .
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
π Stop storing JSON blobs in XB config entities: improve DX Active is in π₯³
- π¦πΊAustralia larowlan π¦πΊπ.au GMT+10
Added a link to π Version component prop definitions for SDC and Code components Active to item 2.1. Wim and I basically came up with the same thing independently - nice!
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
WRT dependency/usage tracking: π Calculate field and component dependencies on save and store them in an easy to retrieve format Active landed, and so did its follow-up π [PP-1] Evaluate storing XB field type's "deps_*" columns in separate table Active . This cleaned up the technical debt around
FieldTypeUninstallValidator
and hence enabled me to close 3 stable blockers, see #3457504-56 for details.
This unblocked:- short term: robustness-related functionality such as π Cannot delete JS components due to component depending on them Active
- long term: future functionality such as β¨ Usage info for code components with unpublished changes Postponed
Only remaining: π [later phase] Support props defaults that have config dependencies Postponed . That is a stable blocker too, but won't require data storage changes. So: I'm assigning that subset of this meta a π’.
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Explain relation to π Allow XB to be used on all node types Active in the issue summary.
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Added @larowlan's π [PP-1] Consider not storing the ComponentTreeStructure data type as a JSON blob Postponed and π [PP-1] Spike: Explore storing a hash lookup of component inputs Postponed .
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Added π Spike: Explore storing component inputs in separate columns (aka field union) Active .
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Removed
debt: we know we'll need to support implicit inputs (e.g. block plugins can declare required + optional "contexts" that allow block plugins behave differently not only to component instance author's explicit wishes ("explicit inputs"), but also based on the component developer's logic ("implicit inputs")
because that doesn't affect the data storage.
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Descoped #3523846 per @catch in #3523846-7: [PP-1] Spike: Explore storing component inputs in separate columns (aka field union) β . We may revisit that after π [PP-1] Consider not storing the ComponentTreeStructure data type as a JSON blob Postponed . π
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Added β¨ [PP-2] Allow users to name components for the specific context Postponed
Clarified π Version component prop definitions for SDC and Code components Active
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Landed π [later phase] Support props defaults that have config dependencies Postponed .
Now going to overhaul this meta issue to clarify what's for beta vs stable β¦
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
π [PP-1] Consider not storing the ComponentTreeStructure data type as a JSON blob Postponed landed!
Next up: π Version component prop definitions for SDC and Code components Active .
Good news! The current issue summary's
Blocked on the above first bringing clarity to the full set of needs first, before starting a potentially enormous refactor
is getting to that point of clarity thanks to @larowlan, @effulgentsia, @catch and I actively having started to push these forward in the past few weeks.
So: time for an overhaul of the issue summary!
Updated the issue summary to more clearly distinguish between:
- beta blockers (i.e. necessary for π± Milestone 1.0.0-beta1: Start creating non-throwaway sites Active )
- stable blockers (i.e. necessary for π± Milestone 1.0.0: Production Sites Postponed )
- post-stable priorities (i.e. important, but after π± Milestone 1.0.0: Production Sites Postponed )
β¦ and within each of those, the areas within "data storage" they target. For beta, that's:
- Dependencies/usage
- Field type storage (content entities) schema
- Configuration storage schema
- Validation
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
- Added π [SPIKE] Prove that it's possible to apply block settings update paths to stored XB component trees Active as a beta blocker β well on its way π
- Added π [PP-1] Use JSON schema type for sqlite and remove text workaround Active as a beta blocker, which is blocked on the π Cannot delete a field which uses JSON type Active core issue. There's a proposal
- Moved #3495625 from stable blocker to post-release priority per #3495625-14: Remove ComponentTreeItemList::ROOT_UUID from hydration and client-to-server conversion β
- Added π [PP-1] Harden UUID validation in ComponentTreeStructureConstraintValidator Postponed as stable blocker β but already completed! π₯³
- Added π Spike: Explore adding configuration options to the tree item formatter to support alternate use-cases Active as stable blocker.
Posted "FYI, captured in the data storage meta" comments on:
- #3523842-16: Spike: Explore storing a hash lookup of the ComponentInputs field property: one hash per component instance β
- #3477428-53: [PP-1] Refactor (or decide not to) the XB field type to be multi-valued, to de-jsonify the tree, and to reference the field_union type of the prop values β
- #3440578-89: [PP-2] JSON-based data storage proposal for component-based page building β
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Per @lauriii: add more features we'll eventually need to support.
And: update the status indicators for all of them.
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Clarify JSON:API support, bump to π’ . Linked (not added!) π ServerClientConversionTrait Active even though it actually belongs under π± [META] Production-ready client-side data model + internal HTTP API Active , because it's how we'll add JSON:API support. This is π’ because after #3468272, the implementation path for it became crystal clear.
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Added π Ensure predictable config export order of config-defined component trees Active .
That doesn't mean I can remove though β that's still true.
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Bumped from π΄ to π‘ based on
#3525746-6: Update the React client preview view β , and based on @penyaskito having proven in π Personalization PoC Active that the single newComponentSource
approach works. That means zero data storage changes are needed!Tempted to mark π’, but would prefer to be cautious instead of overly confident. Will only mark as such once more of it is working.
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Triaged the issue queue component, as well as the
@todo
s in XB's config schema.Added many newly created issues β all of them
stable blocker
s unless marked otherwise:- π [11.2-only] Adopt `AtLeastOneOf` validation constraint for cardinality validation Postponed
- β¨ Stop assuming default Field Widget settings sufficeΒ βΒ add Field Widget settings support to `experience_builder.generated_field_explicit_input_ux: prop_field_definitions` Active
- π Tighten validation of `experience_builder.generated_field_explicit_input_ux: prop_field_definitions` Active
- π Tighten validation of `experience_builder.generated_field_explicit_input_ux: prop_field_definitions.[%key].expression` Active
- π [>=11.1.8] Remove `type: field.value.language` work-around once Postponed
- π [PP-1] Remove BetterConfigExistsContraint and move back to ConfigExistsContraint Active β post-stable priority
- π [later phase] [PP-1] Gracefully handle deleted regions in PageTemplate config entities" Postponed β post-stable priority
- π JavaScriptComponent config entities should have mutable machineNames Active
- π Allow XB to be used on all node types Active
- β¨ [PP-1] Validate that content templates are attached to entity bundles that fulfill XB's requirements Postponed
- β¨ Implement `::onDependencyRemoval()` for `Pattern` and `PageRegion` config entities: when a module providing a field type is uninstalled, replace it with the default StaticPropSource Active β post-stable priority
- π [PP-2] Add ComponentAuditabilityTest Active
- π SdcPropKeysConstraintValidator::validate() should complain about extraneous keys too, not just missing keys Active
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Having reviewed #3519352 and written #3519352-55: Content templates, part 3b: store exposed slot subtrees on individual entities β (specifically the part), I'm bumping from π‘ to π’.
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Per #37, I triaged all config management issues this morning.
Didn't find anything.
So I can move to π₯³ Note that #3526127-5: Ensure predictable config export order of config-defined component trees β might become a beta blocker.
did not have an indicator yet β the sole issue there is well on its way, so marked it π’.
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
#32 failed to update from π’ to β for the cited reasons β .
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
The
untranslatable_inputs
sample diff for post-stable was incomplete. - π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Added π Default content exports are invalid and hence are not correct after importing Active to stable blockers and π Tighten validation of `parent_uuid` and `slot` on XB fields to match the strictness of config Active to beta blockers β thanks @justafish!
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
π Tighten validation of `parent_uuid` and `slot` on XB fields to match the strictness of config Active landed.
- β¨ [PP-2] Allow users to name components for the specific context Postponed was +1'd by @larowlan.
- π Version component prop definitions for SDC and Code components Active massive progress by @larowlan and I both.
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Landed beta blocker β¨ Content templates, part 3b: store exposed slot subtrees on individual entities Active ! π₯³
Added π Spike: explore merits of one XB field per exposed slot in content templates Active as stable blocker.
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
β¨ [PP-2] Allow users to name components for the specific context Postponed landed!
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
π Version component prop definitions for SDC and Code components Active landed!
But two beta-blocking bits were descoped from it:
- Assigned to: larowlan π [PP-1] Ensure deterministic version hashes Active β this one is much narrower in scope :)
- Assigned to: larowlan π Either rename `component_id` field property to `component`, or `version` to `component_version` Active β a bike shed that shouldn't block a ~3K LoC diff π
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Bumping from π‘ to π’ given #46.
Also bumping from π‘ to π’ after discussing with @effulgentsia (he agreed), because A) we've discussed it in depth over the past ~2 weeks, B) @penyaskito has a working PoC of this at π Personalization PoC Active , which he'll be updating tomorrow (since a few blockers have landed for him).
That means everything now is either π’ or β !
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
The two new beta blockers that spun out from #3523841 last night have already landed: π Either rename `component_id` field property to `component`, or `version` to `component_version` Active + π [PP-1] Ensure deterministic version hashes Active β thanks to @larowlan!
Added π Add e2e tests that prove we can edit an old version of a component Active as a stable blocker (again: thanks Lee).
Finally: swapping the overall/catch-all π± [META] Experience Builder Personalization Active for the more tightly scoped π Personalization component source Active as proof for full confidence in this future feature:
- feature: ability to store variants of component trees for use cases like personalization (i.e. different component subtree for Belgians & Brits vs everyone else), responsive design (i.e. breakpoint-specific overrides), and arbitrary future use cases
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Added π Deterministic Component version hash should depend not only on source-specific settings, but also slots + explicit input schema Active as a beta blocker.
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
π Constraint slot names allowed by XB in its component tree Active landed.
Per discussion with @larowlan, moved π Ensure predictable config export order of config-defined component trees Active from stable to beta blocking.
- π¬π§United Kingdom catch
π Calculate field and component dependencies on save and store them in an easy to retrieve format Active and π [PP-1] Evaluate storing XB field type's "deps_*" columns in separate table Active were done before π [PP-1] Consider not storing the ComponentTreeStructure data type as a JSON blob Postponed and π Version component prop definitions for SDC and Code components Active .
I think this means that a lot of (maybe all?) the data that was denormalized into a special database table can now be retrieved by querying the field tables (in combination with component config entity dependencies). Is there a still-open issue where this is being discussed? I wasn't able to find one.
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Added π Component instances in exposed `ContentTemplate` slots must use the exposed slot's machine name Active as stable blocker.
- π¦πΊAustralia larowlan π¦πΊπ.au GMT+10
Added π Revisit storage of dependencies in separate table now we have separate deltas per component Active for #51 @catch
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
π Deterministic Component version hash should depend not only on source-specific settings, but also slots + explicit input schema Active and π Ensure predictable config export order of config-defined component trees Active landed, bringing remaining back down to zero!
In , π [SPIKE] Prove that it's possible to apply block settings update paths to stored XB component trees Active landed, and per #3520923-18: [PP-1] Use `json` schema type for SQLite and remove `text` workaround β , moved π [PP-1] Use JSON schema type for sqlite and remove text workaround Active to the bottom of that section.
Next (and IMHO last) most important issue: π Revisit storage of dependencies in separate table now we have separate deltas per component Active .
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Landed the stable-not-beta-blocking π SdcPropKeysConstraintValidator::validate() should complain about extraneous keys too, not just missing keys Active since @larowlan commented on it: #3525759-9: SdcPropKeysConstraintValidator::validate() should complain about extraneous keys too, not just missing keys β .
Re-checked the list of open issues tagged β and found one not listed here: π \Drupal\experience_builder\Plugin\DataType\ComponentInputs::getPropSources needs to take \Drupal\experience_builder\Plugin\ExperienceBuilder\ComponentSource\GeneratedFieldExplicitInputUxComponentSourceBase::rawInputValueToPropSourceArray into account Active .
Added π Component::onDependencyRemoval() should be an uninstall validator Active as a stable blocker.
(FYI: for beta blockers, the bulk of the attention is going to π Revisit storage of dependencies in separate table now we have separate deltas per component Active .)
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
π \Drupal\experience_builder\Plugin\DataType\ComponentInputs::getPropSources needs to take \Drupal\experience_builder\Plugin\ExperienceBuilder\ComponentSource\GeneratedFieldExplicitInputUxComponentSourceBase::rawInputValueToPropSourceArray into account Active is in, and the core blocker for π [PP-1] Use JSON schema type for sqlite and remove text workaround Active landed upstream! π₯³
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Landed π Component::onDependencyRemoval() should be an uninstall validator Active (not here) which ended up (unexpectedly for me) blocking π Revisit storage of dependencies in separate table now we have separate deltas per component Active .
That means the sole remaining data storage beta blocker is π [PP-1] Use JSON schema type for sqlite and remove text workaround Active , but the upstream core issue is in ( π Cannot delete a field which uses JSON type Active ), so now itβs a trivial update on our end once a Drupal core release ships β which should be trivial to do in the coming ~1.5 month.
New stable blockers:
- π [later phase] Make Component `audit` operation scalable Postponed (long known, just was missing from this meta)
- π Update `ComponentAudit::getConfigEntityDependenciesUsingComponent()` to support component versions Active (follow-up for π Revisit storage of dependencies in separate table now we have separate deltas per component Active
New post-stable priority:
- πΊπΈUnited States effulgentsia
wim leers β credited effulgentsia β .
- πΊπΈUnited States tedbow Ithaca, NY, USA
wim leers β credited tedbow β .
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
As described at #3520923-29: [11.2.0-and-up] Use `json` schema type for SQLite and remove `text` workaround β , that can't happen until
11.2.0
is tagged. It's a trivial MR to land though :)So: THIS IS DONE! π₯³π₯³π₯³ Special thanks to:
- @larowlan, for the constant stream of reviews & MRs
- @catch, for the constant feedback (from a core committer!)
- @tedbow, for the translation test coverage + field type uninstall validator + component tree validation logic that was iterated upon but still lives on, these were all very valuable guardrails!
- @penyaskito, for helping get the dependency calculation logic in place, which in all of the recent iterations by Lee and I had proved crucial!
- @effulgentsia, for the original idea by and his chiming in when really necessary over the past few weeks π
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Signaling this is "done for now", i.e. for π± Milestone 1.0.0-beta1: Start creating non-throwaway sites Active but NOT for π± Milestone 1.0.0: Production Sites Postponed , by marking this .
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
Retroactively adding things that actually did happen for beta, but before this meta issue existed or was sufficiently groomed.
- π¦πΊAustralia larowlan π¦πΊπ.au GMT+10
Ideally it would be good to do a spike on a BC loading layer for Paragraphs/Layout builder to triple check we've got a clean run at that in the future with the current data model
- π§πͺBelgium wim leers Ghent π§πͺπͺπΊ
That would be lovely indeed. But I don't think that's a hard blocker for beta. Let's first make sure we actually achieve π± Milestone 1.0.0-beta1: Start creating non-throwaway sites Active , and then let's do this. I don't see how expressing anything that can be built in Layout Builder/Paragraphs/β¦ (see π± [META] Support component types other than SDC Needs work ) could not be expressed in XB, because XB has more freedom/is much less prescriptive?
Also β¦ didn't we discuss (and in fact you proposed it! π) to only convert existing data into the client-side data model, so you can load an existing component-tree-esque thing (Layout Builder, Paragraphs, whatever) into XB and then choose to publish or not?
IOW: require human judgment, because anything automated is unlikely to A) be perfect (pixel perfect match), B) forgiving (a human could judge that actually, it makes sense to use some other component because it is better or simpler or $reason).
An automated batch migration is likely to result in problems, whereas this allows for a nice gradual migration: they can both live side-by-side until every relevant entity has been switched over to XB.
- π¦πΊAustralia larowlan π¦πΊπ.au GMT+10
Sorry, I wasn't clear, I'm not proposing any change to the discussed approach, just validating that the approach is viable. Will focus on π± Milestone 1.0.0-beta1: Start creating non-throwaway sites Active
- Status changed to Downport
5 days ago 12:32am 27 June 2025 - π¦πΊAustralia larowlan π¦πΊπ.au GMT+10
Added π Audit use of ::loadUnchanged Active which we need for sites running content moderation