[META] Production-ready data storage

Proposed to @longwave to close #3468269: #3468269-23: `ComponentTreeStructure` data type: simplify the stored structure → .

Moved a significant chunk out of here and into 🌱 [META] Production-ready ComponentSource plugins Active .

There is a @todo in ComponentTreeItem linking to 🐛 Cannot delete a field which uses JSON type Active so I think that probably needs to be included here?

Opened 🐛 [PP-1] Use JSON schema type for sqlite and remove text workaround Active .

Additional things I think we need for a 1.0

* plugabble/flexible ComponentTreeLoader so that LB/PG can provide a translation layer for legacy data
* it would be good if default values were stored as an array instead of JSON for the sake of git diff on exported config - same reasons as ✨ Create a schema for "allowed_html" which provides a better config diff Needs work - the current json is going to be very painful for content template git conflicts

I think the last point is being tackled in 📌 Stop storing JSON blobs in XB config entities: improve DX Active .

Created

• 📌 [PP-2] Add paging to Component "Audit" operation content listing Postponed
• 📌 [PP-2] Support translations tracking on Component "Audit" operation content listing when using asymmetric translations Postponed
• 📌 [PP-2] Consider translations field usage when uninstalling the module providing a field Postponed

The third one might be a data integrity issue.

Track explicit input schema version: hash (the normalized) explicit input schema, assign version (sequential integer), store version: {hash, provider_version} mapping in Component config entity.

I don't think anything with automated sequential integers is going to work because we have to handle partial deployments of config between sites. Let's say you export a Section from your site that uses a component that is on version 2 of the schema. The SDCs and component config entities might already exist on the destination site, but if it was installed later than the site it was exported from, the same component might only be on version 1 of its schema.

📌 [later phase] Support matching `{type: array, …}` prop shapes Postponed is in.

🔄 📌 Calculate field and component dependencies on save and store them in an easy to retrieve format Active landed!

Added 📌 Stop storing JSON blobs in XB config entities: improve DX Active .

📌 Stop storing JSON blobs in XB config entities: improve DX Active is in 🥳

Added a link to 📌 Version component prop definitions for SDC and Code components Active to item 2.1. Wim and I basically came up with the same thing independently - nice!

WRT dependency/usage tracking: 📌 Calculate field and component dependencies on save and store them in an easy to retrieve format Active landed, and so did its follow-up 📌 [PP-1] Evaluate storing XB field type's "deps_*" columns in separate table Active . This cleaned up the technical debt around FieldTypeUninstallValidator and hence enabled me to close 3 stable blockers, see #3457504-56 for details.
This unblocked:

• short term: robustness-related functionality such as 🐛 Cannot delete JS components due to component depending on them Active
• long term: future functionality such as ✨ Usage info for code components with unpublished changes Postponed

Only remaining: 📌 [later phase] Support props defaults that have config dependencies Postponed . That is a stable blocker too, but won't require data storage changes. So: I'm assigning that subset of this meta a 🟢.

Explain relation to 📌 Allow XB to be used on all node types Active in the issue summary.

Added @larowlan's 📌 [PP-1] Consider not storing the ComponentTreeStructure data type as a JSON blob Postponed and 📌 [PP-1] Spike: Explore storing a hash lookup of component inputs Postponed .

Added 📌 Spike: Explore storing component inputs in separate columns (aka field union) Active .

Removed

debt: we know we'll need to support implicit inputs (e.g. block plugins can declare required + optional "contexts" that allow block plugins behave differently not only to component instance author's explicit wishes ("explicit inputs"), but also based on the component developer's logic ("implicit inputs")

because that doesn't affect the data storage.

Descoped #3523846 per @catch in #3523846-7: [PP-1] Spike: Explore storing component inputs in separate columns (aka field union) → . We may revisit that after 📌 [PP-1] Consider not storing the ComponentTreeStructure data type as a JSON blob Postponed . 👍